Incident Response: The First Hour After a Key Looks Compromised

Telling an incident from a bug

When a key looks compromised, the first hour follows a fixed order: stop the software, move value beyond reach, revoke delegates and sessions, rotate every key the machine could see, rebuild on clean hardware, and only then reconstruct the timeline. Each step is placed where it is because of what it costs and what it protects. Getting the order wrong is how operators spend the hour revoking approvals on accounts that were emptied while they worked.

Before any of that, be sure you are in an incident. A misconfigured strategy and a stolen key look similar on a dashboard and demand completely different responses. The distinction is not about how alarming the number is. It is about whether the activity could have come from your own software at all, and the honest answer to that question is usually available within a minute if you know what to look at.

Ambiguous cases exist and should be treated as incidents until proven otherwise. Containment is recoverable: if you stop automation and move balances and it turns out the cause was a bug, you have lost some time and some fee lamports. The opposite mistake is not recoverable. Bias toward containment, and write down which way you decided and why, because that note is what makes the eventual finding credible rather than a reconstruction.

Grading severity in the first minutes

Severity determines how far the response has to reach, not how fast you start. You always start immediately. What grading changes is the size of the rotation, the number of accounts that need sweeping, and whether the answer involves a handful of execution wallets or every account derived from one phrase. Read the following as a set of conditions to check rather than as a scale to argue about.

• A seed phrase was exposed, entered anywhere, photographed or stored in a synced location. Every account derived from it is affected, present and future.
• A CLI keypair file was readable by an attacker. That file is the private key, and copying it is all that was required.
• The host that loads signing keys ran unknown code, an unverified dependency update, or a binary from an unexpected source.
• A signing key was pasted into a website, a chat, a support ticket, a screenshot or a cloud note, however briefly.
• Only an API or RPC credential was exposed, with no access to key material. Reach is limited to what that credential could do.
• A token approval was granted to an unknown program, with no key exposure. The exposure is bounded by the delegated amount on that token account.
• A read-only credential or a public address was exposed. There is no signing capability, though it is a privacy and targeting concern.

Grade quickly and grade upward. The cost of treating a bounded approval incident as a full key compromise is a rotation you did not strictly need; the cost of treating a key compromise as a bounded approval is everything the key controls. If two of the conditions above are true at once, respond to the broader one, because the narrower response is contained inside it anyway.

The containment order

The table below is the whole playbook in compressed form. Read the third column carefully, because the ordering is the argument being made: each action is placed where it is because of what would be lost by doing it later. The final column exists to stop a false sense of completion, since every one of these actions leaves something unaddressed that a later action has to pick up.

Two orderings in that table are worth stating plainly because they are the ones people reverse under pressure. Software is stopped before funds are moved, since a running strategy will sign against the same accounts you are trying to empty. Funds are moved before anything is revoked, since revocation is a sequence of transactions across many accounts and a balance does not wait for it to finish.

The first-response timeline

Work these steps in sequence. Do not parallelise them across people unless you have agreed in advance who owns which step, because two operators signing from the same keys at the same time produce failures that look exactly like an attack and waste the minutes you are trying to save. One person drives, one person writes down what happened.

1. Mark the time and the trigger. Write the wall clock time and the specific observation that started this. Every timestamp you collect later is read against that mark, and the trigger itself is often the first clue about the entry point.
2. Stop every signer. Halt automation on every host, not only the one you suspect. A strategy left running will continue to sign, and its transactions will be indistinguishable from an attacker in your later reading of the history.
3. Cut the credentials the software held. Invalidate API keys, RPC keys and service tokens. A configuration file copied off the host keeps working from anywhere until the credentials in it are dead.
4. Isolate the host, do not wipe it. Disconnect it from the network so it cannot act, and leave the disk untouched. Wiping is the single most common way an incident becomes permanently unexplainable.
5. Choose a destination you can prove you control. Use a key generated on hardware the suspect machine never touched, and confirm you can sign with it before sending anything to it. Never improvise a destination address under pressure.
6. Move value, largest balance first. Sweep the accounts holding the most first, then work down. Leave each account enough to pay for the signature that empties it, since a balance below the fee cannot move itself.
7. Revoke every delegate. List the token accounts owned by the affected keys and revoke each delegate. An allowance is per token account and per amount, so one revocation does not cover the rest.
8. Disconnect sessions and site connections. Remove connected sites from the affected wallets. Where a session cannot be positively disconnected, treat it as live and rely on the fact that the account is now empty and the key is being retired.
9. Rotate on clean hardware. Generate replacement keys on a machine you trust, record their addresses in the account map, and retire the old ones formally rather than leaving them in a file marked old.
10. Archive the evidence. Collect signatures, log files, configuration snapshots and shell history into a read-only archive with the collection time noted. Do this before any reinstall touches the machine.
11. Rebuild from known-good media. Reinstall the operating environment rather than cleaning it, and reinstall the automation from a source you control and can verify.
12. Reconstruct and close the gap. Trace backwards from the earliest unauthorised signature to the change that made it possible. The output of the incident is a changed rule, not a restored balance.

Stopping the software before touching funds

Stopping the signer is placed first because it is nearly free and it removes an entire class of confusion. While automation runs, it produces signatures that look like activity, competes for the same lamports you are trying to sweep, and can push a balance below the fee threshold at the exact moment you need it to pay for one more transfer. Kill the process, then confirm it is dead rather than assuming the stop command was honoured.

Credentials come with it. Automation typically holds an RPC key, sometimes an exchange or data credential, occasionally a webhook secret. Any of those copied from the host stays valid until it is invalidated, and none of them are protected by the fact that the process is no longer running on your machine. Invalidate them from a device you trust, not from the host under suspicion, and record which ones you cut and when.

Alerting is the reason this step is often possible at all. If nothing was watching, the first sign of trouble is a balance you noticed by chance, and the hour you are now working started long after the attacker did. The signals worth waking someone for, and the thresholds that keep an alert channel useful, are covered across the monitoring hub, and the difference they make is measured entirely in how early this playbook starts.

Moving value out of reach

Moving value is a race with an opponent who may already have started. Sweep the largest balances first, because the ordering that minimises loss is the one that removes the most value per transaction, not the one that visits accounts in the order they appear in your config. Send to an address whose key was generated on hardware the suspect machine never touched, and prove you can sign with that key before you send anything to it.

Not everything will move. Tokens with no liquid destination, positions locked in a program, and accounts whose fee balance has already been drained will resist a clean sweep. Take what you can, note what you could not, and do not spend the minutes arguing with a stuck account while liquid balances remain exposed. Return to the difficult accounts after the easy value is safe.

Choosing the destination in advance is what makes this step fast, and improvising one is what makes it dangerous. A destination decided under pressure is a destination that gets mistyped, or one whose key is sitting on the same machine you are running from. Keep a designated cold destination recorded in the account map, generated and verified before any incident, following the practice described in key storage and rotation.

Revoking delegates and disconnecting sessions

A delegate is an approval granted on a specific token account for a specific amount, and it survives the account being emptied. That matters because an emptied account is not a closed account, and anything that arrives in it later, including a refund, an airdrop or a mistaken deposit, can be drawn by whoever still holds the approval. Revocation is therefore not optional cleanup; it is the step that stops the incident from continuing quietly after you believe it is over.

Enumerate rather than remember. List the token accounts owned by each affected key and check each one, because approvals are per token account and an operator who granted one approval months ago has usually forgotten the other two. The approve and revoke instructions and their per-account semantics are documented in the SPL Token documentation.

solana config get
solana balance <PUBKEY>
solana transaction-history <PUBKEY>
solana confirm -v <SIGNATURE>
spl-token accounts --owner <PUBKEY>
spl-token display <TOKEN_ACCOUNT>
spl-token revoke <TOKEN_ACCOUNT>
solana transfer <SAFE_PUBKEY> ALL --from <COMPROMISED_KEYPAIR> --allow-unfunded-recipient

Session disconnection is the weakest part of this step and should be treated accordingly. Removing a connected site from a wallet is worth doing, but you cannot always confirm that every session everywhere is gone, and a session you cannot verify should be assumed live. The real protection is that the account is empty and the key is on its way to retirement, which is why this step sits behind the sweep rather than in front of it.

Rotating what the machine could see

Rotation covers everything the suspect host could read, not only the key you saw misused. If signing keys shared a directory, were loaded by the same process, or were unlocked by the same passphrase in memory, they belong in the same rotation. A CLI keypair file is a JSON array of bytes and is equal to the private key itself, so file system read access is the whole attack; no further exploitation was required.

Seed phrases widen the rotation dramatically. Derived accounts use BIP44 style paths from a single seed, and the phrase controls every account derived from it, including ones you have not derived yet. An exposed phrase therefore cannot be partially rotated: every account under it is retired at once, and the replacement is a new phrase generated on hardware that has never been connected to the compromised environment.

Generate replacements on clean hardware and record them properly. A rotation that leaves the old key file lying around under a new name, or that never updates the account map, produces the worst outcome available: an operator who believes they rotated, funding an address that a retired key still controls. Write the new addresses into the map, mark the old ones retired with the date, and confirm the signing capability of each new key before the first funding transfer reaches it.

Evidence before rebuild

Evidence is destroyed by the same actions that make you feel productive. Reinstalling the host, rotating the log files, clearing the shell history, redeploying the automation and tidying the configuration all remove the record of what actually happened. Collect first, then rebuild. The collection does not need to be sophisticated, only complete and untouched afterwards.

Collect the signature of every unauthorised transaction and the block time attached to it, the log lines around the first anomaly, the configuration file as it existed at the time, the shell history, and a list of what was installed or updated in the preceding days. Store the archive read-only, away from the affected environment, with the time of collection recorded. Signatures matter most, because they are the one part of the record an attacker cannot edit and you cannot lose.

Log quality decides whether the timeline is reconstructable at all. A log that records the instruction, the signer, the destination and the outcome makes the first unauthorised action obvious; a log of connection failures and stack traces does not. The schema that supports this, and the fields that must never be written to a log file in the first place, are set out in reading bot logs.

What segmentation limits the loss to

Segmentation does not prevent a compromise. What it does is bound the amount that a single exposed host can reach, and that bound is arithmetic you can work out before anything goes wrong rather than after. The example below uses protocol constants for the account minimums and an arbitrary total to make the shape visible; substitute your own figures and the conclusion changes only in size.

Read that result carefully, because it is a bound on exposure and not a claim of safety. The 10 SOL reserve is only out of reach if its key genuinely never touched the affected machine, and the treasury is only separate if the separate machine was not compromised by the same route. Segmentation that exists on a diagram and not in the key material buys nothing. How the tiers are defined, capped and refilled is the subject of the wallet segmentation strategy.

The second attack

People who have just lost value are targeted again within hours, and the second attempt is usually more effective than the first because it arrives as help. Offers to trace, reverse or recover stolen funds appear in replies, direct messages and search results, and they ask for the one thing that guarantees the remaining balance is gone: a seed phrase, a keypair file, or a signature on a transaction you did not read. Treat every unsolicited offer of recovery as hostile by default.

Hold two rules without exception. A seed phrase is never entered into any tool, site, form or conversation for any reason, including a reason that sounds technical and urgent. A keypair file is never shared with anyone, including someone who appears to work for a service you use, because that file is the key and sharing it is the compromise. Legitimate help never requires either, and anything that does is the attack rather than the remedy.

Close the incident with a written finding rather than a feeling of resolution. Name the entry point if the evidence supports one, say plainly where the evidence ran out if it does not, list the keys retired and the balances moved, and state the one process change that would have prevented it. That document is what makes the next incident shorter, and writing it while the details are fresh is the last step of the first hour rather than a task for later.

Questions the desk gets asked