Module 12 Lesson 2 of 6 🕑 ~60 min

> cat module-12-2-containment-eradication-recovery.md

Containment, Eradication & Recovery

This is the technical heart of incident response — stopping the bleeding, removing the attacker for good, and bringing the business back online without letting them straight back in.

1 Containment Basics

Investigation determines what's happening. Containment tries to stop the attacker causing more damage — isolating an endpoint, disabling a user or admin account, revoking tokens, resetting passwords, blocking an IP or domain, removing a malicious OAuth application, disabling API keys or VPN access, quarantining email, blocking a file hash, segmenting the network, disabling a switch port, restricting firewall rules, or taking a server offline.

Short-term containment
Immediate emergency action — e.g. EDR isolates a laptop, stopping its network communication instantly.
Long-term containment
More sustainable measures while investigation continues — e.g. a compromised application moved behind a restrictive firewall policy, credentials rotated, monitoring increased, a temporary clean server introduced in its place.

2 Containment Is a Business Decision

Technical analysts often default to "compromised server = shut it down." Sometimes that's correct. Sometimes it isn't.

🔮 Think it through

Imagine the compromised server controls hospital equipment, electricity distribution, industrial production, payment processing, aircraft operations, or emergency communications. Why might immediately shutting it down create more risk than temporarily monitoring the attacker?

Reveal the reasoning

For safety-critical or life-critical systems, an uncontrolled shutdown can itself cause harm — a hospital device going dark mid-procedure, a power grid segment dropping unexpectedly, a plant halting in an unsafe state. Containment decisions have to weigh security risk against business risk, safety risk, regulatory risk, operational impact, and evidence preservation together, not security risk alone. Sometimes the right call really is closely monitoring a contained attacker for a short window while a safer shutdown is arranged.

3 Identity & Domain Controller Containment

Identity is now one of the most important containment areas. If an attacker compromises a privileged identity, isolating their laptop accomplishes very little — they don't need that laptop anymore. Responders may need to disable the account, revoke active sessions, reset the password, revoke refresh tokens, remove authentication methods (including any the attacker added), disable suspicious OAuth grants, rotate API keys and service-account passwords and certificates, and review privileged group memberships. For highly privileged compromise, this can mean emergency password rotation across large parts of the environment at once.

A compromised Domain Controller should be treated as extremely serious. Attackers who control Active Directory can create administrators, reset passwords, deploy software via Group Policy, access servers, impersonate users, create persistence, and manipulate authentication itself. Simply deleting malware from one Domain Controller doesn't make the environment trustworthy again — it usually means the organisation needs major identity remediation, not a quick cleanup.

4 Cloud & Ransomware Containment

Cloud containment looks different from on-prem: disable the IAM user, disable the access key, revoke the API token, remove the IAM role, modify the security group, block a network route, quarantine a virtual machine, apply an organisation-level policy, disable a compromised application, rotate cloud secrets. The old advice "unplug the network cable" simply doesn't apply to serverless applications or SaaS services — there's no cable to pull.

Ransomware containment is a speed problem the moment it starts spreading: isolate affected endpoints, disable compromised accounts, temporarily restrict SMB traffic, block known malicious infrastructure, disable attacker-controlled remote-access tools, segment network zones, restrict lateral movement, protect backup infrastructure, disconnect vulnerable systems, disable compromised VPN access, revoke cloud sessions, and preserve forensic evidence throughout. CISA recommends identifying and isolating impacted systems immediately — and where compromise is widespread, broader network-level isolation rather than trying to disconnect thousands of individual devices one at a time.

5 Eradication: Root Cause, Not Symptoms

Containment stops the immediate damage. Eradication removes the attacker and the mechanisms that let them in — malware, malicious accounts, persistence, scheduled tasks, malicious services, vulnerable configurations, rogue OAuth apps, malicious mail-forwarding rules — plus patching, credential rotation, and certificate/API key replacement.

Suppose ransomware is removed from ten servers, but the original compromise happened because a VPN password was reused and MFA wasn't enabled on that account. Unless that VPN account gets secured, the attacker simply comes back through the same door. Effective eradication traces the whole chain:

SYMPTOM             Ransomware encrypts file server
IMMEDIATE CAUSE      Attacker executed ransomware
ENABLING CONDITION   Attacker obtained Domain Admin
EARLIER CAUSE        Credentials dumped from admin workstation
INITIAL ACCESS       Phishing email
CONTROL FAILURE      Admin account used for email + ordinary workstation activity

Fixing only the top line and calling it done is exactly how the same organisation gets hit again six months later.

6 Rebuild vs Clean, and Legacy Eradication

A common decision: clean the system, or rebuild it? For serious compromises, rebuilding from a trusted source is usually safer than trying to surgically remove everything an attacker touched:

Compromised server → Preserve forensic image → Destroy/reformat
   → Deploy clean OS → Patch → Install security tools → Restore validated data

The goal is a known-good state, not a probably-clean one.

Legacy technology makes eradication genuinely harder. If a critical manufacturing application runs on Windows Server 2008 and the vendor no longer supports it, you can't just upgrade overnight. Compensating controls fill the gap instead: network isolation, application allowlisting, strict firewalling, restricted administration, jump servers, enhanced monitoring, virtual patching, or removing Internet connectivity entirely. "Patch everything immediately" is sometimes technically impossible — enterprise security frequently means managing imperfect environments rather than pretending they're perfect ones.

7 Recovery Priorities & the Clean Recovery Environment

Recovery returns systems and business operations to normal — but it doesn't mean "turn everything back on." The organisation should have predefined priorities, respecting dependencies:

1. Identity services   2. DNS   3. Core networking   4. Security infrastructure
5. Backup infrastructure   6. Databases   7. ERP
8. Customer services   9. Internal applications   10. User workstations

Restoring an application before DNS, Active Directory, its database, certificates, API gateway and load balancer are back would simply fail — the dependency chain has to be respected, not just the priority order.

Large ransomware responses often build a separate clean recovery environment — new Domain Controllers, clean servers, validated backups, restored applications, deliberately kept apart from the compromised network — reducing the risk that still-infected infrastructure recompromises whatever just got rebuilt.

8 Backup Recovery, RPO/RTO & Validation

Backups are critical, but backup does not automatically equal recovery. Attackers increasingly try to compromise backup infrastructure before deploying ransomware, so responders need to check: are backups intact? Was the backup admin account compromised? Did attackers delete snapshots or steal backup credentials? Are backups encrypted? Are offline/immutable copies available? When was the last clean backup? Restoring an infected backup can simply reintroduce the attacker you just removed.

RPO — Recovery Point Objective
How much data can the business afford to lose? E.g. RPO = 4 hours means accepting up to four hours of lost data.
RTO — Recovery Time Objective
How long can the system stay unavailable? E.g. RTO = 8 hours means the business expects restoration within eight hours.

These targets shape backup architecture and recovery priorities long before any incident happens. Before restoring to production, verify malware is removed, credentials rotated, patches installed, persistence removed, EDR and logging operational, firewall policy correct, backups validated, applications tested, data integrity checked, and accounts reviewed. Recover progressively — restore 5 systems, monitor, restore 20, monitor, restore the rest — rather than reconnecting 5,000 computers simultaneously and hoping nothing was missed. Afterward, increase monitoring specifically for previous attacker IPs/domains/hashes, unexpected admin logins, persistence mechanisms, recreated accounts, suspicious remote tools, and unusual outbound traffic — the absence of new alerts doesn't automatically mean the attacker is gone.

Lesson Outcome

You should now be able to distinguish short-term from long-term containment, explain why containment is a business decision rather than a purely technical one, describe identity/Domain Controller/cloud/ransomware-specific containment, trace a root cause instead of just removing a symptom, choose between rebuilding and cleaning a system, and explain how RPO/RTO and progressive recovery keep a restoration from reintroducing the attacker. Lesson 3 covers what happens after systems are back up: lessons learned, metrics, and how to communicate through all of this.