Designing, instrumenting and running an incident response exercise: another team takes the SOC analyst shift at a small maple syrup producer and must detect, assess and contain an incident unfolding in real time, without shutting production down.
2 h
SOC shift in real time, at most
5
log sources centralised in Splunk
6 wk
sugaring season, when any shutdown costs the most
What the SOC sees, source by source.
Scroll: each source lights up, along with its devices, as its logs reach Splunk.
Three segmented zones, a single crossing point: the router-firewall.
Internet
Marketplace website
Router-firewall192.168.0.11
Website LANWebsite
2
192.168.1.0/24
Reverse proxy .1
Apache .2
E-commerce application server .3
E-commerce database .4
Administrative LANAdmin
346
192.168.2.0/24
Active Directory .1
Bastion .2
Splunk (SIEM) .3
IT admin workstation .4
HR · Finance · Sales · Management laptops .5–.8
Production LANProduction
5
192.168.3.0/24
SCADA server (supervision) .1
HMI operator station .2
IoT gateway
Valve position sensor
Temperature sensor
Valve control PLC
Érablière Boréale IT
Source — / 06
01Router-firewall
Firewall: every flow between zones.
Nothing moves from one zone to another, or out to the Internet, without crossing the router-firewall. Its logs say who talks to whom.
02Website LAN
Application servers: the exposed face.
Reverse proxy, Apache and e-commerce server: the logs of the only service published on the Internet.
03Administrative LAN
Active Directory: authentications.
Every sign-in and every account change, from servers to office laptops.
04Administrative LAN
Bastion: administration sessions.
The mandatory path to administer a server remotely: every session is recorded.
05Production LAN
Sensors: the physical process.
Valve position and temperature join the logs: production availability and integrity become security signals.
06Administrative LAN
Splunk correlates: one console.
All five sources converge on the central SIEM, the analyst's main source of alerts.
01
Designing and running the exercise
A three-zone architecture behind a router-firewall, logs centralised in Splunk, a bastion for administration, and scripts that inject logs, traffic and alerts at the pace of a real incident. Then the participant brief, the rules of engagement and live facilitation.
02
Defending another team's
The opposite seat: SOC analyst on an environment designed by another team. Separating signal from noise, assessing, requesting proportionate containment and keeping the incident log as events unfold.
Two hours, four stages.
Stimuli injected live, with neither content nor timing known in advance, and a status update roughly every 15 minutes, even when nothing has changed.
How the session runs
T+0 → 10 min
Briefing
Context, access hand-over, tool checks, rules of engagement.
T+10 → 100 min
Active phase
Stimuli come in: Splunk alerts, emails, employee reports, requests from management.
T+100 → 115 min
Wrap-up
Final containment and recovery actions.
T+115 → 120 min
Summary
Incident log finalised, last status updates sent.
The expected cycle, for every stimulus.
A more intrusive measure isn't always the best one: in the middle of sugaring season, isolating production without need can cost more than the incident.
Method and rules of engagement
1
Detect
Separate the signal from the noise, correlate it with other sources, rule out false positives.
2
Assess
Assign a severity and a likely scope (zone, assets affected), and justify it.
3
Contain
Proportionate measures, declared and timestamped. Never from a command line on production.
4
Eradicate
Deal with the cause: the account, access or component behind it.
5
Recover
Confirm things are back to normal before closing the incident.
6
Communicate
Keep stakeholders informed at every significant change in status.
Practise decision-making under pressure: contain an incident without sacrificing production, and leave a record management can act on.