---
source: https://info.defcon.org/defcon34/content/67495
---
- [Lee McWhorter](https://info.defcon.org/defcon34/people/68613 "Lee McWhorter")
- [Alexander "Zombie"](https://info.defcon.org/defcon34/people/68687 "Alexander \"Zombie\"")

## Overview

AI defenses are only as good as the analysts training them. This session exposes how attackers manipulate SOC analyst behavior to silently degrade AI baselines over time, no exploit required. We cover the mechanics of false positive injection, alert fatigue as a weapon, and misdirection. Then attendees step into the analyst's seat in a live browser-based simulation and experience how their own decisions poison the model.

## Notes

### Workshop
- Wasn't able to stay for the full hands-on workshop portion
- Presenter said the presentation and additional notes would be posted at https://largeliabilitymodel.com/, where others can also try the workshop themselves

### OODA Loop
- Observe
- Orient
- Decide
- Act

- Every dismissed alert today trains the AI's Orient step for the future

### Issues
- Fatigue: AI has increased the number of alerts; it handles some of them but leads to more work for human analysts
	- This leads to burnout and fatigue for human analysts
- False positives: what happens when the human analyst and the AI disagree?
- AI is a tool, not a replacement
	- It's bound by its instruction set; a human can deviate and read between the lines better
	- Craft a system where when the AI calls a true positive and the human calls a false positive, those analysts can train the AI to be better

### Red Team Perspective on Manipulating an AI SOC
- Attack in a way that gets the AI to flag an analyst's attention, so the human follows the AI's lead, then follow up with the real attack
- With enough of the behavior persisting, the AI will consider it normal and stop flagging you
- Gauging the SOC
	- See if things get booted or kicked
	- Is the model still in learning mode?
	- Is this an in-house tool or a service?
- Start with small stuff, then escalate to see how long it takes to respond
	- Then a "dismissal" campaign starts with what was learned in recon, tailoring the noise:
		- Match the environment
		- Vary the source, spread out activity
		- Want to generate alerts, but not a full response
		- Measure for investigative scans — were our alerts closed, or not investigated at all?
	- How to:
		- Profile the human/AI setup
		- Identify what the human will dismiss
		- Pick a TTP that forces a human decision
		- Set a cadence the human can dismiss
		- Execute once the AI trusts it

### TTPs That Generate Dismissals
- Blend into scheduled tasks
- Abuse trusted binaries
- Mimic sysadmin work
- Trigger known noisy signatures
- Low and slow enumeration
- Target shift change windows
- Password spraying — dismissed a lot, since people don't type their passwords in very often

### Case Study: Scranton
- EDR vendor shipped an update that changed the log format without updating the AI layer
	- This exhausted the API token budget, and the AI stopped triggering alerts
- Fallout
	- Human analysts had to be reassigned from other tasks
	- Trust in the EDR and AI system eroded

### Timing and Patience as Weapons
- Time works against human analysts
	- Fatigue accumulates
	- Context gets lost, shifts rotate
	- Alert blindness sets in
- Time also works against the AI
	- Confidence builds with repetition
	- The model cannot forget, and every dismissal compounds
	- Retraining windows are your deadline — models get updated, and new training runs can undo what's been built

### How Do You Know the Campaign Is Working?
- Your noise stops generating follow-up activity
- No escalation after repeated triggers
- The analyst is overriding the AI
	- The human in the loop now becomes part of the attack surface
- Your real activity goes quiet — no scan blocks or IR activity