Adversarial ML & Model Security

The boundary. Guardrails and Prompt-Injection Defense covers attacks that arrive as text prompts to an LLM (jailbreaks, indirect injection, output filtering), and Claude Code Sandboxing and Security covers sandboxing and permissions for coding agents. This lesson is the classical ML threat model plus the model supply chain: how someone fools a classifier with crafted inputs, corrupts what it learns, extracts what it knows or what it was trained on, steals it, or ships you a malicious model file. It is defensive education: every attack below is a few lines on a toy model, the point is the measurement and the mitigation, not an operational recipe against any real system. Where an LLM or agent angle exists, Section 8 points to the lessons above.

Interviews ask "how would you secure an ML system?" and score you on whether you start from a threat model instead of a list of attacks, know the difference between a white-box and a black-box attacker, can say what a defense costs, and refuse to claim a model is "robust" on weak evidence. Every number labeled verified comes from a fixed-seed run on a CPU (scripts lived in /tmp/advml/, sample sizes stated; most evasion numbers are single-model runs on 5,000 test points, so differences under about one point are noise; the privacy rows average 10 seeds). All data are synthetic toy tasks: a 20-dimensional Gaussian task, a 9-feature fraud-like table, and so on. They show mechanisms and orders of magnitude, not the success rate of any attack on any real system. Facts about papers, frameworks, library features and law are from memory and hedged. Likelihood and impact scores in the worked example are judgement/ASSUMPTION.