Adversarial ML & Model Security
Threat modeling ML systems — evasion, poisoning, extraction, privacy attacks and the model supply chain
Threat-model an ML system the way a security engineer would and defend the answer in an interview: assets, attacker knowledge and access (white-box versus black-box), a STRIDE-for-ML table and MITRE ATLAS; evasion (FGSM and PGD with an exact linear-model derivation, accuracy-versus-epsilon curves on a 20-dimensional toy task where a logistic regression falls from 96.6 to 82.7 percent at epsilon 0.3 and an MLP to 75.0 percent, transferability to independently trained models, adversarial training that moved MLP robust accuracy from 75.0 to 88.7 percent at almost no clean-accuracy cost until epsilon approaches the signal size, a gradient-masking demonstration where a rounding preprocessor makes a gradient attack report 95.7 percent accuracy while a BPDA attack gets 76.0, and a cost-constrained fraud-feature attack that evades a gradient-boosted model 98.3 percent of the time with three mutable features); poisoning and backdoors (random versus targeted label flips, a 1 percent trigger backdoor with 98.0 percent attack success and unchanged clean accuracy, a spectral-signature filter that only partly worked, provenance filtering that cut attack success to 4.6 percent); privacy attacks (loss-threshold membership inference, AUC and TPR at 1 percent FPR versus generalization gap, output limiting, DP-style training); model theft (extraction fidelity versus query count for logistic and MLP victims, rounding and label-only outputs); supply chain (pickle code execution, safetensors, hub trojans, dependency attacks, SBOM, signing, registry access); serving security including a sponge-input cost multiple measured on an attention-like operation; LLM and agent bridges; governance and incident response; and a worked fraud-API threat model with five mitigations tied to the measured numbers. All attacks run on small synthetic models with numpy, scikit-learn or PyTorch on a CPU; likelihood and impact scores are labeled judgement; library, paper, framework and legal facts are from memory.
Practice questions (6)
-
View →
Threat-Model a Video Recommender Before Anyone Proposes a Defense
Advanced · Free -
View →
Defend a Weekly-Retrained Fraud Model Against Feedback Poisoning
Advanced · Free -
View →
Protect a Pricing Model Exposed Through a Partner API From Extraction
Advanced -
View →
Build a Supply-Chain Checklist for Adopting a Third-Party Model
Advanced -
View →
Assess Membership-Inference Risk Before Releasing a Clinical Model
Advanced