Practice — Adversarial ML & Model Security (6 questions)

You are the ML lead for a short-video platform. A ranking model trained on watch time, likes and shares orders the home feed; creators upload videos and anyone can create accounts; partner apps call a public "similar videos" API that returns scored candidates. The security team asks for a threat model of the recommender before the next launch.

1. List the assets and the attacker profiles (knowledge, access, goal, budget). Do not start from attack names.
2. Walk the trust boundaries with STRIDE and name at least six concrete attacks.
3. Rank them by likelihood and impact, and say how honest those scores are.
4. Pick the top three mitigations and say what each costs.
5. Which parts of this are not this lesson's problem, and where do they go?

A payments fraud model is retrained every week on three label sources: chargebacks (arrive 30 to 60 days late), analyst decisions, and merchant "this was legitimate" feedback submitted through an API. A red team shows that a small group of colluding merchants can submit crafted feedback so that a specific transaction pattern is later scored as safe, while ordinary test metrics do not move.

1. Explain the attack in terms of what the attacker controls, and why the usual offline accuracy gate does not catch it.
2. Design defenses at four layers: data intake, training, promotion gate, and production. Say what each costs.
3. Which of the defenses did the lesson's toy experiments support or fail to support, and what does that mean for how you rely on them?
4. Write the promotion-gate checklist you would actually enforce.