Practice — ML System Design Interview Framework (5 questions)

A product manager at a short-video app says: "We want people to spend more time in the app. Build a model for the For You feed."

1. Propose a concrete ML objective (label, prediction unit, decision) that serves this goal, and name the gap between your proxy and the true business goal.
2. Give one way the naive proxy could be gamed by content creators, and how you would adjust the objective or the re-ranking layer to defend against it.
3. Which offline metric and which online metric would you report, and why are they different?

You are asked to size the ranking tier of a news-feed recommender. Assumptions: 20 M daily active users, 8 feed loads per user per day, peak traffic is 3× the daily average, and each feed load ranks 400 candidates using a model with 150 float32 features per (user, item) pair.

1. Estimate average and peak QPS, and peak item-scorings per second.
2. Estimate the feature bytes that must be assembled per request and the resulting peak feature-read bandwidth. What design decision does that force?
3. If the item catalogue is 5 M items with 128-dimensional float32 embeddings, how big is the item embedding table, and how would you serve it?

Three teams are each starting an ML system. For each, pick the first model you would ship and the next rung you would try, and justify both in terms of data volume, features, and latency budget.

1. A B2B SaaS company with 40,000 customers wants to predict which accounts will churn next quarter. They have 3 years of monthly usage snapshots and a sales team that will act on a list of 200 accounts per month.
2. A marketplace with 50 M listings wants "similar items" on every product page, served in under 30 ms, with 10 M pageviews per day.
3. A payments company wants to block fraudulent card transactions in real time; 0.2 % of 100 M monthly transactions are fraudulent, and chargebacks arrive 30–90 days later.

A candidate designs a fraud model and reports "ROC-AUC 0.98 on a random 80/20 split, so we are ready to ship." The fraud rate is 0.1 % and the manual review team can inspect 500 transactions per day out of 2 M.

1. Identify at least three problems with this evaluation.
2. Propose the evaluation you would present instead, including the split strategy and the metric tied to the review capacity.
3. Explain how you would validate the model online without exposing the business to unbounded fraud losses.

A search product has an end-to-end p99 budget of 250 ms for the results page. Network, page assembly and non-ML services consume 120 ms, leaving 130 ms for the ML path. The corpus is 200 M documents.

1. Sketch a candidate-generation → ranking → re-ranking funnel and assign a rough latency budget and candidate count to each stage.
2. A colleague proposes a cross-encoder model that jointly encodes (query, document) at ~4 ms per pair on the available hardware. Where, if anywhere, can it fit?
3. Name two things you would monitor for this funnel that are specific to ML rather than generic service health.