Case Study: Long-Term Engagement Recommender
"Our feed ranker optimizes click-through rate, and it's working — CTR is up every quarter — but users are churning and nobody can explain why" is a sentence a surprising number of consumer companies eventually say out loud. The mechanism is not mysterious once you look for it: a model trained to predict and maximize the next click will happily learn that outrage, cliffhangers and slot-machine-style variable rewards produce clicks, and it has no way to know that the user who clicked today is quietly souring on the product and will be gone in six weeks. "Redesign the feed to optimize for long-term engagement instead of clicks" is the system design question that tests whether a candidate understands why this happens and, more importantly, what to actually build instead of just diagnosing the problem.
This case study deliberately overlaps with the Long-Term Value & Delayed Reward Systems subject, and says so up front rather than pretending otherwise. That subject teaches the general theory of optimizing for outcomes that mature long after the decision that caused them — surrogate reward construction, delayed-feedback training, and long-horizon measurement — as a standalone topic. This case study does not re-derive that theory. It applies those ideas to one concrete system: a recommendation feed, built on top of the retrieval → ranking → re-ranking funnel taught in the Ranking & Recommendation System Architecture subject. If you have not read Long-Term Value & Delayed Reward Systems and Ranking & Recommendation System Architecture, read them first — this subject's job is to show how their ideas combine in one interview-ready answer, not to teach either from scratch. It also uses the sequential-decision framing from RL Foundations: MDPs & Value Functions and the reward-shaping discipline from Reward Design & Delayed Credit Assignment where the feed's ranking decisions are treated as a sequence rather than independent events.
Read it as the answer you would give in 40 minutes, following the same step structure as the ML System Design Framework subject, then use the follow-up questions near the end to pressure-test yourself.