Choosing an Inference Mode for a Notification Ranker
A social app wants to rank which of a user's pending notifications to surface first when they open the app, and also decide, throughout the day, whether a new notification is worth a push alert at all. There are 80 M daily active users and roughly 15 notification events per user per day.
- Which inference mode(s) would you use for (a) ranking notifications when the app opens, and (b) deciding whether to push-alert a new event as it happens? Justify each.
- Propose a hybrid design that reduces online cost without hurting freshness for either sub-problem.
- What is the concrete risk of batch-scoring all 80 M users nightly for the "push or not" decision?
1. Mode per sub-problem
(a) Ranking notifications on app open depends on the current session (which notifications exist right now, recency, in-app context) and must reflect events that happened seconds ago — this needs an online step at request time; it cannot be fully precomputed because the notification list changes continuously.
(b) Deciding whether to push-alert a new event is triggered by the event itself, not by a user request, and must react within seconds to be useful (a push about something that happened an hour ago is often worthless) — this is a streaming / event-driven inference problem: score each event as it arrives from the event stream.
2. Hybrid design
Precompute a user propensity profile offline/batch (e.g. "how push-sensitive is this user", "preferred hours", category affinities) refreshed daily. The streaming scorer for (b) combines this precomputed profile (cheap lookup) with a few real-time features (event type, time since last push) — avoiding a full feature computation per event. For (a), precompute embeddings for notification types/senders in batch, and do a lightweight online re-rank of the currently pending set (typically under 20 items) using those embeddings plus session context. Both paths reuse the same offline profile, so cost per event/request is small and freshness comes from the thin online/streaming layer.
3. Risk of nightly batch-scoring "push or not"
Events arrive continuously and unpredictably (15/user/day, roughly one every 90 minutes on average, but bursty). A nightly batch score cannot react to an event that just happened — by the time tomorrow's batch score is used, the relevant event is stale or already gone. It also wastes compute scoring hypothetical events that never occur. Batch is the wrong mode whenever the decision is triggered by an event rather than a request, and the acceptable staleness is seconds, not hours.
Share this question