Case Study: Search Ranking (Airbnb / E-commerce Marketplace Search)

"Design the search ranking system for Airbnb" (or Etsy, Booking.com, Amazon Marketplace) is one of the most common ML system design prompts because it forces you to combine almost everything in the track: retrieval, learning-to-rank, biased implicit feedback, multi-objective optimisation, low-latency serving and online experimentation. It also has a twist that generic recommendation prompts lack: the marketplace is two-sided. A ranking that maximises what guests click can starve new hosts of exposure, flood popular hosts with requests they decline, and hurt long-term supply.

This subject is written as a model answer following the 7-step framework from the ML System Design Framework subject: clarify requirements → business-to-ML objective → data and labels → features → model → offline and online evaluation → serving and monitoring. It applies the learning-to-rank and multi-stage ranking theory covered in the Ranking and Recommendation Systems subject rather than re-teaching it; where a concept is used, it is named and moved past.

Read it as a script for a 45-minute interview. Every section says what to state out loud, what numbers to assume, and which trade-off the interviewer is fishing for.

Step 1 — Clarify Requirements and Scale

Never start drawing boxes before pinning these down. Say the assumptions explicitly; interviewers reward numbers even when they are guesses.

Functional requirements

• Input: a query = destination text (or map bounds), check-in/check-out dates, guest count, optional filters (price range, amenities, property type). For an e-commerce marketplace substitute keyword + category + filters.
• Output: a ranked page of listings (about 20 per page, users typically browse one to three pages).
• Personalisation: results may differ per user (past bookings, wishlists, price sensitivity, session behaviour) but must still make sense for logged-out users.
• Two-sided marketplace goals: rank for the guest (find a stay they will book and enjoy) and for hosts / supply health (new listings get a chance, requests go to hosts likely to accept, no single host monopolises page one).

Non-functional requirements

• Latency: end-to-end p99 around 200 ms for the search response; ranking itself gets a slice of that budget.
• Availability: search is the top of the funnel; degrade gracefully (fall back to a heuristic sort) rather than fail.
• Freshness: availability calendars and prices change by the minute; a listing booked five seconds ago must not appear as available.

Scale assumptions (state them, then design to them)

That last row matters: implicit-feedback training data is enormous but bookings (the label you care about) are ~1% of searches. Everything downstream is shaped by that sparsity.