Choosing the Randomisation Unit in a Marketplace

A food-delivery marketplace wants to test a new courier dispatch algorithm that assigns orders to couriers faster. The primary metric is delivery time; the guardrail is courier utilisation.

1. Explain why randomising individual orders (or customers) to the new algorithm produces a biased estimate of the effect of a full rollout.
2. Propose a design that avoids the bias, and describe its main cost.
3. Separately, a colleague suggests randomising by page view for a change to the restaurant-ranking algorithm and reporting 30-day customer retention. What is wrong with that?