Speculative Decoding Break-Even: Which Acceptance Rate and Draft Length Pay Off?

A team wants to enable speculative decoding with a draft model for a 70B target. Draft cost: each draft token costs c = 0.10 of a target decode step. Measured per-token acceptance on their traffic (assume independent): \alpha = 0.65 for open-ended chat and \alpha = 0.85 for code completion. At night the replica runs at low batch; at peak it runs at batch 256, where verifying k+1 tokens per sequence costs \rho \approx 1.7 plain decode steps (at low batch \rho = 1.0; a mid-load value is \rho \approx 1.3).

1. Derive the expected tokens produced per round and the speedup formula, with the draft cost and the verify ratio \rho.
2. Compute the speedup at \alpha = 0.65 and 0.85 for k = 4 at \rho = 1.0 and at \rho = 1.7. Compute the break-even \alpha for k = 4 at \rho = 1.0, 1.3, 1.7.
3. Which k would you choose at \alpha = 0.65 and \alpha = 0.85 when \rho = 1.0?
4. Why does \rho depend on batch size, and what does the team deploy?
5. How would you check that the implementation does not change the output distribution?