Exposure Bias: A Summarizer That Validates Well and Rambles in Production

A seq2seq abstractive summarizer trained with teacher forcing reaches a validation loss of 1.6 nats per token, and a perplexity-based checkpoint choice looks great. In production, free-running summaries often start fine and then repeat a phrase or drift off topic after 30-40 tokens. The ROUGE score computed with free-running beam search is far lower than what the validation loss suggests. A colleague says "that is just overfitting".

1. Explain exposure bias and why it fits these symptoms better than overfitting. How would you confirm it?
2. Give a back-of-the-envelope calculation of how a small per-token error rate compounds over a 60-token summary.
3. Propose two training-time mitigations, how each works, and its cost or risk.
4. Propose two inference-time or evaluation changes, and say which metric you would use to select the checkpoint instead of the teacher-forced loss.