Advanced
Open
Pro
Diagnosing a Position Bias Bug
After launching a new ranker, an analyst notices the model assigns unusually high scores to videos regardless of content quality whenever they appear in slot 1 of the training logs, and the online CTR curve by slot shows position 1 gets 5x the CTR of position 10 even for comparable content.
- Explain the mechanism that causes a ranking model to learn this pattern if position is not handled correctly.
- Describe two concrete training-time fixes and how each works at serving time.
- How would you empirically confirm the fix removed the bias rather than just hiding it?
Share this question