Keep the Evaluation Set Clean Across AL, Weak Labels and LLM Labels

Your labeling program mixes human gold labels, items chosen by active learning, labeling-function (LF) labels, LLM labels and synthetic examples. You are asked to write the plan that keeps the evaluation set trustworthy for the next two years.

1. List the ways the evaluation set can be contaminated or biased in this setup and give a measured or computed example for at least three.
2. Decide the test-set size needed to distinguish two models that differ by 0.02 macro-F1 when the standard deviation of one model's score is 0.030 at 500 items, 0.020 at 1,000 and 0.008 at 5,000.
3. Write the rules (split, labeling protocol, refresh, versioning).
4. Say how you detect that a leak has happened after the fact.