A BatchNorm Bug Between Training and Serving

A CNN with BatchNorm reaches 94% validation accuracy in your notebook, but the service wrapping the same weights reports 71% on the same validation images, and results change if requests are batched differently. Separately, a teammate fine-tuned the model for only 20 optimizer steps on a new small dataset and gets odd accuracy in eval mode.

1. Describe what BatchNorm computes in training mode and in eval mode, and state the most likely bug that explains the service behaviour. Why does the output depend on batching?
2. Why can the service also raise an error for single-image requests?
3. Explain the 20-step fine-tuning anomaly using the running-average update (momentum 0.1, running mean initialized to 0). How do you fix it?
4. Why is BatchNorm a poor fit with very small batches, and what do you use instead?