Transfer-Learning Plan for 2,000 Chest X-Rays

A hospital gives you 2,000 labelled chest X-rays from 800 patients (some patients have several images) for a 3-class problem: normal (70%), pneumonia (20%), other abnormality (10%). Images are single-channel, about 1000x1000. You have one GPU and two weeks. The first attempt, fine-tuning an ImageNet ResNet-50 with Adam at learning rate 1e-3 on all layers and random train/validation image splits, reports 96% validation accuracy but fails badly in a pilot at a second hospital.

1. List everything wrong or suspicious in the first attempt, in order of severity.
2. Design the data split and the evaluation protocol.
3. Give a staged transfer-learning plan: input handling, what to train in each stage, learning rates, BatchNorm handling, augmentations to use and to avoid.
4. How do you use a linear probe to decide how much to fine-tune, and what would you try if pretrained ImageNet features transfer poorly?