Choose an Optimizer and Schedule for a Fine-Tuning Job

You must fine-tune a pretrained 110M-parameter transformer encoder for a 5-class text classification task with 20,000 labeled examples. You have one GPU that fits batch 32, and budget for about three epochs. A junior engineer proposes "SGD, lr=0.1, batch 32, constant LR, no warmup, weight decay on all parameters".

1. Compute the number of steps for three epochs, and propose a complete recipe: optimizer, base LR for the backbone and for the new head, schedule with warmup length, weight decay and which parameters it applies to, gradient clipping, and evaluation cadence. Justify each choice.
2. Explain concretely what goes wrong with the proposed recipe.
3. The loss plateaus at the initial value for the first 30 steps after you launch. Is this a problem? How would your answer change if it had not moved after 600 steps?