The GPU Is 70% Idle: Find the Bottleneck

An image classifier trains at 586 images/s on a single GPU. nvidia-smi shows about 30% utilization with sawtooth gaps. The code uses DataLoader(ds, batch_size=256, shuffle=True) with default num_workers, JPEG decoding plus random-crop augmentation in __getitem__, and calls loss.item() and print every step. Profiling one batch on a single CPU core shows 307 ms to load and augment 256 images; the GPU forward/backward/step takes 130 ms.

1. Reproduce the 30% and 586 images/s from the numbers.
2. Name the bottleneck and state how you would confirm it with measurements rather than guesses.
3. Choose the DataLoader settings and calculate the expected throughput. How many workers and why?
4. List other things that can leave a GPU idle even after the loader is fixed, and how each shows up.
5. After the fix, utilization is 95% but throughput is still lower than the hardware allows. What are your next options?