ONNX Export Works on Your Laptop and Fails in Production: Export, Fusion and Benchmark Hygiene

A model exported to ONNX passes your test at batch 4, runs in ONNX Runtime, and the report says "2.1x faster, mean latency 25 ms". In production it crashes at batch 7, and for some inputs the outputs differ from the PyTorch model by large margins. The team also wants to turn on torch.compile and TensorRT "to be safe".

1. Explain the two export bugs (the crash and the wrong outputs), how you reproduce each, and how you export correctly.
2. A trace shows 985 of 1,000 requests at 20 ms and 15 requests at 400 ms. Compute the mean, p50 and p99 and explain why "mean 25 ms" is a misleading headline. List the other benchmark flaws you would look for.
3. Explain operator fusion with BatchNorm folding, including why it is required before quantization.
4. What do you say about torch.compile and TensorRT, including portability and when they do not help?