W8A8 Collapsed on a 13B LLM: Outliers, SmoothQuant, GPTQ, AWQ and What to Ship

You quantize a 13B decoder to int8 weights and int8 activations with one scale per activation tensor. Quality collapses on long outputs, while weight-only int8 is fine. The activation statistics show 3 of 5,120 hidden channels with maximum magnitude about 60 times the median channel.

1. Explain mechanically why the outlier channels ruin per-tensor activation quantization, with a rough number for how many ordinary activations are lost.
2. Describe SmoothQuant, LLM.int8() and the weight-only route, and say what each trades away.
3. The business wants 4-bit weights. Explain what GPTQ optimizes, how that differs from round-to-nearest, and the surprising consequence for weight error. Then explain the AWQ idea and what can go wrong when picking its scaling exponent.
4. Choose a deployment plan for (a) a single-user local deployment and (b) a high-throughput batch service on H100s, with reasons.