Evaluate an ASR System Fairly Across Accents: Normalization, Slices and Confidence Intervals

A vendor reports "5.1 percent WER" for their ASR. Your callers include several regional accents, and customer success says some groups are misunderstood far more often. You have 3,000 human-transcribed utterances from real calls, labeled with accent group and channel.

1. Explain exactly how you compute WER on this set so that the number is reproducible, including normalization. Illustrate how much normalization can matter.
2. How do you aggregate by accent group, and how do you decide whether a difference between groups is real?
3. Why is WER alone insufficient for a voice agent, and what do you add?
4. A group has only 30 utterances. What do you do?