How Many Evaluators a Heuristic Evaluation Actually Needs
You're scoping a heuristic evaluation of a new settings screen against Nielsen's 10 usability heuristics, deciding how many evaluators to involve before pooling findings.
Per Nielsen's own guidance on running this method, what's the recommended panel size?
The correct answer is "3-5, reviewing independently before pooling findings."
Nielsen's guidance is specific: a panel of 3-5 evaluators, each reviewing the interface independently before findings are pooled, finds the large majority of heuristic-violation problems. A single evaluator misses too much, since different reviewers naturally spot different violations — but past roughly 5, the marginal new findings drop off sharply, the same diminishing-returns curve that governs the 5-user rule for qualitative usability testing, just applied to expert reviewers instead of end users.
The distractors don't match this guidance: a single evaluator is specifically called out as insufficient, not the recommended approach; a panel in the teens is well past the point of diminishing returns this method is scoped for; and heuristic evaluation is exactly the kind of method with a stated, citable panel-size recommendation, not an open-ended "however many the budget allows."
Share this question