How Many Usability Problems the First Test Participant Finds
Per Nielsen's usability-testing sample-size curve, each user has roughly a 31% chance of hitting any given usability problem.
Roughly what fraction of a design's total usability problems does the very first test participant typically surface?
The correct answer is "About a third."
This subject cites Nielsen's curve directly: modeling problem- discovery as a probability function (each user roughly 31% likely to hit any given problem), the first user finds about a third of the problems that exist, and by the fifth user you've typically surfaced around 85%. This is the actual math behind the "5 users" rule of thumb — it isn't an arbitrary number, it's the point where the curve of marginal new findings has already flattened sharply.
The distractors misstate the curve: "nearly all, around 90%" is roughly what the fifth user cumulatively gets you to, not the first, and overstates a single participant's coverage; "essentially none — only after user 3" contradicts the stated 31%-per-user mechanism, which gives even the first participant real coverage; and "exactly half" isn't the cited figure — the subject specifically says "about a third," not half, for the first participant.
Share this question