Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Auditing a Usability Study Report for Hidden Bias

A colleague shares a study report: "We tested the new dashboard with 5 power users from our top-tier enterprise accounts, all of whom use the product multiple times a day. All 5 completed every task with zero errors. We're confident this design is ready to ship to our full user base, including new signups."

  1. Identify the specific bias in this study and explain concretely why it undermines the "ready to ship to our full user base" conclusion.
  2. The report treats "5 for 5, zero errors" as strong evidence. Is the sample size itself a problem here, separate from who was recruited? Explain why or why not, referencing what a 5-user qualitative round is actually good for.
  3. Propose a specific follow-up study design that would address the gap you identified, including who you'd recruit and what you'd change about the task design, if anything.

Share this question

← Back to Usability Testing and Evaluation practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.