Scoping a Two-Week Redesign Evaluation With a Fixed Budget
You're given a two-week window and a fixed budget to evaluate a redesigned settings page before it ships. You can afford either: (a) a heuristic evaluation by 4 evaluators plus 5 moderated usability sessions, spread across the two weeks, or (b) 15 moderated usability sessions and no heuristic evaluation at all, using the full budget on user sessions.
- Which option would you recommend, and what specifically makes it the stronger use of a fixed budget?
- What kind of problem would option (b)'s extra 10 sessions likely find that option (a)'s 5 sessions would miss — and is that gap worth the trade-off?
- What kind of problem would option (a)'s heuristic evaluation catch that no amount of additional user sessions would catch faster or cheaper?
1. Recommendation
Option (a): heuristic evaluation plus 5 user sessions, not 15 user sessions alone. The heuristic pass is cheap (a day or two of expert review) and catches known-category violations — inconsistent iconography, missing error prevention, poor visibility of system status — that don't require a real user's time to find at all. Spending the first portion of the budget there means the 5 user sessions that follow aren't wasted rediscovering things a checklist would have caught for free; they're focused on the harder, unknown problems only real users surface.
2. What the extra 10 sessions in option (b) would find
Beyond roughly the fifth participant, the marginal new problems found per additional session drop sharply — most of what sessions 6-15 surface is likely to reconfirm issues already found in sessions 1-5, rather than reveal genuinely new ones. There's a real gap: a few truly rare edge-case reactions might only show up in a larger sample. But given this is a settings page (not a high-stakes, high-variance flow like checkout or onboarding), that gap is unlikely to justify spending the entire budget on session volume alone, especially when it comes at the cost of never running the heuristic pass at all.
3. What the heuristic evaluation catches faster/cheaper
Any issue that maps to a known category Nielsen's 10 heuristics already anticipate — inconsistent terminology between this page and the rest of the product, a destructive action with no confirmation step, a missing loading state — evaluators can flag in a single review session without needing to schedule, recruit, or wait for a real participant to happen to hit that exact problem. Waiting for a real user to stumble into a known-category issue is strictly slower and more expensive than having a trained evaluator check for it directly.
Share this question