Practice — User Research Methods (5 questions)

Your PM ran a survey asking existing users, "On a scale of 1-5, how easy is it to set up a new project in our tool?" The average score came back 4.2 out of 5 — reassuringly high. Two months later, support tickets show new signups are abandoning setup at a high rate, and a quick usability test with 5 new users shows three of them getting stuck on the exact same step.

1. Using the NN/g qualitative/quantitative × attitudinal/behavioral framework, explain specifically why the survey result and the usability test result don't actually contradict each other, even though they look like they do.
2. What was wrong with using this survey, sent to existing users, to answer a question about setup difficulty in the first place?
3. If you only had one week and had to choose one method to re-investigate the setup problem properly, which would you pick and why?

A team has a vague hypothesis — "we think onboarding is confusing" — but no clear idea what specifically is wrong or why. Instead of exploratory research, they jump straight to building a polished prototype of a redesigned onboarding flow and run a usability test on it. The test goes fine: users complete the new flow without visible struggle. The team ships it. Three months later, the same activation-metric problem that motivated the redesign hasn't moved.

1. What generative-vs-evaluative mistake did this team make, and how does it explain the outcome?
2. What should the team have done first, and what method(s) would you use for it?
3. Is the usability test result ("users completed the new flow without struggle") actually worthless here? Explain what it did and didn't tell them.

You're researching whether a new habit-tracking app feature actually helps people stick to a daily routine over several weeks, or whether people abandon it after the novelty wears off. You have budget for either: (a) one round of 60-minute interviews with 10 current users, or (b) a 3-week diary study with 10 participants logging short daily entries about their use of the feature.

1. Which method fits this question better, and why does the other option specifically fail to capture what you need?
2. What's the single biggest operational risk of running the diary study, and how would you mitigate it?
3. Would you run this generatively or evaluatively? Justify your answer.

A researcher recruited usability-test participants by posting in the company's internal Slack and asking colleagues to "grab five minutes to try our new feature." Five colleagues from the design and engineering org volunteered. The researcher sat next to each participant during the session, taking visible notes, and told them "this is really important, we need this to go well before launch."

1. Identify every specific bias or pitfall at risk in this setup, and explain the mechanism by which each one would distort the findings.
2. Rewrite the recruiting and session framing to remove as many of these risks as practical.

You've just finished 8 interviews about why users struggle with a file-sharing feature. Your raw notes include scattered quotes like: "I wasn't sure if the link would expire," "I didn't know if the other person could edit or just view," "I accidentally shared it with the whole team instead of one person," "I couldn't tell if they'd actually opened it yet," and "I re-shared the same file three times because I forgot I already had."

1. Walk through how you'd use affinity mapping to turn these five scattered quotes into one or two actionable findings, including what cluster(s) you'd form and why.
2. Write the "how might we" problem statement(s) that would come out of this synthesis.
3. Why is presenting these five quotes individually, without this synthesis step, a weaker deliverable than the clustered finding — specifically for the stakeholders who'll act on it?