When Survey Answers and Actual Behavior Point Opposite Ways
Users tell a survey they'd pay for a premium feature. A paywall A/B test then shows near-zero conversion.
Which axis of the NN/g framework explains why these two results diverge, and which one predicts revenue?
The correct answer is "Attitudinal vs behavioral — the behavioral A/B result predicts revenue."
This is the textbook case this subject uses to explain why the attitudinal/behavioral axis matters: attitudinal methods (a survey) measure what people say, behavioral methods (an A/B test) measure what people do, and people are unreliable narrators of their own future behavior — not from dishonesty, but because self-report runs through memory, self-image, and social framing that don't match what actually happens. When the two diverge, the behavioral data is the one that predicts revenue; the survey still has value for explaining why, but it isn't a substitute for observed behavior.
The distractors reach for the wrong axis or the wrong conclusion: qualitative vs quantitative describes data shape (rich vs. numeric), not what's being measured, and doesn't explain this specific divergence; generative vs evaluative is about the stage of the process, unrelated to why stated preference and behavior disagree; and the two results diverging doesn't mean either study is "flawed" — this subject frames the divergence as expected and explainable, not as evidence of a broken study.
Share this question