Advanced
Open
Pro
Critique a Product Dashboard: Ratio Traps, Vanity Metrics and Definition Drift
Part of the Data Science Foundations path →
Part of the Advanced Data Science & Experimentation path →
You inherit the weekly dashboard of a mobile learning app. It shows:
- "Total registered users" (cumulative line, always rising).
- "Average revenue per user" as a single number: $4.20, up from $3.10.
- "Click-through rate on the daily reminder" computed as the average of each day's CTR, with a two-decimal percentage and no interval.
- "DAU/MAU: 31 percent", with a footnote that since last month DAU counts any app open, while MAU still counts completing a lesson.
- "Average session length", sessionized with a 5-minute inactivity timeout on Android and a 30-minute timeout on iOS, blended.
- No comparison period, annotations or owners; 24 charts on one page.
Critique it. For each item say what is wrong, what you would replace it with, and what you would add so that the weekly review produces decisions. Then propose the weekly review ritual.
Share this question