Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Choosing k and the Right Splitter

You are advising three teams:

  • Team 1: 800 rows, a ridge regression model, wants the most reliable possible estimate of test RMSE and has cheap compute.
  • Team 2: 2 million clickstream rows spanning 18 months, building a model to predict next-week purchases; the data has a clear seasonal trend and a known regime change (a pricing policy changed month 10).
  • Team 3: 50,000 rows of medical images, ~8 images per patient, building a diagnostic classifier; new patients arrive continuously in production and no patient in training will ever reappear.
  1. For each team, recommend a specific CV strategy (splitter, k if applicable, window type if relevant) and justify it against their data structure.
  2. For Team 2 specifically, explain why a random 80/20 split would give a misleadingly good offline number, tying your answer to the pricing regime change.
  3. For Team 3, what happens to the validation AUC if they accidentally use plain KFold instead of your recommendation, and in which direction is the bias?

Share this question

← Back to Bias–Variance Trade-off & Cross-Validation practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.