Practice — CI/CD for ML & Data Code (5 questions)
Intermediate
Open
Free
A One-Line Training-Window Change That Isn't a One-Line Change Permalink →
A data scientist opens a PR that changes a single constant in the
training config: TRAINING_WINDOW_DAYS = 90 becomes TRAINING_WINDOW_DAYS = 180. The repo's CI runs linting and unit tests, both pass, and a
reviewer approves the PR within a few minutes because "it's a one-line
change." It merges and the nightly training job picks it up.
- Explain why "it's a one-line change" is a misleading way to reason about the risk of this PR, using the "artifact is model + data + code" mental model.
- Design the CI checks that should have run on this specific PR before merge, and explain what each one is trying to catch.
- Suppose the model gate you designed shows the challenger's aggregate
AUC is unchanged but the
tenure < 30 daysslice regresses by 3 points. Should this PR still merge? Justify your answer.
Share this question
Intermediate
Open
Pro