Bias–Variance Trade-off & Cross-Validation

Every supervised model you ship is a bet that its performance on data you have already seen predicts its performance on data you have not. Two things can break that bet. The model can be too rigid to capture the signal (bias), or so flexible that it captures the noise of the particular training sample it saw (variance). And your estimate of out-of-sample performance can itself be wrong — because the validation data leaked into training, because you tuned on it too many times, or because it was not drawn from the same process as production. Getting either of these wrong is how a model with "0.93 AUC in the notebook" becomes a model with 0.71 AUC in the dashboard.

Interviewers lean on this material heavily because it exposes whether a candidate can reason about generalisation rather than just call .fit(). Typical prompts: "Your training error is 2 % and validation error is 15 % — what do you do?" "Why can't I just use the test set to pick hyperparameters?" "Your users each have many rows — how do you split?" "Why did the model that scored best in CV do worst in production?" All of them are answered by the ideas here.

Sibling boundaries: the metrics you compute inside each fold — accuracy, AUC, RMSE, log loss, calibration — are the subject of Model Evaluation Metrics and are treated as given here. Penalty-based control of variance (ridge, lasso, \lambda) is covered in Regularization: L1, L2 and Elastic Net; here we treat "regularisation strength" as one more knob on the model-complexity axis.