Linear Regression

Linear regression is the model every data scientist is assumed to know cold. It is the workhorse of business analytics ("what drives revenue?"), the baseline for every tabular prediction problem, the engine behind A/B test analysis with covariates, and the language in which most of applied econometrics is written. Interviewers use it as a probe: anyone can call LinearRegression().fit(), but far fewer can say what a coefficient means when there is a log on the left-hand side, why standard errors are wrong under heteroscedasticity, or what a high VIF actually breaks.

This subject covers the model from the ground up: the OLS objective and its closed-form solution, when you would use gradient descent instead, how to interpret every flavour of coefficient, the Gauss-Markov assumptions and the diagnostic plot or statistic that checks each one, what goes wrong when they fail, how to read R^2 honestly, how inference on coefficients works, and how to find the handful of points that are secretly driving your fit.

Two boundaries with sibling subjects in this track: adding L1/L2 penalties to the objective (ridge, lasso, elastic net) is covered in the Regularization (L1/L2) subject, and how to estimate out-of-sample error with held-out data and cross-validation is covered in the Bias-Variance and Cross-Validation subject. Here we focus on the unpenalised model and in-sample diagnostics.