Feature Engineering

A model can only learn patterns that are expressible in the columns you hand it. Give a logistic regression signup_date as a raw timestamp and it learns nothing; give it days_since_signup and is_first_month and it learns churn risk. On tabular data, the gap between a mediocre and a strong result is almost always in the features, not in swapping one gradient-boosting library for another. Practitioners who win competitions and ship models spend most of their time here.

Interviews test this from two directions. The "toolbox" direction: how do you encode a 50,000-value categorical, when do you scale, how do you handle missing values, what is target encoding and how does it leak. The "judgement" direction: here is a dataset and a prediction date — which of these features would you refuse to build, and why. The second direction is where candidates fail, because leakage is not visible in the code; it is visible only if you think about when each value became known.

Exploratory data analysis — distributions, correlations, outlier detection that tells you which transforms to try — is covered in the Descriptive Statistics and EDA subject. Selection via L1 penalties is covered in the Regularization (L1/L2) subject. This subject is about constructing, encoding and validating features.

What a Feature Is, and Why Representation Beats the Model

A feature is a numeric (or numerically encodable) input the model sees for each example. Feature engineering is the set of transformations from raw records to that matrix. Two facts drive everything else:

1. Every model has an inductive bias. A linear model can only add up weighted inputs, so a U-shaped relationship needs an x^2 column and an interaction needs an x_1 x_2 column. Trees can carve up one variable at a time along axis-aligned splits, so a ratio like charges / tenure — one diagonal cut — may take a tree many splits to approximate. The right feature makes the target simple in the model's language.
2. The model cannot see what happened before the row was created. Aggregations, time windows, and history must be computed by you, correctly, as of the prediction time.

A useful mental split:

Raw record ──► Cleaning ──► Per-column transforms ──► Cross-column & historical ──► Selection ──► Model
              (types, NaN)   (scale, encode, log)     (interactions, aggregates,      (drop noise,
                                                       lags, point-in-time joins)      redundancy)

Everything from "per-column transforms" onward must be fit on training data only and applied identically to validation, test and production. That single discipline prevents most leakage.