Intermediate
Open
Pro
Regularised Logistic Regression and the C Parameter
You train LogisticRegression() with default settings on a fraud
dataset with 300 features and 5,000 rows, get a validation AUC of 0.81,
and want to try stronger regularisation and, separately, a sparse model.
- What regularisation is the default
LogisticRegression()already applying? Write down the objective it minimises, and state the relationship between sklearn'sCand the \lambda in the penalised-objective notation. - You set
C=10expecting "more regularisation" and validation AUC drops to 0.78. Explain what happened. Which direction should you have gone, and what grid would you search? - You now want an L1-penalised model that keeps perhaps 30 of the 300 features. What do you change, and what practical constraint does sklearn impose? Why is regularisation more essential in logistic regression than in linear regression when features are plentiful?
Share this question