Intermediate
Open
Pro
Missing Values: Indicator, Impute, or Let the Model Handle It
A lending dataset has annual_income missing for 18% of applicants.
Investigation shows income is missing far more often for gig-economy
applicants (who often don't have a single employer to report to) than
for salaried applicants.
- Is this data missing completely at random, at random, or not at random? Why does the distinction change what you should do?
- Design a missing-value strategy for (a) a logistic regression and (b) a LightGBM model.
- A colleague proposes dropping all rows with missing
annual_incomebecause "we want complete, high-quality data." What's wrong with that?
Share this question