Advanced
Open
Pro
Learning Rate vs Tree Count Trade-off
Two engineers are configuring a LightGBM model for a fraud detection task:
- Engineer A:
learning_rate=0.3,n_estimators=100 - Engineer B:
learning_rate=0.01,n_estimators=3000
Both models train to similar training loss. Engineer B's model has noticeably better validation AUC.
- Explain mechanistically why Engineer B's configuration generalizes better.
- What is the mathematical relationship between learning rate and the number of trees needed?
- Is there a scenario where Engineer A's configuration is preferable?
Share this question