How Often Do You Retrain the Forecasting Model?

The platform scores 245 million rows nightly (8.75 million active series by 28 horizons). Measured costs on an 8-core machine: scoring takes 39 minutes per model head and there are 5 heads (mean plus four quantiles); training takes 26.7 seconds per million rows, and a 5 percent series sample with 52 weekly origins and 5 horizons is 114 million rows per head.

In a stationary synthetic panel you fit one model at day 959 and score later origins with it, with features rebuilt as of each origin:

1. Compute nightly scoring time and the retrain time for all heads. Compare monthly and daily retraining.
2. What does the staleness experiment say, and what does it not say?
3. Propose a cadence and the triggers for an out-of-cycle retrain.
4. What costs more than compute?