1. Abstract
Residuals reveal how well a model explains observed data. Monitoring residual drift over time helps detect emerging structural change, missing drivers, or weakening relationships before forecast performance deteriorates.
2. Context
Apply this best practice during periodic model reviews, particularly for models that are in Production or actively used for forecasting and planning. Residual monitoring is a core component of responsible model maintenance.
3. Content
3.1 Why It Matters
Residuals represent the difference between predicted and actual values. When residuals begin to drift consistently in one direction, it signals that the model may no longer fully capture the behavior of the underlying system.
Residual drift can indicate:
- A newly emerging driver not included in the model
- Structural change in the business or market
- Incorrect lead timing
- Shifts in customer or economic behavior
- Data quality or definition changes
Unlike overall model fit metrics, residual drift often reveals problems earlier, making it a powerful diagnostic tool.
Ignoring residual trends can lead to:
- Gradual degradation in forecast accuracy
- Overconfidence in outdated relationships
- Larger recalibration efforts later
3.2 How to Apply
During regular model reviews:
- Open the model and navigate to the Diagnostics → Residuals view.
- Focus especially on the most recent periods (for example, the last 6–12 months of actual data).
- Look for patterns such as:
- Sustained upward or downward drift
- Increasing variance
- Repeating patterns not previously present
- Determine whether the drift:
- Reflects a temporary anomaly
- Indicates a missing explanatory variable
- Suggests a structural change
- If drift persists:
- Reassess driver selection
- Re-evaluate lead times
- Consider recalibration
- Document findings and actions taken.
3.3 Example
A demand model historically performs well, but recent residuals trend consistently negative. Investigation reveals that supply chain constraints are limiting sales — a factor not included in the model. Adding a logistics capacity indicator reduces residual drift and improves forecast stability.
3.4 Common Pitfalls
- Ignoring gradual residual drift because overall R² remains high
- Overreacting to isolated one-period errors
- Treating seasonal patterns in residuals as random noise
- Failing to differentiate between data issues and structural change
3.5 Expected Results
- Earlier detection of model degradation
- More proactive recalibration decisions
- Improved long-term forecast reliability
- Stronger confidence in production models
