Reliable forecasts come from historical system behavior rather than conjecture about individual work items.
- When predicting outcomes, models based on historical system behavior outperform models based on speculative task estimates.
- Because speculative estimates introduce large variance and bias.
- When many tasks are estimated, variance explodes. 1
- When predicting delivery based on historical throughput, variance exists, but it is bounded by observed system behavior.
- Speculative estimates suffer from Optimism Bias
- Speculative estimates suffer from Unknown unknowns. Estimation assumes complete task knowledge which is rarely ever known.
To-Do
- Right now, this principle is currently heavily skewed toward estimation and forecasting, though I believe it applies at a more general level.
| Domain | Speculation | Evidence |
|---|---|---|
| planning | estimates | throughput |
| product | opinions | experiments |
| management | intuition | metrics |
| strategy | predictions | market signals |