Markets are noisy. Over short periods, luck can dominate skill. Understanding randomness is essential for interpreting any result, including your own.
Luck looks like skill
Simulate a strategy with a 50% win rate and equal wins and losses, and some of the resulting equity curves will look excellent purely by chance. Our Monte Carlo simulator shows how wide the range of outcomes can be even when the underlying edge is fixed.
Overfitting
Overfitting occurs when a strategy is tuned to the noise in historical data. Signs include many parameters, results that change sharply with small parameter changes, and performance concentrated in a few periods.
Defences
- Keep an out-of-sample period untouched until the end.
- Prefer simple rules with few parameters.
- Record every variant you test.
- Use walk-forward analysis and paper trading.
Check your understanding
Name one sign of overfitting.
Performance that changes sharply with small parameter changes (also: many parameters, results concentrated in few periods).
Why record every variant tested?
The more variants tried, the more likely the best result is due to chance; the record lets you discount accordingly.