Every systematic trader eventually meets the same disappointment: a strategy with a beautiful equity curve that stalls, or reverses, as soon as it goes live. Usually the market has not changed. The backtest was simply measuring something other than what the trader thought.

The following seven pitfalls account for most of that gap. None requires advanced mathematics to detect; all require discipline.

In-sample: design & tuningmany variants tested — record them all Out-of-sampleused once, at the end Paper tradinglive data, simulated orders historical data →→ today Validation should move forward in time, never backwards
Figure 1. A disciplined validation sequence. Data used for tuning must never be reused to judge the final strategy.

1. Look-ahead bias

The simulation uses information that was not available at the time. Common culprits are financial statements stamped with their period-end date rather than their publication date, and signals computed on a day's close but executed at that same close. The fix is point-in-time data and an explicit lag between signal and execution.

2. Survivorship bias

Testing only on companies that exist today silently removes the bankruptcies, delistings and acquisitions that a real portfolio would have held. For equity strategies, this can inflate returns materially, particularly in small caps. Use a universe that includes delisted securities.

3. Unrealistic costs and fills

Many backtests assume execution at the mid-price with zero slippage. High-turnover strategies are especially sensitive: a few basis points per trade can turn a positive edge negative. Model spread, commissions and market impact conservatively, and test sensitivity to doubling them.

If a strategy only works with optimistic costs, it does not work.

Optimisers in frameworks such as Freqtrade or any spreadsheet will happily find the parameter set that best fits historical noise. The more combinations tried, the more impressive the best one looks — and the less it means. Record how many variants you tested, prefer parameter regions that are robust rather than single peaks, and reserve data that the optimiser never sees.

5. Too little data, too few regimes

Three years of a rising market is one regime, not three years of evidence. Strategies should be examined across rising, falling and sideways periods, and across high- and low-volatility environments. If history is short, say so in your conclusions.

6. Ignoring capacity

A strategy that trades thinly traded instruments may be profitable at small size and impossible at larger size. Estimate the fraction of average daily volume your orders would represent, and assume impact grows with it.

7. Confusing a backtest with a validation

The final pitfall is conceptual. A backtest can falsify an idea; it rarely proves one. Validation requires out-of-sample testing, walk-forward analysis and, ideally, a period of paper trading under live conditions.

A minimum checklist

  • Point-in-time data with explicit publication lags
  • Universe including delisted securities
  • Costs modelled conservatively, with a sensitivity test at double costs
  • A record of the number of variants tested
  • A held-out period never touched during development
  • Results reported across distinct market regimes
  • Capacity estimated relative to traded volume

Open-source engines such as Backtrader, Zipline Reloaded and LEAN make each of these checks possible. None performs them automatically.

Quorient Editorial Desk

Articles credited to the Editorial Desk are researched and written for Quorient and reviewed by the editor, Callum Reid, before publication, in line with our Editorial Policy. Spotted an error? Request a correction.