Backtest vs Live Results: Why the Gap Exists and How to Discount It
A rule that won 62% of 400 replayed trades wins 51% of the first 40 live ones. Where the difference comes from, which sources hurt and which only add noise, and a way to haircut a backtest before it gets a single contract of real money.
A rule that won 62% of 400 replayed trades wins 51% of the first 40 live ones. Nothing about the rule changed. The numbers are invented, the pattern is not, and the question every trader asks at that point is which of the two figures is the true one. The answer is neither, and the size of the backtest vs live results gap is something you can estimate before the first live trade rather than discover after it. Everything below assumes you can read the bars a test is built from; the chart pillar covers that from zero.
A backtest, for this post, is any measurement of a rule on data where the outcome was already known when you measured it. That includes a Strategy Analyzer run with code, a week of Market Replay sessions, and a Saturday spent scrolling back through charts marking where the setup "would have" fired. The third kind is the most common and the least reliable, and the reasons are the same seven that apply to the other two.
Where the gap comes from
| Source | What happens | Direction |
|---|---|---|
| Fills | The test fills a limit order the moment price touches it; live, the queue in front of you has to clear first, and the touch-and-reverse entries that made the test look good are the ones that don't fill. Market orders pay the spread and slippage the test ignored | Hurts live |
| Hindsight in the rule | A level you mark today was not on the chart at the time (today's POC is only known at the close); a "clean reversal" is only clean once the bar has finished; a rule that uses the bar's close to enter at that bar's open | Hurts live, sometimes badly |
| Selection | You tested the days you remember the setup working, or you tested 40 parameter combinations and kept the best one. The best of 40 is expected to be lucky | Hurts live |
| Data resolution | A stop and a target inside the same 1-minute bar: which was hit first? Minute data can't say, so the test guesses, usually in your favor. Tick data answers it | Hurts live |
| Regime | Six months of test data from a trending market, two live weeks inside a range | Either way, but you notice it only when it hurts |
| The trader | The test takes every signal. You hesitate on the third loser in a row, skip the entry after a news candle, move a stop, take a target early. Discretion leaks into a rule that looked mechanical on paper | Hurts live, and it's the largest term for most discretionary traders |
| Costs | Commissions, exchange fees, data. A test without them overstates a scalping rule far more than a rule that holds for 40 points | Hurts live |
Six of seven point the same way, which is why the live figure is nearly always the lower one. The regime term is the only one that can flatter live results, and it does so briefly.
The size of each term is what decides whether a rule survives. Fill cost on Micro E-mini Nasdaq-100 (MNQ) futures is a tick or two per side in a normal tape, so a rule with a 40-point stop barely notices it and a rule that scalps 6 points is defined by it. Hindsight is either absent or fatal: if the level the rule reacts to did not exist at the time, there is no rule to discount. Selection bias scales with how many things you tried before settling. The trader term is the one nobody can measure in advance, and it is the reason the first live weeks tell you more about yourself than about the setup.
Discounting before you trade it
Practitioners handle the gap in three ways, and they are not exclusive.
A haircut on the numbers. Take the tested win rate and cut it by five percentage points; add two ticks per side to every trade for fills; put the real commission in. Then run the expectancy formula again and see whether anything is left. A rule that tested at 55% winners with an average win of 1.4R and an average loss of 1R has an expectancy of 0.32R per trade. At 50% it drops to 0.20R, and after two ticks each way on a 20-point stop (80 ticks, so the fill cost is 5% of R) it lands at 0.15R. It survives, with half its margin gone. A rule that tested at 52% gives 0.25R before the haircut and about 0.08R after it, which is the kind of edge one bad month erases. The five points and two ticks are round figures, not a law; the exercise is to see how much of the edge is real and how much was the test.
Data the rule has never seen. Keep the last four to six weeks out of the test entirely and run the rule on them once, without touching a parameter. If the result on the unseen weeks is within the haircut of the tested one, the rule generalizes. If it collapses, what you had was a description of the tested period, not a rule. This is the cheapest of the three and the one most people skip, because the temptation to "just check" the untouched weeks after every tweak is strong and each check quietly moves them into the tested set.
A staged bridge to live. Replay first, a simulated account with live data second, one micro contract third, and a fixed number of trades at each stage before moving on: 30 to 50 is enough to see whether the win rate is in the neighborhood. The sim stage catches hindsight, since the level has to be on the chart before the trade; the one-contract stage catches the trader. Record the grade and the MAE and MFE of every trade at every stage, because a distribution that shifts between stages tells you which term in the table is responsible. A win rate alone does not.
The stance from this desk is the third path with the first applied on top: nothing goes onto an evaluation until it has done its 40 trades on replay and its 40 on a sim account with the same charts, and the number that has to survive the haircut is the sim one, not the replay one. Replay is where I find out whether a rule exists; sim is where I find out whether I can follow it.
Educational content, not investment advice. Futures trading involves substantial risk of loss. Examples are for illustration only. Read the full Risk Disclosure.