Most spectacular backtests are not luck. They are bugs: a price from the future, a clock off by an hour, a fill nobody could get. Every lesson below cost us a result we wanted to be true.
A backtest that fills a limit order whenever price touches it keeps the trades that bounce off the level and never misses one. Require price to trade through the level, and do not credit a target in the same minute as the fill. Here the fill rules alone halved the result before any cost was charged, and the costs of a small stop did the rest.
From the test: ICT Silver Bullet (10:00-11:00 New York) →Costs are paid in price, but results are counted in R. With a stop 1.8% away, a 0.1% spread and a few nights of financing cost 0.09R a trade, more than the setup earned before costs even at the most favourable reading. And on daily bars, check which came first on the entry day: here that one assumption moved the result threefold.
From the test: Momentum pullback to the 9 and 21 EMA →A machine-learning model will find any leak in seventy columns faster than you will. Verify each file's time zone against a scheduled event it must contain (an economic release, an exchange open, a settlement), and treat a model that beats a shuffled control but still loses after costs as a bug report, not a near miss.
From the test: Machine-learning intraday forecaster →The gates before the holdout check that a result is not a bug and not obviously luck. Only the holdout checks that it was not tuned to the past. If we had stopped one gate early, this would have gone on a funded account.
From the test: Nasdaq breakouts on high-fear days →Almost any buy rule on the Nasdaq-100 since 2005 shows a profit. The question is whether it beats buying on random days with the same stop and holding period. Without that comparison, a positive backtest says more about the index than about the setup.
From the test: Weak-day reversal (lower high, lower low) →Check the share of flat bars at every anchor minute, year by year, and never fill a stop order at the trigger when the bar opened beyond it. Our simulator now refuses stale anchor minutes and fills gaps at the open; on a random walk with a stale pre-open quote the old loop shows t 10.5, the new one about zero.
From the test: Opening Volatility Stretch on DAX, Bund and Nasdaq →Before trusting a top result from a parameter search, rerun the search on destroyed data and see what the best result looks like there. If your winner does not beat that, it is not a winner.
From the test: We mined 10,248 strategy rules on a GPU →A t-statistic of 3 on daily close-to-close returns can be worth nothing to a trader who pays a spread, uses a stop and sleeps flat. Always measure an academic effect inside the window you can actually trade, at the venue's real cost.
From the test: 60/40 rebalancing pressure →Academic results are measured before costs, on futures or ETFs, often on close prices. A 1.4 bp effect against a 0.6 bp spread leaves a Sharpe ratio too small to survive a stop, a few bad years or a retail account.
From the test: Intraday momentum into the close →Any rule that compares 'this morning' with 'last night' across a date boundary can leak. Test every filter at a lag of one and two days: a real effect survives, a leak disappears.
From the test: USD/JPY Tokyo fix fade →