How we test a strategy against a prop challenge.
Most backtests look better than the strategy really is. These are the checks Aydeus runs on your trades, why each one exists, and what no tool can check for you.
1. Replay the challenge exactly as the firm counts it
A challenge is not judged on your equity curve. It is judged by the firm's accounting, and small details decide whether an account survives:
- The firm's clock. The daily loss limit resets at the firm's own midnight (FTMO uses midnight Central European time; Topstep's trading day ends at 17:00 Chicago time). Your trades are converted from your platform's time zone and counted on the firm's day.
- Intraday drawdown. The limit is tested on equity, including open losses. When your file has no intraday data we assume a full stop-loss excursion; with one-minute bars we rebuild the real worst point of every trade.
- Trailing and end-of-day limits that follow your best balance, consistency rules (best day as a share of total profit), minimum trading days, and wider spreads around the daily rollover.
2. Simulate many challenges without breaking your losing streaks
Shuffling trades one at a time makes losing streaks rarer than they really are, which flatters the pass chance. Aydeus resamples whole blocks of your trading days (a stationary block bootstrap, Politis & Romano 1994) with the block length chosen from your own data (Politis & White 2004). Each simulated challenge runs until it passes or breaks a rule, then through the funded stage, payout cycles and the profit split. The fee, refund policy and several attempts are included in the expected profit.
3. Ask whether the edge is real or luck
- Luck test, deflated for the strategies you tried. If you tested twenty variants and kept the best, its Sharpe ratio is inflated. We use the probabilistic and deflated Sharpe ratio (Bailey & López de Prado, 2014) with the number of variants you tell us.
- Holdout. Your most recent trades are checked on their own. An edge that is strong early and gone lately is flagged as fading.
- Concentration. If removing your best few trades wipes out the profit, the result rests on rare outliers and is not something to size a challenge on.
- Enough trades. A small edge needs many trades to show. At +0.1R per trade with a standard deviation of 1.1R, about 480 trades are needed before chance alone is unlikely to explain it (t ≈ 2).
4. Simulate at half the edge
Measured edges shrink in live trading: costs differ, markets change, and the best of several backtests is the luckiest one. The risk-level scenarios and the rule comparison therefore use half of your measured edge, and show the result at the full edge and at zero edge next to it.
5. Risk levels under the rules
The hard cap is the largest risk per trade at which your worst realistic day (your busiest day's number of trades, each a full loss plus slippage and costs) still fits inside the daily limit. Below that cap, the simulation reports the expected result after the fee at each risk level, and the headroom figure shows how much room the daily limit leaves after each loss.
6. Stress the costs
Every result is recomputed with extra commission and spread (+0.05R per trade) and extra slippage on stops (+0.10R). A strategy that only works at perfect fills is reported as such.
7. Filters are judged on what they remove
“Skip Mondays” or “stop after two losses” always looks good on the trades it keeps. Aydeus only marks an hour, weekday or stop rule when the trades it would remove were clearly worse than the rest (t ≤ −3 on at least 20 trades). Cut results enough ways and some slice always looks bad by chance.
Lessons from our own research
We built these checks because they caught real mistakes in our own strategy research before any money was at stake:
- Price bars stamped one minute late made one strategy look nearly 40% stronger (t-statistic 3.7 instead of 2.7) than on the clock a live trading bot actually sees.
- A currency filter that showed a t-statistic of 7.5 turned out to use information from later in the day (look-ahead). Without it the result was negative.
- A filter that improved the trades it kept added nothing once judged on the trades it removed.
Rule of thumb: a simple rule that looks spectacular is a bug until proven otherwise.
What no tool can check for you
- Whether you will trade the same way with real pressure on the line.
- Whether your firm's execution, spreads and slippage match your history.
- Rule changes after the date the firm's rules were read, news-trading restrictions and other terms specific to your account. Always read the firm's current terms.
How the engine itself is tested
Every calculation has known-answer tests: hand-computed cases for the daily reset across time zones, trailing drawdowns, consistency, the luck test and the simulation. They run in Node and again in the browser before every release.