The 8 robustness tests on every AI strategy

Think of a student. The practice questions are the in-sample data: the search studies them and keeps the rules that did best there. Doing well on practice questions proves little — the answers may just be memorised.

The real exam is the out-of-sample data: the most recent part of your dates, kept hidden during the search. A strategy that still makes money there has probably found something real.

Every strategy is checked against these 8 tests.

• Out-of-sample holds up — it also made money on the unseen data, with enough trades, and over all your dates it meets the return requirement after taxes and slippage, on its starting capital.
• Monte Carlo Trade Resampling — its trades are reshuffled many times; it must stay profitable in almost every order.
• Profitable in most periods — it must make money in most parts of the history, not in one lucky stretch.
• Survives delayed entry/exit — delayed execution must still work.
• Beats random strategies — it must clearly beat strategies built from random rules with the same exits.
• Neighbouring settings also work — slightly different numbers must work too. A strategy that only works with one exact number is probably luck.
• Survives noisy prices — small random changes to the prices must not break it. When it was not run, its column shows a dash.
• Walk-forward — tuned on one stretch of data, it must keep working on the next.

A top strategy passed all 8 tests. When we replayed the product on past years, about 6 in 10 top strategies stayed profitable in the months that followed, against about 4 in 10 random strategies.

So the tests tilt the odds in your favour. They do not guarantee a profit. Study a strategy, understand its rules and trade it small before trusting it.

On a strategy's page, Run robustness tests runs all 8 tests on it again, on the same dates as the search that found it. An unchanged strategy gets exactly the same verdict. If you edit the strategy first, it is a new strategy and may get a different verdict. Each re-test uses 1 credit.