Campaigns: declare the plan at night, read an honest verdict in the morning
A campaign runs the whole validation method for you, unattended. You declare the plan — which strategies, which markets and bar types, which ranges of settings to try, and what the goal is — and go to bed. Overnight the app searches, tests and cross-examines every combination, kills the ones that fail, and in the morning presents one report that says what was tried, what survived, and what it means. It survives interruptions: after a crash or a restart it picks up where it stopped instead of starting over.
What one campaign contains
Each row of the plan is one combination: a strategy, the market (or basket of markets) to trade it on, and the bars to trade it with. Every combination runs the same gauntlet you would run by hand — walk-forward, fixed-parameter audit, perturbation of the winning setting's neighbourhood — and advances only on the verdicts those stages already use: a profitable walk-forward, an audit that at least deployed, a ROBUST perturbation. A combination that fails a stage is culled on the spot, and the expensive later stages are never spent on it.
Each row can be a different strategy, with its own ranges, its own walk-forward window sizes and its own number of attempts. That matters because a fast three-minute basket and a slow five-minute swing strategy are different animals: judged on one shared window length, both can fail for no reason other than being tested in the wrong rhythm. Every member is tested at the pace it actually trades.
The holdout: one shot, spent once
At plan time the campaign reserves the most recent slice of your date range and locks it away. Every search, every window, every audit runs on the data before that slice. At the very end, one candidate gets one backtest on the reserved weeks — and that number leads the report. There is no button to run it again.
The plan form also has "Hold back the last N weeks" (two by default). Those weeks stay out of the whole campaign, holdout included: the holdout is carved from what is left. When the survivor goes into incubation, its first check on new data can run on the held-back weeks right away instead of waiting a month. Set it to 0 to use the whole range.
The reserved weeks are also remembered forever. Plan a second campaign whose holdout overlaps weeks an earlier campaign of the same strategy already spent, and the plan review stamps it — fresh, partially seen, or fully seen, with the attempt number — and that stamp rides the report and anything it certifies, permanently. A plan identical to a finished campaign is refused outright: re-rolling the same dice has no honest purpose. The review always names the honest exit — how many days of data have arrived that no campaign has ever touched.
Two goals
Certify one strategy. The strongest surviving chain gets the holdout shot and a certification against the prop profile you chose. One setting, one verdict.
Build a portfolio — the campaign ends at the book. Nobody trades one setting in isolation; you trade a book, against a prop firm's rules or your own saved standard. With the portfolio goal, the campaign goes all the way there:
- Every combination that survives its gauntlet becomes a member.
- The members are combined into one book — instantly, from the trades already stored, no re-running — and the combined book is immediately judged against the rules you chose, with its modeled odds of passing an evaluation.
- A book that breaks the rules fails, full stop. If the members are individually fine but together they would have blown the daily loss limit on some Tuesday in April, the campaign says so and ends. It will not quietly drop a member and retry until the same data happens to say yes — that is how a meaningless pass gets manufactured. A failed book means a new plan.
- The book — not any single member — takes the holdout shot. Every member is re-run on only the reserved weeks, the results are merged, and the book is graded on two things at once: did it make money, and did it behave (no rule breaches). Membership is frozen before this happens, so there is no way to see the answer first and rearrange the book after.
The morning report
One document, in a fixed order: the headline (the holdout number first, and always "best of N combinations tried" — the more you try, the luckier the best one looks, so the denominator is part of the result); in plain words — what the numbers mean, ending in a single recommendation from a fixed rulebook (deploy carefully / park it in incubation and wait for more evidence / do not deploy, in those words); what held everywhere across stages; each stage's verdict with the caveat it should be read with; the graveyard of everything that died and where; and what the report does not say. The wording never flatters: a thin result is called thin, a negative holdout gets "do not deploy", and a campaign where nothing survived gets "that is the system doing its job — change something real, don't re-roll." Print it or save it as a PDF like any other report.
After the verdict
A surviving setting — or the whole book — moves to the Deployment page with one click, frozen exactly as validated, as deployed (you trade it) or in incubation (parked, proving itself). From there it can be re-checked periodically on only the weeks that arrived after its last test — data that did not exist when it was chosen, which is the one test that can never be cheated.
What campaigns refuse to do, on purpose
No re-running the holdout. No reshuffling members after seeing it. No sizing optimization — finding the contract counts that flatter the past is fitting by another name. No silent denominator: every report says how many things were tried. An honest "nothing survived" is a result the system is proud of; the holdout weeks stay fresh for a better idea.