Perturbation & Monte Carlo: plateau, cliff edge, and what luck could do

Perturbation — plateau or cliff edge?

A real edge sits on a plateau: nudge every setting slightly off its chosen value and it should still work. An edge that only works at one exact magic number is a cliff — and live markets push you off cliffs. AlgoCrucible backtests dozens of nearby parameter sets and re-grades each one on held-out data.

The out-of-sample figure is the one that predicts live behavior — the in-sample number is the whole backtest and always flatters.

Each row is one parameter swept ±10–20% off your value, every step re-graded on the held-out tail. An all-green row is drift-proof on that axis; a red cell is where the edge stops surviving unseen data — treat that setting with respect in live trading.

Monte Carlo — what could luck have done?

A backtest is one ordering of history. Reshuffling the trade sequence thousands of times shows the range of outcomes the same edge could produce — these are the numbers a prop firm's drawdown and daily-loss rules actually act on.

The bold line is your actual backtest; the band is where luck could have taken it. The confidence table turns the fan into planning numbers — at 95% confidence, plan for the lower-tail net and the upper-tail drawdown, not the single backtest's.

Size your account — and your nerves — for the 95th-percentile drawdown, not for the single lucky drawdown one backtest happened to show.