FAQ: results that look wrong
Most "the numbers are wrong" reports turn out to be the software correctly reporting something surprising. This article covers the ones that come up repeatedly, and how to tell a real problem from a real result.
Every walk-forward window picked the same settings
AlgoCrucible warns about this, and it almost always means the parameters never reached the strategy. The search ran, each trial got different settings, and every trial produced the same backtest — so the "winner" is arbitrary.
On MultiCharts the usual cause is a strategy that fetches its inputs with AC_Param but then
uses the raw inputs in its logic. Fetch and use must match:
fastL = AC_Param("FastLen", FastLen); { fetch }
if Average(Close, fastL) > ... { use fastL, not FastLen }
The other cause is the chart running a different strategy than the one you selected.
The P&L is enormous — 100× what it should be
MultiCharts' default position size is 100 shares, which is sensible for stocks and wrong for futures. A futures strategy left on the default trades 100 contracts a signal.
Fix it in Format Signals → Properties → Position sizing — set it to 1 contract. AlgoCrucible detects the default and warns, but the setting lives in MultiCharts, so it has to be changed there.
Two runs with identical settings gave different answers
Check the search ranges, not just the stage settings. A member's search space is inherited from the most recent run of that strategy, so an unrelated optimize you ran in between can silently widen or narrow it. Different ranges, different winners, different results.
The engine itself is deterministic: same strategy, same range, same data, same seed gives the same answer to the dollar. If two runs differ, something in the inputs differed.
Per-member numbers don't add up to the portfolio total
Fixed in the current version — earlier builds reported member rows before slippage and the combined row after it, so the parts didn't sum to the whole. If you see it, the run predates the fix; re-run it.
A strategy I deleted still shows up
NinjaTrader: NT 8.1.8 and later compile with generated assembly names rather than
NinjaTrader.Custom, which left deleted strategies visible to older bridge builds. Update to the
current version and refresh.
MultiCharts: the bridge can't see what's on a chart — it only knows what a strategy last declared about itself when the chart calculated. A stale name means the new strategy hasn't declared anything yet: it isn't wired, or it hasn't recalculated. The detection panel shows how old a declaration is, with a forget link to clear it.
The audit says MIRAGE on a strategy that looked great
That is the audit doing its job. Optimize scores are the best of many attempts, so they're biased upward by construction; walk-forward is honest but still re-fits each window; the audit fixes one setting and tests it on unseen data. A result that survives the first two and dies at the third was fitted to its history — which is exactly what you want to find out before funding it, not after.
Costs
See Cost sensitivity — the break-even slippage is usually the fastest way to tell whether a marginal result is worth more work.