QQ 301, 4 min read

Added on 2026-10-07: full-history validation and the portfolio tools

These landed after sections 1 to 7 were written. Where they change an earlier statement, this section wins.

8.1 Validation runs on the whole dataset

The assistant's run_validation, run_walkforward and run_montecarlo tools used to load only the newest 2,000 bars unless told otherwise, with a schema maximum of 50,000. On 1-minute data 50,000 bars is about five weeks, so an 11-year strategy was judged on a few dozen trades. They now load what a backtest of the same file loads: the whole dataset, or the from/to window. The bars argument is gone.

• Every result states its span. span carries bars, loadedBars, first/last, trades, holdoutBars, whole, capped and one plain sentence, for example "Ran on 4,267,457 bars, 2015-01-02 00:00 to 2026-09-18 23:59, 666 trades." The Checks tab shows it under the verdict.
• A time limit instead of fewer bars. time_limit_s (0 = none; the assistant passes 240 s for the standard suite and 180 s for institutional). A check that reruns the strategy and cannot finish in time, going by the measured pace of the baseline run, is skipped with the reason and counts as a caution. It is never run on a shorter history.
• A capped load is stated. If a load does stop short (a caller's own positive limit, or the 100,000-bar ceiling on remote providers), span.capped says why.
• Grid size. run_walkforward and run_validation accept at most 24 parameter combinations on any dataset over 5,000 bars; a larger grid is refused up front with the reason.
• Measured. On a real 4.27M-bar 1-minute file with a vectorised script: standard suite with a 6-combination grid 25 s, with 24 combinations 56 s; institutional 170 to 190 s.

Source: backtest/validation.py (span_of, time_limit_s), engine/server.py (the research tool handlers), tests/test_validation_span.py.

8.2 Portfolio backtest on one shared balance (run_portfolio)

The single backtest runs one script on one dataset. A portfolio is several legs, each a strategy file on its own dataset, sized off one account.

• Each leg is first measured alone on its own capital, exactly as a single run. All entries and exits are then walked in time order on one balance.
• Sizing rule. With sizing: "balance" (the default) a trade's quantity is multiplied by (shared balance / the leg's own balance) at its entry, times the leg's weight. That is exact when the script sizes each trade as a fraction of a balance that starts at the leg's capital and moves by its closed P&L. With sizing: "fixed" the quantity is used as written. The engine cannot tell which kind a script is; the caller says.
• Balance means starting capital plus the net P&L of trades closed by then. A one-leg portfolio reproduces the single run to 1e-9 of capital.
• Drawdown is mark to market on every bar across all legs, as in a single run.
• Currency. P&L in a quote currency that is not the account currency is converted at the exit rate: from the pair's own price when the account currency is the base (USDJPY on a USD account), from a USDJPY dataset in the library for a cross such as EURJPY. A leg that cannot be converted is listed in currency.unmodelledLegs and the result says so.
• Reported. Net and annual return, max drawdown, profit factor, win rate, trades, per-year returns and losing years, each leg's contribution, and peak leverage, notional and open trades.
• Not modelled. Margin: nothing is refused, peak leverage is reported instead. Legs on different timeframes are ordered by bar open time, so two events inside one coarse bar can be booked in the wrong order.

8.3 Stop-fill slippage stress (run_slippage_stress)

A "level fill" is a fill the script priced itself away from the bar's open: a stop or limit level. On 1-minute bars, filling a stop exactly at its level is optimistic. The stress reruns the book under a table of scenarios: slippage per level fill (default 0, 0.5, 1 and 2, in bps or pips), with and without a per-lot commission, and optionally "fill at the worse of the level and the bar's close". Each row gives net and annual return, max drawdown and losing years. The underlying fill options also work on a plain backtest: level_slippage_bps, level_slippage_price, level_slippage_scope, level_fill: "worse_of_close", commission_per_lot, lot_size.

8.4 Profit concentration (run_concentration)

For the top 1% and 5% of trades (by return or by P&L): how many trades that is, their share of gross profit and of net profit, and the book re-measured with those trades removed. A result that needs a handful of trades is a different claim from one spread over thousands.

8.5 Drawdown and losing-year odds (run_drawdown_odds)

A bootstrap over the book's daily returns (or its trades), in blocks so runs of bad days stay together: the probability of a drawdown of at least drawdown_pct (default 20%) inside one year and the probability of a losing year, each with its standard error, next to what the backtest itself showed in its own calendar years and rolling windows. Default 10,000 paths, seeded.

Annual figures in 8.2 to 8.5 use calendar time. The single run's extended annualizedReturn and its sharpeRatio count a year as bars divided by the median bar spacing, which overstates both on markets that close; the two will not match on the same strategy.

Source: backtest/portfolio.py, backtest/runner.py (the level-fill options), tests/test_portfolio.py.