TRADING STRATEGY TEST: THREE LEGS, ONE QUESTION
We deleted 264 test rows because one leg ran the wrong experiment without erroring. The three-leg ladder replaced that guesswork: a clean baseline, a fixed 150 ms delay, and a robustness run, each with a gate that has an address.
We deleted an entire test wave once. 264 rows, each a cell pairing one range with one phase, all clean on the surface: normal gradients, recorded verdicts, a database that looked exactly like finished work. Every row answered the wrong question.
A trading strategy test earns trust only when it separates the edge from the conditions that flatter it. We run three legs: a clean baseline, a fixed 150 ms delay, and a measured robustness run. When something fails, the failure has an address instead of hiding inside a pile of averages.
The stake is money, not adjectives. A strategy that works only with perfect fills can look excellent in a spreadsheet and fall apart the moment it meets a live order book. We want numbers that survive the ideal case, one fixed-delay deployment assumption, and a deliberately wrong market.
#Three legs, one question each
The ladder works because each leg changes one thing. P1 removes delay and perturbation. P2 keeps the strategy unchanged and adds the same 150 ms delay every time. P3 keeps execution clean and changes the market inputs instead.
| Leg | Condition | Question |
|---|---|---|
| P1 | Clean baseline | Does an edge exist at all? |
| P2 | Fixed 150 ms delay | Does it survive deployment latency? |
| P3 | Skip 20 / Shift 20 / ExitDelay 20, seed 42 | Does it survive a slightly wrong market? |
We start with the cheapest question. There is no point paying for a delayed or perturbed run when the strategy cannot make money in the clean case, and we move down only when the first answer earns the next test.
A range is one period of time: a start date and an end date and nothing else. A cell is one range under one phase, so P1, P2, and P3 results never blend into one score.
#The clean baseline sets the reference row
P1 is deliberately unrealistic. No delay. No robustness perturbation. The tester's default execution mode fills signals instantly at the mid price, and the leg gives us the upper bound everything else gets measured against.
The clean leg must produce positive profit. If it cannot, there is no edge to test under harder conditions. A pass here earns permission to keep going, nothing more.
| P1 proves | P1 cannot prove |
|---|---|
| The logic fires and the arithmetic works | That the result survives a slow fill |
| Positive profit is possible | That the profit is real rather than a fill-model artifact |
| The strategy direction makes sense | Anything about deployment, slippage, or spread cost |
P1 also works as a regression check. It varies no delay or robustness setting, so it should stay still when nobody has touched the instrument, the tape, the contract, or the tester session. A moved clean result is the cheapest alarm we have: something drifted, and every later number should be read as a deviation from this row.
That is why we never call P1 a score. It is the reference from which the latency tax and the robustness loss are measured. A strategy that earns a large clean profit and loses almost all of it after 150 ms is fragile somewhere inside that first 150 ms. This leg cannot say where. Locating the break needs the broader sequence of delay gates, not this one point.
#A fixed 150 ms delay exposes the latency tax
P2 measures the price of hesitation. Every order that filled instantly in P1 now fills 150 ms later, at whatever price the market reached in the meantime. Nothing else changes, which turns the gap between the two rows into a measurement instead of a guess.
The delay is fixed on purpose. Run the leg twice with the instrument, tape, contract, and tester session unchanged, and the same trades should land to the cent. A random delay would describe a distribution of network conditions. This leg describes the strategy's response to one repeatable deployment assumption.
The verdict has two parts: profit must stay positive, and profit must exceed drawdown. Both are read by hand, as the protocol's verdict checks are, so a single headline number cannot hide a blown drawdown. A barely positive result leaves no room for real execution, which is usually worse than the uniform 150 ms lag we just measured.
P2 is where an attractive curve starts to look fragile. A small edge can stay positive while drawdown rises faster than profit, and a gate that compared only one of the two numbers would celebrate the curve and miss the fragility. On manual review, the written gate fails the leg when profit does not clear drawdown, however healthy the profit line looks.
Our four-gate latency ladder shows how this single delay check fits into a broader sequence of delay gates. Apply the same delay every time, or the comparison is not repeatable.
#Robustness perturbs the market, not execution
P3 asks whether the edge can earn money in a slightly wrong world. Execution stays clean (no delay), but the market inputs take the RobustLab treatment: Skip 20 / Shift 20 / ExitDelay 20 with seed 42. Those settings force the strategy to work without perfect inputs.
| RobustLab setting | Failure it probes |
|---|---|
| Skip 20% of ticks | A thin or intermittent tape hides entries |
| Shift 20 points | Entry prices drift and cost arrives as slippage |
| ExitDelay 20% | Orders leave later and exits become worse |
| Seed 42 | The same perturbation returns on every run |
The three changes cover different failure shapes. Skipping ticks can remove a signal the strategy thought it saw. Shifting points can turn a quoted entry into a worse fill. Holding exits longer can damage the exit logic. Together they map to four failure hypotheses: fill cost, missing entries, exit quality, and spread squeeze. Seed 42 makes the perturbation repeatable from one run to the next. The table above names shift as the probe for fill cost. It lists no probe for spread squeeze, and in our tester the price-shift and noise-style axes are currently vacuous, so fill cost and spread squeeze are not measured here. Only skip and exit delay reliably bind.
Where a knob does nothing, we record "not measured here," never "passed." A gate that counts an inert setting as evidence is worse than no gate.
We can say that skip and exit delay changed the result. We cannot claim that a shift or noise knob proved anything if the tester never applied it. An inert control is evidence about the tester, not about the strategy.
#The wave we deleted: robustness that tested nothing
The worst failure in this kind of testing is a leg that runs the wrong experiment without erroring. P3 needs the perturbed configuration. The deleted wave showed what happens without it: run the plain configuration used by P1 and P2 under clean execution and the result is a duplicate of P1, to the cent, trade for trade, not a rough approximation.
We purged 264 rows because one wrong binding sent the robustness leg to the plain configuration.
The database cannot protect us from a bad binding. A row stores a number. The conditions live in the protocol: the configuration, the phase, the execution mode. We now verify those conditions before reading the wave, and our account of the deleted wave records the failure in full.
A known result must also return bit-for-bit before a wave starts. Our bit-identical control catches drift in the instrument or the runner before new results can inherit it.
Verification is cheap compared with explaining a void wave later. We check the phase identity, the setfile binding, the execution mode, and the expected count before we read profit. A mismatch stops the wave at the door.
#What the code enforces today
The current protocol covers structure, binding, immutability, and the terminal journal. Several performance gates are still written but not wired, so we publish both states.
| Written in the protocol | Enforced in code today |
|---|---|
| Each leg must conclude 500 ranges | No comparison between a lane's actual count and the expected count |
| G1–G4 latency gates with numeric thresholds | No gate object, threshold, or evaluation |
| Stage one gates stage two | Stages are parsed and drawn, but they do not gate execution |
| P2 runs at 150 ms and P3 carries RobustLab | The legs can run clean unless the structured blocks are present |
There are 0 numeric latency-gate thresholds enforced in code today. That does not erase the protocol. It changes who owns the check. A reviewer still compares the conditions and verdicts by hand, and we would rather say that plainly than call an unwired rule automatic.
The code does enforce several structural boundaries. A phase needs a valid identifier, deterministic flag, execution mode, and expected count. Unknown keys are refused, not silently dropped. A run names exactly one protocol, a phase database is never overwritten, and the terminal's own journal must contain the passed token before registration.
Results also stay inside the correct range. We compute expectation per range and compare cells within the same range and phase. Averaging across ranges could let one smooth period hide a blown one, which would turn a visible failure into a pleasant headline.
#What we took from it
One wave went wrong, and three checks now stand between a result and our trust in it. We verify the binding conditions before we read a wave. A known result must return bit-for-bit before a new wave starts. We publish the gap between what the protocol promises and what the code enforces.
- P1, P2, and P3 ask different questions: does an edge exist, what does 150 ms of latency cost, and does the edge survive a slightly wrong market.
- Conditions are verified before rows are read. Phase, configuration, and execution mode must match, because a matching number is not evidence on its own.
- A written gate is not an enforced gate. The gap goes on the review list, not into a marketing sentence.
The ladder returns three bounded questions, the assumptions written beside them, and a list of every gap between the written protocol and the code.
Every accepted result carries its evidence. Our DAAVILEFX research dashboard shows the phases, verdicts, and raw deal streams run by run, and the published DAAVILEFX results list what cleared the process. The record exists because a spreadsheet curve and a live order book are not the same test.
#Questions we get asked
#What are the three legs of the trading strategy test?
The first leg is a clean baseline, the second adds a deterministic 150 ms delay, and the third perturbs the market while keeping execution clean. Each leg asks a different question about the same strategy.
#Why does P3 need a different setfile?
P3 must carry the RobustLab inputs: Skip 20 / Shift 20 / ExitDelay 20 and seed 42. A plain setfile under clean execution reproduces P1 instead of measuring robustness.
#What should I check before trusting a wave?
Check the protocol binding, phase, setfile, and execution mode before reading the rows. The declared count of 500 ranges per leg is still checked by hand: code does not compare a lane's actual count with the expected count.
#Is the three-leg ladder fully automated?
Not yet. There are 0 numeric latency-gate thresholds enforced in code today, even though the protocol names G1–G4. The written rules remain part of our manual review.
#Why do you compare results within a range?
A range is one period of time: a start date and an end date. A cell is one range under one phase. Comparing cells within the same range and phase stops one smooth period from hiding a blown one.