Four Trading Ideas That Failed — and What They Taught Us

Verdict: overfit / no economic edge · Asset/TF: BTC and liquid crypto, 15m–1h · Evidence: chronological OOS, realistic costs, walk-forward and block bootstrap

Most trading research that reaches the internet has already passed through a publication filter: winners are shown and failures disappear. That creates a dangerous impression that every reasonable feature becomes a strategy after enough modeling.

We ran four plausible ideas through the same basic discipline: decide the hypothesis before opening the final period, split chronologically, enter on the next open, remove overlapping labels, charge commission plus spread and slippage, select thresholds on validation only, and leave the final result untouched.

All four failed. The failures were useful because each ruled out a different shortcut.

The failed experiments

Failure 1: high precision is not positive expectancy

The selective classifier was the most seductive result. It identified UP events with 66.21% precision against a 46.63% base rate. That is real classification lift.

But the selected trades lost 9.19 bps each after costs. Precision ignored payoff asymmetry: frequent small correct calls were offset by fewer, larger adverse moves and execution costs. A retrospective threshold above 0.80 looked profitable only after inspecting the final test, so it was documented but not promoted.

Lesson: optimize and validate conditional return, not hit rate. A model can classify an event correctly while trading the event remains unprofitable.

Failure 2: predicting movement size does not predict direction

The order-book experiment asked a narrower question: can book state identify breakouts large enough to clear costs? The feature layer modestly improved magnitude ranking. It did not tell us whether a price breakout would continue or reverse.

The final price-plus-book gate lost 21.71 bps per trade. None of 17 assets had positive net expectancy, and the 7-day block-bootstrap interval was [-29.33, -14.67] bps. RLXBT executable proofs on BTC, ETH and SOL were all negative.

Lesson: a volatility or magnitude model is not a directional model. It may be useful for sizing, execution or option selection, but it cannot rescue a direction rule by itself.

Failure 3: incremental improvement can still be economically useless

The FRED layer used VIX, Treasury yields, the dollar index, high-yield spread and the policy rate. Every daily observation was delayed by two calendar days before use.

Price-only signals lost 8.44 bps per trade; price plus macro lost 4.19 bps. The improvement was measurable, but the result remained below zero. Its bootstrap interval crossed zero and only one of three walk-forward windows was positive.

Lesson: compare against zero as well as against the baseline. Making a bad strategy less bad is research progress, not a tradable strategy.

Failure 4: relevant data can operate at the wrong clock speed

Bitcoin network data sounds naturally suited to Bitcoin forecasting. We tested active addresses, block count, native fees, hash rate, transaction count and transfer count from the Coin Metrics Community API. To prevent look-ahead, each UTC daily value became available only two days later.

The network layer reduced direction ROC-AUC from 0.5684 to 0.5548 and large-move ROC-AUC from 0.6601 to 0.6501. Selected trades lost 6.50 bps each. All three walk-forward folds were negative. The RLXBT execution proof returned -7.79% over 204 trades with Sharpe -0.90.

Lesson: domain relevance is not enough. Daily network state may describe the macro regime while contributing no timing information at a four-hour horizon. Historical mempool pressure and block-level events remain a separate, untested hypothesis.

The research protocol that made the failures visible

• Chronological train, validation, inner-test and final splits with purging.
• Thresholds selected on validation only; no final-period reversal or retuning.
• Decisions spaced so forward-return labels did not overlap.
• Next-open entry and fixed holding horizon.
• 11.81 bps round-trip cost: commission, slippage and half-spread.
• Walk-forward windows and 7-day block bootstrap.
• RLXBT engine replay after the local statistical audit.

These controls are not decoration. Removing any one of them can turn several of the rejected ideas into an attractive chart.

What belongs in the knowledge base

A failed backtest should be saved when it contains a reusable boundary:

1. the exact hypothesis and causal data availability;
2. the untouched threshold-selection rule;
3. economic results after costs;
4. uncertainty and regime consistency;
5. the condition under which the idea may be tested again.

For these four ideas, the reopening conditions are explicit: a new untouched period, genuinely historical intraday mempool data, a separate directional mechanism for book features, or a payoff-aware target for selective ML. Until then, tuning the same final sample would manufacture confidence rather than knowledge.

Reproduce

The research artifacts contain frozen parameters, threshold tables, walk-forward results, bootstrap intervals and executable RLXBT strategy graphs. The primary on-chain case uses:

{
  "entry_rules": [
    {"condition": "onchain_probability >= 0.55", "direction": 1},
    {"condition": "onchain_probability <= 0.45", "direction": -1}
  ],
  "max_hold_bars": 16,
  "entry": "next_open",
  "round_trip_cost_bps": 11.8088
}

Tools called: load_dataset → report_research_event → ai_run_backtest. Chronological walk-forward and block bootstrap were run by the reproducible research pipelines before the engine proof.

Negative research is not empty space between successful strategies. It is the map that prevents the next researcher—or the next agent—from spending the same OOS sample twice.