I priced Polymarket's crypto up/down contracts as what they actually are: binary options that settle on a 60-second average, not the last print. Rebuilding that settlement rule reproduced real outcomes 97.9% of the time, against 77.1% for the point-to-point read most people use. The backtest said the model had a big edge.

Then 219 live paper trades said otherwise. The book was perfectly calibrated and my model over-predicted its own side by 12 points. Worse, the bigger the claimed edge, the worse the result: 62% hit rate at +4–9c of edge, 37% at +18–25c. The entry filter was selecting on model error, not information.

Why did the backtest lie? It scored against sparse traded prints instead of the live book, so a model reading live spot beat a stale proxy. An unbiased shadow log of 12,386 observations confirmed it: blending model and book, the best weight on the model was zero.

One thing survived: the exit rule. Selling early beat holding to expiry by +4.85c a trade (t = 5.03), offline and live. Entries are off. The post-mortem is in the repo.

My stock cache was today's S&P 500. That sounds harmless until you remember that today's list is the stocks that won. Anything that went bankrupt, got acquired cheap or dropped out of the index isn't in it.

I measured it directly: backtesting on today's members inflated returns by about 16 points of CAGR a year. For a momentum book, that was all of the alpha. The strategy wasn't picking winners, the universe was.

The fix is point-in-time constituents: test on the index as it was on each day, not as it is now. If a backtest looks too good on a modern ticker list, this is the first thing to check.

A parameter sweep on my earnings-drift strategy came back with Sharpe ratios of 1.34–1.44. I was ready to celebrate.

The catch: it was computed on a cash basis with a zero risk-free rate. Earnings drift trades in bursts, so the book sits in cash most of the time, and cash earning nothing quietly flatters the ratio. Credit the idle cash at T-bill rates, size it the way it would actually run, and the honest number is about 0.70.

That 0.70 is now the bar every variant has to clear. Half the original Sharpe was an accounting choice.

Before touching the hold-out data, I write the hypothesis, the exact test and the pass bar into a ledger. Then I run it once.

Earnings drift, a multi-asset book and a crypto strategy all went through that process this cycle. All three failed. No re-running with new parameters, no "what if we just exclude 2022". A hold-out you mine twice isn't a hold-out anymore.

The rule now: the 2004–2026 history is spent. New ideas get forward-tested on data that doesn't exist yet.

The line in the bottom-left corner is Meridian's earnings-drift bot, running on a $100k paper account. It counts only trades the bot logged in real time, from Sep 17 on. Backfilled history is excluded, so the number can't be flattered by hindsight.

Realized P&L is closed trades only. Open positions aren't marked to market. It's a paper account with simulated fills, not a track record, and it updates when I refresh the site.