What backtesting can (and can’t) tell you

Every trading rule sounds convincing when you describe it out loud. “Buy when it drops 20% below its high.” “Rotate out when the 50-day crosses under the 200-day.” The words are persuasive because they’re tidy, and tidy feels true. Backtesting exists to interrupt that feeling — to replay the rule against real history and show you what it actually did, not what it sounds like it should do.

That’s a genuine service. It’s also where a lot of people quietly go wrong, because a backtest is easy to misread in a very flattering direction. So it’s worth being precise about both halves: what a backtest can tell you, and what it absolutely cannot.

What is a backtest?

A replay of a mechanical trading rule across historical prices. You define rules — an entry condition, an exit condition — and the engine runs them across historical price data, buying and selling exactly when the rules said to. Out the other end comes an equity curve (what your money would have done over time), a drawdown profile (the peak-to-trough declines along the way), and a record of when the strategy traded versus sat idle.

The value is that it replaces a story with evidence. “This should work” becomes “here’s how it behaved across the 2020 crash, the 2022 selloff, and the flat stretch in between.” That’s a real upgrade over reasoning from a single memorable example.

What can a backtest tell you?

How a rule behaved, the drawdown you would have had to sit through, and whether the idea is obviously broken.

How a mechanical rule behaved, mechanically. If your rule is genuinely objective — no judgment calls — a backtest shows its historical behavior faithfully. That’s its home turf.

The drawdown you’d have had to stomach. This is the number that matters most and gets ignored most. The headline return is seductive; the maximum drawdown is honest. A strategy that returned 200% but spent a year down 45% along the way is a strategy almost nobody actually holds through. The backtest tells you the peak-to-trough pain you’d have signed up for — and whether you’d really have sat through it.

Whether an idea is obviously broken. Backtests are better at rejecting strategies than confirming them. If a rule looks bad across history, that’s strong evidence to drop it. A rule that looks good is only a candidate for more scrutiny — not a green light.

What can’t a backtest tell you?

That the future will look like the past, whether you curve-fit it, what costs would have taken, or what you would really have done.

That the future will resemble the past. This is the one that sinks people. A backtest is a statement about history, full stop. Markets regime-shift; the conditions that made a rule work can quietly disappear. Past behavior is evidence about the past and a hypothesis about the future — never a promise.

Whether you curve-fit it into looking great. Give yourself enough knobs — this threshold, that indicator, this lookback window — and you can tune almost any rule to look brilliant on the exact history you tested it on. That’s not a discovered edge; it’s a memorized answer to one specific exam. The more parameters you tweaked while staring at the same data, the less the result means. The tell: a strategy that’s dazzling in-sample and falls apart the moment you test it on a period you didn’t tune against.

The costs that quietly eat returns. A naive backtest can ignore trading fees, bid-ask spreads, slippage, and taxes. A rule that trades often can look wonderful gross and be a loser net. Always ask what frictions the test did — and didn’t — include.

What you’d have actually done. The backtest assumes a disciplined robot that followed every signal without flinching. You are not that robot. The gap between a strategy’s backtested return and its investor’s real return is often mostly behavior — panic sells, skipped signals, second-guessing at exactly the wrong moment.

How do you read a backtest without fooling yourself?

Look at drawdown first, test on data you didn’t tune on, and account for costs.

• Look at drawdown before return. Ask “could I have held this through its worst stretch?” before you ask “how much did it make?”
• Test on data you didn’t tune on. If it only shines on the period you built it against, you found a coincidence, not an edge.
• Prefer fewer knobs. A simple rule that works decently across many periods beats an intricate one that’s perfect on one.
• Account for costs. Re-run the mental math with fees and slippage and see if the edge survives.
• Treat it as a filter, not a forecast. A good backtest earns a strategy a place on your shortlist. It never earns it your money on its own.

Is a backtest a forecast?

No. A backtest is a hypothesis test, not a crystal ball. Used well, it’s one of the best tools you have for killing bad ideas cheaply and understanding the risk profile of the survivors. Used badly, it’s an expensive way to convince yourself of something that was only ever true in the rearview mirror.

Tradune’s portfolio backtesting runs rule-based strategies across historical data and surfaces the equity curve and the drawdowns — including the stretches a strategy underperformed — because the wins were never the part you needed convincing about. It shows you the historical record honestly; the interpretation, and the humility, are yours to bring. As always, this is educational analysis, not a recommendation to trade any strategy.