Guide

Deflated Sharpe ratio explained: the check that catches overfitting

Backtesting fundamentals · ~7 min read

A Sharpe ratio of 3 sounds like a great strategy. It can also be exactly what you'd expect from pure noise, if you tested a thousand parameter combinations and kept only the best one. The deflated Sharpe ratio is the statistic built to tell the two apart.

Quick answer

The deflated Sharpe ratio discounts a strategy's raw Sharpe ratio for how many variations were tried to find it. The more combinations you searched, the higher a Sharpe ratio you'd expect to find by chance alone — deflation subtracts that expected-by-luck component, leaving an estimate of whether the result reflects a genuine edge or just the best of many random outcomes.

The problem deflated Sharpe was built to solve

Say you backtest one strategy with fixed rules and get a Sharpe ratio of 1.4. That number means roughly one thing. Now say you tested 500 combinations of moving-average lengths, RSI thresholds, and stop-loss levels, and the best one scored a Sharpe of 1.4. That's a completely different situation, even though the number on the page looks identical.

With enough attempts, pure random noise will eventually produce an impressive-looking Sharpe ratio by chance — the same way flipping a coin a thousand times will eventually give you a run of ten heads in a row, without the coin being biased. Overfitting is what happens when you mistake that lucky run for a real edge and trade it with real money.

How the calculation works, in plain terms

The deflated Sharpe ratio (introduced by Bailey and López de Prado) takes three inputs: the Sharpe ratio you're evaluating, the number of independent trials or parameter combinations you searched, and the variance across those trials. From these it computes the Sharpe ratio you'd expect to see by chance alone, purely as a side effect of trying that many combinations — then it tests whether your actual result clears that bar by a statistically meaningful margin.

The output is usually expressed as a probability: how likely is it that a genuinely skill-less strategy, tested this many times, would produce a Sharpe ratio at least this good? A high deflated Sharpe probability (people commonly use ~0.95 as a rough threshold) means the result is unlikely to be chance. A result that collapses toward 0.5 or below after deflation is a strong sign you found noise, not an edge.

Why the number of trials matters so much

The core intuition is simple: the more lottery tickets you buy, the more likely one of them wins, even though each individual ticket is still a losing bet in expectation. Backtesting has the same structure. Each parameter combination you test is another ticket. If you only ever look at the winning ticket, you'll systematically overestimate how good your strategy really is.

Why this trips people up Most traders can tell you the Sharpe ratio of their best strategy. Very few can tell you how many strategies or parameter combinations they tried to find it. That second number is exactly what deflated Sharpe needs, and exactly what most backtesting tools don't track at all.

Deflated Sharpe vs out-of-sample testing — different jobs

These two checks are often confused because they both push back against overfitting, but they catch different failure modes and neither replaces the other.

Deflated Sharpe corrects the in-sample number itself for the search process that produced it — it's a statistical adjustment applied to a single number.
Out-of-sample testing checks whether the strategy, unchanged, still performs on data it never saw during development — it's an entirely separate test on fresh data.

A strategy can pass one and fail the other. The most convincing result is one that holds up on genuinely unseen data and survives deflation for the number of variations you tried to get there. Relying on just one of the two still leaves a real gap.

How to actually estimate this in practice

The honest version of this check requires you to log every parameter combination you tested, not just the winner — most people don't do this, which is exactly why the check gets skipped. A close, practical substitute if you haven't tracked every trial: be conservative and assume you tried more combinations than you remember, since manual tuning tends to implicitly search a wider space than a formal grid search would suggest. When in doubt, treat an untracked search as a large one.

What a low deflated Sharpe actually tells you

It doesn't prove your strategy is worthless — it tells you that the evidence you currently have isn't strong enough to distinguish it from noise, given how hard you searched to find it. The response isn't to abandon the idea, it's to demand more evidence: a longer out-of-sample period, a simpler rule set with fewer degrees of freedom, or testing the underlying logic on an entirely different asset or timeframe to see if the edge shows up independently of the exact parameters you first found it with.

Or see the honest version automatically

The Honest Backtest Engine tracks every optimisation run and deflates the headline Sharpe ratio for the number of combinations actually searched — alongside a fixed out-of-sample split you can't quietly override. Two different overfitting checks, both run automatically, every time.

See how it works

Frequently asked questions

What is a deflated Sharpe ratio?

A deflated Sharpe ratio adjusts the raw Sharpe ratio downward based on how many strategy variations or parameter combinations were tested to find it. It answers a different question than the raw Sharpe: not just "how good does this look", but "how good does this look after accounting for the fact that I went looking for the best of many tries".

Why does trying more parameter combinations lower the deflated Sharpe ratio?

Every combination you test is a chance for randomness alone to produce an impressive-looking result. The more combinations you search, the more likely it is that your best result is inflated by luck rather than a genuine edge, so the deflation grows with the number of trials.

What is a good deflated Sharpe ratio?

There isn't a single fixed cutoff, but a common practical read is that a deflated Sharpe probability above roughly 0.95 suggests the result is unlikely to be pure chance, while a raw Sharpe that collapses toward zero after deflation is a strong sign of overfitting.

Is deflated Sharpe ratio the same as out-of-sample testing?

No, they catch different problems and work best together. Out-of-sample testing checks whether a strategy performs on data it never saw. Deflated Sharpe corrects the in-sample number itself for how many attempts it took to find that strategy in the first place.