Regression to the Mean: The Jinx That Is Just Arithmetic

An instructor training pilots noticed something that seemed to teach him a hard lesson about human nature.

When a trainee flew an unusually good manoeuvre and the instructor praised them, the next attempt was usually worse. When a trainee flew badly and the instructor shouted at them, the next attempt was usually better.

His conclusion was straightforward: praise makes people complacent, criticism makes them focus. He had years of consistent observation behind it.

He was wrong, and the reason he was wrong has nothing to do with psychology.

An unusually good flight is unusual precisely because everything happened to go right at once — the trainee's skill plus a favourable combination of wind, timing, and luck. The skill carries over to the next flight. The lucky combination does not. So the next attempt is likely to be closer to the trainee's ordinary standard, which is worse. The same logic in reverse means a terrible flight is likely to be followed by a better one.

This would have happened whether he praised, shouted, or said nothing at all. He was watching a statistical certainty and reading it as a lesson about motivation.

What is actually happening

Almost any real-world result combines two things: something stable (skill, quality, underlying conditions) and something variable (luck, timing, noise, circumstance).

An extreme result — very good or very bad — usually requires both components to line up. To have an exceptional month, a salesperson probably needs to be genuinely capable and have had a few things break their way.

Now run the next month. The skill is still there. The luck is drawn fresh. On average, fresh luck is average. So the result lands closer to the middle.

That is the whole mechanism. Nothing pulls results back toward the average; there is no force involved and no correction happening. It is simply that extreme outcomes need luck, and luck does not repeat on demand.

Two consequences worth holding onto:

Regression happens in both directions. Bad extremes improve for the same reason good ones deteriorate. A disastrous quarter is followed by a better one, on average, with no intervention required.

The size of the effect depends on how much luck is involved. In a nearly pure skill contest, the best performer stays the best and regression is slight. In something heavily influenced by chance, extremes regress dramatically. This gives you a practical rule: the more randomness in a domain, the less a single outstanding result tells you.

The trap it sets

The pilot instructor's error is not a curiosity. It is a systematic trap that catches organisations constantly, because regression makes punishment look effective and reward look counterproductive.

Consider what happens when you intervene only at the extremes — which is what everyone does, because extremes are what get noticed.

You investigate the worst-performing branch, send in consultants, change the manager. Next quarter it improves. Everyone concludes the intervention worked. Some of that improvement — often most of it — would have occurred anyway.

You promote your best salesperson to lead a team. Their numbers decline. Everyone concludes they were promoted beyond their competence. Perhaps. Or perhaps their record year needed luck that did not recur.

A footballer has an extraordinary season, appears on magazine covers, and has a worse season next year. This gets called a jinx or the pressure of fame. It is arithmetic.

A medical treatment is given to patients when symptoms are at their worst — which is when people seek treatment. They improve. Some of that is the treatment; some is that symptoms at their peak tend to subside. This is exactly why controlled trials with a comparison group exist: without one, regression alone will make almost any intervention look effective.

The general rule: whenever you act only when things are extreme, regression will flatter your intervention if things were bad and undermine it if things were good.

How to avoid being fooled

Compare against a control, not against the extreme itself. The question is never "did it improve?" but "did it improve more than a similar case where we did nothing?" This is the only reliable defence.

Expect the follow-up to be more ordinary, and say so in advance. If you predict before intervening that the number will improve partly on its own, you protect yourself from over-crediting the action — and from the reverse error of abandoning something useful because a stellar result was not sustained.

Judge on many observations, not one. A single result is skill plus luck. An average across many is mostly skill, because the luck cancels out. This is why one interview, one quarter, or one match is weak evidence, and why persistent performance over years is strong evidence.

Be careful selecting on extremes. Choosing the top ten performers for a study of what makes people successful guarantees you have selected for both skill and good luck, and their subsequent performance will disappoint. This is closely related to survivorship bias — both are cases where the sample was chosen in a way that guarantees a misleading conclusion.

Remember the effect is bigger where luck is bigger. In domains with lots of randomness — venture returns, short-term markets, sales in a lumpy business — a spectacular result should be discounted heavily. In low-luck domains, less so.

The instructor was not careless. He observed accurately, repeatedly, over years, and drew the only conclusion that seemed available. The information that would have corrected him was not in his data at all — it was in understanding what a data set of extreme results necessarily looks like.