How to Read a Research Study, Part 4 of 7

Trade-offs and limits

Taleb’s lens: ergodicity, the population-vs-individual gap, the Lindy heuristic, the rare ruinous event, skin in the game.

Ask Grace

Want to ask Grace whether a population result actually applies to one person? Ask Grace.

Ergodicity, the population-vs-individual gap

A 46% one-year remission rate in DiRECT is a population figure. It does not say “you have a 46% chance of remission”. It says “of one hundred people who walked the protocol, 46 remitted at one year”. The individual is non-ergodic; the trial mean averages across people who differ in adherence, biology, time-since-diagnosis, support network, what their household eats. Ole Peters’s 2019 Nature Physics paper made the formal argument; Taleb made the practical one.

“Every single bit of risk you take adds up to reduce your life expectancy. If you climb mountains and ride a motorcycle and hang around the mob and fly your own small plane and drink absinthe, and smoke cigarettes, and play parkour on Thursday night, your life expectancy is considerably reduced, although no single action will have a meaningful effect.”

“Never compare a multiplicative, systemic, and fat-tailed risk to a non-multiplicative, idiosyncratic, and thin-tailed one.”

Source: Taleb, Skin in the Game (2018), Chapter 19, The Logic of Risk Taking. The casino does not lose because the casino owns the ensemble; the gambler can lose because the gambler walks one trajectory. The trial owns the ensemble; the person on the trial drug walks one trajectory.

Greenhalgh’s parallel critique of EBM, written in 1996 and unchanged through the seventh edition, lands the same point in clinical language:

“When applied in a vacuum, that is, in the absence of common sense and without regard to the individual circumstances and priorities of the person being offered treatment or to the complex nature of clinical practice and policymaking, ‘evidence-based’ decision-making is a reductionist process with a real potential for harm.”

Source: Greenhalgh, How to Read a Paper, 7th edition (2025), Preface to the first edition (1996). The Taleb point in ergodicity language and the Greenhalgh point in clinical language converge: the population mean is real, the application to one person without judgement is harm.

Black swan, the rare ruinous event

Severe hypoglycaemia, DKA, post-bariatric severe hypoglycaemia, pancreatitis signal on GLP-1 medications, exercise-induced ketosis on closed-loop systems running aggressive targets. These are low-frequency, high-consequence events that population averages hide. A trial powered for mean HbA1c at six months is rarely powered for the one-in-a-thousand event that arrives at month nine; the absence of the event in the trial report is not the absence of the event in the world.

Lindy, the heuristic that survives

Newer is not better. A 2026 single-arm pilot trial does not supersede a 2002 systematic review on the same question. Structured education (DAFNE for T1D, DESMOND for T2D, both well over twenty years old) outperforms most new digital interventions on every outcome that matters because the cohort of clinical hours behind the curriculum is larger than the new intervention’s entire trial. When you are reading a new study, ask what older heuristic it claims to supersede, and whether the new evidence is strong enough to displace twenty years of behaviour.

Via Negativa, subtraction beats addition

If a clinical recommendation lists three reasons, ask which one is load-bearing. The other two are usually padding to make the case feel safer. T2D remission via the Newcastle / DiRECT protocol works by removing ectopic fat from the liver and pancreas. The protocol is not “remove the liver fat AND increase the activity AND optimise the sleep AND …”; it is “remove the liver fat; the rest is hygiene”. A guide that adds three reasons to a recommendation has usually weakened the recommendation.

Better to be roughly right than precisely wrong.

Skin in the game

Per Taleb, the test of any advisor is whether they would carry the consequence of their own advice. Applied to medical research: the trialist who hides negative data does not live with the harms their hidden data would have prevented. The pharma rep selling the closed-loop system does not wear the system. The journalist writing the headline does not adjust their basal insulin on the back of it. The hierarchy of credibility runs in the opposite direction to the hierarchy of confidence: the person with the most certainty in their voice usually has the least at stake.

Part 4 of 7

Trade-offs and limits

Read more on GNL

Ask Grace