The p-value from Arbuthnot’s 82 years of christenings: how a hypothesis test works, what errors and power mean, and what a small p cannot say.
Chapters
0:00 82 years, all boys: what does it prove?
0:34 In this episode
1:06 Arbuthnot, Fisher, Neyman and Pearson
1:26 Arbuthnot’s argument and the p-value recipe
2:34 The lady tasting tea
3:27 Testing a flyer’s sleep claim
5:18 Checking against the true mean
5:56 Drawing the line at 5%
6:49 Two kinds of error
7:46 Power: catching a real effect
9:07 What is a test for?
10:24 p-values when nothing is going on
11:48 Four p-value traps
13:14 Back to 1710
13:46 Key ideas and next time
Previous episode: 1.10 “Confidence Intervals”. Next episode: 1.12 “Paired t-Test”.
Subscribe to Thinking Lab:
Reference: OpenStax Introductory Statistics 2e (Illowsky & Dean, CC BY-NC-SA 4.0, and OpenIntro Statistics 4e (Diez, Cetinkaya-Rundel & Barr, CC BY-SA 3.0, OpenIntro Statistics 5.3 and 7.4 and Introductory Statistics 9.1-9.6.
Historical sources: Arbuthnot (1710), An argument for Divine Providence, Phil. Trans. 27 (OCR text): · Arbuthnot (1710), Philosophical Transactions: · Fisher (1935), The Design of Experiments, first edition: · Fisher (1925), Statistical Methods for Research Workers, chapter 3: · Richardson (2021), the lady tasting tea, Significance 18(5): · Senn (2012), Tea for three, Significance 9(6): · Fisher (1955), Statistical methods and scientific induction: · Neyman (1956), Note on an article by Sir Ronald Fisher: · Lehmann (1993), The Fisher, Neyman-Pearson theories of testing hypotheses: · American Statistical Association (2016), statement on p-values: · Wasserstein & Lazar (2016), The ASA statement on p-values: · Greenland et al. (2016), Statistical tests, P values, confidence intervals, and power:
Narration: AI voice (ElevenLabs); music: AI-generated (Lyria) and original synthesized tracks.
source