Paired t-test vs two-sample test on the same data: why pairing patients with themselves changes p from 0.079 to 0.003, plus Cohen’s d.
Chapters
0:00 Same 20 numbers, two p-values
0:34 In this episode
1:05 Cushny, Student, Fisher
1:29 Shuffling a coffee experiment
2:21 The standard error of a gap
3:47 Treating the drugs as two groups
4:28 Pairing each patient with themselves
5:40 Why pairing wins
6:16 What pairing can and can’t fix
6:55 Student’s 666 to 1 and Fisher’s t
7:11 Comparing two proportions
8:17 Effect size: Cohen’s d
8:59 How often does one group win?
9:42 Significant versus important
11:05 Back to the sleeping drugs
11:34 The mislabeled drug
12:01 Key ideas and next time
Previous episode: 1.11 “The p-Value”. Next episode: 1.13 “Chi-Square Test”.
Subscribe to Thinking Lab:
Reference: OpenStax Introductory Statistics 2e (Illowsky & Dean, CC BY-NC-SA 4.0, and OpenIntro Statistics 4e (Diez, Cetinkaya-Rundel & Barr, CC BY-SA 3.0, OpenIntro Statistics 5.3, 6.2 and 7.2-7.3 and Introductory Statistics 10.1, 10.3 and 10.4.
Historical sources: Senn & Richardson (1994), The first t-test, Statistics in Medicine 13: · Senn & Richardson (1994), Crossref record: · Cushny & Peebles (1905), J Physiol: · Senn, Cushny and Peebles and optical isomers, James Lind Library: · Student (1908), The probable error of a mean, Biometrika: · McGraw & Wong (1992), A common language effect size statistic: · Cohen on small, medium and large effect sizes (Bakeman memo):
Narration: AI voice (ElevenLabs); music: AI-generated (Lyria) and original synthesized tracks.
source