Today we’re going to talk about p-hacking (also called data dredging or data fishing). P-hacking is when data is analyzed to find patterns that produce statistically significant results, even if there really isn’t an underlying effect, and it has become a huge problem in science since many scientific theories rely on p-values as proof of their existence! Today, we’re going to talk about a few ways researchers have “hacked” their data, and give you some tips for identifying and avoiding these types of problems when you encounter stats in your own lives.
XKCD’s comic on p-hacking:
Crash Course is on Patreon! You can support us directly by signing up at
Thanks to the following Patrons for their generous monthly contributions that help keep Crash Course free for everyone forever:
Mark Brouwer, Trevin Beattie, Satya Ridhima Parvathaneni, Erika & Alexa Saur, Glenn Elliott, Justin Zingsheim, Jessica Wode, Eric Prestemon, Kathrin Benoit, Tom Trval, Jason Saslow, Nathan Taylor, Brian Thomas Gossett, Khaled El Shalakany, Indika Siriwardena, SR Foxley, Sam Ferguson, Yasenia Cruz, Eric Koslow, Caleb Weeks, Tim Curwick, D.A. Noe, Shawn Arnold, Ruth Perez, Malcolm Callis, Advait Shinde, William McGraw, Andrei Krishkevich, Rachel Bright, Mayumi Maeda, Kathy & Tim Philip, Eric Kitchen, Ian Dundore, Chris Peters
—
Want to find Crash Course elsewhere on the internet?
Facebook –
Twitter –
Tumblr –
Support Crash Course on Patreon:
CC Kids:
source
I still don’t understand. What does p-value mean, and how do we choose the p-value? Does low p-value mean significant affect, and high p-value means no affect? Is it just cherry picking? Can someone please explain I’m so confused😭
Chad and Stacy studio 💪💪💪💪
Awesome video! Thank you!
Are the peas placed on the laptop because the peas are hacking? (P-Hacking?)
thumbs up
"You feel bad about taking their million dollars."
You should never feel bad about taking Kevin O'Leary's money. p<0.05, effect size = yes.
So what does hacked :p mean?
She reminds me a bit of Terri Garr.
So, in order to believe any statistic we see, and even then the statistic might itself be a chance result, we need Ph.D.'s in statistical analysis, an army of scientist employees, and sophisticated lab and computer equipment to verify claims. Oh, and access to the journals said statistic was published in, each of which will cost a pretty penny.
Not just CC, but in general, the media and experts do a fine job of outlining the problem but rarely give solutions and even when they do, the solutions are impractical or theoretical.
The same media and experts, due to shenanigans like p-hacking, have broken the public's trust in what they have to offer.
My solution: mandatory statistics classes beginning in Kindergarten. This will take forty years for any critical mass of trust to return to the societal influencers because in forty years, today's kindergarteners will have the power to influence and the old guard of prevaricating and unethical doyens will have died or be too old to dictate the direction of society in any meaningful way.
Or, wait for the Matrix plugin seats. They're coming!
Regards Adrian, I am planning to carry out an empirical study with p hacking being the subject at hand, could you please a case study that could be worked upon.
Thanks in advance !!!!!!
i had to pause the damn video to laugh like a goat when i saw the peas on the laptop XD
The bowl of peas sitting at that angle and on a laptop is very distressing!
This crash course is great. The level is fairly high (I believe that most people with a PhD could even learn a thing or two), and it's still comprehensible.
Great job!
Haha a bowl of P's
Plot twist: those peas are actually green jellybeans.
Actually, there is a small misconception in this video (the table in the beginning). When we reject the alternative hypothesis that does not mean that the the null hypothesis is true. It simply means that we do not have statistical evidence to reject the null hypothesis – we cannot say with 100 % statistical certainty that Ho is true. Andy Fields writes in 'Discovering Statistics in SPSS': "If the p-value is greater than 0,05 you can decide to reject the alternative hypothesis but that is not the same as the null hypothesis being true" (Fields, 2018: 76).
That broken laptop annoys me
But that information useless in that it does not offer any details about which variables (jelly beans) are significant or even how many! By adjusting your p-values you've CHANGED your null hypothesis. There are better ways of doing this!
I only found out this statistic video 2 days before my exam dammit
Poor kid… He fell asleep studying and gets called an honest mistake 😀
can you guys do a crash course forensics??
oh no, is it over? =o
Is this the last episode? There hasn't been an upload of this series in 2 weeks…
This puts the sigh in science and the lie in believe the scientists, they all agree.
how to get p-hacking out of "for profit" research?
1-make only "no profit" & massively perreviewed research significant enough for laws.
2-implement UBI and/or have a "national scientific research fund" that allows for scientists to follow through theier research within ethical conditions for themselfs and their results.
This is such an important video
Is there a relationship between p-hacking and Simpson paradox?
Watch out people: the green jellybeans are really peas.
phacking hell
All of that and you didn't mention the common terms "fishing" and "exploratory analyses", nor how such approaches can be used ethically to generate new hypotheses, or be methodologically accommodated, such as with screening and hold-out samples (cross-validation).