Famous Cognitive Psychology Experiments that Failed to Replicate - Social sciences found to be 63% fake and gay

https://aethermug.com/posts/famous-cognitive-psychology-experiments-that-failed-to-replicate
archive.org

The field of psychology had a big crisis in the 2010s, when many widely accepted results turned out to be much less solid than previously thought. It's called the replication crisis, because labs around the world tried and failed to replicate, in new experiments, previous results published by their original "discoverers". In other words, many reported psychological effects were either non-existent—artifacts of the experimenter's flawed setup—or so much weaker than originally claimed that they lost most of their intellectual sparkle.

(The crisis spanned other fields as well, but I mostly care about psychology here, especially the cognitive kind.)

This is very old news, and I've been vaguely aware of several of the biggest disgraced results for years, but I keep on forgetting which are (still probably) real and which aren't. This is not good. Most results in the field do actually replicate and are robust[citation needed], so it would be a pity to lose confidence in the whole field just because of a few bad apples.

This post is a compact reference list of the most (in)famous cognitive science results that failed to replicate and should, for the time being, be considered false. The only goal is to offset the trust-undermining effects of my poor memory—and perhaps yours, too?—with a bookmarkable page.

This can't be a comprehensive list: if a study is not on this page, it's not guaranteed to be fully replicated. Still, this should cover most of the high-profile debunked theories that laypeople like me may have heard of.

Credit: I enlisted the help of Kimi K2, o3, and Sonnet 4 to gather and fact-check this list. I also checked, pruned, and de-hallucinated all the results.

Ego Depletion Effect​

• Claimed result: We have a "willpower battery" that gradually depletes during the day as we exercise self-control. (I remember reading Baumeister's pop-science book and being awed by the implications of their findings; I might have known it sounded too good to be true.)
• Representative paper: Baumeister et al. 1998
• Replication status: did not replicate
• Source: Hagger et (63!) al. 2016

Power Posing Effect​

• Claimed result: Adopting expansive body postures for 2 minutes (like standing with hands on hips or arms raised) increases testosterone, decreases cortisol, and makes people feel more powerful and take more risks.
• Representative paper: Carney, Cuddy, & Yap (2010)
• Replication status: did not replicate
• Source: Ranehill et al. (2015)

Social Priming: Elderly Words Effect​

• Claimed result: People walk more slowly after being exposed to words related to elderly stereotypes.
• Representative paper: Bargh, Chen, & Burrows (1996)
• Replication status: did not replicate
• Source: Doyen et al. (2012) (I like how they prove that the psychological effect was actually in the experimenters, rather than the subjects!)

Money Priming Effect​

• Claimed result: Simply thinking about money makes you more selfish and more likely to endorse free market values.
• Representative paper: Vohs, Mead, & Goode (2006)
• Replication status: did not replicate
• Source: Rohrer, Pashler, & Harris (2015)

ESP Precognition Effect​

• Claimed result: In some cases, people can predict future events "that could not otherwise be anticipated through any known inferential process".
• Representative paper: Bem (2011)
• Replication status: did not replicate
• Source: Galak et al. (2012), Ritchie, Wiseman, & French (2012)

Cleanliness and Morality Effect​

• Claimed result: Being clean or thinking about cleanliness makes people more morally lax.
• Representative paper: Schnall, Benton, & Harvey (2008)
• Replication status: did not replicate
• Source: Johnson, Cheung, & Donnellan (2014)

Glucose and Ego Depletion Effect​

• Claimed result: Connected to the debunked ego-depletion effect, this one claims that adding glucose to your blood "recharges" the willpower battery. (For a while, I may have drunk more orange juice than usual after reading Baumeister's book. At least it's healthy-ish.)
• Representative paper: Gailliot & Baumeister (2007)
• Replication status: did not replicate
• Source: Lange & Eggert (2014)

Hunger and Risk-Taking Effect​

• Claimed result: People exposed to the scent of freshly baked cookies become less sensitive to risk and take more risks to obtain food.
• Representative paper: Ditto et al. 2006
• Replication status: did not replicate
• Source: Festjens, Bruyneel, & Dewitte (2018)

Psychological Distance & Construal Level Theory​

• Claimed result: "Psychologically distant" events are processed more abstractly, while "psychologically near" events are processed more concretely. E.g., you worry about the difficulty of a task if you have to do it tomorrow, but you see the same task's attractive side if it is planned far in the future.
• Representative paper: Trope & Liberman (2010), building on Liberman & Trope (1998)
• Replication status: serious credibility problems
• Source: A collaboration between 73 labs around the world is vetting this theory right now because of many doubts about its validity.

Ovulation & Mate Preferences Effect​

• Claimed result: Women are more attracted to hot guys during high-fertility days of their cycles.
• Representative paper: Gildersleeve, Haselton, & Fales (2014)
• Replication status: did not replicate
• Source: Stern, Gerlach, & Penke (2020)

Marshmallow Test & Long-Term Success Effect​

• Claimed result: Children's ability to resist eating a marshmallow when left alone in a room at age 4-5 strongly predicts adolescent achievement, with those who waited longer showing better life outcomes.
• Representative paper: Shoda, Mischel, & Peake (1990)
• Replication status: did not replicate significantly
• Source: Watts, Duncan, & Quan (2018)

Stereotype Threat (Women's Math Performance) Effect​

• Claimed result: Women risk being judged by the negative stereotype that women have weaker math ability, and this apprehension disrupts their math performance on difficult tests.
• Representative paper: Spencer, Steele, & Quinn (1999)
• Replication status: did not replicate
• Source: Flore & Wicherts (2015)

Smile to Feel Better Effect​

• Claimed result: Holding a pen in your teeth (forcing a smile-like expression) makes you rate cartoons as funnier compared to holding a pen with your lips (preventing smiling). More broadly, facial expressions can influence emotional experiences: "fake it till you make it."
• Representative paper: Strack, Martin, & Stepper (1988)
• Replication status: did not replicate
• Source: Wagenmakers et (54!) al. (2016)

Objective Measurement of Biases​

• Claimed result: You can predict if someone is racist by how quickly they answer certain trick questions.
• Representative paper: Greenwald, McGhee, & Schwartz (1998)
• Replication status: mixed evidence with small effects
• Source: Oswald et al. (2013) shows that the prediction power is small at best.

Mozart Effect​

• Claimed result: Listening to Mozart temporarily makes you smarter.
• Representative paper: Rauscher, Shaw, & Ky (1993)
• Replication status: did not replicate
• Source: Pietschnig, Voracek, & Formann (2010) (What a title!)

Growth Mindset Interventions​

• Claimed result: Teaching students that intelligence is malleable (not fixed) dramatically improves academic performance.
• Representative paper: Dweck, & Leggett (1988)
• Replication status: mixed results - many failed replications but also some successful replications
• Failed replication source: Li & Bates 2019
• Notable successful replication: Yeager et al. 2019 in Nature

Bilinguals Are Smarter​

• Claimed result: Being bilingual provides substantial cognitive advantages in attention, task-switching, and executive control.
• Representative paper: Bialystok, Craik, & Luk (2012)
• Replication status: did not replicate
• Source: Lehtonen et al. 2018

 

[Screamer_2]

Reading that, literally every single claimed result I can instantly go, "yeah, that's bullshit."

 

[Wraithdolf the White]

I dunno, I found that the longer a rape slave was kept in my home dungeon, the more cavalier and based I got about torturing them. But maybe a sample size of 63 isn't enough to produce meaningful results.

2025, Year of the Spoony! I knew you were coming back!

 

[Google.com]

The milgram experiment is still the only cool psychology experiment. But apparently subjecting your subjects to extreme emotional distress is "unethical" fucking liberals.

 

[Otterly]

If this many famous experiments in psychology cannot be replicated, I fear what other experiments cannot be replicated. Science prides itself on being repeatable with the same results. Open source, if you will.

Lots. Possibly most.
In an of itself that’s an issue, but what’s really killing science is the top down imposed dogma you can’t question. The TrustTheScience crowd have done an awful lot of damage to the concept of structured inquiry/challenge.
If a study isn’t replicated, and you try and can’t, then that’s a data point. If you’re not allowed to even challenge it you’ve got a big problem

 

[daffodils]

It's not just psychology...

https://www.cremieux.xyz/p/ranking-fields-by-p-value-suspiciousness

 

[Stabby_Pencil]

I'm surprised any of the gender studies aren't on this list, because I'm pretty sure very or none have replicated.

 

[urr13 account]

Wait until people find out therapy is completely fake and makes people worse.

 

[Gorilla Tessellator II]

wait until you found out how much of MEDICAL research is faked, outright fraud, or not replicable.

 

[NoIdeaWhatImDoing]

Wait until people find out therapy is completely fake and makes people worse.

all the people saying " therapy is great, i got much better" 90% of the time they are terrible people.

 

[Exterminate Leftists]

all the people saying " therapy is great, i got much better" 90% of the time they are terrible people.

Imagine how bad they used to be. Therapy only works if you can swallow your ego and learn better coping methods. Lifting weights is the same thing but less gay.

 

[Coldsteel the Hedgehog]

Talk therapy works for people who are just smart enough to realize something is wrong but not smart enough to be meaningfully introspective. The truth is that a lot of people just need someone who isn't objectively retarded to tell them why they're miserable who have the perceived authority such that they'll believe them and follow their advice.

The fact that certain types of therapy can literally cure BPD and PTSD proves to me that it certainly has its uses, but these uses are mostly relegated to specific subtypes of therapy that are geared towards equally specific things. It's very telling that the fields with the best outcomes are often ones with programs and finite enrollment lengths.

Anyways, it's always been obvious that social sciences are incredibly vulnerable to people being batshit fucking nutso. The problem is when people can't rub two brain cells together long enough to figure out not to trust something blindly just because there's a paper on it.

 

[NoIdeaWhatImDoing]

Imagine how bad they used to be. Therapy only works if you can swallow your ego and learn better coping methods. Lifting weights is the same thing but less gay.

most of the time they just learn therapy speak and use it to abuse and manipulate others.

The fact that certain types of therapy can literally cure BPD

 

[Spooky Dogwood]

I don't know why ego depletion effect is so hard to believe. Anyone who comes home after being at work all day should know what that feels like.

 

[mold]

The milgram experiment is still the only cool psychology experiment. But apparently subjecting your subjects to extreme emotional distress is "unethical" fucking liberals.

the strange situation is also good

 

[One Angry Computer]

I'm surprised any of the gender studies aren't on this list, because I'm pretty sure very or none have replicated.

That's because most, if not all of their tests subjects, cannot reproduce themselves.

 

[NoIdeaWhatImDoing]

I don't know why ego depletion effect is so hard to believe. Anyone who comes home after being at work all day should know what that feels like.

that's just being tired. Ego depletion is about self control, that you get tired by exercising self control.

 

[Coldsteel the Hedgehog]

Ver archivo adjunto 7955016

The problem with BPD isn't that it's impossible to manage: proper treatment has a pretty good success rate at reducing symptoms to sub-clinical levels. It's just that "proper treatment" involves intensive DBT programs that are difficult to access and cluster Bs are notorious for not accepting there's anything wrong with them to begin with.

It's like telling a miserable NEET that he'll stop being depressed if he quits smoking, gets a job, and starts working out. Yeah, it will, but good fucking luck actually getting them to do it.

 

[NoIdeaWhatImDoing]

The problem with BPD isn't that it's impossible to manage: proper treatment has a pretty good success rate at reducing symptoms to sub-clinical levels. It's just that "proper treatment" involves intensive DBT programs that are difficult to access and cluster Bs are notorious for not accepting there's anything wrong with them to begin with.

It's like telling a miserable NEET that he'll stop being depressed if he quits smoking, gets a job, and starts working out. Yeah, it will, but good fucking luck actually getting them to do it.

sure, you can manage it, you can't cure it though.

 

[Hey Johnny Bravo]

Smile to Feel Better Effect

This is the only one on this list that I've ever heard of. The rest sound like absolute gobbledegook.

If this many famous experiments in psychology cannot be replicated, I fear what other experiments cannot be replicated. Science prides itself on being repeatable with the same results. Open source, if you will.

It is less common in the hard sciences because lots of hard sciences rely on practical results. For example, there have been a few replication issues in pharmaceutical and cancer research - researchers or companies fudged their studies to make them look more successful. Most of these people are immediately caught because other researchers try to replicate the experiment to build on the research and advance it. If you cannot get the same result, you're pissed that you wasted your time and - more importantly - your resources. Lab experiments cost way more money. A lot of times someone just reads a bullshit hard science paper and goes "wait a minute, this doesn't make any sense" and the bullshit researchers are immediately discovered. The most appalling cases of academic chicanery happen when an already-established researcher lied and people basically assumed "well, that person's reputation is good, this can't be bullshit."

I'm not saying it doesn't happen and we should not be concerned but, by the nature of the beast, it's less common in the hard sciences because people do check. Universities are especially sensitive to this because paper retractions severely damage their reputation.

My understanding is that, in the social sciences, there is an attitude of "let sleeping dogs lie" because a ton of these bullshit papers are the basis of someone's PhD and not simply a paper. If you want to see another corpus of inane bullshit, go look at papers written by "educators" - not just what they churn out to get their education degrees, but the absolute nonsense they publish for their own self-masturbatory "academic" circles.

 

[TroonsDid911]

Objective Measurement of Biases​

• Claimed result: You can predict if someone is racist by how quickly they answer certain trick questions.
• Representative paper: Greenwald, McGhee, & Schwartz (1998)
• Replication status: mixed evidence with small effects
• Source: Oswald et al. (2013) shows that the prediction power is small at best.

"Alright subject 437, please answer what word you believe I am going to be spelling as quickly as it comes to you...ready?

N.
"