Over 300 scientists set out to repeat the successes recorded in about 100 past research experiments and published by top psychology journals just to see if they’d get the same results obtained by the original experimenters – but alas, they failed half of the time.
“Any one study is not going to be the last word,” said Brian Nosek, a psychology professor at the University of Virginia. He led the project.
Nosek disclosed that the inability of the repeat team to reproduce results obtained by past experimenters does not indicate that the studies were wrong, but only underscores why scientists would say that “more research is needed.”
“Each individual study has some evidence. It contributes some information toward a conclusion. But the real conclusion, when you can say confidently that something is true or false, is based on an accumulation of evidence over many studies,” Nosek said. “Even this project itself is not…a definitive word about reproducibility.”
This new study is published in the journal Science, and it details the efforts of over 300 researchers working with authors of original studies to reproduce their research conclusions. They focused on 100 studies published in 2008 by three major psychology journals – Psychological Science, the Journal of Personality and Social Psychology and the Journal of Experimental Psychology: Learning, Memory, and Cognition.
The repeat study researchers were only able to obtain the same past results 40% of the time.
Nosek explained there could be various reasons why they could not replicate results with experiments. The original study could be wrong, or the repeat experimenters overlooked something during the process by chance. Both studies could also be right, but with different conclusions due to the different manner of conducting the processes.
E.J. Masicampo of Wake Forest University in Winston-Salem, North Carolina, a co-author of the new study, said one of his own experiments was not confirmed by the project. He said research is still ongoing to determine why this is so.