KU’s Robert Ryan participates in unprecedented international psychology project



Associate Professor Dr. Robert Ryan, along with 269 co-authors from around the world, published their findings in Science Magazine following their investigation into the reproducibility of psychological science. Launched nearly four years ago, the Reproducibility Project: Psychology has produced the most comprehensive investigation ever completed about the rate and predictors of reproducibility in a field of science.

The project conducted replications of 100 published findings of three prominent psychology journals, sticking to strict protocol in their reproduction attempts. They found that regardless of the analytic method or criteria used, fewer than half of their replications produced the same findings as the original study. The team suspects these results may have been due to small differences in when, where or how the replication was carried out, simple chance, or that the original result might have been a false positive.

Ryan says that remedies to reduce these false positives are already in progress, including recognizing the importance of open access to all aspects of studies, pre-registering all aspects of studies before collecting data, designing studies with high statistical power, journals being more willing to publish null result studies and continuing periodic reproduction studies.

His involvement in the project started with a simple email from the Center for Open Science, which Ryan subscribes to. Though he had hoped to participate, he suspected that professors at other major research universities would have better resources and more time to conduct the studies required of the project. He later received an email from a member of the project, Jeff Galak of Carnegie Mellon University, asking for help with some statistical analyses, which he provided, and ultimately ended up becoming involved in this historic project.

Since the Reproducibility Project began in 2011, similar projects have emerged in other fields such as the Reproducibility Project: Cancer Biology. As a result, a discipline of metascience is emerging, scientific research about scientific research. These, and the widespread efforts to improve research transparency and reproducibility, indicate the push in scientific fields to be constantly self-examining in order to produce the most efficient and top-quality results possible.

For more information on this project and for the full list of contributors, visit https://osf.io/ezcuj.

Leave a Reply