The Reproducibility Project, as it is called, was not an attack by a group of crusading psychologists bent on discrediting the statistical significance of over half of the studies they attempted to reproduce, but a group of volunteers who worked closely with the original authors of each study in order to most accurately replicate the design and methodology of each study. Cognitive psychology is concerned with basic operations of the mind, and studies tend to look at areas such as perception, attention and memory.

Psychology professor Brian Nosek, coordinator of the study.

Each was the subject of a psychological study that researchers recently found to be either unreproducible or weaker in significance than originally believed.

Science is unique from other ways of gaining knowledge, Gampa said, because it relies on reproducibility to gain confidence in ideas and evidence. “Rather, credibility accumulates through independent replication and elaboration of the ideas and evidence”. As a result, a discipline of metascience is emerging, scientific research about scientific research. “What it takes to be a successful academic is not necessarily that well aligned with what it takes to be a good scientist”. “To thrive in science, researchers need to earn publications, and some kinds of results are easier to publish than others, particularly ones that are novel and show unexpected or exciting new directions”.

One of Nosek’s specialties is exploring the possibilities of reproducibility and replicability, issues that deal with whether findings from research can be independently verified.

In general, surprising effects were less reproducible, say the authors, as were effects for which it was more challenging to conduct the replication.

The replication might have failed, by chance, to detect the original result. The results come from a collaborative effort of 270 scientists working for The Reproducibility Project based at the Center for Open Science in Charlottesville, Virginia.

Nosek said scientists are also under pressure to publish their research regularly and in top journals, and the process can lead to a skewed picture.

In keeping with the goals of openness and reproducibility, each replication project team posted its methods and results on a public website.

[Marcus Munafo, a co-author on the study and professor of psychology at Bristol University] said that the problem of poor reproducibility is exacerbated by the way modern science works.