How the "data whisperer" uncovered a psychology scandal

How the data whisperer uncovered a psychology scandalS

What happens when scientists lie about the results of their work? Often, they never get caught. But now, a new breed of scientific whistleblowers are trying to keep their colleagues honest, sometimes working for years to show that data has been fudged. Here are their stories.

Over at Nature, there's a great piece that profiles three well-known whistleblowers in the scientific community, who range from mild-mannered data analysts to anonymous crusaders.

Here's the way the first profile begins:

Uri Simonsohn sees himself as more of a data-whisperer than a whistle-blower. His day job as a social scientist at the University of Pennsylvania in Philadelphia involves scouring archival data — from house prices to auction records to college admissions — as part of his research into judgement and decision-making. He suspects that this background has predisposed him to catching spurious patterns in other psychologists' results. "With an experiment, you do a t-test and move on," he says. "But people who work with archival data are used to looking at data very carefully."

It was this intuition that stirred when he first came across papers by Dirk Smeesters at Erasmus University Rotterdam in the Netherlands and Lawrence Sanna at the University of Michigan in Ann Arbor in the summer of 2011. In both cases, the data seemed too good to be true, containing an overabundance of large effects and statistically significant results. In one of Sanna's papers, Simonsohn noticed that one experiment — in which volunteers were supposedly split into different groups — produced results with uncannily similar standard deviations. In the results of Smeesters' studies, he saw a suspiciously low frequency of round numbers and an unusual similarity between many of the averages. "If there's too little noise, and the data are too reliable again and again, they cannot be real," he says. "Real data are supposed to have error."

Head over to Nature to read the rest, and to find out about the other whisteblowers who are changing the way science gets done.

Leave a Reply