This week businessman James McCormick was convicted of fraud after making £50m selling fake bomb detectors to security forces in Iraq and many other countries around the world. The detectors were said to work in a similar way to dowsing rods and were claimed to detect explosives up to one kilometre below the ground. Even more incredibly, they could apparently be used to locate drugs, people, elephants – even $100 bills. They didn't work and, in all probability, hundreds of lives were lost as a result of misplaced trust in the phony devices.

In February, a similar device was reported to be capable of detecting liver disease in patients sitting several feet away. The claims were endorsed by Dr Gamal Shiha, "one of Egypt's most respected liver specialists, and one of the device's developers". Sceptics were quick to point out that the claims were not based on peer-reviewed research and that there was no plausible mechanism by which such a device could work. As Síle Lane of Sense About Science put it, such miraculous devices offer "hope and nothing more".

In the mid-1970s, a teacher in Australia claimed to have found a way to allow patients with severe autism, cerebral palsy and other disorders to communicate with the outside world. Known as facilitated communication, the technique relied upon another person, the facilitator, steadying the impaired individual's hands just enough to allow them to type on a keyboard. The technique was introduced into the USA by Douglas Biklen, a sociologist, and soon became an international phenomenon.

From the start, though, many thought the results were too good to be true. For example, many of those who appeared to be able to communicate this way had never had any training in the use of written language – and sometimes they seemed not even to be looking at the keyboard. Sadly, as properly controlled tests subsequently showed, the messages were not coming from the patients at all. They were coming, unwittingly, from the facilitators.

What do these stories have in common, apart from the fact that they are promoting techniques that verge on the miraculous and that, it turns out, do not work? They are all based upon the ideomotor effect, a phenomenon well known to psychologists whereby suggestions, beliefs or expectations cause unconscious muscular movements.

The ideomotor effect also explains a number of other ostensibly paranormal phenomena. For example, "table tilting" – an initially American craze that caught on in Victorian Britain at the height of the popularity of séances. Guests would sit around a table resting their hands upon its upper surface. After a while, the table would move, apparently of its own volition. The movements might be slight jerks, but in a successful session, sitters would find themselves chasing around the room trying to keep up with the table.

Naturally, all of the sitters would deny that they were simply pushing the table. It was claimed that the movements could be used to communicate with the spirit world, and indeed they were generally thought to be brought about by the actions of spirits.

This phenomenon has a special place in the history of "anomalistic psychology" in so far as it attracted the attention of Michael Faraday, the famous English physicist. He carried out a series of ingenious experimental investigations which established that, despite the protestations of the sitters, it was in fact unintentional muscular movements causing the table to move. This was one of the first systematic studies of the ideomotor effect.

The ideomotor effect is also behind the supposed power of Ouija boards to communicate with the dead. Sometimes referred to as 'spirit boards', Ouija boards typically consist of a round board marked with all the letters of the alphabet, the digits one to nine, and the words "yes" and "no". Sitters place their fingers lightly on a specially constructed heart-shaped piece of wood known as a planchette and proceed to address questions to the spirit world. The technique also works simply by using letters and numbers written on pieces of paper and arranged in a circle on a smooth table, along with an upturned wine glass in place of a planchette.

Amazingly, in response to questions, the planchette (or wine glass) often appears to move around, pointing to various letters and numbers to relay the responses back from the spirits. Once again, we are dealing with an example of the ideomotor effect. Although the illusion that the pointer is being moved by some outside force is extremely strong, the truth is that the sitters are actually moving it without realising it.

Dowsing is a centuries-old technique allegedly for locating water, gold, or indeed pretty much anything. Dowsers claim that by holding Y-shaped twigs, pendulums or L-shaped dowsing rods, the chosen device will respond, apparently of its own accord, in the vicinity of the target substance. Indeed, some dowsers claim that they do not even need to be physically near the target substance and can instead dowse by holding a pendulum over a map of the target area.

Readers will not be surprised to learn that dowsing does not work when it is tested under properly controlled conditions that rule out the use of other cues to indicate target location.

Whether the device is a Ouija board, a divining rod or a bomb detector, the ideomotor effect is capable of producing powerful illusions that can be exploited by the unscrupulous. Those whom they fool are usually well-intentioned, often highly intelligent individuals. But the demonstrations used to convince them of the claims are never carried out under properly controlled conditions. If anomalistic psychology shows us anything, it is that intelligence and good intentions are no protection against self-deception. The only way to avoid being taken in by such effects is through the use of properly controlled, double-blind tests.

It's a simple lesson, but one we ignore at our peril.

Chris French is a professor of psychology at Goldsmiths, University of London, where he heads the Anomalistic Psychology Research Unit. On Twitter he is @chriscfrench

Anomalistic Psychology: Exploring Paranormal Belief and Experience by Chris French and Anna Stone will be published in November 2013. This is an edited extract, reproduced with the permission of the publishers Palgrave Macmillan