Fifty years ago today, President John F Kennedy, one of the most charismatic leaders of modern times, was gunned down by a lone gunman in Dallas, Texas. Or was he? The majority of people in the US and many people around the world doubt the "official story" and instead believe that JFK was, in fact, the victim of a conspiracy involving members of the mob. Or the CIA. Or communists. Or extreme right-wingers. Or the military-industrial complex. The list goes on.

The idea that JFK was not the victim of a lone deranged gunman has been referred to as the "mother of all conspiracy theories" although some might argue that this accolade should nowadays go to the claim that 9/11 was an inside job. Although these two conspiracies are probably the most widely held, a wide variety of other conspiracy theories are also endorsed by large proportions of the population. These include the idea that Princess Diana was murdered, that the Americans never actually landed on the moon, that the HIV/Aids virus was deliberately manufactured with genocidal intent, and that the world is, in fact, ruled by giant shape-shifting lizards. What is it about conspiracy theories that makes them so attractive to so many people? And why is it that some people are more drawn to such theories than others?

One possibility, of course, is that all of the above conspiracy theories are true and based upon a cool and rational analysis of available evidence. The only reason that some people do not believe in them might be because they have not looked at the evidence and instead, like mindless sheep, have simply accepted the lies of the establishment.

This seems highly unlikely. For one thing, in this internet age with instant access to rolling news, conspiracy theories arise almost instantly after any major event, often far too quickly for any evidence to have been properly gathered and analysed regarding the real cause of the event in question. Secondly, studies have shown that some people will, upon first encountering them, endorse entirely fictional conspiracies dreamt up by researchers with no supporting evidence whatsoever. Finally, those with a strong inclination towards conspiratorial thinking will even endorse mutually contradictory conspiracy theories. Thus, those who believe that Princess Diana was murdered by the British secret service are also more likely to believe that she faked her own death; those who believe that Osama bin Laden was in fact already dead at the time that the Americans claim to have killed him are also more likely to believe that he is still alive. The explanation is, of course, that those with a conspiratorial mindset are not exactly sure what really happened with respect to these dramatic events. The only thing they are sure of is that the "official story" is not true.

Surprisingly, with a few notable exceptions, it is only recently that psychologists have turned their attention towards this fascinating topic but there are signs this has changed in recent years, with an increasing number of relevant publications and publication of special issues of journals and blogs devoted exclusively to this topic. Some interesting findings have already emerged.

For example, it appears that one of the underlying cognitive factors that inclines us towards belief in conspiracy theories is our intuitive notion that big events must have big causes. This has been referred to as the proportionality bias. Thus we find it hard to accept that JFK was the victim of a lone deranged gunman or that Princess Diana was the victim of a drunk driver. We prefer the idea that such major events must have major causes – such as complex conspiracies by groups of powerful individuals.

Other research has pointed to the relevance of the psychological mechanism of projection. It appears that some people are more inclined to believe in conspiracies than others because those people would themselves feel inclined to engage in conspiracies in similar contexts. The reasoning goes like this: "If I were in that situation, I would probably engage in conspiratorial behaviour. That implies that most people would do likewise. Therefore, this conspiracy probably really did take place."

Another factor that plays an important role is confirmation bias, probably the most ubiquitous cognitive bias of them all. We all have a tendency to pay more attention to evidence that supports what we already believe or want to be true and to ignore, neglect or dismiss evidence that contradicts of favoured beliefs. In one study, participants were presented with information relating to the JFK assassination. Some of the information was more consistent with the lone gunman explanation, some was more consistent with the conspiracy-based explanation. All participants found the information that was consistent with their pre-existing views more convincing than information that was inconsistent with those views. Thus the presentation of the same information had a polarising effect: those who already believed in the conspiracy theory believed in it even more strongly; those who already rejected the conspiracy theory, rejected it even more strongly.

In real life, information about major events is often complex and includes errors and anomalies, meaning that such contradictory interpretations are all too easy. One of our greatest strengths as a species is our ability to find patterns and connections in complex data and to perceive cause and effect relationships between events. The problem is that we sometimes see patterns when they are not really there and deduce cause and effect relationships where they do not exist.

Not surprisingly perhaps, those who believe strongly in conspiracies tend to show higher levels of anomie and paranoia. It is not just that they do not trust governments and official institutions. They are also less likely to trust their next-door neighbours and their colleagues at work. They tend to feel powerless, the victims of uncontrollable external forces, and it is arguably the case that belief in conspiracies gives them at least the illusion of control. Identifying at out-group as the cause of one's misfortune – be it the government, the CIA, the Illuminati or whatever – means there is at least the possibility that this enemy can be defeated. It is also undeniably satisfying to believe that one is in possession of secret knowledge about the way things really are that others are either too stupid or ignorant to believe.

It should be noted that none of the psychologists engaged in this line of research would for one minute maintain that all conspiracy theories are false. A tiny minority of such theories do turn out to be true. The focus of this research is not upon whether any specific conspiracy theory is true or false but instead upon trying to understand the factors that make such theories so easy to believe for so many people. A second focus is upon why some people are more drawn to belief in conspiracies, even entirely fictional or mutually contradictory ones, than others.

In the final analysis though, does belief in conspiracy theories do any real damage? Isn't it the case that such beliefs are just a bit of harmless fun? Although that may be true of belief in most conspiracy theories, the evidence shows conclusively that belief in conspiracies can cause serious harm. For example, it is estimated that unfounded conspiracy-based beliefs in the alleged dangers of treatment for Aids has cost hundreds of thousands of lives. It has also been shown that terrorist groups use conspiracy theories to recruit and motivate their followers. Finally, belief in conspiracy theories is associated with disengagement from democratic processes making believers less likely to vote, for example.

In a democracy, it is essential that citizens examine and question the official version of events if we are to hold our leaders to account when it is appropriate to do so. But proper scepticism does not entail the rejection of all official versions of events but careful rational analysis using critical thinking skills to maximum effect. The assumption that all information from official sources is untrue is a dangerous road to go down.

• Chris French is a professor of psychology at Goldsmiths, University of London, where he heads the Anomalistic Psychology Research Unit. On Twitter he is @chriscfrench