“Extraordinary claims require extraordinary evidence” is a fascinating mantra. It is at once entirely obvious, subtly brilliant, and yet can also serve as fodder for maintaining the status quo.
Today, these salient words are most often associated with Carl Sagan’s memorable delivery on the TV series Cosmos, but the idea isn’t new. In his 1900 work ‘From India to the Planet Mars’, Théodore Flournay said it quite elegantly: “The weight of the evidence should be proportioned to the strangeness of the facts.” This was itself a shortened version of a longer quote from Pierre-Simon Laplace, a late 18th/early 19th century French mathematician and astronomer.1 Sagan himself only slightly reworded the quote from his colleague and fellow CSICOP (Committee for the Scientific Investigation of Claims of the Paranormal) founder Marcello Truzzi, who said “An extraordinary claim requires extraordinary proof.”2 Truzzi is a fascinating character who will come back up later in the article.
If I told you a red car was parked in front of your house, you would likely nod and accept this as a fact without much further investigation. However, if instead I claimed that a hyperdimensional stagecoach carrying the entire pantheon of Greek gods had just landed on your front lawn, you might decide to run to the window and check before believing me. It’s common sense. The weirder the claim, the more evidence we look for. In our everyday lives, this is an intelligent way to discern how much energy we need to spend verifying whether something is likely true.
The fact that it is such an obvious part of everyday life makes it more difficult to see why, as a method for judging or filtering scientific claims, this rule of thumb falls short. Scientific inquiry isn’t our everyday lives, and as such, it has different (higher) standards. It involves laying down biases and approaching each question with an open minded and enthusiastic yearning to be educated by whatever reality tells us. Incorporating the subjective value judgement of what is and isn’t ‘extraordinary’ introduces biases and beliefs into an equation where they have no place.
The Problem of Subjectivity
Are claims of ghosts and after-life communication extraordinary? Given the state of current scientific consensus, they certainly are, but to many people in many cultures around the world and throughout time, they wouldn’t be at all extraordinary. Even today, a third of Americans believe in ghosts while nearly three-quarters believe in some kind of paranormal phenomena,3 all of which would be labelled ‘extraordinary’ by the majority of the academic community. Many believe that with prevalence numbers this high, these paranormal claims couldn’t be considered anything but ordinary, but it all depends on what your criteria is. Of course, this says nothing at all about the reality behind these phenomena, which is exactly my point, but it does give good reason to invest in some serious research. As an aside, it’s worth noting that the majority of today’s academic modern after-life communication researchers report positive results.4
Parapsychology, the study of these ostensibly paranormal effects (like telepathy, mediumship, and pre-cognition), will serve as a useful frame of reference as we move forward.
The luxury of using a measure as arbitrary as ‘extraordinary’ is that it allows for biased parties to move the goal posts at will. In theory, any evidence that is presented for any claimed deemed extraordinary could be written off as not extraordinary enough, and what might meet one persons criteria of extraordinary might not meet another persons.
It has been proposed that Bayes’ theorem, a mathematical framework which can be used for determining the probability of a given hypothesis, could be a way to quantify the rule.5 Bayesian inference, as opposed to the standard (frequentist) model of determining statistical probability, has the ability to take into account a wider array of variables and knowledge (and beliefs) about prior probabilities.
The comic on the right depicts a humorous scenario exemplifying how a Bayesian approach can, in many circumstances, be superior to simple frequentist inference. Since we know that the prior probability of the sun going supernova at any given moment is tiny, we can incorporate that information into our analysis, rather than determining our probability based only on the fact that the chance of rolling two two sixes in a row is fairy unlikely. If the machine were set up to detect something less extraordinary – say, whether or not it’s raining in Vancouver – that would affect the Bayesian analysis and we would be more likely to trust the machine.
Bayesian inference works well when we know a lot about a situation, like how often stars go supernova. But what is the prior probability of whether or not telepathy exists? In a field in which a genuine scientific consensus has not been reached, is it even possible to answer that with any degree of accuracy without letting our subjective opinions and biases interfere with our assessment? It’s hard to see how.
In the end, it seems that while Bayesian analysis can useful in many circumstances, it has also the potential to be used as a pseudo-mathematical method of subjectively defining what is and isn’t extraordinary through he manipulation of the prior probability input. Dean Radin, a prominent parapsychology researcher, has commented on the misuse of Bayesian analysis as a method of dismissing the evidence for psi (parapsychological effects):
“Skeptics jump on Bayesian methods as a convenient way to explain away psi effects, because they can conveniently choose priors that do not allow any new data to shift their initial position. In this way Bayesian methods mathematically prove why orthodoxy persists until the holders of the status quo die. […]we can calculate that as certain people die (the current more visible holders of the status quo), that the Bayesian likelihood that psi is real will progressively increase. And the beauty is that this will happen without any of us having to generate any new data at all.”6
Rather than delving into the subjectively muddy waters of Bayesian prior probabilities, most parapsychology researchers will attempt to control the experiment as well as possible to rule out any conventional explanation, and then simply determine the probability that the anomalous evidence could be the result of chance alone. This isn’t perfect, as it always leaves the door open to the data being skewed by some unprepared for, uncontrolled for alternate hypothesis – but that’s true of a Bayesian approach as well – which is why parapsychology researchers tend to be incredibly stringent when it comes to laboratory experimental controls.
As research progresses, more and more physically-based hypotheses are controlled for, and yet these parapsychological effects, while seemingly quite small, remain stubbornly present. Extraordinary or not, eventually the data begins to speak for itself.
The Double Standard
The science is in. According to standard scientific and statistical measures, ESP effects (not to say anything of their cause) have been demonstrated to happen in a laboratory setting (selected references).7 8 9 10 11 The interesting thing is that many of the skeptics agree! Prominent skeptic and sometimes parapsychology researcher Richard Wiseman said in an interview with the Daily Mail:
“I agree that by the standards of any other area of science that remote viewing is proven, but begs the question: do we need higher standards of evidence when we study the paranormal? I think we do. (…) if I said that a UFO had just landed, you’d probably want a lot more evidence. Because remote viewing is such an outlandish claim that will revolutionize [sic] the world, we need overwhelming evidence before we draw any conclusions.”12
Wiseman later clarified that he was not only talking about remote viewing, but ESP in general.13 Chris French, editor-in-chief of The Skeptic magazine in the UK agrees as well:
“I would probably actually agree with Richard… I’m at pretty much the same point myself. I actually first of all put the question, is there a double standard in science when it comes to parapsychology? And I say actually, yes, there is. Then I go on to say well, I think there should be. Which I think is just really the same point that Richard’s making here.”14
Whatever one’s opinion is on the paranormal research, the bold suggestion that a double standard not only does, but should exist, ought to send up serious red flags for any thoughtful science minded person. The data says what it says, and when the data becomes strong enough that we must admit it meets the standards of science – even if we don’t like it – the progress of scientific information implores us to change our minds rather than trying to change the standards. It’s hard to imagine how two prominent skeptical and scientific researchers can endorse such a blatantly anti-scientific position.
Michael Shermer, editor-in-chief at the US-Based Skeptic magazine has a different take on the issue:
“Well the standards that parapsychology is held to – and should be held to – are no different from than anything else, than any other claim in science- any science. That’s just the way science is done. Although there is one provisor that ‘Extraordinary claims require extraordinary evidence’. It’s not that the standards have changed, it’s just that if the standards are sloppy or some results are let through, the more extraordinary the claim the more people are gonna notice that.15
This is a much more reasonable take on the extraordinary claims adage. New, upcoming, and groundbreaking areas of science should have their research finely and closely scrutinized. If this is done and the data is found to be conclusive according to ‘standards of any other area of science’, as Wiseman and French admit, and parapsychology should not be held to any other standard, as Shermer correctly points out – then we are left with no choice but to accept the data and to find a model which incorporates, rather than ignores it.
Now, I’m not saying that ESP effects are definitely real in a paranormal sense, I think the evidence that they happen is pretty strong, but why they happen is unknown. I am, however, saying that the dismissive attitude many scientifically minded people have is not grounded in the evidence, and is therefore unscientific. Wiseman, French, and many others do science a disservice by claiming that double standards are desirable. Hesitantly, one might conclude that these skeptics seem to be more interested in maintaining a materialistic worldview, regardless of the statistics and evidence, than they are in following the data through to its natural conclusions and arriving at the truth.
Marcello Truzzi, the prominent skeptic who helped to found CSICOP, and from whom Carl Sagan borrowed the titular phrase, himself recognized these problems later in his career. He popularized the term pseudoskeptic as a way to describe reputed skeptics who made negative claims, without meeting the burden of proof for those claims.16
This led him to leave and to offer public criticism toward CSICOP, stating:
“They tend to block honest inquiry, in my opinion. Most of them are not agnostic toward claims of the paranormal; they are out to knock them. […] When an experiment of the paranormal meets their requirements, then they move the goal posts. Then, if the experiment is reputable, they say it’s a mere anomaly.”17
I am not here to promote parapsychology. I do not have strong opinions about the topic either way.
Science is always all about extraordinary claims. When you are making any scientific claim, you are saying that your claim is the very best explanation possible based on all of the given evidence and data related to that claim. On any topic, in any field, that kind of claim is extraordinary.
Science is a method for finding closer and closer approximations of truth and reality based on the evidence. Such brushes with truth are extraordinary by definition and require extraordinary evidence. Luckily science already treats every claim as if it were extraordinary, and it requires extraordinary evidence for every claim. This is why science and scientists place such a high value on publishing and reproducing findings, on peer review, and on strict experimental controls. Any evidence that can survive the brunt of these control measures is extraordinary indeed.
Unfortunately, the phrase about extraordinary evidence is thrown around as if it were a shield that could protect one from having to consider any information that might upset their current worldview. Evidence standards become artificially high for claims these people don’t like, and artificially low for the claims they do like. Even if it isn’t the intent, the function of such a rule becomes the maintenance of the status quo. Those who are looked up to and who’s opinions are thought highly of tend to be the ones whose influence determines what is and isn’t extraordinary. Scientific consensus devolves into an ideological popularity contest.
Max Planck, the father of quantum mechanics, is often famously quoted as saying that science progresses one funeral at a time.18 This outlook, while certainly cynical, rings true to something I think we can all recognize about ourselves: people don’t like to be wrong. Especially when they perceive that being wrong could threaten their identity.
Science is supposed to be the one arena that cuts through all of that baggage, but it is to our peril if we ever believe that we can be completely, or even almost completely, free of it. By avoiding codifying a rule that directly feed into these inherent human weaknesses, we can come closer to, but likely never achieve, the ideal of an unbiased worldview. It seems that perhaps the best we can do is start by attempting to be honest about our biases, and letting the conversation progress from there.
- Théodore Flournay, 1900, From India to the Planet Mars: A Study of a Case of Somnambulism. pg 344–345 (back)
- Marcello Truzzi, 1978, “On the Extraordinary: An Attempt at Clarification”. Zetetic Scholar Issue 1 (back)
- Gallup Poll 2005, Three in Four Americans Believe in Paranormal (back)
- The Reincarnation of Mediumship Research (back)
- Rational Wiki, 2013, Extraordinary claims require extraordinary evidence (back)
- On the (mis)use of Bayesian methods by skeptics (back)
- M. Schlitz & D. Radin, 2002. “Telepathy in the ganzfeld: State of the evidence,” Healing, Intention and Energy Medicine (London: Harcourt Health Sciences) (back)
- J.B. Rhine and J.G. Pratt, 1954. A Review of the Pearce-Pratt Distance Series of ESP Tests (back)
- C. Honorton; S. Krippner, 1969. “Hypnosis and ESP performance: A review of the experimental literature“, Journal of the Society for Psychical Research, 63(3), 214-252 (back)
- R. Sheldrake & P. Smart, 2003. “Experimental Tests For Telephone Telepathy“, Journal of the Society for Psychical Research 67, 184-199 (back)
- Puthoff & Targ , 1976. A perceptual channel for information transfer over kilometer distance: Historical perspective and recent research, Proceedings of the IEEE 64(3), 329-353 (back)
- Daily Mail, 2008, Could there be proof to the theory that we’re ALL psychic? (back)
- PodBlack Cat, 2009, Dr Richard Wiseman On Remote Viewing In The Daily Mail – Clarification (back)
- Skeptiko Podcast, 2009, 83. Dr. Chris French, Extraordinary Psi Claims (back)
- Skeptiko Podcast, 2007, 3. Dr. Michael Shermer on Darwin, Evolution and Creativity (back)
- M. Truzzi, 1987, “On Pseudo-Skepticism”.Zetetic Scholar (12/13) (back)
- Parapsychology, Anomalies, Science, Skepticism, and CSICOP – compiled by Daniel H. Caldwell (back)
- This is a popular paraphrase of the actual quote. More info here (back)