Do Extraordinary Claims Require Extraordinary Evidence?

“Extraordinary claims require extraordinary evidence” is a fascinating mantra. It is at once entirely obvious, subtly brilliant, and yet can also serve as fodder for maintaining the status quo.

Marcello Truzzi. Photo by Steve Kuzma.  Reproduced with permission.

Marcello Truzzi. Photo by Steve Kuzma.
Reproduced with permission.

Today, these salient words are most often associated with Carl Sagan’s memorable delivery on the TV series Cosmos, but the idea isn’t new. In his 1900 work ‘From India to the Planet Mars’, Théodore Flournay said it quite elegantly: “The weight of the evidence should be proportioned to the strangeness of the facts.” This was itself a shortened version of a longer quote from Pierre-Simon Laplace, a late 18th/early 19th century French mathematician and astronomer.1 Sagan himself only slightly reworded the quote from his colleague and fellow CSICOP (Committee for the Scientific Investigation of Claims of the Paranormal) founder Marcello Truzzi, who said “An extraordinary claim requires extraordinary proof.”2 Truzzi is a fascinating character who will come back up later in the article.

If I told you a red car was parked in front of your house, you would likely nod and accept this as a fact without much further investigation. However, if instead I claimed that a hyperdimensional stagecoach carrying the entire pantheon of Greek gods had just landed on your front lawn, you might decide to run to the window and check before believing me. It’s common sense. The weirder the claim, the more evidence we look for. In our everyday lives, this is an intelligent way to discern how much energy we need to spend verifying whether something is likely true.

The fact that it is such an obvious part of everyday life makes it more difficult to see why, as a method for judging or filtering scientific claims, this rule of thumb falls short.  Scientific inquiry isn’t our everyday lives, and as such, it has different (higher) standards. It involves laying down biases and approaching each question with an open minded and enthusiastic yearning to be educated by whatever reality tells us. Incorporating the subjective value judgement of what is and isn’t ‘extraordinary’ introduces biases and beliefs into an equation where they have no place.

The Problem of Subjectivity

Are claims of ghosts and after-life communication extraordinary? Given the state of current scientific consensus, they certainly are, but to many people in many cultures around the world and throughout time, they wouldn’t be at all extraordinary. Even today, a third of Americans believe in ghosts while nearly three-quarters believe in some kind of paranormal phenomena,3 all of which would be labelled ‘extraordinary’ by the majority of the academic community. Many believe that with prevalence numbers this high, these paranormal claims couldn’t be considered anything but ordinary, but it all depends on what your criteria is. Of course, this says nothing at all about the reality behind these phenomena, which is exactly my point, but it does give good reason to invest in some serious research. As an aside, it’s worth noting that the majority of today’s academic modern after-life communication researchers report positive results.4

Parapsychology, the study of these ostensibly paranormal effects (like telepathy, mediumship, and pre-cognition), will serve as a useful frame of reference as we move forward.

The luxury of using a measure as arbitrary as ‘extraordinary’ is that it allows for biased parties to move the goal posts at will. In theory, any evidence that is presented for any claimed deemed extraordinary could be written off as not extraordinary enough, and what might meet one persons criteria of extraordinary might not meet another persons.


Frequentists vs Bayesians –  xkcd webcomic.
License: Attribution-NonCommercial 2.5 Generic (CC BY-NC 2.5)

It has been proposed that Bayes’ theorem, a mathematical framework which can be used for determining the probability of a given hypothesis, could be a way to quantify the rule.5 Bayesian inference, as opposed to the standard (frequentist) model of determining statistical probability, has the ability to take into account a wider array of variables and knowledge (and beliefs) about prior probabilities.

The comic on the right depicts a humorous scenario exemplifying how a Bayesian approach can, in many circumstances, be superior to simple frequentist inference. Since we know that the prior probability of the sun going supernova at any given moment is tiny, we can incorporate that information into our analysis, rather than determining our probability based only on the fact that the chance of rolling two two sixes in a row is fairy unlikely. If the machine were set up to detect something less extraordinary – say, whether or not it’s raining in Vancouver – that would affect the Bayesian analysis and we would be more likely to trust the machine.

Bayesian inference works well when we know a lot about a situation, like how often stars go supernova. But what is the prior probability of whether or not telepathy exists? In a field in which a genuine scientific consensus has not been reached, is it even possible to answer that with any degree of accuracy without letting our subjective opinions and biases interfere with our assessment? It’s hard to see how.

In the end, it seems that while Bayesian analysis can useful in many circumstances, it has also the potential to be used as a pseudo-mathematical method of subjectively defining what is and isn’t extraordinary through he manipulation of the prior probability input. Dean Radin, a prominent parapsychology researcher, has commented on the misuse of Bayesian analysis as a method of dismissing the evidence for psi (parapsychological effects):

“Skeptics jump on Bayesian methods as a convenient way to explain away psi effects, because they can conveniently choose priors that do not allow any new data to shift their initial position. In this way Bayesian methods mathematically prove why orthodoxy persists until the holders of the status quo die. […]we can calculate that as certain people die (the current more visible holders of the status quo), that the Bayesian likelihood that psi is real will progressively increase. And the beauty is that this will happen without any of us having to generate any new data at all.”6

Rather than delving into the subjectively muddy waters of Bayesian prior probabilities, most parapsychology researchers will attempt to control the experiment as well as possible to rule out any conventional explanation, and then simply determine the probability that the anomalous evidence could be the result of chance alone. This isn’t perfect, as it always leaves the door open to the data being skewed by some unprepared for, uncontrolled for alternate hypothesis – but that’s true of a Bayesian approach as well – which is why parapsychology researchers tend to be incredibly stringent when it comes to laboratory experimental controls.

As research progresses, more and more physically-based hypotheses are controlled for, and yet these parapsychological effects, while seemingly quite small, remain stubbornly present.  Extraordinary or not, eventually the data begins to speak for itself.

The Double Standard

The science is in. According to standard scientific and statistical measures, ESP effects (not to say anything of their cause)  have been demonstrated to happen in a laboratory setting (selected references).7 8 9 10 11 The interesting thing is that many of the skeptics agree! Prominent skeptic and sometimes parapsychology researcher Richard Wiseman said in an interview with the Daily Mail:

“I agree that by the standards of any other area of science that remote viewing is proven, but begs the question: do we need higher standards of evidence when we study the paranormal? I think we do. (…) if I said that a UFO had just landed, you’d probably want a lot more evidence. Because remote viewing is such an outlandish claim that will revolutionize [sic] the world, we need overwhelming evidence before we draw any conclusions.”12

Wiseman later clarified that he was not only talking about remote viewing, but ESP in general.13 Chris French, editor-in-chief of The Skeptic magazine in the UK agrees as well:

“I would probably actually agree with Richard… I’m at pretty much the same point myself. I actually first of all put the question, is there a double standard in science when it comes to parapsychology? And I say actually, yes, there is. Then I go on to say well, I think there should be. Which I think is just really the same point that Richard’s making here.”14

Whatever one’s opinion is on the paranormal research, the bold suggestion that a double standard not only does, but should exist, ought to send up serious red flags for any thoughtful science minded person. The data says what it says, and when the data becomes strong enough that we must admit it meets the standards of science – even if we don’t like it – the progress of scientific information implores us to change our minds rather than trying to change the standards. It’s hard to imagine how two prominent skeptical and scientific researchers can endorse such a blatantly anti-scientific position.

Michael Shermer, editor-in-chief at the US-Based Skeptic magazine has a different take on the issue:

“Well the standards that parapsychology is held to – and should be held to – are no different from than anything else, than any other claim in science- any science. That’s just the way science is done. Although there is one provisor that ‘Extraordinary claims require extraordinary evidence’. It’s not that the standards have changed, it’s just that if the standards are sloppy or some results are let through, the more extraordinary the claim the more people are gonna notice that.15

Photo of Storefront Psychic fortuneteller in Downtown Crossing, Boston

Photo of Storefront Psychic. By user Boston on Wikipedia.
License: Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)

This is a much more reasonable take on the extraordinary claims adage. New, upcoming, and groundbreaking areas of science should have their research finely and closely scrutinized. If this is done and the data is found to be conclusive according to ‘standards of any other area of science’, as Wiseman and French admit, and parapsychology should not be held to any other standard, as Shermer correctly points out – then we are left with no choice but to accept the data and to find a model which incorporates, rather than ignores it.

Now, I’m not saying that ESP effects are definitely real in a paranormal sense, I think the evidence that they happen is pretty strong, but why they happen is unknown. I am, however, saying that the dismissive attitude many scientifically minded people have is not grounded in the evidence, and is therefore unscientific. Wiseman, French, and many others do science a disservice by claiming that double standards are desirable. Hesitantly, one might conclude that these skeptics seem to be more interested in maintaining a materialistic worldview, regardless of the statistics and evidence, than they are in following the data through to its natural conclusions and arriving at the truth.

Marcello Truzzi, the prominent skeptic who helped to found CSICOP, and from whom Carl Sagan borrowed the titular phrase, himself recognized these problems later in his career. He popularized the term pseudoskeptic as a way to describe reputed skeptics who made negative claims, without meeting the burden of proof for those claims.16

This led him to leave and to offer public criticism toward CSICOP, stating:

“They tend to block honest inquiry, in my opinion. Most of them are not agnostic toward claims of the paranormal; they are out to knock them. […] When an experiment of the paranormal meets their requirements, then they move the goal posts. Then, if the experiment is reputable, they say it’s a mere anomaly.”17

I am not here to promote parapsychology. I do not have strong opinions about the topic either way.


Science is always all about extraordinary claims. When you are making any scientific claim, you are saying that your claim is the very best explanation possible based on all of the given evidence and data related to that claim. On any topic, in any field, that kind of claim is extraordinary.

Science is a method for finding closer and closer approximations of truth and reality based on the evidence. Such brushes with truth are extraordinary by definition and require extraordinary evidence. Luckily science already treats every claim as if it were extraordinary, and it requires extraordinary evidence for every claim. This is why science and scientists place such a high value on publishing and reproducing findings, on peer review, and on strict experimental controls. Any evidence that can survive the brunt of these control measures is extraordinary indeed.

Unfortunately, the phrase about extraordinary evidence is thrown around as if it were a shield that could protect one from having to consider any information that might upset their current worldview. Evidence standards become artificially high for claims these people don’t like, and artificially low for the claims they do like. Even if it isn’t the intent, the function of such a rule becomes the maintenance of the status quo. Those who are looked up to and who’s opinions are thought highly of tend to be the ones whose influence determines what is and isn’t extraordinary.  Scientific consensus devolves into an ideological popularity contest.

Max Planck, the father of quantum mechanics, is often famously quoted as saying that science progresses one funeral at a time.18 This outlook, while certainly cynical, rings true to something I think we can all recognize about ourselves: people don’t like to be wrong. Especially when they perceive that being wrong could threaten their identity.

Science is supposed to be the one arena that cuts through all of that baggage, but it is to our peril if we ever believe that we can be completely, or even almost completely, free of it. By avoiding codifying a rule that directly feed into these inherent human weaknesses, we can come closer to, but likely never achieve, the ideal of an unbiased worldview. It seems that perhaps the best we can do is start by attempting to be honest about our biases, and letting the conversation progress from there.

  1. Théodore Flournay, 1900, From India to the Planet Mars: A Study of a Case of Somnambulism. pg 344–345  (back)
  2. Marcello Truzzi, 1978, “On the Extraordinary: An Attempt at Clarification”Zetetic Scholar Issue 1  (back)
  3. Gallup Poll 2005, Three in Four Americans Believe in Paranormal  (back)
  4. The Reincarnation of Mediumship Research  (back)
  5. Rational Wiki, 2013, Extraordinary claims require extraordinary evidence  (back)
  6. On the (mis)use of Bayesian methods by skeptics  (back)
  7. M. Schlitz & D. Radin, 2002. “Telepathy in the ganzfeld: State of the evidence,”  Healing, Intention and Energy Medicine (London: Harcourt Health Sciences)   (back)
  8. J.B. Rhine and J.G. Pratt, 1954. A Review of the Pearce-Pratt Distance Series of ESP Tests  (back)
  9. C. Honorton; S. Krippner, 1969. “Hypnosis and ESP performance: A review of the experimental literature“, Journal of the Society for Psychical Research, 63(3), 214-252  (back)
  10. R. Sheldrake & P. Smart, 2003. “Experimental Tests For Telephone Telepathy“, Journal of the Society for Psychical Research 67, 184-199  (back)
  11. Puthoff & Targ , 1976. A perceptual channel for information transfer over kilometer distance: Historical perspective and recent research, Proceedings of the IEEE 64(3), 329-353  (back)
  12. Daily Mail, 2008, Could there be proof to the theory that we’re ALL psychic?  (back)
  13. PodBlack Cat, 2009, Dr Richard Wiseman On Remote Viewing In The Daily Mail – Clarification  (back)
  14. Skeptiko Podcast, 2009, 83. Dr. Chris French, Extraordinary Psi Claims  (back)
  15. Skeptiko Podcast, 2007, 3. Dr. Michael Shermer on Darwin, Evolution and Creativity  (back)
  16. M. Truzzi, 1987,  “On Pseudo-Skepticism”.Zetetic Scholar (12/13)   (back)
  17. Parapsychology, Anomalies, Science, Skepticism, and CSICOP – compiled by Daniel H. Caldwell  (back)
  18. This is a popular paraphrase of the actual quote. More info here  (back)

5 thoughts on “Do Extraordinary Claims Require Extraordinary Evidence?

  1. Hans says

    You did not wrote the whole quote of Wiseman just a part of it:

    “Professor Richard Wiseman, a psychologist at the University of Hertfordshire, refuses to believe in remote viewing.

    He says: “I agree that by the standards of any other area of science that remote viewing is proven, but begs the question: do we need higher standards of evidence when we study the paranormal? I think we do.

    “If I said that there is a red car outside my house, you would probably believe me.

    “But if I said that a UFO had just landed, you’d probably want a lot more evidence.

    “Because remote viewing is such an outlandish claim that will revolutionise the world, we need overwhelming evidence before we draw any conclusions. Right now we don’t have that evidence.”

    The quote is from:

    I know that you are writing about a double standard here but this is important because experiments in the paranormal world are filled with errors, tricks, deception and even lies.

    The AWARE study where Dr. Sam Parnia was trying to prove that we have a soul also failed to meet this standard in the end because no one saw the images which would even destroy the skeptics:

    “During the AHA meeting, Dr. Sam Parnia, head of intensive care at Stony Brook University Hospital in New York, reported early results of a 25-hospital study of how frequently cardiac arrest survivors see or hear things while their hearts are stopped. Of 152 survivors interviewed, 37 percent said they had recollections from the unconscious period. Only two recalled actually seeing events and one described any events that could be verified. None saw images mounted in the treatment room as part of the experiment.”

    The quote is from:

    This is my take on the whole situation. If there is PSI we need better data then there are. So far we did not prove anything in the end and even experiments in telepathy were met with failures:

    Just look here in telepathy under virtual reality:

    There are more failed studies. So I am very doubtful about this whole business.

    • placebonetix says

      It’s just comical that these professional skeptics think remote viewing is “outlandish”. The research shows quite the opposite — what’s more, we probably use RV all the time but don’t notice it. The ordinary person can demonstrate RV to his own satisfaction by getting trained by credible instructors – and then just doing it. Anyone seeking unbiased information about RV would do well by checking out the International Remote Viewing Association,

  2. Hans says

    Did some digging and found how Wiseman meant that quote and he does not agree that Remote Viewing is confirmed even Alex Tsakiris asked him about this in one of his skeptiko podcasts:

    “Alex Tsakiris: Okay. I think that’s going to roll right into the next question I have. I have one question for each of you and then I want to give each of you a chance to respond like we just did. So, Dr. Wiseman, let’s start with you again. Here’s a quote that I think has generated quite a bit of stir and let’s drill into it a little bit. This is you being quoted in the UK’s Daily Mail: “I agree that by the standards of any other area of science that remote viewing is proven. That begs the question do we need higher standards of evidence when we study the paranormal?”

    Then you went on in a subsequent interview and further refined that by saying: “That’s a slight misquote because I was using the term in more of a general sense of ESP. That is, I was not talking about remote viewing per se, but rather Ganzfeld, etc. as well. I think that they do meet the usual standards for a normal claim but are not convincing enough for an extraordinary claim.”

    Now, I think you can see where I’m going with this, but in a most recent article in the Skeptical Inquirer Magazine, titled, “Heads I Win, Tails You Lose,” you begin with: “After more than 60 years of experimentation, researchers have failed to reach a consensus about psi.”

    I guess the question is this. Aren’t these two quotes of yours really at odd? I mean, if parapsychology research has really brought us to the point of proving psychic abilities by the standards of any other area of science, isn’t that quite an achievement? And shouldn’t we be really commending that accomplishment? I’m sure you’re going to have a lot to say about that, so dig into it in any way you like.

    Dr. Richard Wiseman: First of all, I do actually commend parapsychologists. I count myself as a parapsychologist and carry out that research. That is, I was very much part of that community, not so much now. I do commend them for doing the research and doing it in a systematic way and attempting to be as scientific as possible. That has led us to a basic place which is messy in many ways.

    But if you take the general ESP claim, which might include Ganzfeld and some of the other ESP paradigms as well, I think it’s true – I don’t know there’s any kind of objective way that could be measuring this, but my feeling is if that were a claim about the effect of alcohol on memory, then we’d go, yeah, there’s probably something to it. But the claim here is far more radical than that. It would lead to a massive shift within science. It would overturn most of what we know within psychology. I don’t know about other areas, but certainly within psychology. So for me the evidential bar, as it were, needs to be much higher than that.

    I sometimes say to people it’s a bit like if I say, “Outside there on the road there’s a car,” you’d go, “Well, fine. I don’t feel the need to go outside and look at the car.”

    If I say, “Well, it’s a spaceship,” then you might think, ‘Well, actually I do want to look at that. You know, an alien spacecraft.”

    I say, “Yeah, it’s an alien spacecraft.”

    Well, you wouldn’t take my word for it. You’d go outside. You’d require a higher level of evidence. And I think that parapsychologists by not far from 100 years of research have failed to come up with that level of evidence. It’s not to say they couldn’t in the future, but to me there just hasn’t been the level of progress that you would expect given the amount of work that’s been put in. So the article was really about how we might try to push that forward and try and get some closure, i.e., can we produce the level of evidence one would need, and if not, do we agree not to carry on with the endeavor?

    Alex Tsakiris: Boy, before I even let Dr. Sheldrake get in there I want to just fill in to a couple of points there because 1) you seem to be saying that there should be some kind of subjective measure other than the normal means of science, peer review, the normal means we have in place for measuring certain claims that we’re going to label as extraordinary. I’m just wondering how we would and why we would layer on top of the institution that we have on science, why we would layer that on top?

    And then 2) you kind of alluded to some again, subjective sense of how much body of evidence would be required. Those just seem awful fuzzy to me.

    Dr. Richard Wiseman: Well, I think they are. I mean, the basic approach to all of this is that your own belief system magically influences how much evidence you need. So within mainstream science we’ve already said people will be pretty skeptical about the idea of psychic ability because it overturns the current scientific world view. So they will require a very strong database. You can’t really put numbers to it. You can’t say, “Oh, 10 studies coming out of this lab, and 2 studies out here.” It doesn’t really work like that.

    But it’s that notion they will require a very strong database and though it’s kind of agreement I think from a lot of parapsychologists, not all, but a lot is that strength of evidence simply isn’t there. There is a huge problem with replication. And unless you have some sort of phenomenon that replicates reliably outside of proponents running the experiments in particular, you won’t get mainstream science taking psi, as it were, onboard.”

    The quote is from:

    • aaron says

      Yes, I’ve heard that interview. In it he says pretty much the same thing he said in my two line quote. That if it were any other area of science, it would be confirmed, but since it’s telepathy, which is extraordinary, we need a higher standard. I appreciate you taking the time to dig this up, but I’m uncertain about what your point is, given that it confirms exactly what I’ve written about him in this article. Which is not that Richard Wiseman agrees remote viewing has been confirmed, but that he agrees according to the standard of any other science it would have been confirmed, but since he asks for a higher standard, he believes it hasn’t been.

      • Hans says

        I think Wiseman is trying to say that the evidence is not so strong. He is saying this actually that proponents and those who you cited are proponents make the claim that PSI is proven. Wiseman is arguing that this is not the fact and that only proponents take the data as solid and true:

        “But it’s that notion they will require a very strong database and though it’s kind of agreement I think from a lot of parapsychologists, not all, but a lot is that strength of evidence simply isn’t there. There is a huge problem with replication. And unless you have some sort of phenomenon that replicates reliably outside of proponents running the experiments in particular, you won’t get mainstream science taking psi, as it were, onboard.”

        This is actually the problem. When you are a proponent then the evidence is enough for you. However for those who are not believers this is not convincing. Other parapsychologist like Antonia Mills has noted this problem also and even Ian Stevenson that the cultural bias is very strong when it comes to parapsychology.

        Chris French also claims something similar who you quoted here:

        “Dr. Chris French: It’s quite recent, then again, I’d have to look at that in context. But the general thrust of what you’re saying I would probably actually agree with Richard. I mean, I’ve just written a chapter for a book that Stanley Krippner is editing. I’m at pretty much the same point myself. I actually first of all put the question, is there a double standard in science when it comes to parapsychology? And I say actually, yes, there is. Then I go on to say well, I think there should be. Which I think is just really the same point that Richard’s making here.

        Now the reasons for that are because – it gets back, I mean, to put it in a nutshell it’s the Carl Sagan line of you know, “extraordinary claims requiring extraordinary proof.” That kind of thing. Why should we have kind of a double standard like that? Well, I illustrate in this chapter by talking about a real-life example from my own research. I used to do research in the area of cognition and emotion, and without going into the full detail, my wife and I who were working together, we published a paper which offered some support for a new idea, a new hypothesis that had just been put out in this area. And that was published and that was fine. We then found we couldn’t replicate it.

        Now, this happens quite a lot in psychology. Psychological effects are often not – you know, some of them are very reliable but a lot of them aren’t. We tried our damnedest to replicate. We even wrote of our failed replication attempts. We couldn’t get that published. I mean, we would tell people on an informal basis that we weren’t able to replicate. We thought that the reason for it was just a statistical blip that or perhaps something else but we didn’t think it was a genuine effect. But of course, in a sense, it mattered less, because whichever way around that particular, the reality was for that particular hypothesis we were not asking people to overturn you know, basic laws of physics. It was – it mattered less in that sense. And I think that’s why…”

        The quote is from:

        Here it continues:

        “Dr. Chris French: Well, okay, I think it depends what you mean by having – I mean, again, let me clarify. I mean, I think there’s kind of – there’s possibly two way of interpreting that kind of idea of double standards. One is in line with what I think Richard would say and although I’ve not seen the quotation in context so I can’t be sure, and certainly what I was arguing which I think a case can be made there. The other is such a completely unjustifiable nus – idea of double standards, which I think also happens to be true. I think the reason real prejudice within mainstream science regarding any kind of parapsychological claim, I don’t endorse that for one moment.

        I think parapsychologists really do have a kind of bias to fight against. But as I referred in our previous e-mail exchange is that in itself is irrelevant in deciding whether or not paranormal forces really exist. But I do think it’s there. I mean, I think it’s true and I sympathize with parapsychologists over that. But getting back to the more substantive argument, I think we have to have quite a sophisticated view of what science is. And I’m not raising all those kinds of notions about well, what is replicability? We certainly can’t have a situation where you know, we get a few experiments here and there that appear to replicate what the vast majority don’t. That’s not what we mean by replicability.

        And you know, we can look at meta-analyses being rated and very, very fond of meta-analyses, but there are problems in using meta-analysis. I mean, I think the debates that parapsychologists and critics have had about the strengths and weaknesses of meta-analysis have been really, really enlightening and you know, have wider applications beyond just parapsychology. So I think it’s a useful debate and discussion to be having anyway. But at the end of the day, what we really need to do is to get down to some kinds of effects which would actually be reasonably robust and reliable and replicable that pretty much – no you don’t have to have 100 percent replicability.

        We’re talking about some number that you’d have a reasonably good chance of being able to replicate under well controlled conditions if you set up the situation properly. And that is still something that we’re looking for in parapsychology. I mean, it would be great if we find it and…

        Alex Tsakiris: And maybe…

        Dr. Chris French: …some of my times are looking for it but most of my research effort, as you know, go into the anomalistic psychology side and looking at things which look as if they might be psychic but actually aren’t.”

        The quote is from:

        Chris French makes a point why there should be a higher standard:

        “Now, this happens quite a lot in psychology. Psychological effects are often not – you know, some of them are very reliable but a lot of them aren’t. We tried our damnedest to replicate. We even wrote of our failed replication attempts. We couldn’t get that published.”

        The quote is from:

Leave a Reply