Feeling the Future: A Meta-Analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events
📄 Original study ↗📌 Appears in:
Plain English Summary
After Bem's 2011 study ignited a firestorm, this massive follow-up asked: does the effect hold when other labs try it? Across 90 experiments from 33 labs in 14 countries with over 12,000 participants, the answer was a surprisingly firm yes. The Bayes Factor (a measure of how strongly data favor one hypothesis over another) exceeded five billion — wildly past what's considered decisive evidence. Even independent replications held up. Here's a fascinating wrinkle: experiments using fast, intuitive thinking worked significantly better than slow, deliberate ones — suggesting precognition might ride on gut instinct rather than careful reasoning. Seven of eight bias tests came back clean, arguing this isn't just cherry-picked results.
Actual Paper Abstract
In 2011, one of the authors (DJB) published a report of nine experiments in the Journal of Personality and Social Psychology purporting to demonstrate that an individual's cognitive and affective responses can be influenced by randomly selected stimulus events that do not occur until after his or her responses have already been made and recorded, a generalized variant of the phenomenon traditionally denoted by the term precognition. To encourage replications, all materials needed to conduct them were made available on request. We here report a meta-analysis of 90 experiments from 33 laboratories in 14 countries which yielded an overall effect greater than 6 sigma, z = 6.40, p = 1.2 × 10 with an effect size (Hedges' g) of 0.09. A Bayesian analysis yielded a Bayes Factor of 5.1 × 10, greatly exceeding the criterion value of 100 for "decisive evidence" in support of the experimental hypothesis. When DJB's original experiments are excluded, the combined effect size for replications by independent investigators is 0.06, z = 4.16, p = 1.1 × 10, and the BF value is 3,853, again exceeding the criterion for "decisive evidence." The number of potentially unretrieved experiments required to reduce the overall effect size of the complete database to a trivial value of 0.01 is 544, and seven of eight additional statistical tests support the conclusion that the database is not significantly compromised by either selection bias or by intense "p-hacking"—the selective suppression of findings or analyses that failed to yield statistical significance. P-curve analysis, a recently introduced statistical technique, estimates the true effect size of the experiments to be 0.20 for the complete database and 0.24 for the independent replications, virtually identical to the effect size of DJB's original experiments (0.22) and the closely related "presentiment" experiments (0.21). We discuss the controversial status of precognition and other anomalous effects collectively known as psi.
Research Notes
The definitive meta-analytic follow-up to the Bem (2011) controversy, providing the strongest cumulative statistical case for the precognition effect. Central to the Feeling the Future debate (Controversy #2). The fast/slow thinking distinction offers a theoretically motivated moderator that may explain inconsistent replication results.
Meta-analysis of 90 experiments from 33 laboratories in 14 countries (12,406 participants) testing anomalous anticipation of random future events, following up on Bem's (2011) original nine experiments. The overall effect was Hedges' g = 0.09, z = 6.33, p = 1.2 x 10^-10, with a Bayes Factor of 5.1 x 10^9 greatly exceeding the criterion for decisive evidence. Independent replications yielded g = 0.06, z = 4.16, BF = 3,853. P-curve analysis estimated the true effect size at 0.20, closely matching Bem's original d = 0.22. Fast-thinking protocols (g = 0.11) significantly outperformed slow-thinking protocols (g = 0.03, ns). Seven of eight statistical tests indicated the database is not compromised by selection bias or p-hacking.
Links
Related Papers
Meta Analyzes
- Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect — Bem, Daryl J (2011)
- Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect — Ritchie, Stuart J (2012)
- Correcting the Past: Failures to Replicate Psi — Galak, Jeff (2012)
- Feeling the Future Again: Retroactive Avoidance of Negative Stimuli — Maier, Markus A (2014)
- Retro-priming, priming, and double testing: psi and replication in a test–retest design — Rabeyron, Thomas (2014)
- Results from a Confirmatory Replication Study of Bem (2011): Precognitive Detection of Erotic Stimuli? — Wagenmakers, Eric-Jan (2012)
Cites
- Predictive Physiological Anticipation Preceding Seemingly Unpredictable Stimuli: A Meta-Analysis — Mossbridge, Julia (2012)
- We Should Have Seen This Coming — Schwarzkopf, D. Samuel (2014)
- Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice — LeBel, Etienne P (2011)
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
- A Bayes Factor Meta-Analysis of Bem's ESP Claim — Rouder, Jeffrey N (2011)
- Too Good to Be True: Publication Bias in Two Prominent Studies from Experimental Psychology — Francis, Gregory (2012)
- An Agenda for Purely Confirmatory Research — Wagenmakers, Eric-Jan (2012)
Same Research Program
- Does Psi Exist? Replicable Evidence for an Anomalous Process of Information Transfer — Bem, Daryl J (1994)
- Bem's 'Feeling the Future' (2011) Five Years Later: Its Impact on Scientific Literature — Silva, Bruno A (2017)
- Predictive Physiological Anticipation Preceding Seemingly Unpredictable Stimuli: An Update of Mossbridge et al.'s Meta-Analysis — Duggan, Michael (2018)
Cited By
- A Call for an Open, Informed Study of All Aspects of Consciousness — Cardeña, Etzel (2014)
- Inner Experience – Direct Access to Reality: A Complementarist Ontology and Dual Aspect Monism Support a Broader Epistemology — Walach, Harald (2020)
- Precognition as a Form of Prospection: A Review of the Evidence — Mossbridge, Julia A (2018)
- Why Most Research Findings About Psi Are False: The Replicability Crisis, the Psi Paradox and the Myth of Sisyphus — Rabeyron, Thomas (2020)
- The Experimental Evidence for Parapsychological Phenomena: A Review — Cardeña, Etzel (2018)
- Searching for the Impossible: Parapsychology's Elusive Quest — Reber, Arthur S (2019)
- Anomalous Cognition: An Umbrella Review of the Meta-Analytic Evidence — Tressoldi, Patrizio (2021)
- Exceptional Experiences Reported by Scientists and Engineers — Wahbeh, Helané (2018)
Also by these authors
When the Truth Is Out There: Counseling People Who Report Anomalous Experiences
Meta-Analysis of Free-Response Studies 2009-2018: Assessing the Noise-Reduction Model Ten Years On
On the Correspondence Between Dream Content and Target Material Under Laboratory Conditions: A Meta-Analysis of Dream-ESP Studies, 1966-2016
More in Precognition
Experimental Investigation of Precognition in Yoga Practitioners
Future dreams of electric sheep: Case study of a possibly precognitive lucid dreamer with AI scoring
Sentiment and Presentiment in Twitter: Do Trends in Collective Mood "Feel the Future"?
Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project
A Preregistered Multi-Lab Replication of Maier et al. (2014, Exp. 4) Testing Retroactive Avoidance
📋 Cite this paper
Bem, Daryl J, Tressoldi, Patrizio E, Rabeyron, Thomas, Duggan, Michael (2015). Feeling the Future: A Meta-Analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events. F1000Research. https://doi.org/10.12688/f1000research.7177.2
@article{bem_2015_feeling,
title = {Feeling the Future: A Meta-Analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events},
author = {Bem, Daryl J and Tressoldi, Patrizio E and Rabeyron, Thomas and Duggan, Michael},
year = {2015},
journal = {F1000Research},
doi = {10.12688/f1000research.7177.2},
}