Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses?
📄 Original study ↗📌 Appears in:
Plain English Summary
Kennedy scrutinizes decades of parapsychology meta-analyses (studies pooling many experiments) and finds a core problem: most studies were far too small to detect anything. Only 20-33% got significant results, way below the 80% standard. A proper ganzfeld telepathy experiment needs about 201 participants, yet most used around 40 — giving roughly a 1-in-5 shot at finding a real effect. Here's the fascinating twist: in random number generator studies, result strength doesn't grow with sample size as statistics demands. Kennedy's explanation? Maybe experimenters unconsciously influence outcomes through their own psychic abilities — a deliciously circular idea. He calls for pre-registered, properly powered studies.
Actual Paper Abstract
Retrospective meta-analyses are post hoc analyses that have not been effective at resolving scientific controversies, particularly when based on substantially underpowered experiments. Evaluations of moderating factors, including study flaws, small-study effects, and other sources of heterogeneity, do not neutralize confounding as in a well-designed experiment and cannot fully compensate for weaknesses in the original experiments. A group of well-designed experiments with adequate power and reliable results is needed for convincing evidence for a controversial effect. The widely recommended standard for experimental research is adequate power to obtain significant results on at least 80% of confirmatory experiments. Meta-analyses in parapsychology typically have found that 20% to 33% of studies with good methodology obtained significant results. Power analysis during experimental design is needed to achieve much better replication rates. Meta-analyses of RNG studies have consistently found that z value does not increase with sample size—which is contrary to statistical theory and has been and will be interpreted as an indication of methodological problems. This anomalous property and other sources of heterogeneity for parapsychological results must be addressed. Challenging topics such as experimenter effects, goal-oriented psi, and capricious psi-missing can no longer be ignored in research syntheses.
Research Notes
A cornerstone of Kennedy's methodology-critique program. Uniquely bridges the meta-debate with the elusiveness-of-psi problem by arguing that goal-oriented experimenter psi explains z's independence from sample size. Table 1's compilation of replication rates across paradigms is a key library reference.
Argues that retrospective meta-analyses have failed to resolve parapsychological controversies because they aggregate substantially underpowered studies. Compiles replication rates across major meta-analyses (Table 1), finding only 20-33% of well-conducted studies obtain significant results versus the 0.80 standard. A ganzfeld experiment needs N=201 for adequate power, yet the median is 40 trials (power 0.22). RNG meta-analyses consistently show z independent of sample size, contrary to statistical theory. Monte Carlo simulations confirm small-study effects in early ganzfeld data. Proposes goal-oriented psi experimenter effects as a parsimonious explanation and recommends prospective registration, adequate power, and best-evidence synthesis.
Related Papers
Cites
- Does Psi Exist? Replicable Evidence for an Anomalous Process of Information Transfer — Bem, Daryl J (1994)
- Examining Psychokinesis: The Interaction of Human Intention With Random Number Generators—A Meta-Analysis — Bösch, Holger (2006)
- Why Most Published Research Findings Are False — Ioannidis, John P.A (2005)
- Does Psi Exist? Lack of Replication of an Anomalous Process of Information Transfer — Milton, Julie (1999)
- Evidence for Consciousness-Related Anomalies in Random Physical Systems — Radin, Dean I (1989)
- Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology — Storm, Lance (2010)
- Replication and Meta-Analysis in Parapsychology — Utts, Jessica (1991)
- False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant — Simmons, Joseph P (2011)
- Methods for Investigating Goal-Oriented Psi — Kennedy, J.E (1995)
- The Capricious, Actively Evasive, Unsustainable Nature of Psi: A Summary and Hypotheses — Kennedy, J.E (2003)
- A Proposal and Challenge for Proponents and Skeptics of Psi — Kennedy, J.E (2004)
- Reexamining Psychokinesis: Commentary on the Bösch, Steinkamp and Boller Meta-Analysis — Radin, D (2006)
- "Future Telling": A Meta-Analysis of Forced-Choice Precognition Experiments, 1935-1987 — Honorton, Charles (1989)
Same Research Program
- Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud — Kennedy, James E (2024)
- Experimenter Fraud: What Are Appropriate Methodological Standards? — Kennedy, J.E (2017)
- Planning Falsifiable Confirmatory Research — Kennedy, James E (2024)
- Bayesian and Classical Hypothesis Testing: Practical Differences for a Controversial Area of Research — Kennedy, J.E (2014)
- Why Is Psi So Elusive? A Review and Proposed Model — Kennedy, James E (2001)
- Skepticism and Negative Results in Borderline Areas of Science — Kennedy, J.E (1981)
- Methods for Investigating Goal-Oriented Psi — Kennedy, J.E (1995)
Companion
- Testing for Questionable Research Practices in a Meta-Analysis: An Example from Experimental Parapsychology — Bierman, Dick J (2016)
- Mindless Statistics — Gigerenzer, Gerd (2004)
- Why Most Research Findings About Psi Are False: The Replicability Crisis, the Psi Paradox and the Myth of Sisyphus — Rabeyron, Thomas (2020)
Cited By
- Conclusions about Paranormal Phenomena — Kennedy, J.E (2013)
- Is the Methodological Revolution in Psychology Over or Just Beginning? — Kennedy, J.E (2016)
- Searching for the Impossible: Parapsychology's Elusive Quest — Reber, Arthur S (2019)
- Bayesian and Classical Hypothesis Testing: Practical Differences for a Controversial Area of Research — Kennedy, J.E (2014)
- Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis — Watt, Caroline A (2017)
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
Self-Ascribed Paranormal Ability: Reflexive Thematic Analysis
📋 Cite this paper
Kennedy, J.E (2013). Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses?. Journal of Parapsychology.
@article{kennedy_2013_beyond_meta_analyses,
title = {Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses?},
author = {Kennedy, J.E},
year = {2013},
journal = {Journal of Parapsychology},
}