Experimenter Fraud: What Are Appropriate Methodological Standards?
📄 Original study ↗📌 Appears in:
Plain English Summary
Kennedy has a remarkable backstory: he helped catch W. J. Levy faking data in the famous Rhine parapsychology lab, then spent twenty years in pharmaceutical research under strict FDA rules. That gives him a uniquely sharp perspective on fraud in psi research (studies of psychic phenomena). Here's the fascinating problem: in most sciences, you spot fabrication through suspicious statistical patterns -- but in parapsychology, those same weird patterns could be the psychic effects you're looking for! That makes after-the-fact fraud investigations almost useless. Catching Levy required a covert sting during a live experiment. Kennedy's solution is refreshingly practical: stop trying to catch cheaters afterward and make cheating nearly impossible from the start -- duplicate records, two experimenters watching each other, and validated software.
Actual Paper Abstract
Discussions of experimenter fraud in parapsychology have missed a key lesson from the Levy case. The standard procedure for handling scientific fraud is an after-the-fact (post hoc) investigation. Post hoc investigations cannot be expected to be effective in parapsychology because signs of fraud in the data can be attributed to psi, as happened with Levy. In parapsychology, compelling evidence of fraud usually requires direct covert detection of fraud as it occurs during an ongoing experiment, as in the Levy case. However, such covert measures by colleagues are not a practical strategy for addressing fraud and are not expected in other areas of science. The standard that experimental procedures should make fraud by one experimenter very difficult or impossible has long been advocated in parapsychology but has not been implemented in recent decades. This standard was implemented in my experience working in regulated medical research and should eliminate the vast majority of cases of fraud—which start when one experimenter finds data manipulation or fabrication easy and tempting with very little possibility of detection. This standard provides a systematic and effective way to address experimenter fraud and should become part of the new standards for research in the behavioral sciences.
Research Notes
Key paper in the library's methodology/fraud discussion thread (Controversy #10). Kennedy's unique position as both the Levy fraud exposer and a pharmaceutical research veteran gives this paper unusual authority. Directly preceded by Kennedy (2016) on confirmatory methods and followed by Kennedy (2024) on comprehensive fraud prevention strategies.
Drawing on firsthand experience exposing the W. J. Levy fraud at J. B. Rhine's lab and two decades of work in FDA-regulated pharmaceutical research, Kennedy argues that standard post hoc investigations of fraud are uniquely ineffective in parapsychology because statistical anomalies indicative of data manipulation can be explained as psi effects. The Levy case required direct covert detection during an ongoing experiment — a 'sting operation' — to produce compelling evidence. Kennedy advocates adopting the pharmaceutical research standard that fraud by one experimenter should be very difficult or impossible, implemented through duplicate data records, dual-experimenter oversight, and documented software validation. This systematic prevention approach would eliminate the vast majority of fraud cases and is preferable to reliance on after-the-fact accusations and investigations.
Related Papers
Same Research Program
- Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud — Kennedy, James E (2024)
- Is the Methodological Revolution in Psychology Over or Just Beginning? — Kennedy, J.E (2016)
- Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses? — Kennedy, J.E (2013)
- Planning Falsifiable Confirmatory Research — Kennedy, James E (2024)
- Conclusions about Paranormal Phenomena — Kennedy, J.E (2013)
- Options for Prospective Meta-Analysis and Introduction of Registration-Based Prospective Meta-Analysis — Watt, Caroline A (2017)
Companion
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
Self-Ascribed Paranormal Ability: Reflexive Thematic Analysis
📋 Cite this paper
Kennedy, J.E (2017). Experimenter Fraud: What Are Appropriate Methodological Standards?. Journal of Parapsychology.
@article{kennedy_2017_experimenter_fraud,
title = {Experimenter Fraud: What Are Appropriate Methodological Standards?},
author = {Kennedy, J.E},
year = {2017},
journal = {Journal of Parapsychology},
}