Simpson's paradox in psychological science: a practical guide

Abstract

The direction of an association at the population-level may be reversed within the subgroups comprising that population—a striking observation called Simpson’s paradox. When facing this pattern, psychologists often view it as anomalous. Here, we argue that Simpson’s paradox is more common than conventionally thought, and typically results in incorrect interpretations – potentially with harmful consequences. We support this claim by drawing on empirical results from cognitive neuroscience, behavior genetics, psychopathology, personality psychology, educational psychology, intelligence research, and simulation studies. We show that Simpson’s Paradox is most likely to occur when inferences are drawn across different levels of explanation (e.g., from populations to subgroups, or subgroups to individuals). We propose a set of statistical markers indicative of the paradox, and offer psychometric solutions for dealing with the paradox when encountered—including a toolbox in R for detecting Simpson’s Paradox. We show that explicit modeling of situations in which the paradox might occur not only prevents incorrect interpretations of data, but also results in a deeper understanding of what data tell us about the world.