Reading focus
Syllabus: When Do Psychological Findings Generalize?
Keep three levels separate: the selected finding, its results across samples, and the collection of findings. Follow the selection rules and uncertainty as carefully as the pooled results.
Why you’re reading it. Interpret multi-site evidence without turning a replication percentage into a verdict on a discipline.
A question to consider. Choose one reported effect and specify a boundary that the participating sites test, a boundary they do not test, and a new design that would address the latter.
Why this reading
A result can be studied across many samples while much of its procedure remains fixed. Many Labs 2 makes that distinction tangible by examining both average effects and variation across sites for 28 selected findings.
Before you read
The project includes 125 samples from 36 countries and territories, with each protocol administered to roughly half the samples. Heterogeneity describes differences in effect sizes beyond sampling error. Its magnitude, uncertainty and possible causes are separate questions.
Read for
- Track how findings, sites and procedures were selected, including changes documented before and after preregistration.
- Compare pooled estimates with the variation across samples. Distinguish prespecified moderators from exploratory comparisons and attend to uncertainty in heterogeneity estimates.
Consider afterwards
- An effect is similar across the participating sites. Which other materials or settings remain untested, and what study would challenge the strongest remaining generalization?
Editorial context
The findings and sites were not probability samples of all psychological research or humanity. The broad WEIRD/less-WEIRD grouping cannot explain cultural mechanisms. Standardized procedures also restrict the variation assessed. The paper’s replication proportion is not an estimate for psychology as a whole.