Reading focus
Syllabus: When Do Psychological Findings Generalize?
Compare the verbal-overshadowing example with the stimulus and multi-site arguments. Distinguish the target article’s recommendations from a guarantee that a more complicated model will generalize.
Why you’re reading it. Bring the target claim, the sampled dimensions and the analytical assumptions into one defensible argument.
A question to consider. For a finding from Many Labs 2, write a narrower claim supported by its design and name one design change needed for a broader claim. Would changing the statistical model alone suffice?
Why this reading
A statistical analysis can support a narrower claim than the words used to describe it. Yarkoni examines that mismatch across people, stimuli, tasks and sites, then proposes changes to design, modeling and scientific claims.
Before you read
A fixed effect estimates a particular contrast; a random effect represents variation across levels of a factor. Terminology differs across fields. A model cannot estimate variation in a factor that never varies in its data.
Read for
- Separate the simulations, the verbal-overshadowing example and the broader claims about research practice.
- Ask which proposed remedies change the data, which change the model and which narrow the conclusion.
Consider afterwards
- A study uses many participants but one task and one set of instructions. What would have to change before its claim could concern a broad psychological capacity?
Editorial context
This is the complete BBS target article; accompanying commentaries are separate works. Its proposals are contested. Bear and Phillips emphasize systematic stimulus-selection bias; Yarkoni’s reply defends improved models without claiming they solve every generalization problem. Adding random effects does not establish representativeness or validate a construct.