Reading focus
Syllabus: When Do Psychological Findings Generalize?
Examine how the audit distinguishes participant composition from how authors describe the reach of their findings. Keep its journal and time period in view.
Why you’re reading it. Identify whose experiences are absent and where a claim exceeds the population information reported.
A question to consider. Rewrite a claim about “people” for a study with a narrowly described sample. What missing information would matter most for judging its intended reach?
Why this reading
Rad and colleagues turn a broad concern about sampling into an audit of published research. They examine both who participated and how authors described the reach of their conclusions.
Before you read
The study examines one journal in 2014 and a smaller follow-up period in 2017. An article, a study and a sample are different units of counting. Missing demographic information limits what an audit can establish.
Read for
- Track the denominators and coding decisions behind the reported proportions.
- Separate evidence about publication practices from the authors’ proposals for changing incentives and reporting.
Consider afterwards
- How would you rewrite a claim about “people” to identify the population actually sampled?
- What further evidence would be needed to evaluate whether a proposed editorial policy improved generalization?
Editorial context
This dated audit concerns Psychological Science, not every journal or current research practice. Broader participation and more explicit population claims are distinct objectives. The proposed reforms are arguments informed by the audit; the paper does not experimentally evaluate their effects.