Reading focus
Syllabus: How to Read Psychological Science
Connect the selection of target studies, the different replication criteria, the results, and the discussion. Compare effect estimates and uncertainty rather than treating statistical significance as a verdict.
Why you’re reading it. Interpret replication evidence using design, effect magnitude, uncertainty, and study selection together.
A question to consider. If an original result is statistically significant and its replication is not, which effect estimates, uncertainty intervals, and design details would you examine before deciding how much to change your confidence?
Why this reading
The collaboration attempted replications of 100 experimental and correlational studies from three psychology journals. Read it as a large coordinated assessment of selected findings, with several ways of evaluating replication success. Its value is not a single verdict on psychology: it makes the relationship between original results, replication estimates, uncertainty, and sampling decisions available for inspection.
Before you read
This collaboration tests selected prior findings using new observations. Different estimates can reflect sampling uncertainty, differences in implementation or context, or limitations in the original evidence. A discrepancy alone does not identify its cause.
The paper uses several criteria to evaluate replication. Keep the question asked by each criterion separate from a verdict on an entire theory or discipline.
Read for
- Trace the journal, article, study and focal-result selection decisions, including feasibility constraints.
- Compare significance, effect sizes, interval compatibility and other replication criteria: which question does each answer?
- Distinguish correlates of replication outcomes from evidence identifying their causes.
- Follow the authors’ limits on generalizing from this sample and on interpreting individual replication attempts.
Consider afterwards
- A replication estimates a smaller effect than the original. What information would help you evaluate their compatibility and theoretical implications?
- How could two defensible criteria produce different assessments of the same replication, and how should a report make that visible?
Editorial context
The target studies came from a specific set of journals and publication years, not a representative sample of every psychological field or method. Statistical significance, effect-size differences, and compatibility of estimates answer different questions about replication. The original report should therefore be read with attention to both study selection and the definition of success. Avoid converting a result for this sample into a timeless percentage of psychology that is reliable.