Reading focus
Syllabus: How Can Psychological Science Become More Reliable?
Compare the project’s selection decisions and outcome criteria with the preceding conceptual argument.
Why you’re reading it. Interpret a coordinated replication project without treating its headline as a field-wide reliability score.
A question to consider. Which conclusion would require several replications of one effect rather than single replications of many effects?
Why this reading
The collaboration attempted replications of 100 experimental and correlational studies from three psychology journals. Read it as a large coordinated assessment of selected findings, with several ways of evaluating replication success. Its value is not a single verdict on psychology: it makes the relationship between original results, replication estimates, uncertainty, and sampling decisions available for inspection.
Before you read
This collaboration tests selected prior findings using new observations. Different estimates can reflect sampling uncertainty, differences in implementation or context, or limitations in the original evidence. A discrepancy alone does not identify its cause.
The paper uses several criteria to evaluate replication. Keep the question asked by each criterion separate from a verdict on an entire theory or discipline.
Read for
- Trace the journal, article, study and focal-result selection decisions, including feasibility constraints.
- Compare significance, effect sizes, interval compatibility and other replication criteria: which question does each answer?
- Distinguish correlates of replication outcomes from evidence identifying their causes.
- Follow the authors’ limits on generalizing from this sample and on interpreting individual replication attempts.
Consider afterwards
- A replication estimates a smaller effect than the original. What information would help you evaluate their compatibility and theoretical implications?
- How could two defensible criteria produce different assessments of the same replication, and how should a report make that visible?
Editorial context
The target studies came from a specific set of journals and publication years, not a representative sample of every psychological field or method. Statistical significance, effect-size differences, and compatibility of estimates answer different questions about replication. The original report should therefore be read with attention to both study selection and the definition of success. Avoid converting a result for this sample into a timeless percentage of psychology that is reliable.