Reading focus
Syllabus: How Can Psychological Science Become More Reliable?
Keep the claim being tested visible when considering changes in procedures or context.
Why you’re reading it. Distinguish informative confrontation with evidence from literal repetition.
A question to consider. A study supports a broad theory, but a contrary outcome is dismissed as a task failure. What would make the test genuinely diagnostic?
Why this reading
Repeating a procedure is not the only way to test a prior claim. Nosek and Errington propose that replication depends on whether possible outcomes would change our confidence in that claim. Their argument makes the connection between replication and theory explicit.
Before you read
A claim can concern a particular observation or a broader explanation. A diagnostic test bears on that claim. Identical procedures can function differently across settings; changed procedures can sometimes provide a relevant test.
Read for
- Examine why the authors give weight to both confirming and disconfirming outcomes.
- Follow their treatment of changing contexts and boundaries of generalization.
- Ask what theoretical commitments are needed to call a study a replication.
Consider afterwards
- A task is translated for a new population. What would make the study informative about the original claim, whatever its outcome?
Editorial context
This is a proposed conceptual definition, not a universally adopted rule. Deciding whether an outcome is diagnostic itself requires judgment. Agreement between studies need not establish the preferred explanation, and disagreement need not identify its cause.