Reading focus
Syllabus: How Can Psychological Science Become More Reliable?
Compare the analytic questions, models and uncertainty behind the teams’ conclusions.
Why you’re reading it. Extend scrutiny beyond whether a single analysis was planned in advance.
A question to consider. If most teams obtain a significant association, what must still be examined before interpreting it as a causal effect?
Why this reading
An association does not emerge from data without decisions about models, variables and comparisons. This project gives several teams the same football dataset and question, making variation across analyses visible.
Before you read
An odds ratio expresses an association on a particular scale. Different adjustments or models can target different quantities. The dataset is observational; an association between rated skin tone and red cards does not by itself identify its cause.
Read for
- Follow the independent analyses, peer feedback and later discussion as distinct stages.
- Compare model choices with the estimates and uncertainty, rather than only counting significant tests.
- Ask which sources of variation this one dataset can reveal.
Consider afterwards
- Two analyses disagree. What would you need to know before deciding whether they answer the same question?
Editorial context
The 2018 corrigendum clarifies prior dissemination of the findings. A later methodological critique emphasizes specifying questions and evaluating analytic adequacy. Variation does not establish that every analysis is equally defensible.