Reading focus
Syllabus: How Can Psychological Science Become More Reliable?
Trace the selection and coding decisions before interpreting the comparison between publication formats.
Why you’re reading it. Evaluate a reform with the same causal caution applied to other observational comparisons.
A question to consider. What alternative explanation for the difference in positive results would you investigate before crediting the format alone?
Why this reading
A proposed reform needs evaluation. Scheel and colleagues compare support for the first tested hypothesis in conventional papers and Registered Reports, providing an empirical question about the published record rather than assuming that a new format works.
Before you read
A positive result here means support under the authors’ coding scheme. Its frequency can reflect which hypotheses are investigated, how studies are analyzed and which reports appear. It is not the proportion of claims that are true.
Read for
- Trace eligibility, search language and the choice of the first hypothesis through the sample.
- Compare the main result with analyses separating replication and original studies.
- Examine what the observational comparison can establish about publication format.
Consider afterwards
- What additional comparison would help distinguish a format’s effect from differences in the researchers or questions using it?
Editorial context
The groups were not randomly assigned. Selection, unblinded coding and differing research questions limit causal interpretation. A lower positive-result rate is not, by itself, proof of better science or an estimate of how many conventional findings are false.