Reading focus
Syllabus: How Do We Combine Evidence Across Studies?
Compare the published-only and combined evidence within the same treatment contrasts, keeping unavailable data and outcome reporting in view.
Why you’re reading it. Understand why searching a registry or funding record can reveal a different evidence base from searching papers alone.
A question to consider. If all recoverable unpublished trials were added, what remaining problem could still bias the review’s conclusion?
Why this reading
A literature search cannot automatically recover studies that never became papers. Driessen and colleagues start with funded research and trace what was published, then seek missing results. This changes the evidence available for estimating treatment benefit.
Before you read
Publication bias occurs when availability of a study’s results is related to what those results show. Selective outcome reporting occurs within studies. Recovering unpublished studies addresses one problem without necessarily solving the other.
Read for
- Follow the path from grants to initiated trials, publications and recoverable data.
- Compare published-only and combined estimates within the same kind of treatment comparison; inspect uncertainty as well as the point estimates.
Consider afterwards
- Why is a grant-based search informative in a way that a search of journals alone is not?
- What would you still need to know before calling the combined estimate unbiased?
Editorial context
The sample concerns US NIH-funded trials of acute treatment for adult major depression, funded from 1972 through May 2008. Data were unavailable for two unpublished studies, and original protocols were insufficient to assess selective outcome reporting. The review itself had no registered protocol. Its results do not establish the publication pattern of every treatment field or imply that a smaller estimated benefit is no benefit.