Reading focus
Syllabus: How Do We Know When Psychotherapy Helps—or Harms?
Track the denominator of funded, initiated, published and recoverable studies before comparing effect estimates.
Why you’re reading it. Understand how assembling a different evidence base changes a treatment claim.
A question to consider. If recovering unpublished trials reduces an effect estimate, what can you now conclude about benefit, and what remains unknown about selectively reported outcomes?
Why this reading
A literature search cannot automatically recover studies that never became papers. Driessen and colleagues start with funded research and trace what was published, then seek missing results. This changes the evidence available for estimating treatment benefit.
Before you read
Publication bias occurs when availability of a study’s results is related to what those results show. Selective outcome reporting occurs within studies. Recovering unpublished studies addresses one problem without necessarily solving the other.
Read for
- Follow the path from grants to initiated trials, publications and recoverable data.
- Compare published-only and combined estimates within the same kind of treatment comparison; inspect uncertainty as well as the point estimates.
Consider afterwards
- Why is a grant-based search informative in a way that a search of journals alone is not?
- What would you still need to know before calling the combined estimate unbiased?
Editorial context
The sample concerns US NIH-funded trials of acute treatment for adult major depression, funded from 1972 through May 2008. Data were unavailable for two unpublished studies, and original protocols were insufficient to assess selective outcome reporting. The review itself had no registered protocol. Its results do not establish the publication pattern of every treatment field or imply that a smaller estimated benefit is no benefit.