Reading focus
Syllabus: How Do We Know When Psychotherapy Helps—or Harms?
Examine comparator definitions and support for control therapists alongside the effect estimates.
Why you’re reading it. Apply the framework to an evidence synthesis while retaining uncertainty about causal explanations.
A question to consider. A trial reports a large advantage, but only the experimental therapists received supervision: what competing interpretations should a reviewer investigate?
Why this reading
The name of a therapy is only one feature of a trial. This synthesis examines whether estimated treatment differences vary with the comparator and with support for its therapists. It gives an empirical basis for questioning a result’s design before treating its effect size as a property of a therapy.
Before you read
A standardized effect expresses a group difference in standard-deviation units. A comparison across trials is observational even when participants were randomized within each trial. Different design features may occur together.
Read for
- Compare the control categories in Table 1 with how treatment, training, supervision and fidelity monitoring were coded.
- Follow the sensitivity analyses and the distinction between an unreported procedure and a procedure known not to have occurred.
Consider afterwards
- Give an alternative explanation for an association between control-arm supervision and treatment effect size.
- What experiment would more directly test whether changing one implementation feature changes the result?
Editorial context
The review includes 125 adult depression trials, with searches through June 2009. It cannot establish that the coded design features caused the differences between studies. Implementation features overlap, reporting is incomplete, several comparisons are sparse, and publication bias is possible. The findings support careful comparator description; they do not establish a universal ranking of controls or equivalence of therapies.