Reading focus
Syllabus: How Do We Combine Evidence Across Studies?
Compare the model assumptions with the changing weights and intervals in the worked examples.
Why you’re reading it. Understand the quantity a pooled estimate represents before interpreting its precision.
A question to consider. If study-skills interventions differ in format and duration, what would an average effect describe, and what would it leave unresolved about a particular course?
Why this reading
The same observations can answer different questions under different models. Borenstein and colleagues use worked examples to distinguish a common effect from the mean of varying effects and show why the distinction changes weighting, uncertainty and interpretation.
Before you read
An effect size describes a magnitude on a chosen scale. Sampling variance describes uncertainty in an estimate; between-study variance describes variation in underlying effects. A confidence interval for a mean effect is different from a prediction interval for another study’s effect.
Read for
- Compare the college examples and forest plots: what population and quantity does each analysis represent?
- Follow how adding between-study variation changes the role of large and small studies.
- Examine why the authors reject choosing a model solely from a heterogeneity test.
Consider afterwards
- A review reports a precise positive average but effects vary widely across settings. What would you need before using it to predict the result in a particular setting?
Editorial context
The article introduces particular inverse-variance models and estimators. Random effects do not correct selective publication, poor measurement or confounding, and a modeled population of studies does not establish representativeness. With few studies, the amount of variation and resulting uncertainty can be difficult to estimate.