Reading focus
Syllabus: How Do We Combine Evidence Across Studies?
Identify shared participants, shared comparisons and nested samples in the examples.
Why you’re reading it. Recognize why counting estimates is not the same as counting independent evidence.
A question to consider. If five reports come from two cohorts and each gives several outcomes, what information would you collect to avoid treating every result as a new study?
Why this reading
Several results from the same participants are not several independent replications. Cheung distinguishes sources of dependence and shows how retaining their structure can answer questions that averaging or discarding results would obscure.
Before you read
Effect sizes can share participants, comparison groups or a broader study context. Multivariate models represent related outcomes and their sampling covariances. Multilevel models represent nested variation. Those are different sources of dependence, and a study label alone may not capture both.
Read for
- Identify the different dependence structures in the examples before examining the fitted models.
- Compare what is lost when outcomes are averaged, selected or analyzed separately.
- Follow the limits of approximating one structure with another and the information needed about correlations.
Consider afterwards
- A review extracts three outcomes and two follow-up times from each trial. What relationships among those estimates would its analysis need to represent?
Editorial context
The examples introduce multivariate and three-level approaches, with robust variance estimation discussed as another option. They do not cover every crossed or overlapping data structure. A more elaborate model still needs appropriate effect selection and enough information; variance components can be unstable in small collections.