Reading focus
Syllabus: What Can Statistical Evidence Tell Us?
Follow how the denominator in a standardized effect changes with reliability, sampling and design.
Why you’re reading it. Make the quantity behind the comparison explicit.
A question to consider. Two schools have equal raw gains but different standardized gains; what could explain the difference, and which comparison answers the school’s decision?
Why this reading
An effect expressed in standard-deviation units can look comparable across studies while concealing differences in measurement, sampling and design. Baguley examines what is lost or gained when researchers replace the original units.
Before you read
A raw mean difference retains the outcome’s units. A standardized difference divides by a measure of variability. A correlation also depends on variation in the variables. Reliability and range restriction can therefore affect comparisons of standardized estimates.
Read for
- Track which quantity changes when measurement reliability, sample variability or design changes.
- Compare the reading-time and range-restriction examples with the situations in which the author finds standardization useful.
Consider afterwards
- Two classrooms gain the same number of test points but have different score variability. What would a comparison of standardized gains add, and what might it hide?
Editorial context
This is an argument for choosing an effect measure to fit a question. Its preference for original units has exceptions, including some comparisons across instruments. Raw units do not repair confounding, a misspecified model or an invalid measure. The paper’s reporting recommendations invite scrutiny rather than automatic adoption.