Reading focus
Syllabus: How Do We Know What a Psychological Test Measures?
Use the six measurement questions to trace how a construct becomes an operational procedure and a reported score.
Why you’re reading it. Establish the questions that the rest of the syllabus will deepen. Readers returning from Start Here can apply the same framework to a different measurement problem.
A question to consider. For the proposed Persistence Scale, which decisions must be justified before a correlation with course completion can be interpreted?
Why this reading
A precise statistical result can still leave it unclear what was measured. Flake and Fried turn that problem into questions readers can ask about construct definitions, instruments, scoring, and changes to a measure. The article shows why these choices belong in an evaluation of the claim itself, rather than being treated as minor details in the methods section.
Before you read
A construct is the attribute or phenomenon a researcher means to study, such as anxiety. Operationalization specifies how it will be observed or scored. Reliability concerns consistency of measurement; validity concerns the evidence supporting an interpretation and use of the resulting scores. A consistent measure can still fail to support its intended interpretation. Transparency makes decisions inspectable, but disclosure alone does not make a measure valid.
Read for
- Use Table 1 to connect each of the six measurement questions to information a reader would need in a research report.
- Follow the distinction between defining a construct and choosing a measure that represents it.
- Trace how choices about scoring, modifying an existing scale, or creating a new one could affect the conclusion.
- Distinguish missing information from evidence that a measure performs poorly. Neither automatically establishes a researcher's intent.
- Ask why a successful replication would still leave unanswered questions if both studies used an inadequately justified measure.
Consider afterwards
- Imagine that two studies both claim to measure well-being, but one asks about life satisfaction and the other about recent pleasant feelings. What would you need before combining their results?
- What evidence could change your interpretation of a questionnaire score even if its reliability were high?
Editorial context
The article is an argument for scrutinizing and reporting measurement decisions. Its authors explicitly do not offer a comprehensive tutorial on scale development or validity theory, and focus on measurement that produces numerical variables. The six questions help expose missing justification; answering them transparently is a starting point for evaluation, not a guarantee of validity. For historical depth on the relation between constructs, observations, and theory, read Cronbach and Meehl's construct-validity argument.