How Do We Describe and Measure Individual Differences?

Introduction

We readily describe one person as quick to learn, another as dependable, and another as outgoing. Such judgments guide friendships, teaching, hiring, and care, yet they often rest on little more than remembered episodes and an impression of how the person compares with others. Individual-differences research asks what it would take to make such descriptions dependable: what should be observed, how observations become scores, and when a score tells us about the person rather than about the occasion, the instrument, or the observer.

This syllabus follows that question through the making of measures. It begins with what the study of individual differences sets out to do, and with cognitive ability, a domain with a long tradition of measurement. It then follows two instruments as they are built: a personality questionnaire, whose items must capture a defined attribute without being swayed by how people tend to answer, and a test of reasoning ability, whose items must separate people across a wide range of performance.

Along the way, the readings use factor analysis, a family of methods that represents the correlations among many items or tests through a smaller number of underlying dimensions, called factors. They also examine reliability, the consistency with which a measure distinguishes people. Finding a clear structure is useful, but it does not by itself reveal what causes that structure.

The last two units ask how far a score can be trusted beyond the setting in which it was developed. Do a person’s own ratings agree with those of people who know them, and with what they are observed to do? Can scores be compared across languages, methods of administration, and populations? These questions connect the craft of building a measure with the evidence needed to interpret it.

Throughout the syllabus, imagine a research team studying reasoning ability and conscientiousness—the tendency to be organized, responsible, and hardworking—among adult learners. The team can use cognitive tasks, questionnaires, and ratings from people who know the learners. Each unit asks what this evidence would warrant saying, and what would remain uncertain. By the end, you should be able to explain how a construct becomes items and scores, judge evidence for a measure’s structure and reliability, weigh self-reports against other sources, and recognize when a comparison between groups needs further support.

Scope and omissions

This introductory path covers what individual-differences research sets out to do and how its measures are made and checked: construct definition, item development, reliability, factor structure, agreement between sources, and comparability across groups. Intelligence and personality supply the main examples through a cognitive-ability primer, the construction of a personality questionnaire and a public-domain reasoning test, a study of self- and informant ratings, and a test of personality measurement across settings.

Several readings are technically demanding, especially the two measurement tutorials and the invariance review. The guides define the central terms and focus on interpretation; the path does not teach readers to fit factor or item response models, run statistical code, or administer tests. A statistical model of variation across people is also not automatically an account of the processes operating within one person.

This is not a comprehensive theory of intelligence or personality, a history of testing, an account of genetic and environmental causes, or a guide to consequential assessment decisions. How Do We Know What a Psychological Test Measures? develops validity theory, the multitrait–multimethod framework, and the reliability of experimental tasks in greater depth, and What Is a Mental Disorder? asks whether some differences are better described as dimensions or categories. Interests, motives, values, and identity also extend beyond the two principal examples. The optional readings offer a longer history of the field, a broader treatment of reliability, a model of what selves and others know, an alternative to the Big Five, and an account of why self-report and behavioral measures often disagree.

Synthesis

Describing an individual difference well means building and testing an argument. The argument starts with a defined attribute, proceeds through items or tasks chosen to cover it, and gathers evidence that the scores distinguish people consistently, show the structure the definition implies, agree with other sources, and keep their meaning in the settings where they are used. Reliability and factor structure are necessary parts of that argument, not substitutes for it.

The readings also show that the argument is never finished. The BFI-2 and the ICAR were built with care and tested in large samples, yet their development samples leave open how they perform with other populations and methods of administration. Self-reports and informant reports each capture something the other misses. A familiar label guarantees neither the same meaning elsewhere nor freedom from response styles. A person’s standing on a score remains conditional on the evidence supporting that score in that setting.

Return to the adult-learner project. Write a short measurement argument for reasoning ability and another for conscientiousness. For each, state the construct and its boundaries; the items or tasks, with the evidence for their structure and reliability; a second source of evidence and what disagreement with it would mean; and the comparison across groups or settings that still needs support. Name one conclusion the project could defend and one it could not yet defend.

For deeper treatment of validity theory, the multitrait–multimethod framework, and the reliability of experimental tasks, continue with How Do We Know What a Psychological Test Measures?. Readers who have completed that path can use the questionnaire and ability-test examples here to revisit its competing accounts of what makes a measure valid.

To examine cognitive abilities in greater depth, continue with What Is Intelligence?. It takes the distinction between describing differences and explaining them into debates about general intelligence, cognitive processes, brain systems, broader definitions, and changes in performance. How Should We Understand Personality? asks what trait descriptions such as the Big Five leave out of an account of a person.