The Measurement Challenge
Human behaviour can be difficult to observe and measure objectively.
Measurement varies according to the context and the theory underlying its use.
Psychologists must select methods that collect data relevant to the behaviour being studied.
Different forms of evidence have different strengths and limitations.
Reliability, Validity and Credibility
Reliability concerns whether measurement produces sufficiently consistent results.
Validity concerns whether a measure represents the behaviour or construct it is intended to measure.
Credibility concerns how convincing the findings are given the evidence and methods used.
Triangulation of methods uses more than one way of investigating or measuring an issue.
Triangulation can strengthen credibility when different methods provide compatible evidence.
Forms of Data and Evidence
Form | What is measured or collected | Measurement feature |
|---|---|---|
Quantitative data | Numerical information | Can involve large datasets and statistical analysis |
Qualitative data | Non-numerical information | Can be examined using interpretive approaches |
Mixed data | Quantitative and qualitative evidence | Combines different forms of evidence |
Self-reported data | Information reported by participants | Depends on participants reporting their responses |
Empirical data | Evidence collected through research | Produced through observation or measurement |
Evaluating Measurement Choices
Identify what was measured, how it was operationalized and why the chosen method suits the behaviour studied.
Consider whether measurement is direct or indirect and whether the evidence is quantitative, qualitative or mixed.
Evaluate reliability by considering the consistency of measurement.
Evaluate validity by considering whether the measure represents the intended behaviour or construct.
Use triangulation where relevant when discussing the credibility of findings.
Measurement Across Time
Research design | Measurement timing | Key distinction |
|---|---|---|
Prospective | Collects information moving forward from a starting point | Measures later information as it occurs |
Retrospective | Uses information about the past | Looks backward from the present |
Longitudinal | Measures across an extended period | Tracks behaviour or variables over time |
Cross-sectional | Measures at one point in time | Does not track the same measurements longitudinally |
Statistical Significance and Errors
Statistical significance evaluates whether an observed quantitative result would be unlikely if the null hypothesis were true.
A Type I error occurs when a true null hypothesis is incorrectly rejected.
A Type II error occurs when a false null hypothesis is not rejected.
These errors show that conclusions from statistical analysis can sometimes be incorrect.
Statistical significance should therefore be interpreted alongside the quality of the study's measurement and evidence.
Checklist: can you do this?
Can you explain why human behaviour can be difficult to measure objectively?
Can you explain how context and theory influence measurement choices?
Can you distinguish constructs, variables and operationalization?
Can you explain how operationalization relates to reliability and validity?
Can you distinguish direct and indirect measurement using the brain-imaging example?
Can you distinguish quantitative, qualitative, mixed and self-reported data?
Can you explain statistical significance, Type I errors and Type II errors?
Can you distinguish prospective, retrospective, longitudinal and cross-sectional research designs?
Constructs, Variables and Operationalization
A construct is a psychological idea that researchers seek to represent through measurable variables.
Operationalization specifies how a variable will be observed or measured in a study.
Clear operationalization supports reliable measurement by making the measurement procedure consistent.
It also supports valid representation by linking the measure to the behaviour or construct being investigated.
Always identify exactly what was measured and how it was measured when evaluating research.
Direct and Indirect Measurement
Direct measurement records the behaviour or response being studied itself.
Indirect measurement uses another observable indicator to infer a process or variable that cannot be observed directly.
The syllabus example infers the role of a neurotransmitter from brain activity measured using brain-imaging technology such as MRI.
Indirect evidence requires careful interpretation because the indicator is not identical to the underlying process.
Measurement Techniques
Psychologists use techniques including brain imaging, twin studies, virtual reality simulations and questionnaires.
The appropriate technique depends on the behaviour, context, theory and type of data required.
Brain imaging techniques can provide indirect measurements relevant to behaviour.
Questionnaires can collect self-reported data from participants.
Some investigations collect large amounts of quantitative data for statistical analysis; others use qualitative or mixed evidence.
This fMRI image illustrates how brain activity can be represented as patterns across brain regions. Such imaging provides an indirect measure relevant to understanding behaviour. Source
Interpretive Approaches and Content Analysis
Interpretive approaches are relevant when measurement involves analysing meaning in qualitative or textual evidence.
Content analysis provides a way of identifying and organizing information contained within material being studied.
Measurement choices should remain clearly connected to the behaviour or construct being investigated.
Interpretive evidence can also be considered in relation to credibility, reliability and validity where appropriate.