Edexcel Syllabus focus:
'Evaluate the design of studies used to determine health risk factors, including sample selection and sample size for valid, reliable data.'
When biologists assess links between lifestyle and disease, the quality of the study design matters as much as the results. Poor sampling can make a real risk appear weaker, stronger, or completely misleading.
Why study design matters
Studies of health risk factors aim to find out whether a factor such as smoking, diet, or inactivity is associated with disease. To judge whether the evidence is strong, you must look beyond the headline result and examine how the study was designed.
A well-designed study should produce valid data.
Validity: The extent to which a study measures what it is intended to measure and supports a sound conclusion.
If the same method gives similar results when repeated, the data are more reliable.
Reliability: The extent to which repeated measurements or repeat studies give similar results.
In health research, validity and reliability are strongly affected by sample selection and sample size.
Sample selection
Choosing a representative sample
A sample is the group of people actually studied. It should represent the wider population that the researchers want to draw conclusions about.
Representative sample: A sample with characteristics similar to those of the population being studied, so that conclusions are more likely to be valid.
If a study on cardiovascular risk only includes middle-aged men, its findings may not apply well to women, younger adults, or older people. A representative sample should consider factors such as:
age
sex
ethnicity
socioeconomic background
existing health conditions
lifestyle differences
A sample that reflects these variables gives stronger evidence because the results are less likely to be distorted by one unusual group.
Avoiding sampling bias
Researchers must also avoid bias in sample selection. Bias happens when the method of choosing participants makes some outcomes more likely than others.
For example, a study that recruits volunteers from a fitness club is likely to underestimate the effect of inactivity on disease risk. This is because the participants are already more active than average.
Common sources of bias include:
volunteer bias: people who choose to take part may differ from the general population
selection bias: some groups are more likely to be included than others
non-response bias: people who do not reply may have different risk profiles from those who do
survivor bias: severely affected individuals may already have died or dropped out, making the risk seem smaller
Random sampling can reduce some bias, because every member of the target population has an equal chance of selection.
Stratified sampling splits the target population into distinct subgroups (strata) such as age bands or sexes, then draws a random sample from each stratum. This helps prevent important groups from being under-represented, improving the validity of conclusions when risk factors vary across subpopulations. Source
In some studies, stratified sampling is better, because it ensures important subgroups are included in suitable proportions.
Sample size
Why larger samples are usually better
A small sample may give misleading results because chance variation has a bigger effect. One or two unusual participants can strongly affect the average.
Larger sample sizes improve studies because they:
reduce the effect of anomalies
make estimates more precise
increase confidence that patterns are real
allow clearer comparisons between groups
make statistical tests more meaningful
For example, if researchers compare disease rates in 12 smokers and 12 non-smokers, the result may be strongly affected by unrelated differences between individuals. With hundreds or thousands of participants, these random differences are less influential.
Large samples do not fix poor design
A large sample does not automatically make a study good. If the sample is biased, increasing the number of participants may simply produce a very precise but still misleading result.
For instance, a large study based only on self-selected online respondents may still be unrepresentative. Sample size improves reliability, but validity depends on whether the right people were chosen in the first place.
Judging valid and reliable evidence
Features of stronger studies
When evaluating studies of health risk factors, look for design features that strengthen the evidence:
a large sample size
a representative sample
clear selection criteria
a suitable comparison group
methods that reduce bias
repeated measurements or repeated studies
control of important confounding variables
A confounding variable is a factor other than the one being studied that could affect the outcome.
A confounder is a third variable that influences both the exposure (risk factor) and the outcome, creating an apparent association even if the exposure is not truly causal. Seeing the arrows makes it clearer why study design or analysis must control for confounders to make conclusions more valid. Source
For example, if people with high-fat diets also tend to exercise less, it may be difficult to separate the effect of diet from the effect of inactivity.
Researchers can improve validity by matching groups or adjusting for confounding variables during analysis. This does not remove all uncertainty, but it makes the conclusions stronger.
Common weaknesses
Studies are weaker when they have:
very small sample sizes
narrow or unrepresentative samples
poorly matched comparison groups
high dropout rates
vague methods for measuring exposure or disease
reliance on inaccurate self-report data
Self-reporting can be a major problem in health-risk research. Participants may forget, exaggerate, or underestimate behaviors such as alcohol intake, exercise, or cigarette use. This lowers validity even if the sample size is large.
How to evaluate a study in an exam
When given a study, ask yourself:
Who was sampled?
Was the sample representative of the target population?
Was the sample size large enough to reduce chance effects?
Could bias have affected who took part?
Were there confounding variables?
Can the results be generalized?
Would repeating the study likely give similar results?
A strong evaluation does not just say that a study is “good” or “bad.” It explains why the sample selection and sample size increase or reduce confidence in the findings. In Edexcel Biology, the key idea is that claims about health risk factors are only convincing when they come from studies designed to produce valid, reliable data.
Practice Questions
Explain why sample selection is important when investigating a possible health risk factor. (2 marks)
The sample should be representative of the population being studied. (1)
Poor sample selection can introduce bias, making the results less valid or less applicable to the wider population. (1)
A scientist investigates whether a high-salt diet increases the risk of cardiovascular disease. The study uses 40 volunteers from one gym and follows them for 6 months. Evaluate the design of this study. (5 marks)
Sample size is small, so chance variation may have a large effect. (1)
Participants from one gym are unlikely to be representative of the wider population. (1)
Volunteers may introduce volunteer bias. (1)
People who attend a gym may differ in exercise levels or general health, so confounding variables may affect the results. (1)
A longer study and/or a larger, more representative sample would improve validity and reliability. (1)
FAQ
A very large study can still be misleading if the sample is biased.
For example:
the participants may all come from one region
they may be volunteers with unusual lifestyles
important groups may be missing
In that case, the study may give a very precise estimate of the wrong relationship. Large size mainly improves reliability, not validity.
Attrition means participants drop out before the study ends.
This matters because:
the final sample becomes smaller
the people who remain may differ from those who leave
the results may become less representative over time
If dropout is linked to health status or lifestyle, the study’s conclusions about risk factors can become biased.
Matching means choosing participants so groups are similar in factors such as age, sex, or smoking history.
This helps because it reduces the effect of confounding variables. If groups are more alike, differences in disease risk are more likely to be linked to the factor being investigated rather than another variable.
Matching improves study quality, but it cannot control every possible difference.
One study may be affected by chance, unusual sampling, or unnoticed bias.
If several studies in different populations show similar patterns, confidence increases because:
the finding is more likely to be reliable
it is less likely to depend on one unusual sample
the result may be more generalizable
Scientists usually trust a body of evidence more than a single investigation.
A study’s findings may not transfer well if the new population differs in important ways.
Differences may include:
genetics
diet
healthcare access
age structure
environmental exposure
This is why researchers must be careful when generalizing conclusions. A valid result in one population is not automatically valid everywhere.
