AP Syllabus focus:
‘Essential Knowledge UNC-2.A.6: Discuss the law of large numbers and its implication that as the number of trials increases, simulated (empirical) probabilities tend to approach the true probability. Emphasize the importance of this law in the context of simulation and probability estimation.’
The law of large numbers is a foundational idea in probability that explains why long-run patterns stabilize, allowing simulations and empirical data to approximate true underlying probabilities reliably.
Understanding the Law of Large Numbers
The law of large numbers describes how the behavior of a random process becomes more predictable when repeated many times. A random process is any situation in which outcomes are determined by chance but follow an underlying probability structure. The key idea in this subsubtopic is that empirical probability—the relative frequency of an event observed in data—gets closer to the true probability as the number of trials increases.
Empirical Probability: The long-run relative frequency of an outcome observed in repeated trials of a random process.
This principle helps justify why simulations are powerful tools in statistics: simulated outcomes mimic real-world randomness, and increasing the number of simulated trials yields increasingly accurate probability estimates.
Why More Trials Lead to Greater Accuracy
When only a small number of trials are conducted, random fluctuations can cause outcomes to vary widely from the true probability. As trials accumulate, these fluctuations even out. This occurs because each additional trial adds information that reduces the influence of early deviations.
As the number of trials increases, the long-run relative frequency of an outcome tends to get closer to the true probability of that outcome.
A visualization of the law of large numbers showing how running statistics stabilize as the number of trials increases, illustrating the convergence of empirical results toward the true probability. Source.
True Probability: The actual, theoretical probability of an event based on the underlying structure of the random process.
The stabilization of empirical probabilities as trials increase ensures that randomness becomes more manageable at large scales, reinforcing one of the key goals of simulation: producing results that closely resemble actual long-run behavior.
Long-Run Relative Frequency and Convergence
The phrase long-run relative frequency refers to the proportion of trials that result in a particular event over a large number of repetitions. The law of large numbers states that this proportion converges to the true probability. This convergence does not guarantee short-term accuracy, nor does it predict patterns in small samples. Instead, it describes a long-term trend that emerges gradually.
EQUATION
= Count of event occurrences
= Number of repeated attempts in the random process
This expression highlights the mechanism by which empirical probabilities are computed and how repeated observations refine these estimates.
A common misconception is that the law of large numbers ensures outcomes “even out” in the short term or that past outcomes influence future ones. The principle does not imply that rare events become more likely simply because they have not occurred recently. Instead, it focuses purely on long-run stabilization.
Implications for Simulation in AP Statistics
Simulations are essential tools for estimating probabilities, particularly when theoretical calculations are complex. The law of large numbers provides the justification for using simulation to approximate probability values.
Key reasons the law of large numbers strengthens simulation-based reasoning:
Improved accuracy: Larger numbers of simulated trials produce results that more closely reflect true underlying probabilities.
Stability of estimates: As the number of trials increases, the variability of relative frequencies decreases.
Reliability in modeling: Simulation outputs become more trustworthy as long-run patterns emerge.
Connection to real-world behavior: Many real processes exhibit randomness, and the law explains why large datasets reveal consistent patterns.
These implications underscore why the law of large numbers is not merely theoretical but central to statistical practice.
Interpreting Convergence in Practical Terms
Even though convergence to the true probability occurs as trials increase, the rate of convergence can vary depending on the random process. Some processes stabilize quickly, while others fluctuate for longer before settling toward the true value. Despite this variability, the direction of convergence—toward the true probability—is consistent across all properly defined random processes.
The law of large numbers explains why sample means from a random process become more stable and closer to the true population mean as the sample size increases.
Running sample means of IQ scores converge toward the population mean as the number of observations increases, illustrating long-run stabilization predicted by the law of large numbers. Source.
Convergence (in Probability): The tendency of an empirical measure, such as relative frequency, to get arbitrarily close to a theoretical value as the number of trials increases.
Between trial-to-trial variation and long-term stabilization lies the central logic of probability estimation. The more data collected from repeated random processes, the more confidence statisticians can place in probability estimates derived from those data.
Graphs of the cumulative proportion from simulations, such as repeated coin tosses, clearly show how early values can wander but eventually cluster near the theoretical probability as the number of trials grows.
Running proportions of heads in repeated coin tosses converge toward the true probability of 0.5, visually demonstrating the stabilization of empirical probabilities with increasing trials. Source.
Importance of the Law in Probability Estimation
The law of large numbers ensures that probability estimates become more precise with larger datasets, forming the basis for much of statistical inference. Without this principle, empirical data would not reliably reveal underlying patterns, and the use of simulation to estimate probabilities would lack justification.
Its importance can be summarized through:
Consistency in measurement: Long-run frequencies reflect true probabilities.
Support for statistical inference: Sampling-based methods rely on large-sample behavior.
Predictive value: Stable long-run behavior allows for dependable predictions in uncertain contexts.
Foundation for modeling and simulation: Validates the use of repeated random trials to estimate probabilities.
FAQ
The law of large numbers applies only to long-run behaviour and does not imply that outcomes should correct themselves immediately after an unusual sequence. Short-term fluctuations are expected and do not indicate that opposite outcomes are “due”.
In contrast, long-run convergence refers to gradual stabilisation over many repetitions, not immediate balancing. This distinction helps prevent the gambler’s fallacy, where people mistakenly expect short-term compensation for randomness.
Yes. Independence ensures that the outcome of one trial does not influence the next, allowing long-run patterns to emerge consistently.
If trials are not independent, the cumulative proportion may drift or show structural patterns unrelated to the true probability. The principle also assumes identical conditions, meaning the probability of the event must remain the same across trials.
The speed of stabilisation depends on the underlying distribution and variability of the random process. Processes with low variability tend to converge faster because each outcome is closer to the expected behaviour.
High-variability processes, such as those with rare outcomes or extreme values, may need many more repetitions before the cumulative proportion stabilises.
Yes, provided the process is stable over time and meets the assumptions of identical conditions and independence.
In practice, analysts use long-run recorded data to make probability estimates when theoretical probabilities are unknown. This is common in fields such as reliability testing, sports analytics, and quality control.
Both versions describe convergence of empirical measures toward true values, but they differ in how strictly they define convergence.
• The weak form states that convergence happens in probability.
• The strong form states that convergence happens almost surely, meaning with probability 1.
For AP Statistics, only the intuitive long-run stabilisation idea is required, not the formal distinction.
Practice Questions
Question 1 (1–3 marks)
A student runs a simulation of a random process where the true probability of an event is 0.3. After 10 trials, the empirical probability is 0.5. After 1,000 trials, the empirical probability has moved closer to 0.3.
Explain, using the law of large numbers, why the empirical probability changes in this way.
Question 1
• 1 mark: States that the empirical probability can differ significantly from the true probability with few trials.
• 1 mark: States that as the number of trials increases, the empirical probability tends to move closer to the true probability.
• 1 mark: Explicitly references the law of large numbers as the reason for this convergence.
Question 2 (4–6 marks)
A researcher is investigating a random process in which the true probability of success is unknown. They run a computer simulation and record the cumulative proportion of successes after 20, 100, 500, and 5,000 trials. They observe the following pattern:
• The cumulative proportion fluctuates widely at 20 trials.
• It becomes more stable by 100 trials.
• At 500 and 5,000 trials, the cumulative proportion appears to level off near 0.42.
(a) Explain how the law of large numbers accounts for the decreasing variability in the cumulative proportion as the number of trials increases.
(b) Using the law of large numbers, state what the researcher can reasonably conclude about the true probability of success and justify your conclusion.
Question 2
(a)
• 1 mark: States that small samples produce high variability in the cumulative proportion.
• 1 mark: States that larger samples reduce variability in the cumulative proportion.
• 1 mark: Correctly explains that the law of large numbers predicts convergence of empirical proportions to the true probability as the number of trials increases.
(b)
• 1 mark: States that the cumulative proportion stabilising near 0.42 suggests the true probability is approximately 0.42.
• 1 mark: Justifies this by noting that long-run empirical frequencies approach the true probability according to the law of large numbers.
• 1 mark: Provides a clear concluding statement that the true probability is likely close to 0.42 based on the observed long-run pattern.
