How random samples can transform almost any population into a remarkably familiar bell curve.
Imagine a population of observations that is not Normal at all. It might be heavily skewed, irregular or even have several different peaks.
Now repeatedly take random samples from that population and calculate the mean of each sample.
Something remarkable happens:
This is the Central Limit Theorem.
The original population does not need to be Normally distributed. It is the distribution of the sample means that becomes approximately Normal as the sample size increases.
To see the theorem properly, let's start with a population that is clearly not a bell curve.
Here we use a deliberately skewed population. Most observations are relatively small, but a smaller number of observations extend much further to the right.
This is not a Normal distribution. It is strongly right-skewed.
Now let's repeatedly sample from it.
Let's randomly select observations from our population.
The sample mean will vary depending on which observations happen to be selected.
The Central Limit Theorem becomes visible when we repeat this process many times.
Now instead of taking one sample, let's take thousands.
For every sample we calculate its mean and record the result.
The original population was strongly skewed. Yet the distribution of the sample means is becoming much more symmetrical and bell-shaped.
The size of each sample matters.
Try changing the sample size and generating the sample means again.
With very small samples, the distribution can still look irregular.
As the sample size grows, the distribution of sample means becomes increasingly smooth and bell-shaped.
Consider a population containing both unusually high and unusually low observations.
When we calculate an average, these extremes tend to partially cancel each other out.
A single unusually large observation can have a substantial effect on one sample. But across many observations, its effect on the average becomes smaller.
Randomness does not disappear. Instead, averaging tends to produce a much more predictable distribution.
There is another remarkable feature of the Central Limit Theorem.
As the sample size increases, the sample means become less variable.
The standard deviation of the distribution of sample means is called the standard error.
So if we quadruple the sample size, the standard error falls by half.
Standard deviation describes the spread of individual observations.
Standard error describes the spread of sample means.
The theorem is enormously useful because we often care about averages rather than individual observations.
Rather than needing to understand every possible individual outcome, we can often make useful statements about the behaviour of averages.
Financial returns are noisy and unpredictable from one day to the next.
But investors are often interested in what happens over longer periods.
For example, instead of looking at one daily return, we might look at the average daily return over 20, 50 or 250 trading days.
The Central Limit Theorem helps explain why the behaviour of these averages can be much more regular than the individual observations from which they were created.
The Central Limit Theorem does not magically make financial returns themselves Normally distributed.
That distinction becomes very important when we start modelling investment returns.
We have now built a chain of ideas.
But what if instead of simply observing random samples, we deliberately generate thousands or millions of them?
We can use random numbers to create possible futures and examine the range of outcomes that results.
That is the basic idea behind Monte Carlo simulation.
We can simulate uncertainty, repeat the experiment many thousands of times and build a distribution of possible outcomes.