Overview of the Central Limit Theorem
Important facts about the distribution of possible sample means are summarized in the Central Limit Theorem, which can be stated as follows:
If a random sample of N cases is drawn from a population with mean and standard deviation , then the sampling distribution of the mean (the distribution of all possible means for samples of size N)
1) has a mean equal to the population mean x:
2) has a standard deviation (also called "standard error" or "standard error of the mean") equal to the population standard deviation,x divided by the square root of the sample size, N:
3) and the shape of the sampling distribution of the mean approaches normal as N increases.
This last point is especially important: The shape of the sampling distribution approaches normal as the size of the sample increases, whatever the shape of the population distribution.
The facts represented in the Central Limit Theorem allow us to determine the likely accuracy of a sample mean, but only if the sampling distribution of the mean is approximately normal.
If the population distribution is normal, then the sampling distribution of the mean will be normal for any sample size N (even N = 1). If a population distribution is not normal, but it has a bump in the middle and no extreme scores and no strong skew, then a sample of even modest size (e.g., N = 30) will have a sampling distribution of the mean that is very close to normal. However, if the population distribution is far from normal (e.g., extreme outliers or strong skew), then to produce a sampling distribution of the mean that is close to normal it may be necessary to draw a very large sample (e.g., N = 500 or more).
Important note: You should not assume that the sampling distribution of the mean is normal without considering the shape of the population distribution and the size of your sample. A sample with N > 30 does not guarantee a normal sampling distribution if the population distribution is far from normal.
If you wish, you may review a few terms related to the Central Limit Theorem before continuing on to the activities and questions. Or you may skip the review and go directly to Question 1.
- Mean
The mean is the most common indicator of an 'average' score. It is computed by dividing the sum of all scores by the number of scores. If the distribution has extreme outliers and/or skew, the mean may not be very descriptive of a 'typical' score. More | Formula
- Standard deviation
The standard deviation is a common measure of variation of scores. The standard deviation is computed by taking the square root of the variance. The larger the standard deviation (and variance), the wider the distribution and the further the scores are from the mean. Like the mean, the standard deviation is sensitive to outlying scores. Formulas
- Variance
Variance is a measure of how much scores in a distribution vary from the mean. Mathematically, variance is the average of the squared deviations from the mean. Taking the square root of variance results in the standard deviation. Formulas
- Population versus sample
A population consists of all cases in the group of interest. A sample is a group of cases selected from all possible cases in the population. For example, if the group of interest is American working women, the population would include each and every working woman in America. Usually it is impossible to collect data on an entire population. Instead, we use one of many sampling techniques to select a subgroup from the population. This subgroup is a sample.
- Sample size
A sample is a subset taken from the population of interest. The number of sampled cases is called the sample size.
- Sampling distribution of the mean
The sampling distribution of the mean is a theoretical distribution. If you were to draw an infinite number of samples with a particular sample size from a population you would get an infinite number of sample means (one for each sample you drew). The distribution of these means is the sampling distribution of means for your population at that particular sample size. More
- Normal distribution
The normal curve (also called the "Bell curve" or "the Gaussian distribution") is a theoretical distribution mathematically defined by its mean and variance. When graphed the normal distribution has a shape similar to a bell curve (see Figure below). Naturally occurring distributions are rarely normal in shape. However, the distributions of many chance events do approach normal shape. Importantly, the distribution of possible means for a randomly selected sample is approximately normal if the sample is sufficiently large. The area under the curve for a standardized normal curve is exactly 1.00 or 100%, which is useful for finding probabilities.
Figure. Three normal distributions whose means and standard deviations vary.
Question 1
Answer questions by selecting a response and then clicking "Check answer". You will be given feedback (you may have to scroll down to view it). If your response is incorrect, select a different response, using the feedback to guide you toward the correct one.
Ourtown Health Department reported that the height of women in the city is approximately normally distributed with a mean of 5 feet, 4 inches (i.e., 64 inches) and a standard deviation of 3 inches.
Suppose we select a random sample of five women from our school, measure the height of each, and calculate the sample mean. If we wished to know whether the height of women at our school is typical of the height of women in Ourtown, how should we compare our sample data to information we have about the Ourtown population distribution?
We should compare the sample mean to the population distribution of Ourtown women as we do with an individual score.
We should compare each individual score in our sample, one at a time, to the population of individual scores.
There really isn't a way to make any worthwhile comparison.
We should compare this sample mean to a sampling distribution of all possible means for samples of 5 women from the population of women in Ourtown.
Correct! We have to compare our sample mean with a distribution that is made up of all possible sample means (a sampling distribution of means). We want to compare our mean to a distribution of all possible sample means drawn from the population of interest. To compare our sample mean with a distribution of individual scores would be comparing apples to pickles. EXCELLENT!
Question 2
Calculate the approximate proportion of Ourtown women who are 5 feet, 7 inches (i.e., 67 inches) tall or taller.
Hint: Recall that the height of women in Ourtown is approximately normally distributed with a mean of 5 feet, 4 inches (i.e., 64 inches) and a standard deviation of 3 inches. Which answer is correct?
0%
2%
14%
16%
Correct! You answered approximately 16% of the women in Ourtown are 5 feet, 7 inches tall or taller. The z-score that corresponds to 5 feet, 7 inches is +1.00. The proportion of the area under the normal curve that falls beyond this z-score is .1587 or approximately 16% of the population. GREAT WORK!
Question 3
In what way does the sampling distribution of means for a sample with N = 5 differ from a sampling distribution for a sample with N = 100? How do these sampling distributions compare to the population distribution?
There is more variance in the sampling distribution with N = 5 than when N = 100, and both sampling distributions have more variance than the population.
There is less variance in the sampling distribution with N = 5 than when N = 100, and the sampling distribution with N = 100 has a variance similar to the population.
The variance is much smaller for the N = 100 sampling distribution than for the N = 5, and both sampling distributions have less variance than the population.
You're right! We can use the formula from the Central Limit Theorem to show that the sampling distribution with N = 100 has less variance (dispersion) than the sampling distribution with N = 5, and both sampling distributions have lessvariance than the population.
Question 4
Recall that the population mean for the height of Ourtown women is 5 feet, 4 inches (64 inches), the population standard deviation is 3 inches, and the distribution is normal. Now, try ordering the likelihood of the following four instances from the most likely to least likely. It may be helpful to draw pictures.
- You select one woman from Ourtown at random and she is 5 feet, 4 inches (64 inches) tall to the nearest inch.
- You select one woman from Ourtown at random and she is 6 feet, 1 inch (73 inches) tall to the nearest inch.
- You select a random sample of 10 women from Ourtown, and the mean of their heights, to the nearest inch, is 5 feet, 4 inches.
- You select a random sample of 10 women from Ourtown, and the mean of their heights, to the nearest inch, is 6 feet, 1 inch.
Which of the following is the correct order? Why? Consider the reasons before you check the answer.
1, 3, 2, 4
3, 1, 2, 4
1, 3, 4, 2
3, 1, 4, 2
Excellent answer! The population mean of 5 feet, 4 inches is the most likely height for a randomly selected individual woman or for a sample mean. The mean for a sample with N = 10 has much smaller variability than an individual observation (N = 1), so the mean for a sample of N = 10 is more likely to be within an inch of the population mean.
On the other hand, it is extremely unlikely that you will select a sample of 10 women with a mean of 6 feet, 1 inch. In order to get such a large mean, your sample would include many unusually tall women. Since the likelihood is low of selecting even one woman this tall, it is very unlikely you will find many unusually tall women in your sample. If any women in the sample were of average height or below, you would need women that were even taller than 6 feet, 1 inch in your sample to obtain a sample mean of 6 feet, 1 inch. This being the case, it would be very unlikely to select a random sample with such an extreme mean. Selecting a single case this tall would be more likely.
More on the Central Limit Theorem
The previous questions lead us to two important implications of the Central Limit Theorem. Compared to its parent (population) distribution, a sampling distribution has:
- the same mean, but
- a smaller variance.
Since samples contain many scores, extreme scores tend to cancel each other out and leave you with a pretty middle-of-the-road mean. So, when all these means are put in a sampling distribution there are very few means out in the extremes - the means tend to pile up around the middle. This is especially true when the sample size is large.
Below we've superimposed the sampling distribution of means for N = 81 over the population distribution. As you can see, the sampling distribution of the means is very skinny and very tall compared to the population distribution. Notice the relative probability of an observation of 5' 7" in each of these distributions. If we selected one woman at random from the population (where SD = 3), we would not be very surprised if her height was 5' 7". That height is only one standard deviation above the mean, so it is not very unusual.
However, if we took a random sample of N = 81 women and found the average height to be 5' 7", we would be very surprised. The distribution of means is much less variable, so a mean height of 5' 7" is extremely unlikely. A sample mean of 5' 7" represents a z-score of z = 9.00 (or a score that is nine standard deviations above the mean) on the sampling distribution of means, an extremely unlikely score on a normal distribution.
Figure. Sampling distribution of means, N = 81, superimposed over population distribution.
Question 5
We saw that the standard deviation of the sampling distribution is smaller when the sample size is larger. The Central Limit Theorem gives us an exact formula. The standard deviation of the sampling distribution of means equals the standard deviation of the population divided by the square root of the sample size. The standard deviation of the sampling distribution is called the "standard error of the mean."
The formula for the standard error of the mean, x, is
, where x is the standard deviation of the population and N is the sample size.
We can use this formula to calculate the standard error of any sampling distribution if we know the sample size and the population standard deviation. Imagine you draw a sample of 100 cases from a population with a mean of 57 and a standard deviation of 20.
What are the mean x and the standard error x for the sampling distribution?
x = 57, x = 20
x = 57, x = 200
x = 57, x = 4
x = 57, x = 2
Yes, excellent! The sampling distribution has less variance and is narrower with larger sample sizes. We divide the population standard deviation, x, by the square root of the sample size, N, to get the standard error of the mean x.
Question 6
Imagine that you select a sample of N = 25 women from Ourtown and find that the mean height of the sample is 5 feet, 7 inches (67 inches). The population mean is known to be 5 feet, 4 inches (64 inches) with SD = 3 inches. Would a sample mean this large be more likely or less likely if N = 5? Why?
Equally likely. Sample means are randomly distributed.
More likely. The sampling distribution of the mean would be wider if the sample were smaller, so sample means would be more likely to be farther from the population mean.
More likely. The sampling distribution is wider for larger samples resulting in more extreme obtained means.
Less likely. The mean of the sample will increase as the sample size increases.
Yes, excellent! The shape of the sampling distribution is narrower with larger sample sizes. You can observe that the curve gets thinner as the sample size increases. As a result, a sample mean score 3 inches larger than the population mean is farther in the tail of the sampling distribution (and less likely) when the sample size is larger.