12
Chapter 2 – Organization and Description of Data
When data are in their original form, as collected, they are called raw data. The first task to be done with raw data is clean-up. This is always done. The data must be double-checked to see that it was collected accurately. Any unusual data values should be followed up to see whether they resulted from errors in data collection or from unusual members of the sample. When the data is entered into a calculator or spreadsheet, it should be double-checked to see that it was entered correctly.
After the clean-up procedure, the next task is to describe the data. There two kinds of methods for summarizing and describing data – graphical techniques and numerical summaries. We will discuss some graphical techniques first.
With non-numeric data, we often want a graph which is a variation on the histogram, called a Pareto chart. This type of graph is useful in quality control and process improvement studies, in which the data often represent the different types of defects or failure modes. A Pareto chart graphs the frequencies of occurrences of the different types of defects, ordered from the most frequent to the least frequent. The purpose of a Pareto chart is to focus on the main causes or modes of failure.
Example: We have data, listed below, on number of accidents between 1959 and 1999 for each of a number of different types of aircraft, as well as the number of accidents per million flights.
Aircraft type / Actual no. ofhull losses / Hull losses/million
departures
MD-11 / 5 / 6.54
707/720 / 115 / 6.46
DC-8 / 71 / 5.84
F-28 / 32 / 3.94
BAC 1-11 / 22 / 2.64
DC-10 / 20 / 2.57
747-Early / 21 / 1.90
A310 / 4 / 1.40
A300-600 / 3 / 1.34
DC-9 / 75 / 1.29
A300-Early / 7 / 1.29
737-1 & 2 / 62 / 1.23
727 / 70 / 0.97
A310/319/321 / 7 / 0.96
F100 / 3 / 0.80
L1011 / 4 / 0.77
BAe 146 / 3 / 0.59
747-400 / 1 / 0.49
757 / 4 / 0.46
MD-80/90 / 10 / 0.43
767 / 3 / 0.41
737-3, 4 & 5 / 12 / 0.39
The Pareto chart is shown below. To construct the graph using Excel, we enter the data, with the categories listed in the first column, and the frequencies or relative frequencies listed in the second column. Highlight the data, and choose Insert, Chart, Column.
In this case, of the 22 types of aircraft, we see that the MD-11 had the highest accident rate, followed by the Boeing 707/720 and the DC-8. The latter two are no longer in service in most of the world. The years of service of the MD-11 were 1990 – 1999.
Frequency Distributions and Histograms
For numeric data, there are a number of different graphical techniques available. The author presents several, including the dot-plot. We will not include the dot-plot, as other types of graphs, such as histograms and stem-and-leaf plots, are equally useful.
Often, with univariate data (resulting from a single measured characteristic of a sample), there are too many different data values for a listing of the raw data to be useful in visualizing the characteristics of the data. It is common to divide the interval of values of the data into a relatively small number of subintervals, called classes, and to tabulate the data using the frequencies. Each frequency is the number of occurrences of data values within a subinterval. We sometimes want also to use relative frequencies. The relative frequency for a class is found by dividing the frequency for that class by the size of the entire data set.
Defn: A histogram is a graph that displays numeric data by using vertical bars of various heights to represent the frequencies of occurrence of data values within a subinterval.
Characteristics of a histogram:
1) The classes are listed in order along the horizontal axis.
2) The vertical axis provides a scale for the frequencies.
3) A bar is drawn for each class having width equal to the class width and height equal to the class frequency.
4) The axes are labeled and the graph is titled.
Note: The number of classes, or subintervals, depends on the size of the data set. A good rule of thumb is to choose 5 classes for a small data set (n = 25) and 20 classes with a large data set
(n = 1000).
Note: The class width is found by dividing the range of the data by the number of classes and rounding up slightly, so that the largest data value will be included in the last class.
The class limits are the uppermost and lowermost data values that could be included in the class (note that there may be no actual data values equal to the upper- or lower-class limit for any given class).
Since we may do the histogram with the calculator or with Excel, we do the histogram first, followed by the grouped frequency distribution.
Example: Compressive strength, in pounds per square inch (psi) of specimens of a new aluminum-lithium alloy undergoing evaluation for possible use in aircraft structural components. The data are listed in the following table.
105 221 183 186 121 181 180 143 97 154 153 174 120 168
167 141 245 228 174 199 181 158 176 110 163 131 154 115
160 208 158 133 207 180 190 193 194 133 156 123 134 178
76 167 184 135 229 146 218 157 101 171 165 172 158 169
199 151 142 163 145 171 148 158 160 175 149 87 160 237
150 135 196 201 200 176 150 170 118 149
We will construct a histogram for the data using Excel (Instructions for constructing a histogram using the TI-83/TI-84 are included in the calculator handout). We have a data set with n = 80. We will choose to use 7 classes. The range is 245 – 76 = 169. Therefore the class width will be
Class width=RangeNo. of classes↑=1697↑=24.142857↑=24.2.
The lower limit of the first class will be the smallest data value, 76 (the author sometimes chooses a different value for the lower class limit of the first class). To construct the histogram in Excel:
1) Enter the data.
2) Enter a second column giving the upper class limits for all classes except the last class – 100.2, 124.4, 148.6, 172.8, 197.0, 221.2.
3) Choose Tools, Data Analysis, Histogram.
4) The input range will be a1..a80. The bin range will be b1..b6.
5) The output range will be c1.
6) The type of output will be chart output.
Below is the resulting histogram, followed by the grouped frequency table, constructed using the information from the histogram (In the table, relative frequencies are included).
Class (psi) / Frequency / Relative Frequency76.0 – 100.2 / 3 / 0.0375 = 3.75%
100.3 – 124.4 / 8 / 0.1000 = 10.00%
124.5 – 148.6 / 12 / 0.1500 = 15.00%
148.7 – 172.8 / 28 / 0.3500 = 35.00%
172.9 – 197.0 / 17 / 0.2125 = 21.25%
197.1 – 221.2 / 8 / 0.1000 = 10.00%
221.3 – 245.4 / 4 / 0.0500 = 5.00%
Looking at a histogram of a data set can sometimes provide a quick way of answering questions about data, by simply noting the characteristics of the graph.
Example 1: p. 18
It is immediately apparent from the graph that there are two superimposed distributions, perhaps due to two different operating processes.
Example 2: p. 19
It is immediately obvious from the histogram that most of the interrequest times are relatively small, with only a few very large times.
Sometimes we want to do a relative frequency histogram of a data set (sometimes called a density histogram, for reasons to be covered in Chapter 6).
Example: pp. 19 – 20
The density histogram shows an approximately symmetric, bell-shaped distribution for the compressive strengths.
A simple graphical display of relatively small data sets may be done with a stem-and-leaf plot, (Excel does not do this kind of plot.)
Example: The original aluminum-lithium alloy compressive-strength data set.
7 | 6
8 | 7
9 | 7
10 | 1 5
11 | 0 5 8
12 | 0 1 3
13 | 1 3 3 4 5 5
14 | 1 2 3 5 6 8 9 9
15 | 0 0 1 3 4 4 6 7 8 8 8 8
16 | 0 0 0 3 3 5 7 7 8 9
17 | 0 1 1 2 4 4 5 6 6 8
18 | 0 0 1 1 3 4 6
19 | 0 3 4 6 9 9
20 | 0 1 7 8
21 | 8
22 | 1 8 9
23 | 7
24 | 5
It is clear from the graph that the distribution of compressive strengths is approximately bell-shaped, centered at a value of approximately 160 psi.
Numerical Descriptive Measures
One type of numerical summary describes, in some sense, the location of the center of a data set. There are several measures of central tendency, the most important of which is the mean.
Defn: For a variable X measured for every member of a finite population of size N, yielding a set of values x1, x2, …, xN, the mean, or average, is given by . For a sample of size n chosen from the population, yielding a set of values x1, x2, …, xn, the sample mean, or average, is given by .
Sometimes, the sample mean is not the most useful measure of central tendency. For example, sometimes a data set has some extreme values (either very large or very small). These extreme values are called outliers (more on this topic later). The value of the sample mean may be strongly affected by these outliers. In such a case, a more useful measure of central tendency may be the sample median.
Defn: The sample median, , is the center of the data set when the data are ordered from smallest to largest. If n is odd, then the median is the middle item of data. If n is even, then the median is the average of the two middle items of data.
The median is not usually affected by outliers (Example on page 26).
Example: In the original compression strength data set, n = 80, so
psi.
In addition to locating the center of the data set, we want to describe the dispersion of the data values.
The simplest, although least useful, measure of dispersion is the range of the data set.
Defn: The range of a data set is the difference between the largest and smallest values of the data; the range is a simple measure of the dispersion of the data.
Example: For the compression strength data,
Range = 245 psi – 76 psi = 169 psi
The range cannot distinguish between the dispersion of two data sets that have the same largest and smallest values, even though the values in between may be quite different from one data set to the other. For this reason, we need a measure of dispersion that takes into consideration the location of each data value relative to the center of the data set.
Consider a data set with data values x1, x2, x3, …, xn. For each data value xi, we define the deviation from the mean as xi-x. This value gives the (directed) distance of the ith data value from the mean of the sample data. We may consider using the sum of all of these deviations as our measure of dispersion. However, it would be useless to do so, as you will show in Exercise 2.50.
Instead, we define two other measures of dispersion, the variance and the standard deviation.
Defn: For a variable X measured for every member of a finite population of size N, yielding a set of values x1, x2, …, xN, the variance of the data is given by , and the standard deviation is given by . For a sample of size n chosen from the population, yielding a set of values x1, x2, …, xn, the sample variance is given by , and the sample standard deviation is s.
Note: In the above definitions, and are parameters; these two quantities have fixed but usually unknown values. The two quantities and s are statistics; the values of these two quantities depend on the particular sample chosen from the population.
If all of the data values in a data set are the same, then the variance and standard deviation are both 0. If there are any differences among the data values, then both the variance and standard deviation are positive; the greater the differences among the data values, the greater the values of the variance and standard deviation.
Note: While the defining formulae for the population mean and the sample mean have the same form, the defining formulae for the population variance and the sample variance differ. For the population, the variance is the mean of the squared deviations of the data values from the mean value. For the sample, the variance is almost the mean of the squared deviations of the data values from the mean value. Instead of dividing the sum of squared deviations by the sample size, we divide by n – 1. The reason for doing so has to do with the fact that we want the sample variance to be a good estimator of the population variance. A better estimator is given by dividing by n – 1, rather than by n. Statistically, we say that there are n – 1 degrees of freedom associated with the sample variance.