The Forecast Process, Data Considerations, and Model Selection

Full file at

CHAPTER 2

THE FORECAST PROCESS, DATA CONSIDERATIONS, AND MODEL SELECTION

CHAPTER OVERVIEW

This chapter establishes a practical guide to implementing a successful forecasting process stressing evaluation of data for trend, seasonal, and cyclical components. In addition, basic applied statistics are reviewed.

LEARNING OBJECTIVES

• Establish Framework for a successful Forecasting System
• Introduce the Trend, Cycle and Seasonal factors of a Time Series
• Review basic statistics such as Normal and t distributions, Confidence Intervals, and Hypothesis Testing
• Introduce Correlation and its role in the Model Selection Process
• Introduce the concept of Autocorrelation and Estimation of the Autocorrelation Function

NOTES TO TEACHERS

1.This chapter introduces some basic statistical concepts that underlie modern quantitative forecasting methods and is therefore necessary reading for future material. We stress to students that statistics can be thought of as the science of uncertainty. Since virtually all business decisions are made under conditions of uncertainty, students should understand the important role of forecasting in reducing this uncertainty.

2.Instructors requiring forecasting projects should recommend to students the nine-step process as a reasonable way to proceed to generate forecasts. This text primarily concerns itself with the fifth through seventh steps of the nine-step process, which include the model selection phase, model evaluation, and forecast preparation. The model selection aspect of the forecasting process is conveniently presented in Table 2.1 (page 58), which serves as a model selection checklist with respect to four important factors.

3.The distinction between "fit" and "accuracy" should be noted as well as a sample "holdout-period" used to evaluate forecast model accuracy. These are important for students to understand.

4.Students should appreciate that any time series can be decomposed into trend, cyclical, seasonal, and irregular components. The point is that statisticians have found ways to exploit time-series data patterns, aimed at increasing forecast accuracy.

A particularly important example is the difference between regular and "seasonally adjusted" time series, which is addressed in the discussion of Figure 2.2 (page 62). In this course, students will learn how to deseasonalize data and make forecasts based upon modeling seasonal behavior using regression or time series techniques.

5.No matter how powerful the computer, or the sophistication of forecasting technique, unless the student understands the basic concepts that underlie the generated output, the resulting forecasts are likely to be suboptimal. Accordingly, basic statistics is a foundation upon which forthcoming chapters build. For example, the distinction between population and sample means is analogous to the distinction between the true regression model and the fitted version. Discussion of sampling distributions leads naturally to the t distribution, which is applied extensively in the chapters on regression.

When a population is unobservable, statisticians have developed estimators or rules on how to use sample data to evaluate a population. Since any inference about a population from a sample necessitates error, we must concern ourselves with the type of error (type I or II) and the relative probability of occurrence (the level of significance). This leads naturally to confidence intervals and hypotheses testing which are used to build forecasting models.

Some helpful hints include:

i)Statistical inference and hypothesis testing are basic tools in forecast model selection. Accordingly, this part of the course deserves several examples. In addition, we usually present hypothesis testing as a four-step procedure: step one is formulating the null and alternative hypotheses; step two is selecting the appropriate test statistic and distribution for the problem at hand; step three establishes the appropriate critical regions (asking students to show a graph of the particular distribution helps them understand the process); step four is the decision to reject or not reject the null.

ii)Students should approach statistical inference by first clearly establishing the null and alternative hypotheses. The null should be formulated as to be rejected; allowing the researcher to set the level of type I error, i.e., the level of significance.

iii)Students will also find it helpful if they display graphically critical regions involved in hypothesis testing and that these tests are consistent with confidence intervals for the population parameter of interest. Students often ask what determines whether a one- or two-tailed test is appropriate, and which tail should contain the critical region if the test is one-tailed. The answer is simple: When one is only interested in whether there is any difference (direction is not important) then the sign in the alternative hypothesis is an unequal sign (≠) and the test is a two-tailed test. Otherwise, a one-tailed test is appropriate. Accordingly, for a one-tailed test the entire significance level () goes in the one tail of the distribution that is indicated by the direction of the inequality sign in the alternative hypothesis.

iv)Often, if two variables are significantly correlated, then the behavior of one can be used to forecast the behavior of the other, e.g., sales and personal income. Testing the null hypothesis that the true population correlation is zero is accomplished by the t-distribution when using the sample correlation coefficient. It is often useful to point out how the test statistic employs a centering and a scaling transformation analogous to the standard normal transformation.

6.Serial correlation (sometimes called autocorrelation)can lead to spurious inference if ignored when applying regression analysis to time series data. Accordingly, testing for autocorrelation is important and can be accomplished using estimated autocorrelation coefficients. The appropriate null is zero autocorrelation at lag k. In most applications, an approximation is used in which the critical value for rejecting the null of zero autocorrelation is two divided by the square root of sample size.

A convenient way to examine time series behavior is by using a correlogram, which visually displays the sequence of auto- and partial autocorrelation coefficients. Students will appreciate the ease of making inference from correlograms since FORECASTXTM prints a 95 percent confidence interval around zero in a diagram allowing inference through visual means. From correlograms, students will learn how to diagnose trend in a series. In addition, seasonal patterns are shown by significant autocorrelations at lags 4 and 8 for, say quarterly data, and are amazingly persistent after data differencing. Students often find estimating the data in first differences is awkward and confusing. Accordingly, it helps to point out that, when appropriate, first differencing the data removes the trend in a series, while preserving other data attributes allowing the transformed series to be forecast with methods designed for stationary data.

ANSWERS TO END-OF-CHAPTER EXERCISES

1. The null hypothesis states that mean sales, per representative are $24,000. The alternative hypothesis is that mean sales are not $24,000.

Since the sample standard deviation is reported, the appropriate test statistic follows a

t distribution:

However, because the sample size of 100 is so large, we can apply the standard normal table. Since the alternative hypothesis is one of inequality, with level of significance being .05, we need the area making up two and one-half percent of the distribution in each tail. From the Standard Normal Table we see that the probability that a Z-variable exceeds 1.96 is .025. Accordingly, our rejection region is for values of the test statistic that lie either above 1.96 or below -1.96.

Using the sample mean, sample standard deviation, and sample size, the calculated value of the test statistic is:

Since this lies outside the rejection region, we fail to reject the null at the 95% level of confidence and conclude that the reported mean level of sales is statistically indistinguishable from $24,000.

2. State the null hypothesis as the inventory level is the same as the industry average, and the alternative as one of inequality:

Since both the population mean and variance are unknown, the appropriate test statistic is based on the t-distribution:

Given that n = 120, = 310, and s = 72; the absolute value of the test statistic under the null is:

Since 2.28 > 1.96, we can reject the null and conclude the manufacturers mean tire inventory is significantly different from the industry norm.

3.a) The mean is the arithmetic average of all the numbers in the data set. The median is the value that splits the respondents into two equal parts when they are arrayed from the lowest to the highest value. The mode is the value that occurs most frequently.

Using FORECASTXTM, the following descriptive statistics were obtained (it may be quicker to get these directly from Excel however):

Statistics for Solution
Credit Hours
Mean / 8.3
Median / 9
Mode / 9
Standard Deviation / 2.6378
Sample Variance / 6.9579
Range / 11
Minimum / 2
Maximum / 13
Standard Error / 0.5898

b) Since the claim is that business graduate students take fewer credit hours than the average graduate student, we state the null such that its rejection merits the claim. Specifically, the null states that the mean number of credit hours taken by the business graduate student is equal to or larger than the mean number of credit hours taken by typical graduate students. We state the alternative as mean credit hours taken is less than the mean credit hours taken by a typical graduate student:

Since both the population mean and variance are unknown, the appropriate test statistic is based on the t-distribution.

The critical region for a one-tailed alternative at the 5% level of significance with sample size 20 is given by:

Given that n = 20, = 8.3, and s = 2.638, we can calculate the sample value of the test statistic under the null hypothesis:

Since 1.356 < 1.725, we fail to reject the null hypothesis and conclude that business graduate students do not typically take less credits per quarter than non-business graduate students.

4.The issue is whether or not ACC’s sales staff is comparable to those of other producers in the same industry. Accordingly, state the null as ACC’s mean sales per salesperson equals that of other producers, with the alternative of inequality:

The following statistics are used to test this assertion.

Statistics for the Solution
Sales
Mean / 219,017.5625
Standard Deviation / 76,621.2783

Since both the population mean and variance are unknown, the appropriate test statistic is based on the t-distribution:

The critical region for a 5% level of significance, when the sample size is 16 under a two-tailed alternative, is given by:

Given that n = 16, = 219,018, and s = 76621.3, we can calculate the sample value of the test statistic under the null hypothesis:

Since 1.878 < 2.131, we cannot reject the null and conclude that ACC’s sales staff are comparable to that of other producers.

5.a) Since the mean is 205 pounds, and the normal distribution is symmetric about the mean, half of the population should lie above the mean. Therefore 50 percent of the players would weigh more than 205 pounds.

b) To make statistical inference from the probability distribution of football player weights, we use the standard normal distribution with the appropriate transformation. The appropriate Z value for 250 pounds is:

Z = (250 - 205)/30 = 1.5.

Accordingly, what percent of players weigh less than 250 pounds is the same as asking:

P(Z < 1.5) = P[Z ≤ 0] + P[0 ≤ Z < 1.5] = .5000 + .4332 = .9332.

Using the relative frequency interpretation of probabilities this would imply that 93.3% of the players would weigh less than 250 pounds.

c) From Table 2-4: Z.10 = -1.285. Accordingly, 90% of the area under the standard normal density lies above the Z-value of –1.285:

P[Z > -1.285] = [(X - 205)/30 > -1.285] = .90

P[X - 205 > -38.55] = P[X > 166.45] = .90.

Therefore 90% of the players would weigh more than 166.45 pounds.

d) P[180 ≤ X ≤ 230] = P[180 - 205 ≤ X - 205 ≤ 230 - 205]

= P[-25 ≤ X - 205 ≤ 25] = P[-25/30 ≤ Z ≤ 25/30] = P[-.8333 ≤ Z ≤ .8333]

= P[-.8333 ≤ Z ≤ 0] + P[0 ≤ Z ≤ .8333] = .2967 + .2967 = .5934

Therefore, 59.34% of the players would weigh between 80 and 230 pounds.

6.a) Ms. Wharton’s hope is that her bank is viewed more favorably than the average bank, which has an approval rating of 7.01. We formulate the null such that its rejection merits Ms. Wharton’s hopes:

Since data were derived from a market research survey, we view both the population mean and variance as having been estimated. Accordingly, the appropriate test statistic is based upon the t-distribution:

To find the calculated value of the test statistic based upon this sample, we note the following information: Sample Mean = 7.25, 0 = 7.01, s = 2.51, and n = 400.

The critical region for significance level .05 and sample size 400, under a one-tailed alternative, is given by:

P[t399 > 1.645] = .05.

Since our calculated t-value falls into the critical region, we can reject the null hypothesis at the 5% level of significance.

c) The calculated value of our t-statistic would now be:

Hence, we cannot reject the null with this smaller sample, other things held constant.

Why? The key is to examine the variance of the sampling distribution of the sample mean, which depends on the sample size. As sample size increases, we can be more confident in our estimate of the population mean.

7.a) Using FORECASTXTM, the following descriptive statistics about class size were obtained.

Statistics for Solution
Number of students
Mean / 40.1600
Median / 42.0000
Mode / 20.0000
Standard Deviation / 13.1424
Sample Variance / 172.7233
Standard Error / 2.6285
Range / 54.0000
Minimum / 10.0000
Maximum / 64.0000

b) The standard error of the sample mean is 2/n. Estimating the population variance by the sample variance and noting the sample size, the standard error of the estimated sample mean is (172.72/25)1/2 = 2.628.

c) The sampling distribution of the sample mean shows that the sample mean is an unbiased estimator of the population mean. Accordingly, 40.16 is our point estimate for the population class size.

d) Since the population mean and variance are unknown, we use the following confidence interval involving the t-distribution to make probability statements about the unknown population mean:

For a 95 percent confidence interval, using t24,.025 = 2.064, we get:

P[34.74 <  < 45.58] = .95.

For 90% confidence interval, using t24,.05 = 1.711, we get:

P[35.66 <  < 44.66] = .90.

The 95% confidence interval is a wider since we are statistically more confident.

8.a) To examine whether there has been an upward trend in annual larceny thefts in the United States, a time-series plot of annual data from 1972 through 1994 was prepared.

As shown in the time-series plot, there is a positive trend over the sample period.

b) The ACF and PACF estimates and correlograms for THEFTS are reported below.

rk = 2/(sqrt n) so in this case rk = 2/(sqrt 23) = .417.
ACF Values For Larceny Thefts
Obs / ACF /
/ Upper Limit / Lower Limit
1 / .7986 / .4087 / -.4087
2 / .5594 / .4087 / -.4087
3 / .4045 / .4087 / -.4087
4 / .3298 / .4087 / -.4087
5 / .2844 / .4087 / -.4087

To test the significance of individual autocorrelation parameters, we use the following 95% confidence approximation, which states that the critical value for rk is: where n is the sample size. For estimated autocorrelation coefficients in excess of rk in absolute value, we reject the null hypothesis of zero autocorrelation assuming a two-tailed alternative at the .05 level of significance.

Since sample size is 23, the appropriate critical value for testing the null of zero autocorrelation is .417. Examining the autocorrelation function, we see that we can reject the null of zero autocorrelation at lags 1, and 2, since their coefficient estimates exceed .417. Autocorrelations at lags 3 through 5 are not statistically different from zero. Accordingly, because of trend the data are nonstationary.

c) To account for trend, Holt’s exponential smoothing or a regression trend model should be considered.

9.A time-series plot of mobile home shipments (MHS) is shown below.

As indicated by the time-series plot, there is significant seasonality to mobile home sales as shown by the regular periodic variation in the data arising at the same time each year. Specifically, the plot shows seasonal downturns in quarters four and one compared to quarters two and three. The plot also reveals periods of upward trends followed by downward trends, presumably related to business cycle factors such as interest rates and unemployment.

The autocorrelation structure of MHS and correlogram is reported below.

Obs / ACF
1 / .7779
2 / .5593
3 / .6165
4 / .6911
5 / .4623
6 / .2359
7 / .2494
8 / .3154
9 / .1103
10 / -.1118
11 / -.0954
12 / -.0136

Using the 95% approximation for testing the null of zero autocorrelation at lag k, we can reject the null if the estimated autocorrelation coefficient exceeds

With our sample size of 60, the critical value for rk is .258199. Examining the estimated autocorrelations in the table above show that we can reject the null of zero autocorrelation at lags 1, 2, 3, 4, 5, and 8; it is not until lag six that the autocorrelation function falls below .258199 (r6 = .235934). Accordingly, the autocorrelation results suggest a trend in the data. The seasonality may cause a distorted result here. Other analysis of the trend, like a simple linear trend, would show that the slope of the trend line is not different from zero. However, one can argue that there are two main trends, one down and then up which a simple trend line through all the data would not capture.

In addition, the relatively large autocorrelation coefficients for lags of 4 and 8 quarters (r4 = .691087 and r8 = .315445) indicate significant seasonality in the MHS series.

Forecasting methods that might be suggested as good candidates for MHS, based on Table 2-1 in the text would include Winters’ exponential smoothing, time series decomposition, and a causal regression model.

10.a) Private housing starts data (PHS) are plotted below.

The time-series plot of private housing starts (PHS) shows no significant trend and significant seasonality.

b)Using FORECASTXTM, the estimated autocorrelation coefficients and correlogram for PHS are reported below.

Obs / ACF
1 / .8632
2 / .7620
3 / .8098
4 / .8618
5 / .7154
6 / .5929
7 / .6169
8 / .6475
9 / .5093
10 / .3876
11 / .4087
12 / .4455

With our sample size of 64, the critical value for rk is .25. Examining the estimated autocorrelations in the table above show that we can reject the null of zero autocorrelation at lags 1through 12 using the approximate 95% confidence rule.