Lecture #3 The t-statistic and the t-test
Suppose that we have the model
for t = 1, 2, 3, ..., T
We can use OLS to estimate the constant term βo and the slope coefficient β1. We write the estimators of βo and β1 as
and
and these are random variables with means and variances. However, just estimating the two parameters is not enough. We need to ask whether Xt really affects Yt linearly. It should be obvious to you that if β1 = 0, then Xt cannot possibly affect Yt linearly. It may be possible that there is a nonlinear relation between Xt and Yt, but our LINEAR regression cannot tell us much about that without looking at the residuals. We are asking whether Xt affects Yt linearly. Therefore, we are asking whether or not β1 = 0.
Now, to make this very clear, we state it as a hypothesis. A hypothesis is something we are asserting as a fact, but which we only have probable knowledge. In our regression we assert that Xtdoes not affect Yt linearly. This is called the null hypothesis and we write it as
Ho: β1 = 0
Note how very, very precise this is. We are not asserting that β1 is almost zero—rather we are asserting that β1 is EXACTLY zero. Naturally, very few things in Life are so precise and this is one reason we have so much trouble using precise mathematics and statistics to discuss society around us. Society is rarely precise and almost never stable. Nevertheless, we try our best to use these tools that mathematicians and statisticians have created for us.
If Ho is false, then β1≠ 0. This means that β1 > 0 or β1 < 0. We don’t really care here which of these hold. We only care that β1≠ 0. This is called a two-tailed test, since β1 can be positive or negative. We write the alternative hypothesis as
Ha: β1≠ 0
If we reject the null hypothesis, then we accept the alternative hypothesis.
However, we never accept the null hypothesis. The reason is simple. If we accept that Ho is true, then we are saying β1 = 0. But, we could also test Ho: β1 = 0.00001 and we would probably find that we again accept the null! How can we say that β1 = 0 and β1 = 0.00001? Because of this problem, we simply say that we “do not reject the null hypothesis”. Once again, we never accept the null. We merely say that we do not reject the null. In other words, we say that the null is consistent with the data. If all of this bothers you, then just go ahead and use the term “accept the null”. Later, you will realize that accepting the null is not entirely right, but it is not harmful to think in that way.
So, how do we test Ho versus Ha?
We now come to the famous t-test of statistical significance for regression coefficients. This test is used everyday thousands of times around the world to test whether Ho or Ha is closer to the truth given the data. All testing of regression equations begin with the t-test and it is clearly the most famous and most important of all tests in regression analysis. The test is sometimes called Student’s t-test since the original paper was written by a statistician who used the (fake) pseudo-name “Student” to publish his paper. His real name was Gosset, but that should not concern us here. By the way, Mr. Gosset who was English worked at a large brewery in Dublin, Ireland that made beer. That brewery was called the Guinness brewing company and it is still in business today. You can even buy Guinness beer in Taiwan. I prefer Thai beer, but that is my preference.
Here are the steps for testing Ho: β1 = 0
Step 1: Assume Ho is true.
You might think that this is crazy, since we are trying to test Ho—why should we simply assume it is true? The answer is that we must assume that Ho is true in order to see whether we get a very unlikely event when we compute the test statistic using our data. If the test statistic is very far from zero, then we will think it is highly unlikely that Ho is true. Assuming Ho is true and then getting a very unusual result makes us want to reject Ho. That sounds right.
Step 2: Get the t-test statistic
Here we must memorize something. In a higher level class we would derive this test statistic and study its distributional properties. Not here. We only state the way it is computed. That will suit our purposes for now.
The t-statistic for the hypothesis Ho: β1 = 0 is easy to write down and remember.
The numerator of this statistic is just the estimator of β1 and the denominator is the square root of the estimated variance of , which we call the standard error of . We subtract 0 in the numerator because the null hypothesis is Ho: β1 = 0. If the null was Ho : β1 = C, then our t-statistic would be
Step 3: Get the distribution of
Here is where step 1 comes in. When we assume Ho is true we can derive the pdf or distribution of the t-statistic. It has a pdf that look very much like a normal density. In fact, if we have many observations (lots of data) then the density converges to a standard normal random variable.
The t-distribution looks like the following
We note that the half way point is equal to 0. The two tail areas, when summed together equal 0.05 (also called the significance level or the size of the test). There are t-tables that can be found on the Internet that will allow you to compute the probability of being less than or equal to a particular number. We usually do not need these tables when using GRETL. The only thing we need to know to draw this pdf is the number of degrees of freedom. For the t-test, the degrees of freedom ( df ) are determined by the following formula
df = Number of Observations
– Number of Coefficients estimated
= T - 2
The pdf above is called thet distribution with T- 2 degrees of freedom. Note that we subtract the number of coefficients estimated in the regression equation, including the constant.
Step 4: Calculate the statistic
If the calculated , which we may call , is between the two tail sectionsof the t-distribution, then we do not reject Ho. By not rejecting Ho we are saying that there is strong evidence that Ho could be true. We are closer to accepting Ho than before the test.
By contrast, if the calculated , which we now call , is in either the left or right tail section of the t-distribution, then we reject Ho and we conclude that Ho is false.
That’s all there is to it. The t-test is easy to perform and it tells us a lot. It tells us whether a coefficient which we have estimated is statistically significant. It tells us whether or not an explanatory variable really affects Yt linearly. Naturally, it is extremely important to know whether some economic or business variables are related to other variables. The t-test helps us to decide this on an objective and impartial basis. Remember that it is only a test and there are many things that can go wrong with this test. Some basic assumptions needed to use the test might not be met. In addition, we generally need more than the t-test to help us decide whether a variable is related to Yt. In fact, our theory might be very persuasive despite the test.
The last thing which we need to discuss in order to finish this lecture is the p-value which is given in GRETL each time we have a t-test.
The p-value (probability-value) is easy to understand. If we calculate the t-statistic,, then we can place it on the graph and see where it lies.Clearly, if the calculated t-statistic is out in the tails, then we should reject Ho. The p-value is defined as double the area formed by the calculated . More formally, the p-value is the probability of observing a t-statistic larger than || given that Ho is true. If the p-value ≤ 0.05, then we should reject Ho. If the p-value is > 0.05, then we should not reject Ho. In the graph below, the p-value (black tail area) is less than 0,05 (red tail area) and therefore we should reject Ho.
Example: Here is a GRETL regression output with t-tests and p-values.
Suppose that we think smoking is positively related to cancer. Obviously how much people smoke over 20 or 30 years will determine the incidence of cancer in the population. To see if this is true we can regress cancer rates against average cigarettes consumed per person 30 years before. Our regression becomes
Cancer Ratet = βo + β1( Cigarettes/Person)t-30 + εt
If β1 > 0, then we are saying that smoking more 30 years ago increases your chances of cancer today. Here are the results of the GRETL estimation using US annual data 1930-2003.
Model 1: OLS estimates using the 74 observations 1930-2003
Dependent variable: malignant neoplasms
VARIABLE COEFFICIENT STDERROR T STAT P-VALUE
const 117.009 1.40479 83.293 <0.00001 ***
cigs30 0.0199448 0.000531048 37.557 <0.00001 ***
Mean of dependent variable = 159.297
Standard deviation of dep. var. = 32.5653
Sum of squared residuals = 3759.69
Standard error of residuals = 7.2262
Unadjusted R-squared = 0.951435
Adjusted R-squared = 0.950761
Degrees of freedom = 72
Durbin-Watson statistic = 0.167692
The estimate for β1 = 0.0199448 and is statistically significant since the p-value is well below 0.05. On this basis alone we would reject the hypothesis that β1 = 0 and conclude that smoking 30 years before increases one’s chances of getting cancer. This study has many problems which we cannot discuss here, but it illustrates how we can use GRETL to quickly analyze a regression and conduct t-tests.