Statistics 512 Notes 2

Point Estimation Continued

Example: Suppose that an iid sample X1,...,Xn is drawn from the uniform distribution on [0,] where is an unknown parameter and the distribution of Xi is

Several point estimators:

1.

2. . Note: Unlike W1, W2 is unbiased because .

3. W3=2. Note: W3 is unbiased,

How to compare the estimators? Mean square error is a natural criterion.

A Formula for Computing Mean Squared Error

Theorem: The mean squared error of an estimator of

can be written as

Proof:

where we have used the fact that

Notes:

  • The decomposition of the mean-squared error into a bias and variance component is called the bias-variance decomposition.
  • For an unbiased estimator h(X1,...,Xn),

Comparison of three estimators for uniform example using mean squared error criterion

1.

We found the sampling distribution for W1 in Notes 1 to be

Also we found

To calculate , we calculate and use the formula .

Thus,

.

2.

Note .

Thus, ,

and

Because W2 is unbiased,

3.

To find the mean square error, we use the fact that if iid with mean and variance , then has mean and variance (see Examples 4.1.1 and 4.1.2 in textbook).

We have

Thus, , and

and .

W3 is unbiased and has mean square error .

The mean square errors of the three estimators are the following:

W1 /
W2 /
W3 /

For n=1, the three estimators have the same MSE.

For n>1,

So W2 is best, W1 is second best and W3 is the worst.

Application: During World War II, the US army was interested in estimating the amount of German production of certain war equipment. Every piece of German equipment, whether it was a V-2 rocket, a tank or just an automobile tire was stamped with a serial number that indicated the order in which it was manufactured. If the total number of, say, Mark I tanks produced by a certain date was N, each would bear one of the integers from 1 to N. As the war progressed, some of these numbers became known to the Allies – either by the direct capture of a tank or from records seized when a command post was overrun. The problem was to estimate N using only the sample of “captured” serial numbers, Y1,...,Yn.

The following model was used to estimate N. It was assumed that the sample of n captured serial numbers was a sample of size n without replacement from the numbers 1,..,N. In our notation, the model is

where is the distribution of a simple random sample of size n from 1,...,N.

This application is similar to the above example on estimating from an iid sample from the uniform distribution on [0,] except that the uniform distribution is on the positive integers 1,...,N and the sampling is without replacement.

Review of Point Estimation

  1. Consistency, meaning that the point estimate converges to the true parameter of interest as the sample size becomes large, is a property that any reasonable point estimator should have.
  2. Mean square error is a criterion for judging between reasonable point estimators.

Some open questions that will be addressed later in the course (Chapters 6 and 7):

  1. What are general methods for finding point estimators?
  2. Is there a best point estimator? For example, in our example, we found that W2 is better than W1 or W3. But is there a better point estimator thanW2?

Review of moment generating functions (mgfs)

To evaluate the properties of point estimators, we often need to find the sampling distribution of a point estimator. A useful tool for doing this for certain families of distributions is the moment generating function (mgf). You should be familiar with these from your probability course. This is a brief review. The book discusses moment generating functions in Chapter 1.9.

Definition: Let X be a random variable such that for some h>0, exists for –h<t<h. The moment generating function of X (abbreviated mgf) is defined to be the function for –h<t<h.

Properties of mgfs

  1. Mgfs uniquely characterize a distribution: Let X and Y be random variables with mgfs and respectively, existing in open intervals containing 0. Then the cumulative distribution functions (cdfs) of X and Y are equal if only if for all for some h>0.
  2. Let be independent random variables with moment generating functions respectively. Let . Then .
  3. Let Y=aX+b. The mgf of Y is

Application to showing that sample mean of iid normal sample has normal distribution: