Statistics 510: Notes 21
Reading: Sections 7.4-7.5
Schedule: I will e-mail homework 9 by Friday. It will not be due in two weeks (the Friday after Thanksgiving).
I. Covariance, Variance of Sums and Correlations (Chapter 7.4)
The covariance between two random variables is a measure of how they are related.
The covariance between X and Y, denoted by , is defined by
.
Interpretation of Covariance: When , higher than expected values of X tend to occur together with higher than expected values of Y.
When , higher than expected values of X tend to occur together with lower than expected values of Y.
Example 1: From the Excel file stockdata.xls, we find that the correlations of the monthly log stock returns, in percentages, of Merck & Company, Johnson & Johnson, General Electric, General Motors and Ford Motor Company from January 1990 to December 1999 are
CovarianceMerck / J&J / 36.17205
Merck / GE / 26.85792
Merck / GM / 15.92461
Merck / Ford / 16.19535
J&J / GE / 48.45063
J&J / GM / 37.28136
J&J / Ford / 40.37623
GE / GM / 27.72861
GE / Ford / 26.63151
GM / Ford / 84.76252
Correlation: The magnitude of the covariance depends on the variance of X and the variance of Y. A dimensionless measure of the relationship between X and Y is the correlation:
.
The correlation is always between -1 and 1. If X and Y are independent, then but the converse is not true.
Generally, the correlation is a measure of the degree of linear dependence between X and Y.
Note that for ,
(this is what is meant by saying that the correlation is dimensionless – if X and Y are measured in certain units, and the units are changed so that X becomes aX and Y becomes bY, the correlation is not changed).
Example 1 continued: From the Excel file stockdata.xls, we find that the correlations of the monthly log stock returns, in percentages, of Merck & Company, Johnson & Johnson, General Electric, General Motors and Ford Motor Company from January 1990 to December 1999 are
Properties of covariance:
By expanding the right side of the definition of the covariance, we see that
Note that if X and Y are independent, then
Thus, if X and Y are independent,
(1.1)
The converse of (1.1) is not true. Consider the sample space
with each point having equal probability. Define the random variable X to be the first component of the sample point chosen, and Y the second. Therefore, and so on. and Y are dependent, yet their covariance is 0. The former is true because
To verify the latter, note that
Thus,
.
Proposition 4.2 lists some properties of covariance.
Proposition 4.2:
(i)
(ii)
(iii)
(iv) .
Combining properties (ii) and (iv), we have that
(1.2)
If are pairwise independent (meaning that and are independent for ), then Equation (1.2) reduces to
.
Example 2: Let be a hypergeometric random variable – i.e., is the number of white balls drawn in n random draws from an urn without replacement that originally consists of m white balls and black balls. Find the variance of X.
Let .
Then and
To calculate , note and so that .
To calculate , we use the formula . Note that if both the ith and jth balls are white, and 0 otherwise. Thus, =P(ith and jth balls are white). By considering the sequence of experiments look at the ith ball, look at the jth ball, then look at the 1st ball, look at the 2nd’s ball, ..., look at the i-1 ball, look at the i+1 ball, ..., look at the j-1 ball, look at the j+1 ball, ..., look at the nth ball, we see that
P(ith and jth balls are white) =
.
Thus, and
and
Note that the variance of a binomial random variable with n trials and probability of success for each trial is
, so the variance for the hypergeometric is smaller by a factor of ; this is due to the negative covariance between and for the hypergeometric.
II. Conditional Expectation (Chapter 7.5)
Recall that if X and Y are joint discrete random variables, the conditional probability mass function of X, given that , is defined for all y, such that , by
.
It is natural to define, in this case, the conditional expectation of X, given that for all values of y such that by
.
The conditional expectation of X, given that , represents the long run mean value of X in many independent repetitions of experiments in which .
For continuous random variables, the conditional expectation of X, given that , by
provided that .
Example 3: Suppose that events occur according to a Poisson process with rate . Let N be the number of events occurring in the time period [0,1]. For , let X be the number of events occurring in the time period . Find the conditional probability mass function and the conditional expectation of X given that .
III. Computing Expectations and Probabilities by Conditioning (Section 7.5.2-7.5.3)
Let us denote by that function of the random variable whose value at is . Note that is itself a random variable. An important property of conditional expectations is the following proposition:
Proposition 7.5.1:
(1.3)
If Y is a discrete random variable, then equation (1.3) states that
(1.4)
If Y is a continuous random variable, then equation (1.3) states that
One way to understand equation (1.4) is to interpret it as follows: To calculate , we may take a weighted average of the conditional expected value of X, given that , each of the terms being weighted by the probability of the event on which it is conditioned. Equation (1.4) is a “law of total expectation” that is analogous to the law of total probability (Section 3.3, notes 6).
Example 4: A miner is trapped in a mine containing 3 doors. The first door leads to a tunnel that will take him to safety after 3 hours of travel. The second door leads to a tunnel that will return him to the mine after 5 hours of travel. The third door leads to a tunnel that will return him to the mine after 7 hours. If we assume that the miner is at at all times equally likely to choose any one of the doors, what is the expected length of time until he reaches safety?
Example 5: A random rectangle is formed in the following way: The base, X, is a uniform [0,1] random variable and after having generated the base, the height is chosen to be uniform on . Find the expected area of the rectangle.
IV. Conditional Variance (Section 7.5.4)
The conditional variance of is the expected squared difference of the random variable X and its conditional mean, conditioning on the event that :
.
There is a very useful formula for the variance of a random variable X in terms of the conditional mean and conditional variance of :
Proposition 7.5.2:
.
Proof:
By the same reasoning that yields , we have that
. Thus,
(1.5)
Also, as , we have that
(1.6)
Hence, by adding Equations (1.5) and (1.6), we have that
.
Example 6: Suppose that by any time t, the number of people that have arrived at a train depot is a Poisson random variable with mean . If the initial train arrives at the depot at a time (independent of when the passengers arrive) that is uniformly distributed over , what is the mean and variance of the number of passengers that enter the train?
1