Appendix 2 Multivariate Normal Distribution
In this chapter, the following topics will be discussed:
Moment generating function and independence of normal variables
Quadratic forms in normal variable
Let . Then, the density function is
Definition (Multivariate Normal Random Variable):
A random vector
with has the density function
Since is positive definite, , where is a real orthogonal matrix () and . Then,
where . Further,
Therefore, if we can prove and are mutually independent, then
The joint density function of is
Therefore, the density function of
Therefore, and are mutually independent.
2.2Moment Generating Function and Independence of Normal Random Variables
Moment Generating Function of Multivariate Normal Random Variable:
Then, the moment generating function for Yis
If and C is a matrix of rank p, then
Let . Then,
Since is the moment generating function of ,
where Tis an orthogonal matrix.
If , then the marginal distribution of subset of the elements of Y is also multivariate normal.
, then , where
Y has a multivariate normal distribution if and only if is univariate normal for all real vectors a.
Suppose . is univariate normal. Also,
is the moment generating function of , thus Y has a multivariate distribution .
By the previous theorem. ◆
2.3Quadratic Form in Normal Variables
If and let P be an symmetric matrix of rank r. Then,
is distributed as if and only if (i.e., P is idempotent).
Suppose and . Then, Phas r eigenvalues equal to 1 and eigenvalues equal to 0. Thus, without loss generalization,
where T is an orthogonal matrix. Then,
Since and , thus
are i.i.d.normal random variables with common variance . Therefore,
Since P is symmetric, , where T is an orthogonal matrix and is a diagonal matrix with elements . Thus, let . Since ,
That is, are independent normal random variable with variance . Then,
The moment generating function of is
Also, sinceQis distributed as , the moment generating function is also equal to . Thus, for every t,
By the uniqueness of polynomial roots, we must have . Then, by the following result:
a matrix P is symmetric, then P is idempotent and rank r if and only if it has r eigenvalues equal to 1 and n-r eigenvalues equal to 0. ◆
Let and let and be both distributed as chi-square. Then, and are independent if and only if .
If , and is semi-positive definite, then
If and let
If , then and are independent and .
We first prove ., thus
Since , is any vector in . Therefore, is semidefinite. By the above useful lemma, is idempotent. Further, by the previous theorem,
We now prove and are independent. Since
By the previous important result, the proof is complete. ◆