Christian’s Study Notes for Exam P
Kind of like Cliff’s Notes, except these are Christian’s notes.
Complements
Either A occurs, or it does NOT occur:
Unions
You also must the know the extension of this formula to three random variables, which is
Independence
If A and B are independent, then
If A and B are mutually exclusive, then
De Morgan’s Laws
1.
2.
Conditional probability
, which can also be expressed as .
The Law of Total Probability
If you can carve up the probability domain into different non-overlapping (i.e. mutually exclusive) regions, then the probability of a random variable is the sum of the probabilities of the intersections of this random variable. In other words, if and for all i,j, such that , then . Also from the conditional probability formula, .
Bayes’ Theorem
Expected Value
There are two ways to calculate the expected value. The basic, most direct route is one of the things you have to know:
For a discrete distributionf(x),
For a continuous distribution:
But there’s also another way to calculate the expected value that can be faster, based on the information you are given in the problem.
For a discrete distribution:
For a continuous distribution,
where , where F is the cumulate distribution function.
You have to know the basic way to calculate the expected value; the other method is also nice to know and can save you valuable time in the heat of the exam.
Variance
The variance of a random variable X, denoted Var(X), is given by the formula
Covariance
The covariance between two variables is .
Also, you need to know that and that and
The standard deviation, denoted , is simply the square root of the variance, .
Coefficient of variation: This is simply the ratio of the standard deviation to the mean, that is .
Double Expectation
E(X) = EY(E(X|Y=y))
Var(X)=VarY(E(X|Y=y))+EY(Var(X|Y=y))
Wasn’t on my exam, but you never know.
Probability Distributions
Mean, median, and mode
The three most tested probability distributions are the uniform, exponential, and Poisson. You also need to know the binomial, geometric, the negative binomial, and the hypergeometric.
The Uniform Distribution
This is the simplest of the continuous distributions. You are given an interval(a,b), for which the likelihood of any point in the interval is just as a likely as any other. The probability density function is .
The mean of the uniform is and the variance is .
The Poisson Distribution
The Poisson distribution is used to model waiting times.
The important stuff:
The mean =
The variance =
The mode is equal to lambda, rounded down to the nearest integer. For example, a Poisson distribution with mean equal to 3.2 has a mode equal to 3. A Poisson with mean equal to 3 also has mode equal to 3.
Also good to know is that the sum of two Poisson distributions with means 1 and 2 is a Poisson distribution with mean = 1 + 2.
It gets a little trickier if two Poisson distributions or more are involved. A shortcut that can save you a significant amount of time is recognizing that the sum of two or more Poisson distributions is also a Poisson distribution. For example, supposed that you are asked the following question:
A business models the number of customers for the first week of each month as a Poisson distribution with mean = 3, and for the second week of each month as a Poisson distribution with mean = 2. What is the probability of having exactly two customers in the first two weeks of a month? The long way to do this is to figure out all the different combinations –
Case I – one customer in week one, one customer in week two.
Case II – two customers in week one, no customers in week two.
Case III – no customers in week one, two customers in week two.
The easy way to do this is to use the fact that the sum of two Poisson distributions is also Poisson. So the sum of the Poisson distributions from weeks one and two is Poisson with mean = 5. The probability of exactly two customers is
The Exponential Distribution
This is another of the essential distributions. The exponential distribution is used to measure the waiting time until failure of machines, among other applications.
The mean equals , and the variance equals . This is an important distinction from the Poisson, where the mean is equal to the variance. For the exponential, the mean is equal to the standard deviation, so the variance is equal to the mean squared.
Some useful integration shortcuts that can save you valuable time on the exam:
The Gamma Distribution
It’s good to have a passing familiarity with the Gamma distribution. The sum of exponential distributions is a gamma distribution. The exponential distribution is tested very heavily on the exam, and there has been at least one recent exam question where it would have been helpful to know that the sum of two exponentials is a gamma. That’s about all you’ll need to know, but you might get tested on the gamma outright, so listed below are some relevant formulas for the gamma. If pressed for time, skip this and focus on the basics instead.
Gamma pdf:
The Bernoulli Distribution
Discrete distribution, the simplest probability distribution, either an event occurs, or doesn’t occur. A probability is given for the event that the probability occurs.
E(X) = p
Var(X) = p(1-p)
Binomial Distribution
where
Mean = np, Variance = np(1-p)
Geometric Distribution
Perform Bernoulli trials until success, then stop and count the total number of trials – this is the geometric random variable.
The tricky part about this is that there can be two different formulations, based on whether you count the number of trials before the first success, or the number of failures before the first success.
X = # trials until first success:
Y = # failures before first success:
Negative Binomial Distribution
Hypergeometric Distribution
Used for sampling without replacement. Finite population with n objects, k are special, n-k are not. If m objects are chosen at random, the probability that out of m, x are special is
It looks a little complicated but once you’ve worked several problems, this is not too hard.
Normal Distribution
Continuity correction factor for binomial or Poisson or uniform approximations:
Bivariate Normal Distribution
Lognormal Distribution
Other distributions on the syllabus include the beta, the Pareto, the Chi-Square, and the Weibull. I have not presented them here.
Marginal Density
Order Statistics
Conditional Density
Moment Generating Functions
MX(t)=E(etX)
MaX(t)=MX(at)
Mb(t)=ebt
X,Y independent => MX+Y(t)=MX(t)MY(t)
MX(0)=1
MGF for Bernoulli: pet+q
MGF for Binomial: (pet+q)n
MGF for Poisson:
MGF for Standard Normal:
MGF for Normal:
MGF for Exponential:
MGF for Gamma:
Note that based on a comparison of the MGFs for the exponential and gamma distributions, it’s easy to see that the gamma is the sum of n exponential distributions.
Joint MGFs:
A couple of other important formulas that can come in handy:
Chebyshev’s Theorem
Benefit Distributions
E(X) = qE(B)
Var(X) = qE(B) – (qE(B))2
Miscellaneous Formulas
Kth central moment of X : E((X-E(X))k)
Correlation coefficient: