Bayes’s Rule
The Reverend Thomas Bayes (1702 - 1761), an English Presbyterian minister and mathematician, developed an expanded form for conditional probabilities. This expanded rule, called Bayes’s rule, allows us to revise (or adjust) the probabilities assigned to events in accordance with new information.
BAYES'S RULE
Formula 4.9
P(G | CG) =
where A1, ... , An is an all-inclusive set of possible outcomes given B.
Let’s take another look at Illustration 4.14 to explain Bayes’s rule.
ILLUSTRATION 4 - 15
Consider the situation described in Illustration 4.13 and suppose that only items classified as good after inspection are shipped. What percentage of the items shipped are good, P(G | CG)? What percentage of those shipped are defective, P(D | CG)?
SOLUTION
First, let’s match up the events of the problem to the Bayes’s rule notation. The events Bayes identifies as Ai are A1 = G (good) and A2 = D (defective), an all-inclusive set of events. The given or conditional event B is CG, classified good.
A tabular approach will be used to help organize the solution. To construct the table, start by listing all possible outcomes Ai that can occur given event B (that is, “shipped”) in the first column. See Table 4.5. In the second column, list the initial probabilities of the Ai outcomes. In the third column we list the conditional probability that B happened for each Ai , P(B | Ai). For our illustration, P(B | A1) = P(CG | G) and P(B | A2) = P(CG | D). These first three columns represent the information obtained from the problem.
TABLE 4.5 Tabular Presentation of Given Information
(1)(2)(3)
Ai, Possible OutcomesP(Ai) P(B | Ai)
A1, item good0.80.9
A2, item defective0.20.1
Total1.0 ck
To solve for the conditional probabilities P(Ai | B), the first calculation is to multiply each number of column (2) by the number in the same row of column (3). This product is placed in column (4) of the table (see Table 4.6). The column is labeled P(Ai and B). The values calculated represent the probability that both Ai and B will occur. Thus, 72% of the items produced will be good and classified good; 2% of the items will be defective and classified good.
TABLE 4.6 Tabular Solution of Bayes’s Rule
(1)(2)(3)(4)(5)
Ai,P(Ai and B) =
Possible OutcomesP(Ai) P(B | Ai)P(Ai) P(B | Ai)P(Ai | B)
A1 , item good0.80.90.72= 0.973 = P(G | shipped)
A2, item defective0.20.10.02= 0.027 = P(D | shipped)
Total1.0 ck0.74 = P(B)1.000 ck
The second step is to add column (4). The sum represents P(B). Thus, 74% of the items produced will be classified good.
Finally, the answers we are looking for, the conditional probabilities P(Ai | B), are obtained by dividing each number in column (4) by the total of column (40. The results are placed in column (5) and are the answers. Thus 0.973 is the proportion of items classifies good that are good, and 0.027 is the proportion of items classifies good that are actually defective.
NOTE The totals of columns (2) and (5) must equal 1. The total of column (3) need not equal 1.
Bayes’s rule is of special interest because it gives us a mechanism to review initial probability estimates when new information is learned, as we see in the next illustration.
ILLUSTRATION 4 - 16
Consider the situation in which we feel that the probability that a stock is a good buy is 0.4. That is, our prior (before new information) probabilities are P(good buy) = 0.4 and P(bad buy) = 0.6. Now we find out that an investment service that has a record of being right 80% of the time recommends the stock. What should be our revised, or posterior (after new information), probability that the stock is a good buy, that is, P(good buy | investment service recommends it), or P(Ai | B)?
SOLUTIONUsing the Bayesian analysis, we find that the revised, or posterior, probability is 0.727 that the stock is a good buy and 0.273 that it is a bad buy (see Table 4.7).
TABLE 4.7 Tabular Analysis for Illustration 4.16
(1)(2)(3)(4)(5)
AiP(Ai) P(B | Ai)P(Ai) P(B | Ai)P(Ai | B)
A1 , good buy0.40.80.32= 0.727 = P(good buy|recommended)
A2, bad buy0.60.20.12= 0.273 = P(bad buy|recommended)
Total1.0 ck0.44 = P(B)1.000 ck
In reviewing the use of Bayes’s rule to revise prior probability estimates in light of new information, we note the following relationship: The stronger the prior probability, the less effect the new information has on changing the probabilities. Also, the more conclusive the new information, the greater the impact on the revised probabilities.
Exercises
1.Given the following:
P(A1) = 0.2P(A2) = 0.4P(A3) = 0.3P(A4) = 0.1
P(B|A1) = 0.5P(B|A2) = 0.4P(B|A3) = 0.2P(B|A4) = 0.1
Find:a.P(A1|B)b.P(A2|B)c.P(A3|B)d.P(A4|B).
2.In an article titled "Why Quitting Means Gaining" (Time, March 25, 1991), it was reported that giving up cigarette smoking often results in gaining weight. In examining a group of quitters, the following data were found.
Weight gain
MajorSignificantModerateSlight
Men 9% 14% 22% 55%
Women* 12% 11% 26% 50%
*Due to rounding, number for women do not total 100%
Suppose the group were 60% men and 40% women. If a participant were randomly selected and found to have experienced
a.a major weight gain, find the probability that it was a man.
b.a slight weight gain, find the probability that it was a woman.
3.Given the information in the accompanying table, compute P(A1|UF) and P(A2|UF) by filling in the rest of the table. (UF = unfavorable survey results)
P(Ai)P(UF|Ai)P(Ai and UF)P(Ai|UF)
P(A1|UF), profitable 0.6 0.4
P(A1|UF), not profitable 0.4 0.7
4.Immediately following the scandals that scarred the Clinton administration, many experts predicted that the Republicans would regain control of the executive branch of the federal government in the year 2000 election. At one time, the general consensus among political scientists was that if the Republicans fielded their best presidential candidate, they would have a 65% chance of winning the 2000 election. In addition, a poll of several thousand voters from across the nation indicated that 54% intended to vote Republican. Revise the experts’ opinion in light of the information from the poll using a Bayesian tabular analysis.
5.Solve this exercise using Bayes's rule in tabular form. The treasurer's initial opinion is that there is a 30% chance that an investment will exceed expectations, a 50% chance that it will equal expectations, and a 20% chance that it will return less than expected. A private investment consulting service reviews the investment and reports that it should equal expectations. In the past the consultants were correct 60% of the time, underestimated the return 10% of the time, and overestimated the return 30% of the time. What should be the treasurer's revised probabilities?
6.All her life, Cindy George has enjoyed drinking Fizzle, her favorite brand of soda. For years she has dreamed that on the day her savings account obtained a $10,000 balance, she would buy stock in the company that bottles Fizzle. Cindy is convinced that there is only a 20% chance that the stock will drop during the five-year planning horizon that she intends to keep the stock before selling it at what she feels will be a substantial profit. When she called her broker to move forward on her investment, she was horrified to hear that the company was not recommended by his investment advisory committee. The committee, he explained, has a 65% track record of selecting stock investments, and they predict firmly that the corporation that manufactures Fizzle will likely fail sometime during the next three years due to the highly competitive nature of the carbonated soft drink industry. Revise Cindy’s probability estimates in light of the broker’s information using a Bayesian tabular analysis.
7.Ninety percent of the insulators produced by Superior Insulator Company are satisfactory. The firm hires an inspector. The inspector inspects all the insulators and correctly classifies an item 90% of the time; P(classify good | good) = P(classify defective | defective) = 0.9. Items classified good are shipped and those classified defective are scrapped.
a.What percentage of items shipped can be expected to be good?
b.What percentage of items scrapped can be expected to be good?
8.The firm is Exercise 7 hires a second inspector, who has the same accuracy record. The second inspector inspects all insulators independently of the first inspector. What percentage of items shipped and what percentage of items scrapped can be expected to be good if items are shipped only if
a.both inspectors independently say they are good?
b.at least one inspector says they are good?