Bayesian Analysis
Slide 1
Previous lectures indicated that it was important for managers to assess the value of conducting research. If the cost exceeds the value, then the manager shouldn’t order research. Typically, managers rely on an informal process for assessing net worth; instead, I suggest that you think in terms of a more formal procedure. I assure you that in all the consulting research I did for the hospitality industry, not once did a client ask me to show that the value of my proposed study exceeded the price being charged.
I recommend that you think in terms of a Bayesian approach, which is a more formal approach to assessing worth. The point of this lecture is to alert you to what that analysis would entail.
Slide 2
As this cartoon suggests, Bayesian analysis is not a panacea for assessing the value for worthiness of marketing research. The trick is to assess the numbers that must be plugged into the analysis. However, despite its difficulties, I recommend a formal procedure as opposed to an informal seat of the pants procedure.
Slide 3
The next five slides contain a basic reading on the use of Bayesian analysis for assessing the value of information. I urge you to read these slides carefully before you proceed to the examples.
Slides 4 to 8 (No Audio)
Slide 9
Here’s example #1, which is an example of using Bayesian analysis and research to make a simple pricing decision. Let’s pretend we’re a marketing manager who must decide about a pricing strategy for a new product. This slide indicates the payoff table for that pricing decision. You’ll notice that this table contains alternatives, in terms of the marketing choices for a pricing strategy. The states of nature, which are possible demand levels for the product, are assumed as one of three possibilities: light demand, moderate demand, and heavy demand. The consequences are the intersections between those strategies or alternatives and possible states of nature. You’ll notice that the three possible levels of demand differ. The probability of light demand, all else being equal, is 0.6 (60%); the probability of moderate demand is 0.3 (30%); and the probability of heavy demand is 0.1 (10%). These are prior probabilities, in the sense that no research has been done to improve these estimates and the assumption is that they are based on historical data.
Looking at this combination of pricing strategies and possible levels of demand, we can perform the expected value calculation that is shown at the bottom of the slide. You’ll see the expected value, in terms of return to the company—we’ll assume millions of dollars—of choosing a skimming pricing strategy for this new product. There’s a 60% chance that demand will be light, in which case the company will earn $100 million; there’s a 30% chance that demand will be moderate, in which case the company will earn $50 million; and there’s a 10% chance that demand will be heavy, in which case the company will lose $50 million. If you add these weighted profits and loss together, you’ll find an expected $70 million profit. In doing the same calculation for the intermediate pricing strategy and the penetration pricing strategy, you’ll find values less than the expected $70 million profit. Thus, our optimal choice before conducting any research is the skimming pricing strategy because, on average, it will yield the highest expected profit of $70 million dollars.
Slide 10
Now let’s assume we can purchase some marketing research prior to making a decision about a skimming, intermediate, or penetration pricing strategy for this new product. The table indicates the likelihood of each test market result, given that ultimately what will occur is light demand, moderate demand, or heavy demand. You’ll notice that the numbers in each column sum to 1.0, but they don’t sum to 1.0 across rows. That’s because once light demand has occurred, then the probability is 1.0 (100%) that the test market result—assuming a test market has been conducted—was either disappointing or moderate or highly successful. If demand is moderate, there’s a 100% chance that the test market result would have been one of these three ways. If demand is ultimately heavy, there’s also a 100% chance that the test market result would have been one of these three ways.
Assuming demand ultimately will be light, there’s a 70% chance that a test market to predict demand will be disappointing, a 20% chance that a test market to predict demand will be moderately successful, and a 10% chance that a test market to predict demand will be highly successful. In this sense, you can see that the third result—the highly successful test market—is a very erroneous prediction of ultimate demand because demand ultimately will be light, and yet the test market suggested that demand will be heavy. You can read down the remaining columns and get the same sense. For another example, consider the last column. If you ran a test market to predict what demand will be and demand ultimately will be heavy, then there’s only a 10% chance that the test market will be disappointing; a 30% chance that the test market will be moderately successful, and a 60% chance that the test market will be highly successful.
Slide 11
Although the previous slide contains useful information—in this case, the historical accuracy of a test market for predicting ultimate demand—that’s just historical data and an intermediate step. We want to know the probability of a certain level of demand occurring, given a certain test market result. In other words, given a certain level of demand will occur, what is the probability of each test market prediction. This slide shows how to calculate these probabilities.
Given that we commissioned a test market and it produced a certain result, what are our revised probabilities for the ultimate level of demand: light, moderate, or heavy? This slide indicates how you would perform that calculation. Column #1 under each test market result lists the three different levels of demand. Column #2 and #3 list the prior probabilities that appeared in the previous slide. You may recall, based strictly on managerial intuition and historical results from products of this type, there’s a 0.6 or 60% chance that demand will be light, a 0.3 or 30% chance that demand will be moderate, and a 0.1 or 10% chance that demand will be heavy.
Column #4 also reflects the previous slide about the relative accuracy of such test markets to predict ultimate levels of demand. In this case, reading across the rows from the previous slide, there’s a 70% chance the test market will be disappointing, a 20% chance demand will be moderate, and a 10% chance that demand will be heavy. Notice that the three probabilities for each type of test market result in Column #4 need not sum to 1.0; they do by coincidence for disappointing test market result, but they do not for moderate or highly successful result. As in the previous slide, the probabilities sum to 1.0 down a column but not across a row.
To calculate the joint probability, think about the odds associated with flipping a coin. The odds of a coin coming up heads on one flip are 50%, and the odds of it coming up heads on two consecutive flips are 25%. The way we calculated that probability was by multiplying 50% by 50%. We assumed a fair coin and a fair flipper—the heads-versus-tails odds are even and the coin flips are independent—so we multiplied the probabilities of both outcomes to determine the probability of those outcomes occurring jointly. In essence, Column #5 provides that calculation for Columns #3 and #4. If the prior probability of light demand is 0.6 and the probability of a disappointing test market result given eventual light demand is 0.7, then their joint probability is 0.6 x 0.7, or 0.42. We can perform the same calculation for the remaining eight joint probabilities; those probabilities are 0.06 and 0.01 for the remaining disappointing test market result, 0.12, 0.18, and 0.03 for the moderate test market result, and 0.06 for each highly successful test market result.
Although an intermediate step, notice the probability of each test market result. The probabilities with an asterisk to the right are 0.49, 0.33, and 0.18, which sum to 1.0 because the probability of some test market result, assuming a test market is run, is 1.0.
The probability we really want is in Column #6; specifically, we want to know the probability of a certain level of demand (state of nature) given that we receive a certain test market result (rather than the probability of a certain test result given an eventual level of demand, as in Column #4). To calculate this last probability, we need to standardize the numbers in Column #5; in other words, force the probabilities associated with all possible levels of demand under each test market result to sum to 1.0. We make this calculation by taking the three probabilities grouped under each test market result in Column #5 and dividing them by the sum of those three probabilities. For example, there’s a 49% chance of the disappointing test market result (Z1). So take 0.42, 0.06, and 0.01 and divide each one by 0.49 to calculate those first three probabilities in Column #6. Note that those probabilities—0.858, 0.122, and 0.020—sum to 1.0. The results of the same calculations for a moderately successful test market result and a highly successful test market result produced the remaining probabilities listed in Column #6.
From a managerial perspective, it’s important to recognize the degree to which each research result revises initial estimates about probabilities for the different levels of demand. Before we commissioned any research, we were 60% confident that demand would be light, 30% confident that demand would be moderate, and 10% confident that demand would be heavy. Those probabilities appear in Column #3. Now combine those initial probability assessments with the test market results. If the result is disappointing, then all we’ve done is reinforce our initial assessment: we’re now 85.8% rather than 60% certain that demand will be light, 12.2% rather than 30% certain demand will be moderate, and 2% rather than 10% certain that demand will be heavy. So, if we commission a test market given our prior beliefs and the result is disappointing, then all we’ve done is reinforce our initial beliefs.
However, if we commission a test market and it’s highly successful—which is summarized at the bottom right of this slide—our assessment changes markedly. We’re now clueless about the likely level of demand; we’ve gone from 0.6—0.3—0.1 probabilities to equal (1/3rd) probabilities for light, moderate, and heavy demand. So, if the test market is disappointing, given our prior beliefs, then we’d probably be comfortable assuming light demand and choosing a skimming pricing strategy. If the test market is highly successful, then we’re totally uncertain about the likely level of demand and optimal course of action. In this latter case, we’d probably want to conduct additional research to forecast the level of demand.
Slide 12
Here are those expected value calculations, with revised probabilities for the ultimate level of demand, based on managerial intuition plus the results of the research. We can see that if the test market results are disappointing, then we recalculate the expected value accordingly. The expected value is even higher—only $70 million on Slide #9—at roughly $90 million for the skimming pricing strategy. If the test market is disappointing, then a skimming pricing strategy remains the optimal decision. However, the expected value calculation indicates that the optimal strategy shifts if the test market results is either moderately or highly successful; instead, the optimal decision is the intermediate pricing strategy. We can see that the research doesn’t contribute to a change in the decision if the test market result is disappointing, but if the test market result is either moderately or highly successful, then the manager is advised to modify the chosen pricing strategy; rather than selecting a skimming pricing strategy, an intermediate pricing strategy should be selected.
Slide 13
My point with the first example was merely to demonstrate how one would use expected values and research to revise marketing decisions. I never mentioned the value of the research. That particular dimension is added to this second Williams Company example. Before I begin, please note the following. First, Williams Company manufactures soft drinks and is considering running a special promotion that would cost $100,000, but it hasn’t had any meaningful experience with running such promotions and has no idea whether or not that $100,000 will be wasted or will be spent successfully. The Williams Company manager has no clear idea about the likely consumer response to this promotion. The manager’s confusion is reflected in the probabilities of S1, S2, and S3 in the middle of the slide; he or she is 30% confident that the promotion will cause a 10% or more increase in market share, is 40% confident that the promotion will cause a 5-10% increase in market share, and is 30% confident that the promotion will represent totally wasted money because it won’t increase sales. This is about as uncertain as you can get about the efficacy of a $100,000 promotional campaign. So what is this manager to do given this high level of uncertainty? As stated in the last paragraph on the slide, Williams Company could run a marketing research study that would cost $25,000 and include some copy testing. So, the manager’s decision seems straightforward: either (1) proceed to spend $100,000 on the promotion, which has a 30% chance of gaining no customers and costing $100,000, or (2) spend $25,000 on research first—a seemingly reasonable amount—to gain a better sense about the likelihood of a $100,000 loss. As a first step in deciding whether (2) is the best approach, the manager needs a sense of these copy tests’ historical accuracy in forecasting consumers’ response to such promotions.