USING BAYESIAN ANALYSIS TO ASSESS THE VALUE OF CONDUCTING MARKETING RESEARCH
It makes business sense for managers to assess the value of conducting research. If the cost exceeds the value, then the manager shouldn’t conduct the research. After all, why pay more for something than it’s worth! That said, Mike can attest that in all the research he conducted for the hospitality industry, not once did a client ask him to show that the value of his proposed study exceeded the price being charged. Typically, those hospitality managers relied on an informal process for assessing net worth. Instead, we suggest that you rely on a more formal procedure. Specifically, we recommend that you use a Bayesian approach, which is a more formal way to assess likely value.
Although helpful, Bayesian analysis is not a panacea for assessing the value of marketing research. Certainly, such analyses are susceptible to the GIGO—garbage in, garbage out—problem. If estimates based on historical data and managerial intuition are inaccurate, then a formal procedure for combining those estimates will produce poor forecasts. Like all marketing research, Bayesian analysis is only as valuable as the data on which it’s based; regardless of method, you can’t make a good decision based on bad data. (Of course, you can always make a ‘lucky’ decision.) Hence, it’s vital that you plug well-founded estimates into your analysis.
Guessing or forecasting the likelihood of alternative outcomes may help you to make the most appropriate decision. Secondary data can assist in assigning a probability of a particular event or outcome. As such, you can calculate the expected value of an outcome by multiplying the probability of that outcome by its associated payoff (or loss). In essence, Bayesian analysis revises prior probabilities based on new information.
The following two examples show Bayesian analyses. The first example shows how marketing research can revise a basic pricing decision, and the second example shows how to assess the value of conducting marketing research. The Excel spreadsheet included with this CD will allow you to conduct an analysis similar to the second example.
Example 1
Here’s an example of using Bayesian analysis and marketing research to make a basic pricing decision. Assume you’re a marketing manager who must decide about a pricing strategy for a new product. Table 1 indicates the payoff table for that pricing decision: the likely profits for each pricing strategy (high, intermediate, and low) related to each possible level of demand (light, moderate, and heavy). The probability of light demand, all else being equal, is 0.5 (50%); the probability of moderate demand is 0.3 (30%); and the probability of heavy demand is 0.2 (20%). (Remember, probabilities may be expressed as percentages summing to 100% or decimal fractions summing to 1.0.) These are prior probabilities, in the sense that no research has been done to improve these estimates and the assumption is that they are based on historical data and/or managerial intuition.
Combining pricing strategies and possible levels of demand, you can compute the expected value (EV) for each strategy (noted beneath Table 1). The EV is what you would expect to occur ‘on average’, which is another way to say that you’re summing all possible outcomes after weighting them for their likelihood of occurrence. By adding these weighted profits and losses together, these calculations show that the high-price strategy is optimal, as you would expect the highest profit ‘on average’ from that strategy.
Table 1
Payoff Table for Pricing Decision
(Numbers in each cell in thousands of dollars)
Alternative / State of NatureS1: Light Demand
P (S1) = 0.5 / S2: Moderate Demand
P (S2) = 0.3 / S3: Heavy Demand
P (S3) = 0.2
A1: High Price / $100 / $55 / -$55
A2: Intermediate Price / $60 / $95 / -$30
A3: Low Price / -$40 / $0 / $70
EV (A1) = (0.5) ($100) + (0.3) ($55) + (0.2) (-$55) = $55.5 (optimal choice)
EV (A2) = (0.5) ($60) + (0.3) ($95) + (0.2) (-$30) = $52.5
EV (A3) = (0.5) (-$40) + (0.3) ($0) + (0.2) ($70) = -$6.0
Now, let’s assume that you can purchase marketing research prior to deciding on a high, intermediate, or low pricing strategy for this new product. Table 2 indicates the likelihood of each test market result given that ultimately what will occur is light demand, moderate demand, or heavy demand. In other words, how accurate is the research in forecasting eventual demand (or assuming a certain outcome, what is the likelihood of each research result.) Notice that the numbers in each column sum to 1.0, but they don’t sum to 1.0 across rows. The reason: once light demand has occurred, then the probability is 1.0 (100%) that the test market result—assuming a test market has been conducted—was either mildly or moderately or highly successful. If demand is moderate, there’s a 100% chance that the test market result would have been one of these three outcomes. If demand is ultimately heavy, there’s also a 100% chance that the test market result would have been one of these three outcomes. (Don’t be disappointed if you need to read this paragraph a few times.)
Assuming demand ultimately will be light, there’s a 0.6 (60%) chance that a test market to predict demand will be mildly successful, a 0.3 (30%) chance that a test market to predict demand will be moderately successful, and a 0.1 (10%) chance that a test market to predict demand will be highly successful. The third result—the highly successful test market—is a very erroneous prediction of demand because demand ultimately will be light, and yet the test market results suggest that demand will be heavy. For another similar example, consider the last column in Table 2. If you ran a test market to predict demand and it predicts demand will be heavy, then there’s only a 0.1 (10%) chance that the test market will be mildly successful, a 0.2 (20%) chance that the test market will be moderately successful, and a 0.7 (70%) chance that the test market will be highly successful.
Table 2
Conditional Probability of Getting Each Test Market Result Given Each State of Nature
Test Market Result / Level of DemandS1: Light / S2: Moderate / S3: Heavy
Z1: Mildly Successful / 0.6 / 0.1 / 0.1
Z2: Moderately Successful / 0.3 / 0.6 / 0.2
Z3: Highly Successful / 0.1 / 0.3 / 0.7
Σ=1.0 / Σ=1.0 / Σ=1.0
Although Table 2 contains useful information—in this case, the historical accuracy of a test market for predicting ultimate demand—it’s merely an intermediate step. If the potential marketing research supplier has conducted many past test markets, then this table could be constructed based on the previous accuracy of those test markets to forecast demand. However, what you really want to know is the probability of a certain level of demand given a certain test market result. In other words, given a certain level of demand will occur, what is the probability of each test market prediction? Table 3 shows how to determine those probabilities.
In Table 3, Column #1 and #2 list the test market success and three different levels of demand, respectively. Column #3 and Column #4 list prior and conditional probabilities, respectively. Column #5 reflects the joint probability of demand and success (derived from multiplying column 3 and 4). To calculate joint probabilities, think about the odds associated with flipping a coin. The odds of ‘heads’ on a single coin flip are 50%; the odds of heads on two consecutive coin flips are 25% (that is, 50% x 50%). If the coin and flipper are ‘fair’—the heads-versus-tails odds are equal and the coin flips are independent—then you multiply the probabilities of each outcome to determine the probability of those outcomes occurring jointly. Note that the three probabilities for each type of test market result need not sum to 1.0, although they may by coincidence. Therefore, the probabilities sum to 1.0 down a column but not across a row.
The values in Columns #3 through #5 are intermediate values. The probability of interest in Table 3 is found in Column #6; specifically, you want to know the probability of a certain level of demand (state of nature) given that you receive a certain test market result (rather than the probability of a certain test result given an eventual level of demand, as in Column #4). To calculate this probability, you must standardize the numbers in Column #5; in other words, you must force the probabilities associated with all possible levels of demand under each test market result to sum to 1.0. You make this calculation by taking the three probabilities grouped under each test market result in Column #5 and dividing them by the sum of those three probabilities. For example, there’s a 35% (0.35) chance of the mildly successful test market result (Z1). So, take 0.30, 0.03, and 0.02 and divide each one by 0.35 to calculate those first three probabilities in Column #6. Note that those probabilities—0.857, 0.086, and 0.057—sum to 1.0. The results of the same calculations for a moderately successful test market result and a highly successful test market result produce the remaining probabilities listed in Column #6.
It’s important to recognize the degree to which each research result revises initial estimates about probabilities for the different levels of demand. Before you commission any research, you are 50% confident that demand will be light, 30% confident that demand will be moderate, and 20% confident that demand will be heavy. Those probabilities appear in Column #3. Now combine those initial probability assessments with the test market results. If the result is mildly successful (Z1), then you’ve only reinforced your initial assessment: you’re now 85.7% rather than 50% certain that demand will be light, 8.6% rather than 30% certain demand will be moderate, and 5.7% rather than 20% certain that demand will be heavy. So, if you commission a test market, given your prior beliefs, and the result is only mildly successful, then all you’ve done is reinforce those beliefs.
Table 3
Revision of Prior Probabilities in Light of Possible Test Market Results
Test Market Result / Level of Demand / Prior Probability / Conditional Probability / Joint Probability / Posterior ProbabilityZK
(1) / SJ
(2) / P(SJ)
(3) / P(ZK|SJ)
(4) / P(ZKSJ)
(5)=3x4 / P(SJ|ZK)
(6)=5/Σ of 5
Z1: Mildly
Successful / S1: Light / 0.5 / 0.6 / 0.30 / 0.857
S2: Moderate / 0.3 / 0.1 / 0.03 / 0.086
S3: Heavy / 0.2 / 0.1 / 0.02 / 0.057
Σ=0.35 / Σ=1.000
Z2: Moderately
Successful / S1: Light / 0.5 / 0.3 / 0.15 / 0.405
S2: Moderate / 0.3 / 0.6 / 0.18 / 0.486
S3: Heavy / 0.2 / 0.2 / 0.04 / 0.108
Σ=0.37 / Σ=1.000
Z3: Highly
Successful / S1: Light / 0.5 / 0.1 / 0.05 / 0.179
S2: Moderate / 0.3 / 0.3 / 0.09 / 0.321
S3: Heavy / 0.2 / 0.7 / 0.14 / 0.500
Σ=0.28 / Σ=1.000
Σ=1.00
However, if you commission a test market and it’s highly successful, then your assessment changes markedly. You’ve gone from 50%—30%—20% (0.5—0.3—0.2) probabilities to 17.9%—32.1%—50.0% (0.179—0.321—0.500) probabilities for light, moderate, and heavy demand, respectively. So, if the test market is mildly successful, given your prior beliefs, then you’d probably be comfortable assuming light demand and choosing a high pricing strategy. In contrast, if the test market is highly successful, then you’re far less certain about the likely level of demand and optimal course of action. In this latter case, you’d probably want to conduct additional research to forecast the level of demand.
Table 4 summarizes the expected value calculations with revised probabilities for the ultimate level of demand based on managerial intuition plus the research results. If the test market results are mildly successful, then the expected value is recalculated accordingly and the expected value is even higher—$87,000 versus $55,500 in Table 1—for the high-price strategy. If the test market is mildly successful, then a high-price strategy remains the optimal decision. However, the expected value calculation shows that the optimal strategy shifts if the test market results are either moderately or highly successful; instead, the optimal decision is the intermediate-price strategy (A2) for a moderately successful test market and the low-price strategy (A3) for a highly successful test market. The post-research EV calculation shows that the test market result doesn’t encourage a strategy change if the test market is mildly successful; however, if it’s either moderately or highly successful, then you are well-advised to shift from a high-price strategy to an intermediate-price and low-price strategy, respectively.
Table 4
Expected Value (EV) of Each Alternative Given Each Research Outcome
(Numbers in thousands of dollars)
Z1: Mildly Successful (Disappointing) Test MarketRevised Probabilities: P(S1) = 0.857; P(S2) = 0.086; P(S3) = 0.057
EV(A1) = [($100) x 0.857)] + [($55) x (0.086)] + [(-$55) x (0.057)] = $87.2 (Best choice)
EV(A2) = [($60) x (0.857)] + [($95) x (0.086)] + [(-$30) x (0.057)] = $57.8
EV(A3) = [(-$40) x (0.857)] + [($0) x (0.086)] + [($70) x (0.057)] = -$30.2
Z2: Moderately Successful Test Market
Revised Probabilities: P(S1) = 0.405; P(S2) = 0.486; P(S3) = 0.108
EV(A1) = [($100) x (0.405)] + [($55) x (0.486)] + [(-$55) x (0.108)] = $61.2
EV (A2) = [($60) x (0.405)] + [($95) x (0.486)] + [(-$30) x (0.108)] = $67.2 (Best choice)
EV (A3) = [(-$40) x (0.405)] + [($0) x (0.486)] + [($70) x (0.108)] = -$8.7
Z3: Highly Successful Test Market
Revise Probabilities: P(S1) = 0.179; P(S2) = 0.321; P(S3) = 0.50
EV(A1) = [($100) x (0.179)] + [($55) x (0.321)] + [(-$55) x (0.50)] = $8.0
EV(A2) = [($60) x (0.179)] + [($95) x (0.321)] + [(-$30) x (0.50)] = $26.2
EV(A3) = [(-$40) x (0.179)] + [($0) x (0.321)] + [($70) x (0.50)] = $27.8 (Best choice)
Example 2