Statistics for MBA Students: The Goal of MBA 8211
Rajiv Vaidyanathan
[Adapted from an article by Gary Smith(1998b)[1]]
Statistical thinking will one day be as necessary for efficient
citizenship as the ability to read and write.
—H.G. Wells
What do incoming MBA students need to learn about statistics in order to be more effective consumers of statistical information? A basic statistics course is usually a pre-requisite for admission into the program. Regardless of whether the content of the introductory statistics course (usually taken by the students several years ago) is fresh in their minds or not, they need to develop a better understanding not of how to perform statistical analyses, but on how to evaluate statistical information they may encounter in their positions as managers. Understanding and analyzing data involves an understanding of how analytical and graphical techniques can be employed to summarize, display, and analyze data. Additionally, the scope of this course includes an evaluation of how research design can affect the quality of reported results, irrespective of the quality of the statistical analysis. In effect, the focus of this course is on critically evaluating the quality of data even when reported statistical results lend it an aura of incontrovertibility.
Developing this course for the MBA class forced a reevaluation of the objectives of this course. This course has variously been taught as a research methods class, an intro to statistics class, and an advanced statistical applications class. Even when I taught the class earlier, I skimmed through some basic research design issues and maintained the focus of the class on applying advanced statistical techniques (e.g., conjoint analysis, multidimensional scaling, decision-tree analysis, etc.) in business decision making. Since then, I have rethought the goals of this class and decided it is far more managerially relevant and immediately useful to the students to focus on how commonly occurring reports of data can be extremely misleading and bias managerial responses to the data. Clearly, an objective was to try and overcome the widespread perception succinctly stated by Hogg (1991): “students frequently view statistics as the worst course taken in college.”
In many disciplines, introductory courses pose big questions and provide logical answers that students can grasp and, more importantly, retain without the use of sophisticated tools. Advanced courses examine the fine detail behind these answers and provide increasingly complex and subtle answers using ever more powerful techniques. The Data Analysis & Statistics course in the MBA curriculum was intended to provide these “broad tools” that will stand students in good stead as they progress through the more discipline-specific courses in their curriculum. For this reason, MBA students are encouraged to take this course early in their program.
There are many advantages to this approach—from the rough to the fine, from the coarse to the subtle, from the intuitive to the rigorous. One is that students who will never take another course in the discipline have an opportunity to learn some powerful principles that they can use throughout their lives. A second advantage is that students can be more easily persuaded that these concepts are powerful and useful.
It is sometimes argued that students who will go on to become professional statisticians should take a very different kind of introductory course. It is certainly true that professional statisticians should at some point take mathematically rigorous statistics courses. However, they too can benefit from an introductory course that emphasizes the application of statistical reasoning and procedures. Math in a vacuum can be misapplied. In addition, an introductory statistics course that shows the practical application of statistical reasoning may be just what is needed to persuade the mathematically inclined that statistics is a worthwhile career. James Tobin (1985), a Nobel laureate in economics, wrote of how introductory economics courses lured him into becoming a professional economist: “the subject was and is intellectually fascinating and challenging; at the same time it offered the hope that improved understanding could better the lot of mankind.” Wouldn’t it be wonderful if we could say the same of introductory statistics courses.
Some argue that the way to reform introductory statistics courses is to emphasize data analysis rather than mathematical technique (e.g., Bradstreet 1996; Cobb 1991; Hogg 1991); others argue that hands-on activities should replace passively received lectures (e.g., Fillebrown 1994; Gnanadesikan 1997; Ledolter 1995; Snee 1993). No single course will fit all; instead, a great deal of time was spent on thinking seriously about the course’s long-run objectives.
Most MBA students will be consumers of statistics, rather than producers. In order to be useful and memorable, the MBA 8211 course should prepare them for how they will encounter statistics in their careers and daily lives – it should prepare them to evaluate empirical evidence critically.
The central question is: What do we want to give our students in this one course, which may be their only opportunity to develop their statistical reasoning, that they will find useful long after the course is over? Surely not mathematical proofs, memorized formulas, and numerical computations. The most important lessons that students will take away from a statistics course will enable them to distinguish between valid and fallacious statistical reasoning. Thus, I have designed this course to focus on how data in general and statistics in particular can be misapplied, misinterpreted, and used to mislead managers into drawing incorrect conclusions about the “truth” in terms of actual marketplace conditions.
Each of us can make up a list of important statistical concepts that should be familiar to every educated citizen. Here areGary Smith’s (1998b) top-10 list, not in order of importance. Some are obvious candidates and self-explanatory. Those that are not are justified in more detail. We will discuss each of these concepts in this course. I fully expect many readers to look at some of the “errors” explained here and wonder what is wrong with the conclusions – this shall be answered during the course of this class.
1. Graphs: good, bad, and ugly Graphs can be used to summarize data and to reveal tendencies, variation, outliers, trends, patterns, and correlations. Useful graphs display data accurately and fairly, don’t distort the data’s implications, and encourage the reader to think about the data rather than the artwork. Because visual displays are intended to communicate information, it is not surprising that they, like other forms of communication, can also be used to distort and mislead. Whether using words or graphs, the uninformed can make mistakes and the unscrupulous can lie. Educated citizens can recognize these errors and distortions.
2. The power of good data Seemingly small samples can yield reliable inferences; seemingly large samples can yield worthless conclusions. It is important to understand the variation that is inherent in sampling (and how a margin for sampling error can be used to gauge this variation) and to recognize the pitfalls that cause samples to be biased.
A particularly widespread problem is the reliance on data from self-selected samples. A petition urging schools to offer and students to take Latin noted that, “Latin students score 150 points higher on the verbal section of the SAT than their non-Latin peers.” A psychology professor concluded that drunk-driving accidents could be reduced by banning beer pitchers in bars; his evidence was that people who bought pitchers of beer drank more beer than did people who bought beer by the bottle or glass. A study found that people who take driver-training courses had more accidents than people who had not taken such courses, suggesting that driver-training courses make people worse drivers. A Harvard study of incoming students found that students who had taken SAT preparation courses scored an average of 63 points lower on the SAT than did freshmen who had not taken such courses (1271 versus 1334). Harvard’s admissions director presented these results at a regional meeting of the College Board, suggesting that such courses are ineffective and that “the coaching industry is playing on parental uncertainty” (The New York Times, 1988). A survey sponsored by American Express and the French tourist office found that most visitors to France do not consider the French to be especially unfriendly; the sample consisted of 1000 Americans who had traveled to France for pleasure more than once during the preceding two years. A useful first exercise is to find the reasoning error in each of these examples. Can you see any problem with these conclusions? Why did these presumably intelligent and well-educated people draw conclusions which could be quite invalid?
3. Confounding effects In assessing statistical evidence, we should be alert for potential confounding factors that may have influenced the results. A 1971 study found that people who drink lots of coffee have bladder cancer more often than do people who don’t drink coffee. However, people who drink lots of coffee are also more likely to smoke cigarettes. In 1993, a rigorous analysis of 35 studies concluded that there is “no evidence of an increase in risk [of lower urinary tract] cancer in men or women after adjustment for the effects of cigarette smoking” (Viscoli, Lachs, and Horowitz, 1993). This problem is also related to the idea that correlation does not imply causation – addressed in more detail later.
4. Using probabilities to quantify uncertainty Probabilities clarify and communicate information about uncertain situations. Confidence intervals and p values clearly require probabilities. So does a useful assessment of any uncertain situation. Whenever we make assumptions that may be wrong, we can use sensitivity analysis to assess the importance of these assumptions and use probabilities to communicate our beliefs about the relative likelihood of these scenarios.
While it is not essential that students learn counting rules and other formulas that can be used to determine probabilities, they should be able to interpret probabilities and to recognize the value of using numerical probabilities in place of vague words. A memorable classroom exercise would be to ask students to write down the numerical probability they would assign to a medical diagnosis that a person is “likely” to have a specified disease. The answers will vary considerably. When sixteen doctors were asked this question, the probabilities ranged from 20 percent to 95 percent (Bryant and Norman, 1980). If the word “likely” is used by one doctor to mean 20 percent and by another to mean 95 percent, then it is better to state the probability than to risk a disastrous misinterpretation of ambiguous words. This is just one of many issues we will discuss while attacking the issue of questionnaire design.
5. Conditional probabilities. Many people do not understand the difference between P[A | B] and P[B | A]. Moore (1982) has argued that conditional probabilities are too subtle and difficult for students to grasp. I think that they are too important to neglect.
The application of contingency tables to an important issue can demonstrate conditional probabilities in a memorable way. One example is this hypothetical question that was asked of 100 doctors (Eddy, 1982): In a routine examination, you find a lump in a female patient’s breast. In your experience, only 1 out of 100 such lumps turns out to be malignant, but, to be safe, you order a mammogram X-ray. If the lump is malignant, there is a 0.80 probability that the mammogram will identify it as malignant; if the lump is benign, there is a 0.90 probability that the mammogram will identify it as benign. In this particular case, the mammogram identifies the lump as malignant. In light of these mammogram results, what is your estimate of the probability that this lump is malignant?
Of the 100 doctors surveyed, 95 gave probabilities of around 75 percent. However, the correct probability is only 7.5 percent, as shown by the following two-way classification of 1000 patients:
Test Positive / Test Negative / TotalLump is malignant / 8 / 2 / 10
Lump is benign / 99 / 891 / 990
Total / 107 / 893 / 1000
Looking horizontally across the first numerical row, we see that when there is a malignant tumor, there is an 80 percent chance that it will be correctly diagnosed; however, looking vertically down the first numerical column, we see that of the 107 patients with positive test results, only 7.5 percent actually have malignant tumors: 8/107 = 0.075.
According to the person who conducted this survey, “The erring physicians usually report that they assumed that the probability of cancer given that the patient has a positive X-ray...was approximately equal to the probability of a positive X-ray in a patient with cancer.....The latter probability is the one measured in clinical research programs and is very familiar, but it is the former probability that is needed for clinical decision making. It seems that many if not most physicians confuse the two.”
The solution is not for doctors and patients to stop using conditional probabilities, but to become better informed about their meaning and interpretation.
The popular press often confuses conditional probabilities. A Denver newspaper concluded that women are better drivers than men because more than half of the drivers involved in accidents are male. Los Angeles removed half of its 4,000 mid-block crosswalks and Santa Barbara phased out 95 percent of its crosswalks after a study by San Diego’s Public Works Department found that two-thirds of all accidents involving pedestrians took place in painted crosswalks. Researchers concluded that anger increases the risk of a heart attack because interviews with 1623 heart-attack victims found that 36 persons reported being angry during the two hours preceding the attack compared to only 9 who reported being angry during the day before the attack. The National Society of Professional Engineers promoted their national junior-high-school math contest with this unanswerable question: “According to the Elvis Institute, 45% of Elvis sightings are made west of the Mississippi, and 63% of sightings are made after 2 p.m. What are the odds of spotting Elvis east of the Mississippi before 2 p.m.?”
6. Law of averages The law of large numbers states that as the number of binomial trials increases, it is increasingly likely that the success proportion x/n will be close to the probability of success p. Too often, this is misinterpreted as a fallacious law of averages stating that in the long run the number of successes must be exactly equal to its expected value (x = pn) and, therefore, any deficit or surplus of successes in the short-run must soon be balanced out by an offsetting surplus or deficit. A gambler said that, “Mathematical probability is going to give you roughly 500 heads in 1000 flips, so that if you get ten tails in a row, there’s going to be a heavy preponderance of heads somewhere along the line” (McQuaid, 1971). Edgar Allan Poe (1842) argued that “sixes having been thrown twice in succession by a player at dice, is sufficient cause for betting the largest odds that sixes will not be thrown in the third attempt.” Explaining why he was driving to a judicial conference in South Dakota, the Chief Justice of the West Virginia State Supreme Court said that, “I’ve flown a lot in my life. I’ve used my statistical miles. I don’t fly except when there is no viable alternative” (Charlotte, West Virginia, Gazette, July 29, 1987).
The sports pages are a fertile source of law-of-averages fallacies. After a Penn State kicker miss three field goals and an extra point in an early-season football game, the television commentator said that Joe Paterno, the Penn State coach, should be happy about those misses because every kicker is going to miss some over the course of the season and it is good to get these misses “out of the way” early in the year. At the midpoint of the 1991 Cape Cod League baseball season, Chatham was in first place with a record of 18 wins, 10 losses. The Brewster coach, whose team had a record of 14 wins and 14 losses, said that his team was in a better position that Chatham: “If you’re winning right now, you should be worried. Every team goes through slumps and streaks. It’s good that we’re getting [our slump] out of the way right now” (Molloy, 1991).
A sports article in The Wall Street Journal on the 1990 World Series ended this way: “keep this in mind for future reference: The Reds have won nine straight World Series games dating from 1975. Obviously, they’re heading for a fall” (Klein 1990). In March of 1992, the Journal reported that, “Foreign stocks--and foreign-stock mutual funds—have been miserable performers since early 1989, which suggests a rebound is long overdue” (Clements, 1992). Four months later, the Journal repeated its error, this time reporting ominously that the average annual returns over the preceding ten years on stocks, long-term Treasury bonds, and Treasury bills had all been above the average annual returns since 1926. Their conclusion: “after years of above-average returns, many investment specialists say the broad implication is clear: They look for returns to sink well below the average” (Asinoff 1992).