Statistical and Psychological Factors Affecting Clinical Decisions
By:
Afshan Mirza
Mentors:
Mr. Jeffrey Madura
Dr. Keith Carroll
April 2, 2007
------
5/20/2007
Hi Mr. Madura,
So I went to the Scholars Banquet last night and got a very pleasant surprise and thought I'd share it with you... Everyone who graduated as a Scholar with Distinction was presented with plaques. When it was my turn, Dr. Mikula announced that I had the highest IRP grade ever! EVER!! Wow...it was so amazing!My whole family is so proud ofme and I feel a great sense of achievement too. If it wasn't for your help, I would've quit the program a long time ago and would never have discovered this hidden potential. So thank you very much once again.
Have a great summer!
Afshan Mirza
Table of Contents
Introduction…………………………………………………………………………... / Pg. 3Prevalence of Misdiagnoses………………………………………………………….. / Pg. 4
Misdiagnosis Due to Inadequate Understanding of Statistics
The Problem……………………………………………………………………….. / Pg. 4
The Solution……………………………………………………………………….. / Pg. 8
Misdiagnosis Due to Psychological Barriers
The Problem……………………………………………………………………….. / Pg. 12
The Solution……………………………………………………………………….. / Pg. 18
Conclusion…………………………………………………………………………….. / Pg. 24
Bibliography………………………………………….……………………………….. / Pg. 26
Medical Misdiagnosis
Introduction
A person faces choices from the moment of waking to the moment of falling asleep. Sometimes these choices are insignificant, like what color socks to wear; but at other times, they can mean the difference between life and death, like whether to drink and drive or to take a cab instead. All individuals tackle choices in their personal and professional lives. Personal choices often have an impact only on the person who makes them, whereas professional choices usually affect others.
Among the most important decisions made by professionals are those made by doctors. Physicians make choices in the testing, diagnosis, and treatment of patients many times a day. These decisions, when made well, can save a patient’s life or significantly reduce suffering. On the other hand, when made poorly they can cause extreme stress, pain, new illnesses, and even death. In order to reduce the possibility of thesenegative outcomes, doctors need to understand probability and statistics in medical testing and be able to interpret test results correctly. The required knowledge in this area can be provided through continuing education and application of Bayes’ Theorem. The use of computer programs designed to help with diagnosis and interpretation of test results will also assist doctors.
In addition to knowledge deficiencies, doctors often must overcome psychological hurdles that stand in the way of good decisions. The psychological pitfalls in decision-making under uncertainty need to be learned and taken into consideration in order to reduce occurrences of misdiagnosis and mistreatment in medicine. Currently, eighty percent of medical errors are the result of predictable mental traps or cognitive errors (Gorman). Being aware of one’s own mindset and ability to handle risky situations is the key to better medical decisions. Medical school curricula should include programs to teach students the risks of cognitive errors. In addition, doctors should use a formal decision-making approach to ensure their personal weaknesses do not get in the way of their professional decisions. Education, awareness, and action are necessary steps towards better medical care and safer treatment for all.
Prevalence of Misdiagnoses
Misdiagnoses are an unavoidable part of medicine. Any process that involves humans is bound towards imperfection. The problem lies not only in the presence of misdiagnoses, but more so in the prevalence of them. Each misdiagnosis may result in an increase of a patient’s pain, an increase in a patient’s medical bills, or even the patient’s death. According to a study done by the Institute of Medicine, close to 100,000 Americans die annually due to medical errors. Furthermore, at least 1.5 million people per year are harmed by medical errors, at an annual cost of around $3.5 billion (Kalb). Autopsy studies show that doctors seriously misdiagnose fatal illnesses approximately 20 percent of the time. Astonishingly, there has been no improvement in this rate since the 1930’s, according to an article published in the Journal of the American Medical Association (Leonhardt). The problem of misdiagnosis affects millions of people and their families each year. This is an issue that needs to be addressed immediately. But before something can be improved, it is essential to understand the causes of it.
Misdiagnosis Due to Inadequate Understanding of Statistics
The Problem
Doctors almost always rely on medical testing to diagnose a patient’s condition. From strep throat to leukemia, doctors use tests to determine the cause of the patient’s suffering. The cause of misdiagnosis, however, is not testing; it is the incorrect interpretation of the test results by inadequately-trained doctors. Every medical test has two main characteristicsthat must be understood in order to accurately analyze the test results. The first is the sensitivity of the test and the second is the specificity. Both of these factors need to be taken into account before reaching a diagnosis based on test results.
Sensitivity describeshow good a test is at correctly identifying people who have the disease. It is calculated by dividing the number of true positives (the number of people who tested positive for the disease and did in fact have the disease) by the total number of people who were sick (Loong).
Consider a population of 100 people, 40 of whom are sick with a certain illness.
Example 1: / Sick / Well / TotalPositive Test / 30 / 30 / 60
Negative Test / 10 / 30 / 40
Total / 40 / 60 / 100
In this example, the sensitivity would be 30/40, or 75%. Although it may seem like a test with 100% sensitivity would be ideal, this is not the case. A type of test with 100% sensitivity could be a test that is designed to always produce a positive result (Loong). A test like this might be meaningless due to an excessively high number of false positives, and would not help doctors make wise decisions. Medical tests should, however, aim to have a reasonably high sensitivity because it reduces the chances of false negatives.
False negatives are a very serious problem. Patients are always hopeful of walking out of the doctor’s office or the hospital with the assurance that they are perfectly healthy. A false negative gives them this artificial consolation and if they are sick, their disease goes undetected for a longer period of time. Depending on the disease, the time between the false negative and actual detection of the disease’s presence could prove to be fatal. Whether it is fatal or not, the growth of the disease during this time is inevitable, making it more difficult and perhaps more painful to treat. False negatives have greater chances of being discovered as such by taking the test again. But most people do not think of the possibility that although the test may have indicated that they are disease-free, they may still have the disease and the test could be wrong. Therefore, many people do not ask to be retested in the case of a false negative. For this reason and many others, medical tests should be designed to have a high sensitivity to decrease the number of false negatives.
The second characteristic of a medical test is specificity. This describes how good the test is at correctly identifying people who are well (Loong). It is calculated by dividing the number of true negatives (people who tested negative for a disease and were truly well) from a test by the total number of people tested who were well. In the above example, thirty well people tested negative and the total number of well people was sixty. The specificity of the test is therefore 30/60, or 50%. Tests with higher specificitiesare better because they reduce the chances of producing a false positive (healthy people who test positive for a disease).
False positives, like false negatives, also pose a threat to patient safety. If a patient is inaccurately diagnosed as having the disease, then there is a strong possibility that a wrong treatment will be given to him/her. Improper treatment could pose a danger to the patient’s life, as in the case of a four year old boy from Georgia. This child was diagnosed with leukemia and chemotherapy was scheduled to start for him in a few days. What the doctors did not realize, however, was that the little boy had a rare form of the disease, which chemotherapy did not cure. Furthermore, each round of chemotherapy had a high risk of killing him since he was so weak already (Leonhardt). Although dangerous, false positives are often less of a threat than false negatives. Because no one likes to be diagnosed with a disease, false positives have a greater chance of being identified by means of retesting and second opinions. On the other hand, the rate of their discovery is not high enough to undermine the need of reducing their occurrences. The aim to achieve higher specificity in tests should still be a top priority to decrease the number of false positives. The real goal for those who develop medical tests should be both high sensitivity and high specificity.
Sensitivity and specificity are important concepts to understand and properly utilize in order to make wise medical decisions based on testing. Many doctors, however, do not fully understand these ideas. Sixty doctors from HarvardMedicalSchoolteaching hospitals were given data about a test and asked to calculate what the chances were that a patient who tested positive did actually have the disease. They were told that the prevalence of the disease was 1 in 1000, the sensitivity of the test was 100%, and that the specificity of the test was 95%. Here is a table in the same format as the previous example to illustrate the data (the large numbers are needed to avoid fractional people):
Example 2: / Sick / Well / TotalPositive Test / 100 / 4,995 / 5,095
Negative Test / 0 / 94,905 / 94,905
Total / 100 / 99,900 / 100,000
Only 18% of the doctors answered correctly that a patient with a positive test result had less than a 2% (100/5,095 = .019627 = 1.96%) chance of having the disease. The low prevalence was the key factor in this scenario. Because the prevalence was so low, there were bound to be more false positives (4,995) than true positives(100) (Pradhan). Eighty-two percent of the doctors from a prestigious medical school known for its high standards of education were unable to analyze the situation correctly. This number would probably be higher if this test was given to students of other schools.
Another statistical error that occurs in medical decision making is conjunction fallacy. Conjunction fallacy occurs when a person concludes that the conjunction of two events is more likely than one of the events alone (Elstein). This cannot be true because in probability theory, the probability of the intersection of two events cannot be greater than the probability of one of the events alone. This faulty thinking can lead to a diagnosis that is more threatening than the true condition of the patient therefore putting the patient at risk of overtreatment.
The Solution
The best and simplest way of reducing the number of misdiagnoses due to an inadequate understanding of statistics is byeducating doctors in these matters. Providing physicians with the correct information on the significance of different statistical measures, especially sensitivity and specificity, and their correct applications will give them the resources they need to make intelligent decisions. Education on Bayes’ Theorem will be one of the keys to lowering misdiagnosis rates and moving towards safer healthcare.
Bayes’ Theorem is an easy way of calculating a conditional probabilitybased on the inverse conditional probability, which is known. For example, it is possible to calculate the probability that the patient has the disease given that the test is positive from knowledge of the probability that the test is positive for a patient who has the disease. In addition to this, two more pieces of information are needed for the calculation: the probability that a person has the disease and the probability of a false positive test result. The equation is as follows:
P = the event that the test gives a positive result
D = the event that the patient has the disease
W = (well) the event that the patient does not have the disease
P(W) = 1 – P(D)
Then
P(D|P) = P(P|D)P(D) .
P(P|D)P(D)+P(P|W)P(W)
The following is a sample application of the Bayes’ Theorem:
Problem: A test is known to detect a disease in a patient correctly 95% of the time (sensitivity = 95%). However, 15% of all disease-free patients who take this test also test positive (specificity = 85%). 10% of the population has the disease. Patient X has taken this test and has tested positive. What is the probability that he does in fact have the disease?
Solution: P(P|D) = 0.95 (sensitivity)
P(P|W) = 0.15 (1 – specificity)
P(D) = 0.10 (prevalence)
P(W) = 0.90 (1 – prevalence)
Using Bayes’ Theorem, P(D|P) can now be calculated using the probabilities above.
P(D|P) = (0.95)(0.10) .
(0.95)(0.10)+(0.15)(0.90)
P(D|P) = 0.095
0.23
P(D|P) = 0.4130
The probability that patient X does in fact have the disease, given that his test was positive, is only 41.30%(Waner).
This table reflects the facts above:
Example 3: / Sick / Well / TotalPositive Test / 95 / 135 / 230
Negative Test / 5 / 765 / 770
Total / 100 / 900 / 1000
Of the 230 patients with positive test results, only 95, or 41.30%, are actually sick. Here is Bayes’ formula using the terms sensitivity, specificity, and prevalence:
P (D|P) = (sensitivity)(prevalence) .
(sensitivity)(prevalence) + (1 – specificity)(1 – prevalence)
Bayes’ Theorem is a very important concept to be familiar with when dealing with any kind of testing, but especially when it comes to medical testing. If doctors were to assume that patient X had the disease without realizing that there was a 58.70% chance that he did not have the disease, then his health could have faced serious risks. Education in school and training in the workplace are the starting points of a better future for healthcare.
Another tool, aside from education, that will help in reducing the number of medical misdiagnoses is computersoftware designed for the purpose of aiding doctors with making wise medical decisions. These computer programs are commonly referred to as clinical decision support systems. They are defined as “active knowledge systems which use two or more items of patient data to generate case-specific advice” (“Clinical Decision Support Systems”). These systems can make a doctor’s decision-making process easier and more reliable in stressful or complicated situations.
There are four main functions of clinical decision support systems. The first is administrative. The systems support clinical coding and documentation. In addition, they authorize procedures and referrals. The second function is to manage clinical details and complexity. As a part of this function, systems keep track of referrals, follow-up, and preventive care of patients. The third function is cost control. Here, the systems are used to monitor medication orders and avoid unnecessary or duplicate testing. The fourth function of a clinical decision support system is decision support itself. This is perhaps the most important function of a decision support system. It supports the processes of clinical diagnosis and treatment plans. Furthermore, it promotes the use of “best practices, condition-specific guidelines, and population-based management” (“Clinical Decision Support Systems”).
The use of clinical decision support systems should be encouraged by hospitals and demanded by patients. Doctors need these systems to either help them make their decisions or to reconfirm them. Since doctors are expected to memorize and remember so much information, the chances are highthat they may overlook essential information at the time when it is needed the most. Statistical analysis formulas, such as Bayes’ Theorem, take time to work out and are sometimes difficult to remember. Computer programs make it more convenient and easier to apply these and other sophisticated case-management tools.
A study was done to test the significance of clinical decision support systems. In this study, the human and computer-aided diagnoses of 304 patients suffering from abdominal pain were compared. The most senior member of the clinical team achieved a 79.6% accuracy, while the computing system scored a diagnostic accuracy of 91.8% (“Clinical Decision Support Systems”). Not only is this an impressive percentage, but it is also an enormous improvement from the human performance level. This study successfully proved the value of computer-aided diagnoses in the medical sector.
Bayes’ Theorem and computer programs designed to assist in medical decision making are essentialin order to reduce the number of misdiagnoses due to incorrect statistical analysis. Using these two simple methods, doctors can provide their patients with a safer healthcare environment and a more secure feeling towards their medical concerns.
Misdiagnosis Due to Psychological Barriers
The Problem
Incorrect statistical analysis is not the only factor contributing to the high rates of misdiagnosis. Psychological barriers also play a large role in its prevalence. Unlike statistical errors, psychological blunders are more difficult to recognize and avoid. Technical errors are more commonly discussed while thinking errors are largely ignored (Gorman). A few simple steps, however, can ensure the path to a wise decision. But before a solution can be discussed, the problem has to be identified. This is often the most challenging task of the process. Yet, at the same time, it is often the most important.