Chapter 7
Lesson 13: The Sampling of Study Participants
Let's consider two independent studies of an (imaginary) antidepressant medication called "Zobutril." One study found that Zobutril reduced symptoms of major depression. A second study found no reduction of depressive symptoms. There is a contradiction here: it seems that Zobutril both reduces and fails to reduce the symptoms of depression. But, according to the law of noncontradiction (see Lesson 5), this is not possible: the antidepressant either improves or doesn't improve depression, it can't do both. How would you explain this apparent contradiction?
In Lesson 3, we looked at the issue of multifactorial causation: psychological phenomena are affected by a number of interacting factors. For example, in the case of major depression, a large number of psychological, interpersonal, sociocultural, and biological factors have been implicated in the development of this disorder (for a large collection of articles, see the Depression page on The Psychology Student web site). Furthermore, the symptoms of major depression vary from one person to another; and some people develop more severe depressions than others, presumably because they experience different combinations and strengths of important causal factors. For instance, a person, who has had a good upbringing in a nurturing family in which there is no evidence of severe mood disturbances, may develop a mild depression after the breakup of a romantic relationship. A second person, who lost both parents at an early age and was raised by an aunt who suffered from bipolar ("manic-depressive") disorder, may develop a severe depression after being given a pay cut at work. The difference in the responses of these two people to very different stressors may be explained by their unique combinations of causal factors.
Does this help you to answer the question about why the two studies produced different results? One thing you would want to look for are differences in the people who participated in each study. Let's say that the study that found an effect of Zobutril used hospitalized patients as subjects, and the study that found no effect used unhospitalized patients. How might hospitalization status have affected the results? Well, people who are hospitalized for depression probably have more severe symptoms than those who are not hospitalized. Thus, perhaps Zobutril works best for severe depression. Furthermore, biological factors may be more important for severe depression. Thus, perhaps Zobutril works best for depression that is caused primarily by biological problems. On the other hand, the two studies may have differed in their results simply because hospitalized patients are monitored more closely than unhospitalized patients, which may result in hospitalized patients taking their medications more consistently than unhospitalized patients. We would expect that consistent use of an antidepressant is important for its effectiveness.
Sampling From a Population
This example shows that, when interpreting the results of a psychological study, it is important to consider the characteristics of the people who are participating in it. The participants in a study represent a much larger group of individuals who are of interest to the investigators. In psychology, the term population refers to a group of individuals about whom we want to understand something. The population being studied sometimes is "all humans." But, often, psychological researchers are focused on more limited populations, such as "people with dyslexia" or "elderly Americans." In the Zobutril example, the population of interest was "people with major depression."
It should be obvious that, unless a population is very small, it is impossible to study every single member of that population. Thus, we need to "sample" the population. A sample is a set of people, drawn from a population of interest, who participate in a study. Researchers study the sample in order to understand something about the population. Thus, it is important that the sample be a "representative" one. A representative sample is a sample containing individuals who are similar in essential ways to the population of interest. There are many ways to achieve a representative sample. Most often, some sort of random sampling procedure is used, in which each person in the population has an equal chance of being selected for the study. For example, we might put everyone's name into a big drum, mix them up, and then select fifty people to be participants.
Biased Samples Limit Our Ability to Generalize
If we want to conclude something about a population of people based on the results of a particular study--to "generalize" from the sample to the population--it is essential that the sample be a representative one. For example, you all are aware of election polling in which a relatively small sample of people (often, only a few thousand) are asked whom they are going to vote for. Based on the results obtained from the sample, a generalization is made to the voting preferences of the entire national population. If the sample is not a representative one, however, the results are meaningless.
A famous example of this problem occurred just before the 1936 presidential election in the United States. Franklin Roosevelt was the incumbent president who was running against a challenger by the name of Alfred Landon. Landon was a Republican who was supported primarily by those who had survived the initial economic losses of the Great Depression and were still relatively well-to-do. The Democrat, Roosevelt, received the most support from people who were hit hard by the Depression. In order to predict the outcome of the election, a magazine called Literary Digest sent questionnaires to about 10,000,000 Americans (Classic Polling Surprises, 2002; Goodwin, 1995). Their sample included subscribers to the magazine as well as a large number of people selected from phone books and motor-vehicle registration records. The pollsters received responses from about 2.5 million people, which was an extraordinarily large number, especially considering that the population of the United States was much smaller than it is today. Almost 60% stated that they were going to vote for Landon, whereas only about 40% were voting for Roosevelt. Thus, the researchers predicted that Landon would win in a landslide. Another pollster by the name of George Gallup questioned only about 300,000 Americans that he had selected with a random sampling procedure. He predicted that Roosevelt would win. The results of the election: Roosevelt won in a landslide in which he received 60% of the popular vote.
What went wrong with the magazine's polling? It may not be immediately obvious today--a time period in which virtually everyone has at least one telephone and virtually every family owns at least one car. But in the middle of the Depression, car and telephone ownership were much less common because of the costs involved. In other words, wealthier people were more likely to appear in telephone books and in car-registration records. What the Literary Digest had done was poll primarily the well-off and Republican in a country that was mostly poor and Democratic. Another problem was that only about 25% of the original questionnaires were returned. It seems likely that there is a difference between the minority who would take the time to fill out a questionnaire and send it back and the majority who probably tossed it in the garbage. So, even having a sample of 2.5 million people does not guarantee that your results will provide an accurate picture. This very large number of people represented a biased sample, which is a sample that is skewed in a particular direction because of a bias in the selection process. In this case, the sample was skewed towards those with more money and who were Republican because of the bias introduced by sampling "highbrow" magazine subscribers, car owners, and telephone owners.
Representative Samples Are Not Always Used
Although a representative sample is ideal if we want to "generalize" our results to a wider population, there are other considerations that may make this goal difficult to achieve. First, it simply may be too expensive and time-consuming to randomly sample an entire population. In this case, the sample selected often consists of individuals who are readily available, such as college students taking an introductory psychology course. For many studies, this may not be an important limitation if there are good reasons to believe that most people would respond in similar ways to those in the sample. Although interpersonal communication (the main topic of Chapter 7) is strongly influenced by factors such as gender and culture, there are some aspects of interpersonal communication that probably can be studied in just about anyone, regardless of their gender, ethnicity, age, or nationality. For example, just about everyone uses their hands to emphasize what they are saying--even people who have been blind from birth and, therefore, could not have learned to do so by imitation (Corballis,1999). Furthermore, just about everyone uses "filler" sounds such as "um" and "er" in their speech--sounds that seem to have meaning in conversations (Whitfield, 2002). Thus, phenomena such as these may be studied profitably in "convenient" populations of participants, such as college students, without worrying too much about gathering a representative sample.
Second, researchers may want to eliminate from their sample those people who could make it more difficult to obtain clear results. For example, alcohol is thought to reduce the effectiveness of antidepressants. Because of this possibility, many studies of antidepressants exclude people who abuse alcohol. Although this procedure allows researchers to find an antidepressant effect more easily, it should be apparent that they can conclude nothing about the effectiveness of the antidepressant for the large number of people with major depression who abuse alcohol. In fact, studies of antidepressant medications often exclude people who have one or more of the following characteristics common in people with major depression: those who have only mild symptoms; those who have a history of drug or alcohol abuse; those who pose a suicide risk; those with particular anxiety disorders; and those whose mood disturbances include manic episodes or psychosis (Cole, 2002; Pirisi, 2002). One estimate suggests that between 80 and 90% of all depressed patients are excluded from antidepressant drug trials. Thus, it seems that our information on the effectiveness of antidepressant medications is based on only a fraction of the people who suffer from major depression. Some have argued that, because of this, we know very little about whether or not antidepressants are effective treatments for the majority of people who are prescribed them!
Critical Thinking Questions
Question 13-1
Let's say that you wanted to determine the percentage of spades, hearts, clubs, and diamonds in a deck of 1000 cards. Given what you have learned in this lesson, what would be the best way to go about doing this without having to look at all the cards? How many cards do you think you would need to draw to get a good estimate of the percentage of cards of each suit in the deck?
Suggested Answer
Question 13-2
Many studies in psychology, especially in the past, sampled primarily from the population of male college students in the United States. If this had been true of studies of nonverbal communication (discussed in Chapter 7), how might this have affected the results of the research?
Suggested Answer
Question 13-3
In a study of the life expectancy of male homosexuals in the United States, researchers examined obituaries and news stories about recent deaths in urban gay-community papers (reported in Olson, 1997). Based on the ages of death reported in these sources, the researchers estimated the average life expectancy of American gay men to be 43 years. Think carefully about this study's procedure and discuss whether or not the estimate of life expectancy should be trusted.
Suggested Answer
Question 13-4
When we talk about sampling, we often are speaking of the sampling of individuals from a larger population of individuals. In psychological research, however, the situation in which a study takes place also can be thought of as being one situation sampled from a much larger population of situations. Furthermore, we often find that results obtained in one research situation are different from those obtained in other research situations.
Provide one example from Chapter 7 in which a broader sampling of research situations resulted in an unforeseen complexity in communicative behavior.
Suggested Answer
Bibliography and References
Classic Polling Surprises. (2002). MathSoft Engineering & Education. Retrieved June 2, 2002, from http://www.studyworksonline.com/cda/content/new_worksheet/0,,EXP545_NAV2-76_SWK543,00.shtml
Cole, K. (2002, March 1). Antidepressant drug trials turn away most of the depressed population. Brown University News Service. Retrieved June 2, 2002, from http://www.brown.edu/Administration/News_Bureau/2001-02/01-091.html
Corballis, M. C. (1999, March/April). The gestural origins of language. American Scientist, 87(2). Retrieved June 3, 2002, from http://americanscientist.org/articles/99articles/corintro.html
Goodwin, C. J. (1995). Research in psychology: Methods and design. New York: John Wiley & Sons.
Nation, J. R. (1997). Research methods. Upper Saddle River, NJ: Prentice Hall.
Olson, W. (1997, December 19). William Bennett, gays, and the truth. Slate. Retrieved June 2, 2002, from http://slate.msn.com/?id=2098
Pirisi, A. (2002, March 2). Antidepressant drug trials exclude most "real" patients. The Lancet, 359(9308). Retrieved June 2, 2002, from http://www.thelancet.com/journal/journal.isa
Whitfield, J. (2002, May 28). 'Er' cautions listeners to stay on side. Nature Science Update. Retrieved June 3, 2002, from http://www.nature.com/nsu/020527/020527-2.html