CHAPTER FIVE
Sampling and Probability
NOTE TO INSTRUCTORS
In this chapter, students learn a number of fundamental concepts including probability and inference testing. Many students find inference testing confusing. Because students will need to use inference testing in future chapters, you should plan to spend extra class time on this concept. Make sure that students understand the differences between Type I and Type II errors by using everyday examples in your explanations, many of which can be found in the textbook. For example, consider using the jury decision example or the pregnancy test example described, as well as the classroom activity entitled “Application Through Study Design” and its corresponding handout (Handout 5-2), to help students apply their knowledge of Type I and Type II errors.
OUTLINE OF RESOURCES
I. Samples and Their Populations
Discussion Question 5-1 (p. 42)
Classroom Activity 5-1 : What’s Your Sample? (p. 42)
Discussion Question 5-2 (p. 43)
Classroom Activity 5-2: Sampling Scavenger Hunt (p. 43)
II. Probability
Discussion Question 5-3 (p. 44)
Discussion Question 5-4 (p. 44)
III. Inferential Statistics
Discussion Question 5-5 (p. 45)
IV. Type I and Type II Errors
Discussion Question 5-6 (p. 45)
Classroom Activity 5-3: Application Through Study Design (p. 45)
Online Resources (p. 45)
Additional Readings (p. 46)
V. Next Steps: The Shocking Prevalence of Type I Errors
VI. Handouts
Handout 5-1: Sampling Scavenger Hunt (p. 47)
Handout 5-2: Application Through Study Design (p. 48)
Chapter Guide
I. Samples and Their Populations
1.Samples can exist in two forms: random samples and convenience samples. A random sample is one in which every member of the population has an equal chance of being selected in our study. A convenience sample is one that uses participants who are readily available. Although random sampling is ideal, it is usually not done because it is expensive, time-consuming, and often impossible.
2.Generalizability or external validity refers to the researchers’ ability to apply the findings from one sample or in one context to other samples or contexts.
3.Replication reduces the risks and increases the rewards of sampling because we are attempting to duplicate our scientific results, ideally in a different context or with a sample that has different characteristics.
4.Although convenience sampling is more often used, it limits our external validity because we can never be certain that our findings will apply to the population as a whole. One way to deal with this limitation is to replicate our study. If we find the same results, we can be more confident in our findings.
Discussion Question 5-1
Why is a random sample typically not collected? Develop a research question, and determine how we would need to organize the study to use random sampling.
Your students’ answers should include:
Random sampling is expensive and is not easy to achieve without bias.
An example of a good research question would be: For the elderly living in the Boston area, does the context in which they recover affect the time it takes them to recover?
Classroom Activity 5-1
What’s Your Sample?
This activity would make an excellent homework assignment, or it can be used as an in-class activity if you have access to electronic journals in your class.
Have your students retrieve a primary-source psychology research article in an area of interest. Ask them to report to the class what sample was used in the study.
How many articles used true random sampling versus convenience sampling? Include in the discussion the issues of generalizability. Have the students discuss the different types of convenience sampling represented in the articles.
5.You should be especially critical of volunteer or self-selected samples. These are convenience samples in which participants actively choose to participate.
Discussion Question 5-2
What is a volunteer sample? What is a real-life example of when you would use this?
Your students’ answers should include:
Volunteer sampling is a selection method in which participants actively choose to participate in a study. A volunteer sample is a kind of convenience sample that is suspicious in nature because volunteers are likely to be very different from a randomly selected sample and may have many characteristics in common, such as income, education, geography, personality, and needs.
Real-life example of when to use the method would be:
Volunteer sampling is very often used because it is the most convenient method of all. If we want to just obtain a quick 100 participants, this is usually the best way to proceed. So, if we want to get a quick sense of what productivity is like in our corporations, we would ask for volunteers to participate in our study. However, we’d need to wonder whether there is something special about those workers who decide to participate in our study as opposed to those who decide not to participate.
Classroom Activity 5-2
Sampling Scavenger Hunt
In this activity:
Students will search the psychology literature or newspapers for examples of convenience sampling and answer questions about the sampling methods. See Handout 5-1, found at the end of this chapter, for suggested questions to ask.
Alternatively, as the instructor, you could provide the students with examples and use the questions from the handout in your discussion.
II. Probability
1.Confirmation bias is our usually unintentional tendency to pay attention to evidence that confirms what we already believe and to ignore the evidence that would disconfirm our beliefs.
2.An illusory correlation is the phenomenon of believing one sees an association between variables when no such correlation exists.
3.One type of probability is personal probability, which refers to an individual’s opinion or judgment about the likelihood that an event will occur. Although this sense of probability is often used in everyday language, when mathematicians or statisticians use probability, they’re referring to something more precise.
4.Probability is the likelihood that a certain outcome will occur out of all possible outcomes.
5.Statisticians use the term expected relative-frequency probability for the expected outcome if an experiment is repeated many, many times.
Discussion Question 5-3
How does subjective (or personal) probability differ from expected relative-frequency probability? Why do mathematicians or statisticians prefer to use expected relative-frequency probability rather than subjective probability?
Your students’ answers should include:
Subjective probability is our best guess, or a personal estimate, that an event will occur whereas expected relative-frequency probability is the expected outcome if an experiment were repeated many, many times.
Expected relative-frequency probability is more objective and, in the long run, quite predictable. Subjective probability is not scientific, and results from one guess to the next are unpredictable.
6.In probability, we use the term trial to refer to each occasion that a procedure is carried out. Our result of the trial is known as the outcome, and success refers to an occurence of the outcome for which we’re trying to determine the probability.
Discussion Question 5-4
Imagine that you flip a coin 50 times. How would you use the terms trial, outcome, and success to describe this task?
Your students’ answers should include:
Trial would refer to the number of times you flipped the coin.
Outcome would refer to the number of heads and tails when tallied.
Success would depend on what is being tested: If you are testing for the probability of heads, then success is heads.
7.Probability refers to the proportion that we expect to see in the long run. The proportion is the number of successes divided by the number of trials. In contrast, the percentage is just the probability or proportion multiplied by 100.
8.The term independence in probability refers to when the outcome of each trial does not depend in any way on the outcome of previous trials.
III. Inferential Statistics
1.A control group is a level of the independent variable that does not receive the treatment of interest in a study. It’s designed to match the experimental group (the group receiving the treatment or intervention of interest) in all ways but the experimental manipulation. In other words, the experimental group receives the experimental manipulation whereas the control does not.
2.The null hypothesis is a statement about populations that most often postulates that there is no difference between populations or that the difference is in a direction opposite from that desired by the researcher.
3.In contrast, the research hypothesis, or alternative hypothesis, postulates that there is a difference between populations or that there is a difference in a certain direction.
4.When we perform an experiment and obtain results that we analyze, we can either reject the null hypothesis or fail to reject the null hypothesis. We do not use the word accept during formal hypothesis testing.
Discussion Question 5-5
What is the difference between the null hypothesis and the research, or alternative, hypothesis? Why do we never accept the null hypothesis or the research hypothesis?
Your students’ answers should include:
The null hypothesis and the research, or alternative, hypothesis are opposites: The null hypothesis postulates that there is not a difference between populations or that the difference is in a direction opposite from that desired by the researcher; whereas the research hypothesis postulates that there is a difference between populations or, sometimes, that there is a difference in a certain direction, positive or negative.
We never accept either hypothesis because there are too many ways in which a real difference in the population might not get picked up by a sample.
IV. Type I and Type II Errors
1.A Type I error occurs when we reject the null hypothesis but the null hypothesis was correct. Type I errors are considered to be extremely detrimental to research because people might take action (e.g., perform additional experiments) based on a mistaken finding.
2.A Type II error occurs when we fail to reject the null hypothesis, but the null hypothesis was false.
Discussion Question 5-6
Why are Type I errors considered to be particularly detrimental to research?
Your students’ answers should include:
A Type I error is like a false-positive in a medical test. Researchers consider the consequences of a Type I error to be particularly detrimental, possibly even harmful, because people often take action based on a mistaken finding.
Classroom Activity 5-3
Applications Through Study Design
In groups or as a discussion with the entire class, develop a research question and design a study. Use Handout 5-2, found at the end of this chapter, to apply concepts from the chapter to your study design.
V. Next Steps: The Shocking Prevalence of Type I Errors
1.Because of the flaws inherent in research, numerous null hypotheses are rejected falsely, resulting in Type I errors. Some researchers have suggested that nearly half of published medical findings are Type I errors!
2.It is important to be aware of our own confirmatory biases that might affect our tendency to believe research findings without appropriate questioning.
Online Resources
This interactive Web site covers the basic logic of probability; use it in class as a demonstration or have students work on it on their own:
http://www.mathgoodies.com/lessons/vol6/intro_probability.html
This interactive probability Web site includes a number of different problems and their interpretations:
http://www.cut-the-knot.org/probability.shtml
Use this resource to help teach probability:
http://www-stat.stanford.edu/~susan/surprise/
Additional Readings
Nickerson, R. S. (2004). Cognition and Chance: The Psychology of Probabilistic Reasoning. Mahwah, NJ: Lawrence Erlbaum Associate Publishers.
One of the problems in teaching statistics to many of our psychology students can be traced to a more general problem in human cognition. This book focuses on cognition, raising issues related to our understanding (or lack thereof) of probabilistic reasoning.
Packel, E. W. (2006). The Mathematics of Games and Gambling, 2nd ed. The Mathematical Association of America.
This book discusses games of chance (e.g., roulette, craps, blackjack), social games (e.g., backgammon), and other gambling activities (e.g., lotteries), bringing out their mathematical aspects. It also includes information on game theory and game-related exercises.
PLEASE NOTE: Due to formatting, the Handouts are only available in Adobe PDF®.