Chapter 11 – Non-Reactive Research Methods (pp. 269-295)

Overall teaching objective: To introduce undergraduate criminal justice research methods students to various non-reactive research methods and to demonstrate their applications.

Note to instructors: This chapter is presented in two sections. The first section provides an overview of the research method. The second section uses a research report to demonstrate how previous researchers applied this method to a project relevant to criminal justice practice. In both sections the material is organized by the generic research process that was presented in Chapter 2.

  • In Chapter 9 (Experimental Design Research Methods), you were introduced to a concept called reactivity.
  • Reactivity occurs when research subjects change their behavior because they become aware that they are being watched or measured.
  • To overcome this common threat to external validity, researchers may want to observe and measure the behavior of their research subjects in a natural setting. ‘Natural setting’ does not mean in the woods. It means observing how people behave normally in social situations.
  • To capture human behavior in natural settings, researchers have developed numerous non-reactive research methodologies.
  • Non-reactive research is a collection of research methods that unobtrusively gather information from research subjects that is, without their knowledge.
  • Because the research subjects are unaware that they are being observed, they are less likely to change their behavior.
  • Of course there are certain ethical considerations in this research method, but when done carefully, non-reactive research methods can be instructive to the body of knowledge.

Making Research Real 11.1 – Who Buys Gasoline Here?(p. 269)

  • This research project begins when a researcher observes that the buttons on the gasoline pumps are worn out in different places from one chain convenience store to the next.
  • In the lower socio-economic section of town the button over “Pay Inside Cash” is worn while in the higher socio-economic section of town the button over “Pay Outside Credit” is worn.
  • The instructor uses these erosion measures to make conclusions about the differences in the patrons in these two stores.

Non-Reactive Research Methods Basics (p. 271)

  • Non-reactive research methodologies have in common their ability to collect information from and about human beings without their knowledge.
  • These methods can include;
  • evaluating the things people possess (i.e., accretion measures),
  • studying how things are used (i.e., erosion measures),
  • observing how individuals or groups behave (i.e., unobtrusive observation), and
  • analyzing information collected and made available by someone else (i.e., archival data)..

Types of Non-Reactive Methods (p. 272)

  • There are three major types of non-reactive methods: field research, secondary analysis, and content analysis.
  • In field research, the researcher observes behavior(s) in a natural setting.
  • Another unobtrusive method is secondary analysis. Here, the researcher analyzes previously collected or archival data. Sometimes the data are collected for the purpose of another research project. Sometimes the data are collected and made public for scholars and practitioners to analyze as they see fit. In either case, a researcher would obtain the data set and analyze all or parts of it.
  • A third and final type of non-reactive method is content analysis. In a content analysis, the researcher analyzes existing textual information to study human behavior or conditions.

Making Research Real 11.2 – Field Research in Criminal Justice (p. 272)

  • A police captain responds to a neighborhood association’s complaint that somebody is stealing their morning papers.
  • There appears to be no pattern to these thefts.
  • The captain assigns an officer in an to (covertly) observe the neighborhood to catch the culprit.
  • The culprit turns out to be a dog.

Making Research Real 11.3 – Secondary Analysis in Criminal Justice (p. 272)

  • A warden is not supportive of a recent proposal to triple bunk prison cells.
  • The warden conducts a review of injury reports involving staff in the prison for the past 20 years.
  • The warden finds that as the population of the prison increases the frequency and severity of staff injuries also increases.

Making Research Real 11.4 – Content Analysis in Criminal Justice (p. 274)

  • An English professor is interested in how police officers communicate in writing.
  • She evaluates police reports and identifies common words and phrases.
  • Using this information she develops a training program to help the officers write more precisely.

The Benefits and Limitations of Non-Reactive Research Methods (p. 274)

Table 11.1 - The uses of non-reactive research. (p. 275)

:
  • When research subjects are likely to change their behavior after learning that they are being observed
  • When the researcher wants to observe behavior in its natural setting and/or as it naturally occurs
  • When the data has been previously collected by a different individual, group or organization

Table 11.2 - The limitations of non-reactive research. (p. 276)

  • Ineffective for studying the internal motivations, beliefs and attitudes that underlie some behavior
  • Potentially unethical if it involves breaching research subjects’ right to or expectation of privacy
  • Impossible in cases where secondary or archival data do not already exist or are unavailable
  • Non-reactive studies are often hampered by internal and external validity problems

Note to instructors: In this section the research method is discussed within the context of a relevant research project. The material is organized by the generic research process that was presented in Chapter 2. The story (i.e. the research project that provides the context) is presented in a series of set out boxes called Developing the Method. These are repeated here to allow instructors to use the research to teach the concepts, tools and techniques related to this research method.

The Non-Reactive Research Process (p. 276)

  • As in the previous two chapters, we will take a look at the actual research process to understand non-reactive methods in greater depth.
  • Following the steps of the research process outlined in Chapter 2, we will review the process by which a researcher might conduct field research, secondary analysis and content analysis.
  • To illustrate the process further, I will highlight one research study that involved a content analysis. We begin with a general introduction of this case study.

Developing the Method 11.1 - A Case Study in Non-Reactive Research (p. 276)

Several years ago, when my children were young, my wife announced, “From now on, we are not going to watch the evening news! It’s too violent.” She was right. We were living in a large city at the time and it seemed like the evening news was filled with stories about murder, rape, assault, robbery and other violent acts. “If it bleeds, it leads” is a common saying in news rooms. In short, crime ‘sells’.

The tendency of media outlets to focus on crime and violence has the unfortunate effect of creating false perceptions regarding crime. After a few weeks of watching the evening news, one might think that crime is both common and getting worse. Butthe statistics indicate that crime is both uncommon and decreasing. In fact, since 2001, violent crimes have decreased 13.2 percent.

To better understand media portrayals of crime, Melissa Hickman Barlow, David E. Barlow and Theodore G. Chiricos set out to conduct a content analysis of media content on crime. They wanted to know whether economic conditions had an effect on how the media portrayed crime. In 1995, they published their research in the scholarly journal Crime and Delinquency under the title “Economic Conditions and Ideologies of Crime in the Media: A Content Analysis of Crime News”. Their research is an excellent example of content analysis and non-reactive methods in general.

Ask a Research Question (p. 277)

  • Most non-reactive research is exploratory or descriptive nature. Because it is often difficult to measure the underlying cause of behavior using a non-reactive research method, non-reactive methods are less often used in explanatory research.

Developing the Method 11.2 - Asking a Research Question in Non-Reactive Research (p. 277)

Barlow et al. (1995) state that their study is exploratory, which would be appropriate for a content analysis. However, the authors propose that political and economic conditions might help explain how the media portrays crime. Thus, it appears that there are pieces of this study that are explanatory in nature. In their own words, they wanted to know “whether media accounts misrepresent crime in ways that support dominant class interest and whether misrepresentation changes in relation to conditions in the political economy” (Barlow et al. 1995: 6). In other words, they are asking whether the media reports on the crimes that most of us want to read about because of what is happening at the time both politically or economically.

Conduct a Literature Review (p. 278)

  • Reviewing the research methods and mistakes of previous researchers who used non-reactive research techniques may help define concepts, access certain populations, and avoid detection in non-reactive research.

Developing the Method 11.3 - Conducting a Literature Review inNon-Reactive Research (p. 278)

In their review of the literature, Barlow et al. (1995) discovered a rather robust research history. They found numerous sources that confirmed what we all suspected: “crime news distorts and/or frames crime and crime control in ways that support institutions of power and authority” (Barlow et al. 1995: 3). In short, there are political and economic interests involved in how crime is portrayed in society. Though the literature suggested that media reports on crime painted a distorted picture of crime, previous research had not determined how these distortions changed with changing political and economic conditions. So, the researchers decided to proceed with their study.

Refine the Research Question (p. 279)

  • In non-reactive research that is exploratory or descriptive in nature, researchers may simply pose research questions and/or general statements about what they expect to find in the study.
  • In non-reactive research that is explanatory in nature, researchers will develop more formal hypotheses.

Developing the Method 11.4 - Refining the Research Question in Non-Reactive Research (p. 279)

Barlow et al. (1995) predicted that newspaper coverage of violent crime would not only overstate the problem of violent crime, but that coverage would be most intense during times of unemployment and economic stagnation. Thus, they presented two sets of hypotheses at the outset of their article. The first set of hypotheses focuses on the depiction of violent crime in news articles:

Hypothesis 1: The proportion of news articles that are about crimes of violence is significantly larger than the percentage of violent crimes known to the police.

Hypothesis 2: The proportion of news articles on crimes of violence is greater during high-unemployment years than during low-unemployment years.

Hypothesis 3: The proportion of news articles on crimes of violence is greater during periods of economic stagnation than during periods of economic expansion.

The researchers’ second set of hypotheses focused on the “characteristics and images of offenders within crime news articles” (Barlow et al. 1995: 10):

Hypothesis 4: A significantly larger proportion of non-White offenders are depicted in the news articles than the proportion of non-White offenderswho are actually arrested.

Hypothesis 5: When the social class of the offender is mentioned, the largest proportion of social class descriptions is in the lower class category.

Hypothesis 6: The proportion of negative images of offenders is greater during high-unemployment years than during low-unemployment years.

Hypothesis 7: The proportion of negative images of offenders is greater during periods of economic stagnation than during periods of economic expansion.

In general, the researchers predicted that newspaper coverage would highlight the non-White and lower class characteristics of criminal offenders during times of unemployment and economic stagnation. That is, the media would focus disproportionately on non-White, lower class criminal offenders during economically difficult times.

Define the Concepts and Create the Measures (p. 280)

  • For the most part, the actual process of conceptualization and operationalization is the same in non-reactive research as it is in other research methods.

Developing the Method 11.5 - Defining Concepts and Creating Measures in Non-Reactive Research (p. 281)

Barlow et al. (1995) propose that two elements of crime reporting will vary by economic conditions: (1) the type of crimes that is reported and (2) the characteristics of offenders that are highlighted. In essence, they define the concept of ‘media coverage of crime’ in two parts.

To measure the type of crime reported by the media, the researchers differentiated between two types of crime: violent and non-violent. Thus, when the researchers analyzed media content, they coded the content as focusing on violent or non-violent crimes. To measure the characteristics of offenders that are highlighted, the researchers included the following variables: age (under 30 or 30+), race (White or Non-White), Gender (Male or Female), social status (Lower, Middle, or Upper), Employment (Unemployed, Blue-collar, or White-collar), Marital status (Married or Not married), Family history (Positive or Negative), Education (High school + or Less than high school), Friends (Has friends or Is isolated), and Religion (Religious or Not religious). It is likely that the researchers selected these variables because their review of the literature suggested that factors like age, race and gender influence whether and how criminals are depicted in the media.

The authors conceptualize ‘economic conditions’ in two ways: (1) the level of unemployment and (2) the level of economic stagnation. The authors do not explicitly define what they mean by ‘high’ or ‘low’ unemployment. Instead, they decide to analyze media coverage during the year 1953, when there was a relatively low unemployment rate (2.9%), and 1958, when there was a comparatively high unemployment rate (6.8%). Likewise, the authors do not define what they mean by ‘economic stagnation’ and ‘economic expansion’. Again, they decide to analyze media content during years that are popularly known to be periods of economic stagnation. Those years were 1975, 1979, and 1982, during which the unemployment rates were 8.5%, 5.8%, and 9.7%, respectively (Barlow et al. 1995: 6).

Design a Method (p. 282)

  • Non-reactive field research involves covert observation of and, in some cases, covert participation in the behavior that a researcher is interested in studying. In this method, it is important to prepare for all possible contingencies that may arise in the field and all possible threats to researchers’ safety.
  • The use of secondary data is appealing to most researchers because of its availability and low cost. However, the secondary data must be responsive to the researcher’s data needs and research question(s).
  • The most important step in designing a content analysis is to create strong conceptual definitions and operational measures for the phenomena that are being studied. In some cases, this may involve training research assistants.

Developing the Method 11.6 - Designing a Method in Non-Reactive Research (p. 284)

To study media coverage of crime at different periods of time, Barlow et al. (1995) decided to study articles appearing in Time magazine. The researchers chose Time because it was a widely circulated news magazine and “the best available representative of mainstream media as an influence on and reflection of popular consciousness concerning contemporary social issues” (Barlow et al. 1995: 6). In short, the authors suspected that the magazine might reflect changing public opinion and cultural mores.

An additional consideration for selecting Time magazine was that the magazine had a long publication history, making it a suitable source of longitudinal information on changing portrayals of crime. Their selection of this publication enabled the authors to examine “the content of crime news articles…at different points in the postwar period” and to analyze “media representation of crime in relation to changes and developments in the political economy” (Barlow et al. 1995: 6).

Collect the Data(p. 284)

  • Coding sheets provide researchers with an effective and efficient mechanism for recording information in content analyses and field observations.
  • Recording observations is a central part of field research. Video- and audio-taping observations is the most accurate recording method, but these are also the most reactive and, in some cases, they are also unethical. Recording observations on paper or a laptop are the least accurate, but also the least reactive. Researchers must find the appropriate balance between accuracy and obtrusiveness.

Developing the Method 11.7 - Collecting Data in Non-Reactive Research(p. 287)

During the years on which their analysis focused (1953, 1958, 1975, 1979, and 1982), Barlow et al. (1995) chose articles that were “completely or substantially about crime, criminals, or criminal justice.” Only articles that were at least one column in length were chosen. A total of 175 articles met these criteria. Technically speaking, this would be a purposive sample. Only the articles on crime appearing during the years they chose were included in their analysis. The researchers in this case did not use a random sample presumably because no sampling frame existed. In this case, the researchers would have had to identify every article on crime ever published in a written and widely circulated American news source during the selected years. Though this was arguably possible, it was not feasible or efficient. Instead, the researchers chose a representative news source to study media portrayals of crime. Though their sampling strategy was not random, it provided important insight into human behavior.

Once the articles were identified, the researchers turned their attention to conducting the actual content analysis. To do this, they read each article and collected information related to the variables in which they were interested. First, the researcher differentiated between articles about violent and non-violent crime, which was relatively straightforward (Barlow et al. 1995). Collecting information about the characteristics of the offenders was a bit more problematic. Few articles included information on each of the ten variables in which the researchers were interested. For example, very few of the articles included information about the offenders’ education, relationships with friends, religion or marital status. In short, their analysis did not yield information about every variable related to the characteristics of the offender.