Chapter 6 Measurement (Pp. 113-136)

Chapter 6 – Measurement (pp. 113-136)

Overall teaching objective: To introduce undergraduate criminal justice research methods students to the measurement process in social science research.

· Measurement is a process by which social science researchers determine how to assess the social phenomena they study.

Making Research Real 6.1 – I Can’t Measure It, But That Don’t Mean It Ain’t Real (p. 113)

· The author tells a story about his visit to a colleague.

· The colleague lives, by choice, in the house he grew up in economically disadvantaged neighborhood

· The author discusses how he felt unsafe in the neighborhood.

· Although it was not possible to objectively measure this perception – it was real.

· This is the challenge of the social scientist – to measure real things that cannot be readily observed.

Conceptualization and Operationalization (p. 114)

· Conceptualization is the process by which researchers develop precise definitions for vague concepts.

· Operationalization is the process by which researchers decide how to measure the variables as they are defined conceptually.

Making Research Real 6.2 – What is an Adult? (p. 114)

· Adulthood is traditionally measured objectively by the number of years since a person’s birth.

· ‘Adults’ are defined in most states as persons who are 17 or more years old or 21 or more years old.

· ‘Adulthood’ is a more complicated concept than an objective measure of age.

· One strategy for measuring adulthood is to establish the conditions that define adulthood.

o Completed schooling

o Left the parents’ home

o Financial independence

Making Research Real 6.3 – Operationalizing Our Definitions of Adulthood (p. 116)

· This story illustrates how a researcher might operationalize the previous concept of adulthood.

· The example uses survey questions and suggests a scale to measure adulthood.

Levels of Measurement (p. 117)

· Before discussing the levels of measurement it is important to learn a few new definitions.

· A variable is anything that varies. For example, age, height, weight, gender and income are all variables because they vary from person to person or from time to time within the same person.

· The variations within a variable are called attributes. So, there are two attributes for the variable gender – male and female. Theoretically, there are an infinite number of attributes for the variable income because a person’s income is unlimited.

Note to instructors: Variables and attributes are discussed in more detailed in Chapter 7. These concepts are discussed here to provide context for the discussion on the levels of measurement.

· The level at which a variable is measured has important implications. Generally speaking, higher levels of measurement provide more detailed information and allow for more precise analytical techniques.

· In the social sciences there are four levels of measurement: nominal, ordinal, interval and ratio.

o Nominal variables are merely names or categories. The attributes of nominally measured variables cannot be arranged in any logical order or sequence.

o Ordinal variables are also names, but their attributes can be ordered in a logical sequence.

o The attributes of interval variables can also be arranged in a logical order. But interval level variables have equal differences between their attributes.

o Ratio level variables are the same as interval level variables except that their attributes include an absolute zero.

· It is possible to convert a variable measured at a higher level of measurement into a variable measured at a lower level of measurement. However, it is not possible to convert a variable measured at a lower level of measurement into a variable measured at a higher level of measurement.

· There is another commonly used method for classifying the levels of measurement that bears mentioning.

o Some researchers define variables as either discrete or continuous. Discrete variables are similar to nominal or ordinal variables. These variables have a fixed number of attributes. For example, gender is a discrete variable because it only has two attributes – male or female.

o Continuous variables are similar to interval or ratio variables. These variables have an almost unlimited number of values.

Table 6.1 - Levels of measurement. (p. 119)

Levels/
Characteristics / Name only / Rank order / Equal distance / Absolute zero / Examples
Nominal / Yes / No / No / No / The races or ethnicities of adult offenders
Ordinal / Yes / Yes / No / No / Categories for the likelihood of reoffending
Interval / Yes / Yes / Yes / No / The number of years of formal education
Ratio / Yes / Yes / Yes / Yes / The number of dollars a person earns in one year

Making Research Real 6.4 – Converting from a Higher to a Lower Level of Measurement (p. 119)

· This story describes how and why a researcher might want to convert a variable (income) originally measured at the ratio level into an ordinally or nominally measured variable.

Making Research Real 6.5 – Trying to Convert from a Lower to a Higher Level of Measurement (p. 120)

· This story illustrates why it would not be possible for a researcher to convert a variable (income) originally measured at the ordinal level into an intervally or ratio level variable.

Validity and Reliability (p. 121)

· Social scientists rely on the concepts of validity and reliability in order to improve the quality of their measures.

· Validity refers to the ability of a measure to accurately measure the concept it claims to measure. There are four types of validity – face validity, criterion or predictive validity, construct validity and content validity.

Table 6.2 - Common types of validity. (p. 123)

Type / Definition / Example
Face / The extent to which a measure makes sense / A researcher wants a measure for an adolescent’s propensity to become a delinquent. They use the number delinquent peers as their measure. A peer reviewer argues that the measure has face validity because it makes sense as a measure of adolescent propensity to become delinquent.
Criterion/
Predictive / The extent to which a measure relates to another, more direct measure of the same concept / The Ames Pro-Social Inventory is intended to measure an adolescent’s ability to make pro-social (i.e. non-delinquent) decisions. To verify the criterion validity of this measure, a researcher might evaluate whether delinquents score lower on this inventory than non-delinquents.
Construct / The extent to which a measure corresponds to other variables that are related to the underlying construct / Researchers at the state parole commission develop the Parole Success Indicator (PSI) to measure a parolee’s potential post release success. They find that years married, years of steady employment and number of children are positively related to post release success. The PSI score would have construct validity if it was related to these three variables.
Content / The extent to which a measure includes all the dimensions of the concept / A psychologist believes that risky sexual behaviors are related to delinquency among juveniles. He designs a survey wherein he asks high school students about their sexual behavior. The researcher spends a lot of time creating questions that get at all aspects of risky sexual behavior so as to ensure that the measure has content validity.

· Reliability refers to the ability of a measure to consistently measure the concept it claims to measure. There are four methods for determining the reliability of a measure: test-retest, inter-rater, split-half, and benchmarks (concurrent reliability).

Table 6.3 - Common methods for assessing reliability. (p. 126)

Methods / Descriptions / Examples
Test-Retest / The researcher administers the measure twice to the same group to see if the results are similar. / A researcher has developed a fifty question test to measure a juvenile’s propensity to experiment with drugs. The researcher may administer this test to a sample of 50 juveniles. To evaluate the reliability of the test, the researcher may reorder the questions and re-administer the reordered test to the same group juveniles a week later. If the results are similar, then the measure is reliable.
Inter-rater / Multiple researchers independently measure the same thing and compare their results. / A team of three researchers observe cars at a busy intersection to determine the race or ethnicity of the drivers. At the end of the day they compare their results to see they perceived and classified the drivers’ races or ethnicities similarly.
Split half / The researcher splits a measure in half and administers each half to two similar groups. / A probation department develops a test to measure an offender’s potential for success during probation. The test measures fifteen dimensions of the potential for success. Each dimension is measured by two questions for a total of thirty questions. To assess the reliability of the test, the researcher splits the test in half, ensuring that each of the fifteen dimensions is included in each half. They administer one half to a sample of probationers and the other half to a similar sample of probationers. If the results are similar between the two groups, the measure is considered reliable.
Concurrent reliability (benchmarks and baselines) / The researcher uses previous research or generally agreed upon standards to test the reliability of a new measure. / Dissatisfied with the cost of the commercially available intelligence tests, a prison education program develops its own test. They administer the old and new test on the next group of inmates and compare the results. If the results are similar (i.e., intelligence levels are similar for each inmate on both exams), the new test is considered a reliable indicator of intelligence.

Making Research Real 6.6 – Are Searches an Accurate Measure of Racial Bias? (p. 122)

· This story discusses the use of searches as a measure of racial bias.

· A researcher proposes to use the frequency of searches as a measure of racial bias on the part of police officers.

· The researcher alleges that if police officers search a higher percentage of racial minorities during traffic stops then this is evidence of racial bias.

· But is this a valid measure?

· Not likely, one must consider the differences between the types and motivations for searches.

Making Research Real 6.7 – Testing the Personality of Police Officer Applicants (p. 124)

· An applicant is denied employment with a police department because the results of his MMPI suggest he is a psychopathic deviant.

· The applicant applies to another department, takes the same test and receives the same results.

· This suggests that the MMPI is a reliable measure.

Indexes, Scales and Typologies (p. 127)

· An index is a measurement device that compiles various measures into a single number.

· A scale is a method for quantitatively measuring a single phenomenon or variable using a rating system.

o The most commonly used scale in social science research is the Likert scale.

· A typology is a method for classifying observations, people or situations into nominal categories.

Making Research Real 6.8 – How Crime Ridden is Your City? (p. 127)

· This example illustrates how a researcher develops an index to measure the rate of crime in a community.

· It is an index because it measures only one thing – Part I crime.

Making Research Real 6.9 – Sorting Inmates by Behavioral Categories (p.129)

· A prison administrator develops a means to differentiate inmates into three risk of violence categories (low, medium and high).

· This typology uses criteria (age, drug used, criminal history and gang affiliation) to define each level of violence risk.

Quantitative Versus Qualitative Measures (p. 129)

· Quantitative measures rely on numbers and qualitative measures rely on descriptions to measure social phenomena. Often both are necessary to fully understand a concept.

Making Research Real 6.10 – A Framework for Understanding Poverty (p. 129)

· Traditionally, poverty is measured by income.

· Individuals with incomes less than the defined poverty line are considered impoverished.

· However, poverty also has several behavioral and structural definitions.

· Ruby Payne, an educator, proposes that poverty can be defined into a typology that considers financial, emotional, mental, spiritual, physical, support systems, relationships and knowledge of hidden rules.

Getting to the Point (Chapter Summary)

· Measurement is a process by which social science researchers determine how to assess the social phenomena they study.

· Conceptualization is the process by which researchers develop precise definitions for vague concepts.

· Operationalization is the process by which researchers decide how to measure the variables as they are defined conceptually.

· In the social sciences there are four levels of measurement: nominal, ordinal, interval and ratio. Nominal variables are merely names or categories. The attributes of nominally measured variables cannot be arranged in any logical order or sequence. Ordinal variables are also names, but their attributes can be ordered in a logical sequence. The attributes of interval variables can also be arranged in a logical order. But interval level variables have equal differences between their attributes. Ratio level variables are the same as interval level variables except that their attributes include an absolute zero.

· Social scientists rely on the concepts of validity and reliability in order to improve the quality of their measures.