Why Is It Relevant to Study Research

UNIT I: Knowledge Base

Why is it relevant to study research?

Evaluation

Effectiveness

CSWE

Development

Techniques

A social worker should be able to:

How do we learn?

Western perspective:

Types of knowledge and understanding:

Values:

Intuition and experience:

Revelation:

Authority:

Media:

Science:

The foundation of our knowledge and learning is always linked to observation in some way. In the scientific method, observation is an active part of learning. Measurement is used in the process to help reduce the likelihood of inaccurate observations.

So what is the scientific method?

In social work research there are two main approaches to using the scientific method.

Positivist (closely aligned with quantitative method)

Interpretive (closely aligned with qualitative method)

Ethical issues are the concerns, dilemmas, and conflicts that arise over the proper way to conduct research.

Types of scientific misconduct NOT involving a human subject:

Research fraud:

Plagiarism:

Misuse of Power:

Scientific misconduct involving a human subject:

Core ethical issues for research:

Do no harm:

Confidential versus anonymous:

Informed consent:

When a person cannot give informed consent:

GRACE-ful ethics:

UNIT II: Measurement

Research may be conducted at four main levels of knowledge:

1. Exploratory:

2. Descriptive:

3. Explanatory:

4. Developmental (also called Intervention or Evaluative):

These levels form the research knowledge continuum (from low level to highest). It is important to establish what knowledge level is being addressed in a project before a research question is designed.

Designing a good research question is fundamental to conducting good research.

How is a good research question developed - what is involved?

Literature review:

Unit of Analysis:

Ecological Fallacy:

Exception Fallacy:

Reductionism:

Spuriousness:

Tautology:

Teleology:

Concepts and the process of conceptualization:

Variables:

There are two types of variables in research:

1. Independent variable:

2. Dependent variable:

The way that a variable is conceptualized will impact the level of measurement used in a study. Levels of measurement in research categorize the degree of precision that can be obtained about a given phenomenon.

There are four levels of measurement (think of them as a continuum from lowest level to highest level)

1. Nominal:

2. Ordinal:

3. Interval:

4. Ratio:

It is important to know the measurement level before Operationalization occurs for a study (as measurement level can impact statistical analysis and comparison). Operationalization is:

Summary of a “good” research question versus a “bad” or even “mediocre” one:

Feasibility:

Social importance:

Scientific relevance:

We will discuss how to construct a good question in more detail in Unit IV.

UNIT III: Research Design

What is a needs assessment? Why would a needs assessment be helpful?

In designing a needs assessment the first thing to consider is what level of interaction the assessment will address.....whose need is being studied?

Needs assessments are also used in other types of research:

Cross-sectional study:

Longitudinal study:

Trend:

Cohort:

Panel:

The population’s views and personal concerns should always be foremost in a needs assessment. Not the needs and concerns of the researcher or funding agency.

What is a program evaluation? Why would a program evaluation be helpful?

There are two types of program evaluations:

Formative:

Summative:

Social workers often want to make inferences about causality from their research. There are three conditions that must be met for causality to exist between two variables.

When we consider the extent to which a study or evaluation permits causal inferences to be made about the relationship of variables, the concept of “validity” becomes important. Validity is “the extent to which we are measuring what we think we are measuring.”

There are two types of validity: internal and external. Each type of validity has threats to the confidence one has in the outcome of the measurement.

Internal validity is:

There are nine threats to internal validity.

1. History:

2. Maturation:

3. Testing:

4. Instrumentation error:

5. Statistical regression:

6. Selection bias:

7. Mortality:

8. Diffusion:

9. Reactive effects:

External validity is:

There are six threats to external validity:

1. Pretest - treatment interaction:

2. Selection - treatment interaction:

3. Multiple - treatment interference:

4. Researcher bias:

5. Reactivity (the Hawthorne effect):

6. Placebo effect:

Understanding validity is important to considering different types of research designs. Let us begin with a discussion of research shorthand.

“X”

“O”

“X ”

“O ”

“R”

“A”

“B”

“C - through Z”

The most basic research designs are known as “pre-experimental”. These designs are most appropriate for exploratory and descriptive level research studies.

Cross-sectional survey design:

One group post-test only (“one shot” case study):

One group pretest / post-test:

Static group comparison:

“Quasi-experimental” designs are more complex types of research and address more threats to internal and external validity than pre-experimental designs.

Interrupted time series:

Pretest / post-test comparison:

“Experimental” designs attempt to control for threats to internal validity more accurately than quasi-experimental designs do - by allowing for better manipulation and isolation of the independent variable. Because of this, the researcher can make the strongest claims of causality using experimental designs.

Experimental designs use random assignment to groups. Randomly assigned groups are called “control” groups.

Pretest / Post-test control group (“classical design”):

Post-test only control group:

Solomon Four group:

Whether using quasi-experimental or experimental designs there is are some ethical questions for social workers:

Characteristics of a good selection for the projects research design include:

What is practice evaluation? Why would practice evaluation be helpful?

There are descriptive designs and explanatory designs for practice evaluation.

Descriptive designs:

Explanatory designs:

The most common form of explanatory practice evaluations are known as “Single subject designs”:

Baseline period:

Experimental (intervention) phase:

In order to make measurements and therefore look at causality, three issues must first be addressed in single-subject designs:

Basic designs for single-subject (explanatory studies) include:

AB:

ABC:

ABAB:

Single subject designs (specifically AB) can be expanded to study multiple target behaviors, multiple clients, or to include multiple settings. This is called a “Multiple baseline design,” and it is useful because:

In single subject and multiple baseline designs the question is raised as to who is best qualified to be doing the measurements:

Key questions in interpreting single subject or multiple baseline designs include:

Knowing the results can be beneficial to the practitioner as well as the client. Results can verify positive changes, feelings.... therefore encourage continued progress toward goals.

Single subject and multiple baseline designs rely heavily on the use of graphs. We will discuss different methods of graphing in Unit IV.

UNIT IV: Sampling and Data Collection

Concepts relevant to selecting participants:

Population:

Parameter:

Sample:

Sampling Frame:

Element:

The ultimate purpose of sampling is to select a set of elements from a population in such a way that the descriptions of the elements (units of observation, statistical representation...) accurately portray the parameters of the population. Random selection is the key to this process. Random selection allows equal opportunity for an element to be chosen for the sample.

There are two main methods of sampling: probability and non-probability

Probability sampling:

There are four main types of probability sampling techniques:

Simple random sampling:

Systematic Random sampling:

Stratified random sampling:

Cluster sampling:

Non-probability sampling (also called purposive):

There are many types of non-probability sampling techniques. The textbook discusses 8 techniques, we will focus specifically on four.

Availability sampling:

Quota sampling:

Snowball sampling:

Key informants:

What influences the sample size needed for a research study?

Why is neutrality an issue in data collection?

Use of interviews as a data collection technique:

unstructured:

semi-structured:

structured:

If possible, interviews should be conducted face-to-face, why?

Advantages of interviews:

Disadvantages of interviews:

Other types of interviews and effectiveness:

Telephone:

Computer Assisted:

Use of questionnaires as a data collection technique:

Mailed:

+/-

Face-to-face:

+/-

Group:

+/-

What are some issues to consider when constructing a questionnaire (or interview guide)?

Be sure to consult table 9.1 on page 172 in the textbook.

Use of observation(s) as a data collection technique:

Unstructured:

Structured:

Participant observation:

Non-participant observation:

Use of logs and journals as a data collection technique:

Systematic Reviews:

Use of Meta Analysis:

Content Analysis:

Use of secondary data:

Use of scales as a data collection technique:

Likert scale:

Thurstone scale:

Bogardus Social Distance Scale:

Semantic differential scale:

Guttman scale:

Goal Attainment Scaling:

When more than one form of data collection is used in a study it is referred to as “triangulation.” Triangulation is especially common in validating data from qualitative research techniques.

In data collection, it is important to address the issues of reliability and validity.

Reliability is:

“When assessing the reliability of an instrument, you need to determine whether there is evidence of certain sources of error.” (p. 185)

Unclear definitions:

Use of retrospective information:

Variations in collection conditions:

Structure of instrument:

“Reliability is determined by obtaining two or more measurements and assessing how closely the measurements agree. Four methods are used to establish the reliability of an instrument” (p. 186):

Test-retest (stability):

Alternate form:

Split-half:

Representative:

Observer reliability (interrater):

We have already discussed validity in how it applies to program and practice evaluations (internal and external validity). For data collection, the concept of validity is expanded upon. Validity of an instrument used in measurement reflects the extent to which you are measuring what you think you are measuring.

There are several types of validity in assessing a measurement instrument - each type is tested in a different way.

Face validity:

Criterion validity:

Content validity:

Construct validity:

* In measurement, reliability can exist without validity. You can consistently measure the wrong thing.... but validity CAN NOT occur without reliability. You will not be accurate in your measurements if you are not consistent.

Sources of measurement error include:

Random error:

Systematic error:

- demographic variables:

- response set errors:

Response set errors due to personal styles of respondents:

Social desirability:

Acquiescence:

Deviation:

Response set errors due to reactions of observers:

Contrast error:

Halo effect:

Error of leniency:

Error of severity:

Error of central tendency:

Once the data has been collected the next step in the research process is to organize the information. To quote the textbook (p.202), “you need to be thinking about how the data will be organized as early in the research process as possible. This is especially important when you use a questionnaire to collect data, because the way questions are structured can influence the way data can ultimately be organized.”

Organizing quantitative data:

Coding the data:

The coding procedures for quantitative research studies are often developed before any data collection takes place. Once coding has taken place the next step in quantitative research is to do a statistical analysis (Unit V).

Organizing qualitative data:

Filing and coding:

Most qualitative researchers have colleagues or additional researchers verify the codes and categories before moving on to the analysis of the information.

UNIT V: Analysis of Non-numerical and Numerical Data

In qualitative research:

Qualitative data analysis is less standardized than quantitative analysis because:

“Grounded Theory”:

The basis of qualitative analysis is often a description - this includes:

The analysis process occurs by:

“Folk Terms” versus “Cover Terms”:

Domain Analysis is:

In qualitative analysis the emphasis is on finding patterns, understanding events and using models to present what is found.

To validate qualitative research, an alternative hypothesis is used. This hypothesis is:

Triangulation is important because:

Looking for missing information is as important in qualitative research as is connecting existing data. The missing information is often referred to as “Negative Evidence”. The nonappearance of something can reveal a great deal about a situation and can provide additional insights.

Common forms of negative evidence include:

- events that do not occur:

- a population not being aware of events:

- a population wanting to hide certain events:

- a population overlooking common place events:

- effects of preconceived notions:

- unconscious non-reporting:

- conscious non-reporting:

In addition to these forms of negative evidence, it is also important to consider the following when conducting non-numerical data analysis:

Results of analysis should:

Analysis of quantitative data often involves using descriptive statistics. Descriptive statistics:

Video: “Statistics at a Glance”

Descriptive statistics are used to:

What is a frequency distribution?

A vertical line in a graph is called (this is the Y axis).

The horizontal line is called (this is the X axis).

How can a frequency distribution be graphed?

What is a normal curve?

What is a skewed curve?

There are three measures of central tendency:

the mean:

median :

mode :

What does variability show?

Standard deviation is a measure of…..

A relative position of a score is called .

What does this show?

A percentile rank shows…..

Correlation is measured by using a .

How does this show correlation?

Correlation coefficients range from to .

One of the easiest ways to describe numerical data is to use a frequency distribution. Things to consider for a frequency distribution include:

At times you may want to look at the average of a value and to summarize information into a single perspective - this can be done by using the measures of central tendency.

Mode:

Median:

Mean:

If the values form a normal distribution it will present as a “bell-shaped” curve. In a normal distribution, the three measures of central tendency are equal.

If the values are not equal the result is a skewed distribution. (skewed distribution is “a distribution in which most of the scores are concentrated at one end of the distribution rather than in the middle” - textbook definition).

The measures of central tendency summarize the characteristics of the middle of the distribution, other characteristics of a distribution include the spread, dispersion and deviation. These characteristics are referred to as the measures of variability or measures of dispersion.

Range:

Percentile:

Standard deviation:

We have been discussing data analysis with research involving one variable (called univariate) to this point, but what kind of analysis occurs when the data involves two or more variables (called bivariate or multivariate analysis)?

Cross-tabulation:

Correlation:

In using descriptive statistics (whether univariate or multivariate) the most effective way to describe them is to display the results visually.

Interpreting graphs:

Level:

Stability:

Trends:

In graphing data -the X axis =

the Y axis =

Types of graphs include:

Bar graph:

Histogram:

Stem and Leaf Plot:

Frequency Curve:

Pie Chart:

Scatterplot:

Graphs and statistics can be deceiving. Be aware of any graph that does not have an absolute zero point for “Y” or that has a discontinuous scale.

Going beyond simply describing the data with descriptive statistics, you will use inferential statistics to test a hypothesis. Inferential statistics:

There are two forms of hypothesis:

Two-tailed (non-directional):

One-tailed (directional):

From the two-tailed or the one-tailed hypothesis, a null hypothesis is created. The null hypothesis:

A finding is considered to be statistically significant when the null hypothesis can be rejected and the probability that the result is due to chance falls at or below the study’s given significance level.

The power of a statistical test is its ability to reject the null hypothesis, and the power to reject the null hypothesis increases with the sample size.

Errors in judgment concerning the acceptance or rejection of the null hypothesis are referred to as Type I and Type II errors.

Type I error:

Type II error:

There are many types of statistical tests to determine which hypothesis is correct. The textbook in chapter 12 discusses steps to take to decide on a specific test and what conditions are appropriate for each. However, we will look at a few of the specific tests.

When a research study has a normal distribution and the dependent variable is measured at least at the interval level and the independent variable has been measured at the nominal level - the differences between the groups of data can be found using a T-Test.

A T-Test:

To calculate a T-Test (and many other statistical tests) you must know the degrees of freedom in your data (df).

The T-Test is a very efficient method of testing the significance of the difference between two means. However, not all research has hypothesis involving means that can be simplified to just two samples of scores. In social work, it is common for research to involve three or more samples of data to compare. This is when an analysis of variance is used (ANOVA).

An ANOVA:

Even though ANOVA looks at the differences between three or more groups - do not lose the fact that we are still involved with bivariate analysis at this point (one dependent variable and one independent variable).

Another way to analyze bivariate data is to examine the strength of the relationships between variables using correlation coefficients. Correlation coefficients:

Pearson’s r:

The regression line in a scatterplot graph is used to show how much change has occurred in the dependent variable are due to changes in the independent variable (prediction of change). Multiple regression analysis produces a coefficient that allows each particular outcome (interaction between variables) to be evaluated in direction and the amount of change.

Chi-Square analysis (X ):

As a descriptive statistic:

As an inferential statistic:

To use chi-square:

There are three types of significance that are encountered in analysis of group data and also single-subject studies.

1. Practical or clinical significance:

2. Visual significance:

3. Statistical significance:

The problem of “auto-correlation”:

Methods that can be used to lessen the chance of auto-correlation include using a celeration line:

In looking at the same results on a curve distribution, if the intervention mean is more than two standard deviations from the baseline mean, then there is a statistically significant change.