Some of the Most Popular Designs in Educational Research
All research requires thoughtful writing, but quantitative research findings are presented in numbers, treated with statistical procedures. Researchers in this milieu carefully plan their methodologies. Part of their credibility rests on how rigorously those methodologies are applied. Quantitative research involves deductive logic (general to specific), to obtain information that will help solve a particular problem. Controls to facilitate objectivity are emphasized. In education, quantitative research is usually pursued through quasi-experiments, survey or questionnaire studies, standardized tests, and/or observations, using samples of subjects that are randomly selected.
By contrast, qualitative research involves inductive logic (specific to general), to understand the meaning of a situation and its importance to the human condition. Findings are presented in words. Qualitative researchers apply emergent designs – they revise their plan and methodologies as new data become available. Qualitative research often uses interviews, observations, and/or artifacts, obtained from a carefully selected social scene or through work with purposefully selected informants. Qualitative researchers learn from and through their immersion in the study. This is accomplished through adept use of participant language in naturalistic settings, with a reliance on researcher skill rather than through inherent capabilities of a data collection instrument. In qualitative research, the researcher IS the instrument!
There are three headings on the syllabus chart, but all twelve designs can be considered in two general categories: quantitative and qualitative. The attributes of your study plan will be established by the quantitative/qualitative categories. Seven of the designs are essentially quantitative, because all of the program evaluation designs, except Naturalistic Evaluation, are quantifiable. The other five designs are qualitative.
Label your design, so your audience will know the general parameters of your study. Be sure it includes either objectives or a complete data collection instrument, projected sample size and selection procedure, and your strategy for obtaining permission for the study. In addition, quantitative designs should identify an appropriate statistical treatment procedure, and qualitative designs should have clearly articulated foreshadowed problems:
QUANTITATIVE DESIGNS PROGRAM EVALUATION QUALITATIVE DESIGNS
Relationship survey Objectives-based evaluation Historical study
Descriptive survey Discrepancy Evaluation Model Oral history project
Quasi-experimental study Naturalistic Evaluation Ethnographic case study
Delphi probe Evaluability Assessment Policy study
Summary of Selected Statistical Procedures
The following material is based on the assumption that the widespread use of statistical computer software has diminished the need for introductory students to memorize mathematical formula. It presents information about some of the most commonly used quantitative summary procedures: Name, Reference (from the McMillan and Schumacher text), Function, an Example of how it can be used, Relevant Terms, and Attributes. See also Locke, L. F., Silverman, S. J., & Spirduso, W.W. (1998) Table 6.1, pp. 126 - 127.
Procedure / Name / Reference Function Example of What it Can be Used to Study
t-test, To compare the means of Attitudes of 4th and 6th graders,
pp. 363-368 two groups. pre and posttest scores, etc.
Chi-Square, pp. 379-382 To compare observed frequencies The number of students who used
with expected frequencies, especially counseling center services,
when the independent variable is subdivided. according to grade level.
Analysis of Variance To compare two or more sample groups The effects of three different
(ANOVA), pp. 368-370 with one independent variable. treatments on posttest achievement
Factorial Analysis of To compare two or more sample groups Whether type of treatment and
Variance, pp. 371-376 with two or more independent variables. level of anxiety improve
achievement.
Analysis of Co-Variance 1) To adjust initial, uncontrolled group 1) When two groups have different
(ANCOVA), differences related to the dependent pre test means, ANCOVA can
pp. 376-378 variable, or help identify the significance of
pre/posttest contrasts.
2) To increase the likelihood of finding 2) When intact groups are used
a difference in the means of small groups. with randomization--however,
ANCOVA cannot “equalize” them
(cannot match or randomize).
Multivariate Analysis of In the generic sense appropriate Whether the effect of many specific
Variance (MANOVA), to introductory research, for component attitudes toward science
pp. 380-384 comparisons in studies affect a general attitude (enjoyment,
with many variables. appreciation of physics, respect for
chemistry, opinion of dissection,
benefits of field trips, and so forth).
Multiple Correlation To add together the predictive The predictive value on teacher
Tests (Multiple Regression) power of several independent effectiveness of 1) teacher
evaluation scores,
pp. 226-29, 290 variables, and express each so 2) college GPA, 3) ratings by
they can be compared and contrasted. references, and 4) interview
ratings, and 5) written self-report
by teacher.
Wilcoxon Rank Sum To check whether two populations haveIs student achievement greater in a
the same medians, especially sample of independent schools, or in
when it is necessary to compare the population of schools in
data from one population with statewide school districts.
sensitive data from another sample. Note: Rank sum procedures are not to extreme outlying
Meta-Analysis, To statistically summarize results of Review of 282 studies to identify
pp. 147-148 prior independent studies.strategies that can help reduce
recidivism (“What works?”).
Relevant Terms
INDEPENDENT VARIABLE: (AKA experimental or manipulated variable.) The measured data that precedes, or is antecedent to, the dependent variable; the cause--time directed to study, motivation, voluntary enrollment, etc...
DEPENDENT VARIABLE: The measured data that is the consequence of another measured variable; the effect--achievement scores, grade point average (GPA), diminished disciplinary reports, numbers of objectives attained, etc...
UNIVARIATE DATA: Measured data that involve only one variable within a population, with all variables held constant except the one studied.
MULTIVARIATE DATA: In the generic sense, measured data that involve more than one variable within a population.
PARAMETRIC TESTS: Statistical procedures that are based on the assumption of normality (the “bell-shaped curve”--homogeneity of score dispersion).
NONPARAMETRIC TESTS: Statistical procedures that do not require normality.
Attributes of the Procedures Outlined in this Summary
Note 1: Inferential procedure names are underlined.
Note 2: Asterisked (*) procedures are based dispersion or variance.
VARIABLES | NORMALITY
Name of Procedure Univariate Multivariate| Parametric Nonparametric
|
t-test X | X
Chi-Square X | X
ANOVA* X | X
Factorial ANOVA* X | X
ANCOVA* X | X
MANOVA* X | X
Multiple Correlation Tests X | X (either) X
Wilcoxon Rank Sum X | X
Meta-Analysis X (both) X | X (both) X