Acceptable SOE Research Designs

Acceptable SOE Research Methods and Research Designs

Last updated 10/05/2012

Introduction

This document will identify and describe research methods and research designs that are acceptable for dissertation research in the School of Education.

Quantitative Methods

In quantitative research designs educational researchers confirm or verify general statements or predictions. Lodico, Spaulding, and Voegtle (2010, p. 10) refer to this type of research as employing “the hypothetic-deductive method, which begins by forming a hypothesis – a tentative explanation that can be tested by collecting data.” The hypothesis is formed from theory or a synthesis of the results of related previous research. Quantitative methods are frequently placed in three main categories:

True Experimental Designs – The assignment of control and treatment groups and the manipulation of selected variables while controlling all others. Acceptable research design examples include:

· Pretest-Posttest Control Group Design.

· Solomon Four Group Design.

· Posttest-Only Control Group Design.

· Within Subjects Design.

While true experiments, which require random selection of participants, are not typical in educational settings, there might be instances when random selection is possible.

Quasi-experimental Designs – An experimental approach (control and test groups) where participants are not randomly assigned or groups such as classes are assigned either randomly or not randomly. Acceptable research design examples include:

· Non-randomized [Non-equivalent] Control Group Pretest-Posttest Design.

· Solomon Four Group Design.

· Simple Times-Series Design.

· Control Group, Time-Series Design.

· Reversal Time-Series Design.

· Alternating Treatment Design.

· Multiple Baseline Design.

The most common quasi-experimental design done in educational settings is the non-randomized control group pretest-posttest design.

Non-experimental designs. The researcher examines the phenomenon under investigation to explain it or describe it with observation or measurement without manipulating or controlling the environment or variables.

· Descriptive Designs (Survey research).

· Observational Designs.

· Ex Post Facto Designs [e.g., causal-comparative design, factorial design, combined experimental and ex post facto design all of which require partial controls].

· Correlational Designs [e.g., bivariate analysis, simple linear regression, simple logistic regression, multinomial or multivariate logistic regression, multiple linear regression, discriminant function analysis, canonical regression, path analysis, factor analysis].

· Comparative Descriptive Designs.

· Secondary Data Analysis (Records-based Research).

The most common non-experimental designs conducted in educational settings are survey research, correlational designs and comparative descriptive designs.

Note 1: Quantitative descriptive research designs might put the learner-researcher at risk of adding little to the existing body of knowledge and thus are rarely appropriate for SOE dissertations. Descriptive research designs should be considered in light of the criterion provided the Capella Dissertation Manual (2011):

The dissertation is not just descriptive; it has a sound extant basis or a well-developed conceptual basis that leads to the question(s) under investigation. This basis serves as the origin for conclusions and inferences that lead to further research, to enhanced theoretical understanding, and to recommendations for organizational improvement in cases of action science. (p. 8)

Thus it would be unusual or unlikely that a sound extant or conceptual basis would lead to an examination of simple descriptions or opinions or that simple descriptions or opinions will advance theoretical understanding.

Note 2: P re-experiments [e.g., one-shot experimental case study, one-group pretest-posttest design, and static group comparison] that are done alone; that is, without being a preliminary component of either a true experimental or quasi-experimental design are unacceptable research designs for dissertations. This unacceptability is because “In pre-experimental designs, it is not possible to show cause-and-effect relationships because either (a) the independent ‘variable’ doesn’t vary or (b) experimental and control groups are not comprised or equivalent or randomly selected individuals. Such designs are helpful only for forming tentative hypotheses that should be followed up with more controlled studies” (Leedy & Ormrod, 2005, p. 223).

Note 3: If questions arise as to the applicability of a particular research design, then the Mentor should contact the SOE Research Chair for direction.

Qualitative Methods

Qualitative research designs in the SOE typically focus include exploratory studies attempting to describe, explain, or better understand social or educational phenomena. Such exploration is necessary when there is not enough knowledge regarding the particular phenomenon that a quantitative instrument cannot be developed or that the exploration seeks an in depth understanding or insights that cannot be obtained via a quantitative study with existing instruments. Qualitative research designs primarily focus on the participants’ experiences, the meaning participants ascribe to those experiences, and/or understanding a process from the insights of a process participant. Merriam (1998) provides a broad definition of qualitative research “Qualitative research is an umbrella concept covering several forms of inquiry that help us understand and explain the meaning of social phenomena with as little disruption of the natural setting as possible”(p. 5). The centrality of meaning appears again and again in the qualitative methods literature. Acceptable research design examples include:

Action Research – “The practice of a ction research is a fairly common mode of investigation in educational research especially among those researchers interested in classroom teaching practices” (Berg, 2004, p.195). Action research is “intended to uncover or produce information and knowledge that will be directly useful to a group of people (through research, education, and sociopolitical action) . . . that provides people with the means to take systematic action in an effort to resolve specific problems” (Berg, 2004, p. 197). Action research will always involve developing, implementing, and evaluating an intervention.

Basic Qualitative Design – Merriam (2009) describes a basic qualitative research study as having philosophically been derived from constructionism, phenomenology, and symbolic interaction and is used by researchers who are "interested in “(1) how people interpret their experiences, (2) how they construct their worlds, and (3) what meaning they attribute to their experiences. The overall purpose is to understand how people make sense of their lives and their experiences" (p. 38).

Case study – “A qualitative case study is an intensive, holistic description and analysis of a single instance, phenomenon or social unit” (Merriam, 1988, p. 21); thus the case occurs in some form of bounded context. For example a case could be a classroom, a program, a process, etc. A strong case study is often the end product of field work research and thus relies heavily upon observations over an extended period of time. Interviews are typically used to clarify and verify observations and artifacts. Although case studies do not need to have this longitudinal aspect and could be done without observations, what they have in common is multiple data collection methods to develop thick rich holistic cases. “Understanding the case in its totality, as well as the intensive, holistic description and analysis characteristic of a case study, mandates both breadth and depth of data collection. Data collection in a case study is a recursive, interactive process in which engaging in one strategy incorporates or may lead to subsequent sources of data” (Merriam, 1998, p. 134).

Ethnographic Study – An ethnography focuses on society and culture attempting to uncover and describe beliefs, values, and attitudes that structure behavior in a group. Its analysis is unique from other forms of qualitative research in that a sociocultural interpretation is applied to the unit of analysis (Merriam, 1998). Ethnographies are rare as dissertations in the SOE because of the length of immersion in the studied culture. Although methodologists there is some disagreement on the length of immersion and observation, six months to a year or longer appear to be the standard.

Grounded Theory – The researcher derives a substantive theory grounded in the experiences and views of participants. Substantive theory is not formal or grand theory but rather “has is referent specific, everyday-world situations such as an innovative middle school science program, the coping mechanisms of returning adult students, or stages of late-life development. A substantive theory has specificity and hence usefulness to practice often lacking in theories that cover more global concerns” (Merriam, 1998, p. 17). A salient feature of grounded theory is the application of theoretical sampling to reach theoretical saturation, which could lead to a sample size of 20 to 50 participants.

Phenomenological Design – SOE qualitative researchers focus on the essence or structure of an experience or phenomenon. Patton (1990) describes this type of research as based on:

the assumption that there is an essence or essences to shared experience. These essences are the core meanings mutually understood through a phenomenon commonly experienced. The experiences of different people are bracketed, analyzed, and compared to the identity of the essences of the phenomenon, for example, the essences of loneliness, the essence of being a mother, the essence of being a participant in a particular program. The assumption of essence, like the ethnographer's assumption that culture exists and is important, becomes the defining characteristic of a purely phenomenological study. [p. 70, emphasis in original]

Phenomenology is focused on individuals' meaning making as the quintessential element of the human experience (Patton, 2002). The important findings derived from phenomenology are an understanding of a phenomenon as seen through the eyes of those who have experienced it. Phenomenological inquiry holds the assumption that there is an essence or essences to shared experience (Patton, 2002). The focus of a phenomenological study is in uncovering and interpreting the inner essence of the participants’ cognitive processing regarding some common experience. Thus meaning is central to the experience. In general “a phenomenological research is well suited for studying affective, emotional, and often intense human experiences” (Merriam, 2009, p. 26).

In general “a phenomenological research is well suited for studying affective, emotional, and often intense human experiences” (Merriam, 2009, p. 26).

The most typical qualitative research designs done in educational settings are the basic qualitative research design and the case study design.

Note 4: Whatever qualitative research design the learner proposes it is imperative that the learner follow, cite, discuss, and reference an authoritative guide for the particular research design and corresponding data analysis [e.g., Action Research – Carr & Kemmis, Glanz, Kemmis & McTaggart, Sch?n; Basic Qualitative – Merriam, Miles & Huberman; Case Study - Merriam, Stake, Yin; Grounded Theory – Charmaz, Corbin, Corbin & Strauss, Glaser & Strauss; Ethnography – Denzin, Fetterman, LeCompte & Preissle; Phenomenology – Colaizzi, Giorgi, Moustakas, van Manen].

Mixed Methods

Mixed methods research is “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using either qualitative and quantitative approaches or methods in a single study or program of inquiry” (Tashakkori & Creswell, 2007, p. 4). “While mixed methods have an intuitive appeal, they also demand that the researcher be expert in both approaches to research or work with a team that has such expertise” (Mertens, 2010, p. 293). Therefore a mixed methods study requires the support of a mentor or a committee member who is familiar with either mixed methods research, or has sufficient knowledge and skills in both quantitative and qualitative research. Examples of acceptable mixed methods designs or approaches include:

· Sequential Explanatory

· Sequential Exploratory

· Sequential Transformative

· Concurrent Triangulation

· Concurrent Embedded

· Concurrent Transformative

· Dominant-Less Dominant

Other Acceptable Research Designs

This category includes research designs that might be either quantitative or qualitative or blended as in mixed methods or mixed modes.

· Action Science Research ( ASR ) – ASR is research into the structure and processes of systems at every level for the purpose of bringing about change and improvement. Systems in ASR may be individuals, small groups such as classes, organizational systems, and large-scale social systems. The conceptual framework of ASR is drawn from systems theory, action science theory, or change theory. Essential to ASR is a carefully designed intervention for bringing about change at a specific site (often the workplace of the researcher). The intervention is followed by assessment of the effects of the intervention on the system, and a sharing of those results with stakeholders of the system. While continuation of the cycle of improvement is typical of ASR projects in the real world, dissertation research limits the study to a single cycle and uses the last chapter of the dissertation to describe plans for later cycles. Generalizations can be made from the results at a particular site to theory and/or to practice at other similar sites.

In education, ASR is often called participatory action research (McIntyre, 2008), or planned change (Schein, 2011).

· Delphi Technique – The Delphi technique is “a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem” (Linstone & Turoff, 1975, p. 3). Delbecq, Gustafson and Van de Ven describe it as “a method for the systematic solicitation and collation of judgments on a particular topic through a set of carefully designed sequential questionnaires interspersed with summarized information and feedback of opinions derived from earlier responses” (1975, p.10). Also called a Jury of Executive Opinion, it is a process that integrates the judgments of a group of experts to produce a product such as a strategic plan or an assessment of technology innovations, policy impact, or social change. Key features of this technique include forecasting, involving a group or panel of experts, the use of remote group communication, the use of an iterative research process, and a goal of reaching or attempting to reach group consensus.

· Program Evaluation – Lodico et al. (2010) describe program evaluation as follows: The field of education is filled with programs designed to improve both learning and teaching. Examples of these programs include a reading intervention program design to help struggling readers or a teacher-training program designed to help teachers integrate technology into lessons. Program evaluation is designed to make decisions about such programs. Although program evaluation uses quantitative and qualitative methods, its overall purpose is different from most other types of research. Whereas quantitative and qualitative researchers study programs, findings from such studies typically are slow to change or improve the programs themselves. In program evaluation, however, findings are often used for ongoing or short-term decision-making purposes, and programs can be changed or “improved” based on the results of a single evaluation. (p. 41)

Other Research Designs that Require P ermission on an Exceptions Basis

Other possible research designs not typically done in the School of Education are listed below. These designs, if proposed by a learner, would require permission on an exceptions base and the learner would need to provide evidence of doctoral level knowledge of and expertise in the design. The mentor and specialization chair would need to pre-approve any of these research designs if proposed by a learner. Most learners are encouraged to use a more traditional design from the list of acceptable research methods above.