Impact Measurement Approaches and Methodologies: Action Research
Rowan Popplewell, INTRAC
Application
Action Research (AR) can be used for a wide range of purposes including research, individual and organisational learning, monitoring and evaluation (M&E) and impact evaluation (IE).
Approach
AR is a label that covers a broad family of approaches that share similar characteristics: they are typically values based, action oriented and participatory. AR approaches have not emerged from a single academic discipline; rather they have slowly developed over time within a wide range of disciplines and professions, with new approaches continually emerging and existing ones being refined (Brydon-Miller, 2003).
AR approaches have common ontological, epistemological and methodological foundations. Firstly, Action Researchers are subjective not objective; they are value laden and morally committed (McNiff and Whitehead, 2011). Secondly, they understand knowledge as socially constructed and plural (Hinchey, 2008). Finally, their research is practitioner focused (either led by or conducted in close collaboration with practitioners) and is highly participatory, with stakeholders involved at all stages of the research process (Bradbury-Huang, 2010). Consequently, AR approaches are classified as interpretivist, and are cast by some as a critique of conventional positivist approaches to social science (Brydon-Miller, 2003).
In practice, most AR approaches use a variant of a methodological process known as the ‘Action Research Cycle’ (see Fig. 1). This is essentially a cycle with 3 steps: planning action, taking action and evaluating action, which is repeated throughout the research process. Stakeholders should be included at all stages of the research process; however in practice this is often difficult.
Advantages
AR approaches are meant to be highly participatory with researchers and stakeholders collaborating in the diagnosis of a problem and designing, implementing and evaluating interventions to resolve it. Within development, participation is regarded as a normative good, and increasingly, as a means of improving impact. This instrumental view of participation is based on the assumption that participation generates ownership, making interventions more targeted and relevant and increasing the ‘agency’ of stakeholders who help ‘cause’ successful outcomes though their own actions and decisions (Stern et al, 2012).
Consequently, in contrast to conventional approaches to IE, which can be extractive, donor-driven and concerned with upward accountability (i.e. to donors, organisational management), AR approaches facilitate and promote organisational and individual learning, reflexivity and downward accountability (i.e. to stakeholders, participants and beneficiaries). AR approaches can also contribute to IE through providing in depth understanding of local contexts and communities, thus improving construct validity (whether you are actually measuring what you think you are measuring) of the evaluation (Stern et al, 2012). Further, AR approaches may be particularly appropriate for evaluating complex programmes.
Challenges
AR approaches are often criticised for lacking quality and rigor, particularly by positivist social science, which critiques AR approaches for lacking objectivity, validity (whether the research findings really are about what they say they are about) and generalizability (the applicability of research findings outwith the specific context studied). Action Researchers have sought to counter these critiques in a variety of ways from devising alternative quality assurance mechanisms (McNiff and Whitehead, 2011) to rejecting them outright for judging AR against a set of criteria that are not suitable (Coghlan and Brannick, 2010).
In IE, participatory approaches have been criticised for lacking rigor as they take a very different approach to establishing causal inference to that of experimental or statistical approaches which use counterfactuals or statistical correlations to demonstrate causality, and are therefore deemed more ‘rigorous’. As with all approaches, there are some situations where AR approaches are appropriate and others where they are not. Further, AR approaches do not need to be used in isolation; other designs and methods can be used to triangulate or complement findings established through AR or vice versa, thus improving ‘rigor’ (Stern et al, 2012).
Bradbury-Huang (2010) “What is good Action Research?: Why the resurgent interest?” in Action research Vol. 8 Issue 1
Brydon-Miller et al. (2003) “Why Action research?” in Action Research Vol.1 Issue 1
Coghlan and Brannick (2010) Doing Action Research in Your Own Organization (London: SAGE)
Hinchey (2008) Action Research Primer (New York : Peter Lang)
McNiff and Whitehead (2011) All You Need to Know About Action Research (Los Angeles: SAGE)
Stern et al. (2012) “Broadening the Range of Designs and Methods for Impact Evaluations” DFID Working Paper 38 (London: DFID)