Learning from Practice, Changing Lives

Programmes Unit:

Draft Note for Discussion

Learning from Practice, Changing Lives:

An Approach to Impact Assessment in ActionAid India

In ActionAid India (AAI), whereas vertical impact assessment of the past has been appropriately dismantled, it is yet to be replaced by a comprehensive alternative practice. Alps[1] had identified the principles of a participatory and democratic system of review and reflection[2]. Deriving from Alps, AAI published its operations manual titled Tracking Change[3]. More recently, ActionAid-UK detailed indicators[4] for assessing progress on Fighting Poverty Together (FPT). Despite these and other commendable efforts, we are yet to evolve an organization-wide impact assessment (IA) practice that authentically describes the changes in the lives of poor and marginalised women, men, girls and boys.

Directed towards strengthening our accountability and shared learning practice, this discussion note addresses issues of approach and methodology for participatory impact assessments. Also, deriving from AAI’s Country Strategy Paper 2000-03, it suggests a results framework and possible means for their verification, to strengthen and standardise our impact assessment practice.

Why Impact Assessment?

In AAI we are committed to strengthening rights, equity and justice for poor and marginalized people. We build partnerships and collectively struggle against denial of rights and entitlements, which is the primary reason for poverty and marginalisation. Notwithstanding our noble intentions, the point remains that we are obliged to prove the “good” things accomplished through our actions/interventions. In other words, we must be accountable for what we do. And to this end, we must capture learnings from our interventions to strengthen the worthwhile elements and rethink our not so worthy initiatives.

It is in this context that questions such as - Are we unambiguously taking sides with the most poor and marginalized peoples? What difference has our interventions made in poor people’s lives? Are our interventions oriented to addressing basic causes of poverty and marginalisation? - become important. We are accountable for our practice. And we must make collective efforts to learn from it so as to improve it further. In addition, we are obliged to share our learnings both, within and outside AAI.

Are our actions consistent with our strategic objectives? Are we effectively promoting rights, justice and equity? Are the positive changes in poor people’s lives sustainable? What more can we do to achieve our goals? As an action-learning organisation, these are some of the other questions that we have to answer. And this requires an impact assessment practice that is genuinely participatory, in other words, inclusive, empowering and oriented to the future.

Rights Based Approach

In ActionAid India, our shared understanding of poverty and marginalisation in India has changed considerably in the last few years. This has had a bearing on our approach and strategy for development action. We now believe that our actions/interventions for a just and equitable order must address both, the various manifestations of poverty, as also the social conditions that cause and perpetuate it. Strategically this is significant. For now our mission, requires us to engage in sustained partnership with poor people and their institutions, and contribute to their daily struggle against denial of rights and entitlements.

In sync with the emerging understanding, our strategy papers at all levels mandate actions to promote basic rights of poor people, build their organisations, strengthen their capacities to struggle against denial of rights, as also their immediate poverty needs. The rights perspective requires us to look beyond charity and into poor people’s empowerment through conscientisation, organisation building, capacity building, and advocacy. Also, this implies direct engagement with the most poor and marginalized communities.

What is Rights Based Approach?

·  First, it means clearly understanding the difference between a right and a need.A right is something to which a person is entitled solely by virtue of being a human.It is that which enables her to live with dignity. Moreover, a right can be enforced before the government and entails an obligation on the part of the state to honour it.A need, on the other hand, is an aspiration, which can be quite legitimate, but may not necessarily be associated with an obligation on the part of the state to cater to it; satisfaction of a need cannot be enforced. Rights are associated with "being,” whereas needs are associated with "having.”
·  Second, a rights approach is not just about defending or attacking the form of government, on making statements for or against the victim’s political inclination, or on the motivations - alleged or actual – of those violating human rights. Its focus is the rights being violated and the apparatus that makes the violations possible.In other words, a rights approach cannot attack or support a particular type of political system, even though it cannot ignore its resistance as a factor which blocks or favours the effective exercise of human rights.
·  Third, and as a consequence of the foregoing, a right is defined on the basis of dignity, that is to say, on the basis of "being,” not "having” or the social or economic programme of a political party or a government.A political programme can – and should - be negotiated, but dignity is non-negotiable. Political programmes are necessary to honour human rights, but they cannot be a substitute for them.Political programmes are subject to change in social and economic dynamics that is, what is important today may not be important tomorrow. The dignity of the individual is immutable; it is the same at all times and in all places, and its essence transcends cultural particularities.

The Challenge of Understanding Change

Impact assessment of rights based action is a difficult task. As already mentioned, rights action necessitate initiating processes, such as group formation, capacity building, conscientisation, advocacy, policy influencing, empowerment, etc. Their appreciation varies between contexts and stakeholders and often impacts may not be visible in the short term. Therefore, conventional impact assessment tools and techniques designed around quantitative measurement and rooted in notions of objectivity are redundant. Because rights action involves a number of stakeholders, attributing change to a particular initiative/effort is problematic and contestable.

Further, rights based development is not always visible - like a plant growing under the surface - a great deal maybe happening that is not readily apparent. For example, in many instances rights and entitlement standards are not generally recognized. People resort to collective action (e.g. public hearings, protest marches, demonstrations, mass rallies, public interest litigations, etc.) to get the state and other stakeholders to recognise them. Frequently such efforts do not achieve their manifest goals but lead to other impacts which may not be readily seen, particularly by outsiders.

Accordingly, the first challenge in assessing rights based action is to build/strengthen our own ability to see change wherever it is happening and in whichever form. And to this end, sharpen our skills to analyse the pace, course and direction of our interventions. We must possess the necessary curiosity, openness, and a willingness to learn, to comprehensively understand what is happening in poor people’s lives. Without this basic stance, it may not be possible to see development, no matter what techniques are adopted, as our own capacity for absorbing impressions, data and evidence is blocked.

This may appear quite simple, but it is indeed a very difficult skill and requires continuous striving. We need to be mindful of the easy assumptions we make, the ways in which we close down possibilities before they have had a chance of expressing themselves, the ways in which we shut ourselves off from seeing and hearing that which does not make immediate sense to us. It is only after we have strengthened our ability and capacity, can we meaningfully participate in any IA process.

In conclusion, impact assessment of rights based actions/interventions is not a one-off technical skill that can be learnt and then implemented in a linear fashion. It requires opening ourselves to gaining better insights, constant maintenance and cultivation. This is best pursued through frequent processes of reflection and learning as regular points within our work (e.g. community immersion programs, shared learning ambassadors, etc.). Also, our programs and organization initiatives should be so designed that collective review and reflection is an integral part of their annual cycles.

Impact Assessment in ActionAid India

Over the years, approaches to impact assessment in AAI have changed with an increasing emphasis on involving the primary stakeholders in the identification and analysis of impacts. In the early days the approach was more technocratic. There was a strong, almost total focus on quantitative data, the use of written information sources and expert observation and opinion. The IA design, identification of relevant information and analysis of data was very much seen as the domain of the expert social impact analyst.

Later, during the 1990s, AAI started recognizing the importance of drawing on the views, values and priorities of community members. This change grew out of the community development paradigm and consultation emerged as a principal tool in impact assessment. But, even with this focus, participation of the community members in the IA process remained partial, as the process continued to be designed, managed, and interpreted by the expert. Another lacuna was that consultations were usually with a few stakeholders and many others affected by our program decisions were conveniently excluded.

It was only after the conceptualization and acceptance of Alps, that impact assessment in AAI changed significantly. Today, participation of the primary stakeholders (e.g. the community) is mandatory. Also, our definition of participation has widened and means community members must lead all decisions pertaining to the timing, conduct, analysis and evaluation. AAI’s role as also that of the NGO partners and other stakeholders is limited to providing the necessary facilitation and support to the community.

Is IA in AAI participatory? Have we internalized Alps principles? These questions are being increasingly asked both, within and outside AAI. The answer lies in our impact assessment policy and practice, which indicate the following shifts:

·  The focus is now on collecting and analyzing qualitative information flowing from the community in the form of group discussions, stories, case studies, etc. and not on impersonal quantitative measurements;

·  All possible efforts are made at different levels to ensure that impact assessment design is empowering and not extractive;

·  Impact assessments are now conducted for the purpose of empowerment and not for making external (AAI management) judgment on program/initiative performance;

·  Community at the centre – it is the community that does the assessment, identifies impacts, program successes and failures, and who uses the information/knowledge produced for future action;

·  Accountability and shared learning goals – The rationale for impact assessment is to strengthen AAI’s accountability and learning, so the orientation is towards the needs of all stakeholders, e.g. community members, partners, AAI, donors and policy makers;

Participation in Impact Assessment

Participation strengthens impact assessment in several ways. It helps obtain better quality information for evaluative purposes, ensures mutual accountability and learning and helps bring the perspective of different stakeholders, particularly the most poor and marginalized people. Therefore, maximising participation in impact assessment should be our primary concern.

We know that our interventions affect different people in different ways. Those affected include primary stakeholders (e.g. poor and marginalized people) and secondary stakeholders like, non-poor people in the intervention area, market, state, AAI and NGO partner staff, etc. The list of potential stakeholders is generally very long. Moreover each stakeholder category is often an umbrella for different sub-categories e.g. by gender, age, income, caste/tribe, culture and so on.

Accordingly, whether an impact is positive or negative, sustainable or unsustainable, depends on who assesses (a village head, men, women, the community, external researcher, NGO/iNGO staff, policy-maker, IA consultant, etc.) and their interests (economic, social, ecological). Therefore the same impact may be positive for some while many others may consider it negative.

Alps mandates, “…poor and marginalized members of the community in particular to be involved in monitoring, reviewing, and evaluating what progress has been made within programs”.[5] It implies that, poor people must be central to the impact assessment process and they should decide on when and how to monitor, evaluate, analyse, and communicate. This is increasingly the practice in AAI. By following this principle, IA in AAI has emerged as both, an empowerment program and a management tool.

Selection of Participatory Tools and Techniques
Estrella and Gaventa[6] summarise the approach to tools and techniques as follows:
“In general, tools and techniques should:
·  Complement the approach and philosophy of the [organisation];
·  Be perceived by community participants as a way to help them address their questions and problems, not simply as information about them gathered by or for outsiders;
·  Involve end-users in both data gathering and in analysing data;
·  Match the skills and aptitudes of participants;
·  Adapt to fit peoples’ day-to-day activities and normal responsibilities;
·  Provide timely information needed for decision-making;
·  Produce results, which are reliable and, even if not quantitative, credible enough to convince others;
·  Be consistent in complexity and cost to match the level of evaluation called for;
·  Reinforce community solidarity, co-operation and involvement;
·  Be gender-sensitive
·  Only obtain the information that is needed”

Guidelines for Participatory Assessment

In keeping with the Alps mandate, regional teams in AAI have introduced innovations in the statutory annual program reviews to strengthen participation and learning. For example, some review teams (along with the community members, and local program staff) also had Project Directors from adjacent projects. Their presence helped in project networking and sharing of experiences. Similarly, other reviews involved program staff from other initiatives/projects as review team members. Such exposure ensured sharing and learning within the organisation. Reviews have also involved community members from other project, but from the same community. This has helped in highlighting the differences made within the community.

However, these innovations have not yet crystallised into an organization-wide practice. They have remained one off exercises. In this section, based on our learnings from statutory review practice and innovations we will elaborate on a set of guidelines for participatory assessment.