Advancing Theory and Practice of Evaluation in South Asia

Community of Evaluators South Asia

First Six Monthly Progress Report

July 2012

Submitted to

International Development Research Centre

February 2010

Submitted by
Vrutti
#19, 1st Main, 1st Cross
Aswath Nagar, RMV 2nd Stage
Bangalore - 560094, India

Contents

Contents

Abbreviations and Acronyms

1Introduction

2Objectives and Result Areas

3Progress of Work

3.1New Knowledge

3.2Capacity Development

3.3Networking and Advocacy

3.4Professionalisation of Evaluation

3.5Institutionalisation of COE

3.6Resource Mobilisation

3.7Project Management Support

3.8Governance and Decision Making

4Ways Forward

4.1New Knowledge

4.2Capacity Building

4.3Networking

4.4Institutionalisation

5Annex

5.1List of Key Note

5.2List of Workshops

5.3List of Panels

5.4Logistics related tasks

5.5Media related tasks

5.6Outreach

5.7Fund Raising

5.8Details of Tasks for second phase


Abbreviations and Acronyms

3ie / International Initiative on Impact Evaluation
AEA / American Evaluation Association
CLEAR / Regional Centers for Learning on Evaluation and Results
CMS / Catalyst Management Services
COE / Community of Evaluators South Asia
DECI / Developing Evaluation Capacity in ICT4D
DFID / Department for International Development
DIN / Director Identification Number
ECDG / Evaluation Capacity Development Group
IDOBRO / Marketplace for Impact, Event Manager for Evaluation Conclave
IDRC / International Development Research Centre
INR / Indian Rupees
IOCE / International Organisation for Cooperation in Evaluation
M&E / Monitoring and Evaluation
MES / Malaysian Evaluation Society
PACT / International Development Organisation
SAT / Strategic Advisory Team
SLEvA / The Sri Lanka Evaluation Association
UNDP / United Nations Development Programme
UNICEF / United Nations Children's Fund
UNWOMEN / United Nations Entity for Gender Equality and the Empowerment of Women
USD / United States Dollar

1

Advancing Theory and Practice of Evaluation in South Asia, Phase - II, First Six-monthly Progress Report

1Introduction

Community of Evaluators in South Asia (COE) was constituted in October 2008 by evaluation practitioners from Afghanistan, Bangladesh, India, Nepal, Pakistan and Sri Lanka as part of a project titled "Advancing Evaluation Theory and Practices in South Asia – Building a Community of Experts”. The project was technically and financially supported by International Development Research Centre, Canada. The Goal of the COE was to promote and enhance the quality of the theory and practice of evaluation in South Asia and to contribute to the same, particularly from a South Asian perspective, globally.

In the first phase, the COEsuccessfully organised an international conclave on "Making Evaluation Matter" in 2010 in Delhi, which was participated by practitioners, experts, donors and commissioners of evaluation from different countries. COE also organised several regional events in different countries of South Asia and members attended conferences and seminars on evaluation and represented COE. The membership also increased and institutional membership was given.

COE is in the second phase of implementation and is being hosted by Catalyst Management Services (CMS), which is a professional services organisation based in Bangalore, India working in the development sector for the last 17 years. CMS manages the project under one of its group organisations, Vrutti. Vrutti is a registered not-for-profit development organisation and is the legal holder of the project. Vrutti also provides grant management, communication and administrative support to the COE for all its activities.This is a progress report for the first six month of operation, i.e., from January 2012 to June 2012.

COE is formally supported by IDRC, Canada.

2Objectives and Result Areas

The goal of COE is to promote and enhance the quality of the theory and practice of evaluation in South Asia and to contribute to the same, particularly from South Asia perspective, globally.

The contribution areas and the corresponding objectives of COE in the second phase are:

  • New Knowledge: Promote and catalyse the use of new knowledge to promote the quality and practice of evaluation;
  • Capacity Building: Build capacities for improving quality of evaluation;
  • Networking and Advocacy: Build a strong network and advocacy mechanism to improve theory, quality and use of evaluation in South Asia.;
  • Professionalisation of Evaluation: Professionalise evaluation through code of conduct, ethical standards and protocols for engagement;
  • Institutionalisation of COE: Acquiring legal status and formalisation of operations;

The programme framework for the second phase of work has been summarised in the following schema:

3Progress of Work

In the following section, the progress of work under each of the five contribution areas has been documented. The five contribution areas are: (i) new knowledge, (ii) capacity development, (iii) networking and advocacy, (iv) professionalisation of evaluation and (v) institutionalisation of COE. In addition, progress under (vi) resource mobilisation, (vii) project management support and (viii) governance and decision making has also been elaborated.

3.1New Knowledge

Activities listed under new knowledge are: (a) writing and publication of articles; (b) documentation of innovation; (c) use of web-site as an interactive platform and (d) support to evaluation panels.

The edited volume on evaluation was compiled in the first phase of work. The editorial for the volume is being drafted. A line editor has been contracted to edit all papers. The volume is expected to be published by September 2012 and would be released during the Evaluation Conclave 2013.Members floated the idea that COE members can constitute a panel in the upcoming Evaluation Conclave in 2013. Some minor changes have been made to the COE website but it is yet to become a dynamic and interactive platform for communication and knowledge sharing.

The progress under the contribution area has been slow. The activities are expected to pick up in the next six months.

3.2Capacity Development

Activities listed under capacity development are: (a) Evaluation Conclave; (b) Regional events; (iii) moderated discussion groups; and (iv) networking with other institutions and groups involved in capacity building.

3.2.1Evaluation Conclave

After multiple rounds of discussion, COE took following decisions regarding the Evaluation Conclave, which would be conducted in 2013.

Location: The conclave would be held in Kathmandu Nepal.

Dates: The conclave would be held from 26-Feb-2013 to 1st March 2013.

Duration:The duration of the conclave would be of four days, out of which, three days would be spent on the actual conclave work and one day would be spent on COE meeting.

Venue: Three different hotels in Kathmandu were shortlisted for the purpose. Then COE members conducted a reconnaissance of the hotels and held discussion with the COE Nepal members. The rates proposed by the hotels, services available and the availability of break-away rooms were discussed and communicated to the members. Finally Hotel Hyatt Regency was selected as the venue. Advance requisition of the venue has already been done and acknowledgement has been received.

Theme: It was decided that to keep focus and include different domains, it would make sense to have a broad unifying theme with several sub themes. The broad theme was identified as “Evaluation for Development”, which positions evaluation as leading to something greater, rather than just evaluation of development projects. Following sub-themes were suggested for the conclave: (i) inclusiveness; (ii) equity; (iii) transparency, accountability, corruption; (iv) right to food, right to education; (v) citizenship; (vi) gender and violence.

Key Note Speakers:Several names were proposed for key note speech. A list of the proposed names is given at the annexes.

Workshop: Several capacity development workshops would be conducted as a part of the conclave. These workshops would be either three-hours duration or six-hours duration. The primary objective of the workshops would be to develop professional capacities in the practitioners. While three hours is very short for an workshop, it may be fit for some of the topics. Alternately, the resource person might be requested to decide the duration of the workshop. The list of workshops identified and resource persons for the workshops has been given in the annex. The resource persons would be offered an honorarium of USD 2,000 for a six-hour workshop and USD 1,000 for a three-hour workshop. The resource persons would also be requested, if they could waive the honorarium. In addition, the travel and accommodation of the resource persons would be paid by COE. Registration of the participants in the workshops would be on a first-cum-first serve basis.

Panels: Several individuals and institutions were identified to constitute panel discussion. It was decided that the panels needs to be self-funded. The list of proposed panels has been given in the annex.

Coffee Shops: Coffee Shops would be short sessions on various issues of importance, where a facilitator would facilitate the discussion and members would share their views, experiences and opinions. The Coffee Shops would also be used for networking. Invitees for Coffee Shops would be decided, three months prior to the conclave.

Networking - Specific time and space would be allowed for networking. Since networking was not organised properly in the last conclave, it would be included in the programme schedule.

Budget - An estimated budget of 80 lakhs INR (160,000 USD) has been proposed for the conclave, out of which, forty lakhs INR is available as contribution from IDRC and 20 lakhs INR is available as surplus money from the last conclave which is from unused scholarship. This money is with Sambodhi. Donors who supported the first evaluation conclave, would be approached for the second conclave. The donors are: Rockefeller Foundation, IDRC, DFID, UNICEF, CLEAR, UN Women, 3ie, Gates Foundation. However, these donors might like to pursue their own agenda and the conclave content should be flexible to accommodate such requirements. For funding support, two models were considered. The first is the direct sponsorship (i.e. donor giving money to the organizers) and the second is sponsoring participants to participate in the conclave. A detailed budget including the event management cost is being worked out and would be presented soon.

Total participants -It was decided that around 400 participants would be targeted for this conclave.

Logistics -The management of the conclave would be given to an event management company. Two companies, one in India (i.e. IDOBRO) and one in Nepal (i.e. Young Enterprise Services) were identified for the purpose and they were requested to make presentation in the Face to Face meeting held in Nepal. Finally IDOBRO was selected to provide the event management services. The logistics would be outsourced to IDOBRO, which had the relevant experience of conducting large events in the development sector in India. They did not have similar experience in Nepal. IDOBRO was suggested to select and work with a local event management company.

Task Teams - Five different Task Teams were identified for the purpose and related tasks were detailed out. The Task Teams and their broad tasks have been listed below:

  • Content Development - The content for the conclave would be developed by CMS. Asela from COE and the Event Manger would support CMS in developing the content. The introductory communication to key note speakers, workshop facilitators and panelists would be done by the members, who volunteered for the purpose. Subsequent communication would the done by CMS unless the members wish the continue the communication.
  • Logistics - The Event Manager would be primarily responsible for the overall logistics. CMS would provide the oversight. However, the Event Manager would need to work with the Logistics Task Team which includes Gana Pathi, Rashmi and Ramesh from COE and Thanga from CMS. The details of the logistics related tasks, as identified, have been given in the annex.
  • Media - Media activities would be managed on a day to day basis and would be led by Riffat Lucy and Gana Pathi from COE, the Event Manager and CMS. The details of the media related tasks have been given in the annex.
  • Outreach - Outreach would be done through both online and offline activities. Regional Events would be organised for generating awareness about the conclave in member countries. Efforts would be made to increase the outreach in Afghanistan and Pakistan. Outreach would be increased through registration of new members, which would be rolled out soon. The outcome of the regional events would be consolidated and the highlights would be presented in the Conclave in a panel or other platforms as it fits. Following would constitute the Task Teams for the outreach related tasks: Asela would lead the Online Outreach and Chella would lead the Off-line Outreach activities. Bhabatosh and Mallika would be responsible for outreach in Bangladesh and Sri Lanka respectively. Dad Mohammad and Abdul Ghani would be requested to increase outreach in Afghanistan. Mallika would get in touch with practitioners in Pakistan to increase outreach. Sandip from CMS and Kultar from Sambodhi would be part of this task team. The details of outreach related tasks have been given in the annex.
  • Fund Raising - A simple fund raising strategy would be developed. Efforts would also be made to raise fund in the member countries. Sambodhi approached International Initiative for Impact Evaluation (3ie) and Rockefeller Foundation. Rockefeller Foundation has committed 50,000 USD as Core Grant and 50,000 USD for workshop. Following institutions were identified by the members, which would be interested in supporting the conclave: Packard Foundation, Rockefeller Foundation, MasterCard Foundation, World Bank, EDP Nepal, ALNAP, UNICEF, UNWOMEN, UNDP, SLEvA, PACT and IOCE. Task team for fund raising or resource mobilisation include Shiv from CMS, Kultar, Katherine, Ram Chandra Khanal, Mallika and Khairul Islam. The details of the fund raising related tasks have been given in the annex.
  • Coordination - CMS would be responsible for coordination of activities and would work closely with COE Nepal.

Fee Structure and Collection of Fees - Following rates have been proposed for the fee:

  • Multi laterals and donors – 650 USD
  • NGOs and local organisation – 550 USD
  • Students, individuals – 450 USD

Fees would be collected both online (through payment gateway) and offline. CMS has identified two payment gateways i.e., Event Avenue Payment Gateway and EBS Payment Gateway, which agreed to provide services for the same. A final selection would be made after comparing their services and credibility of their operations.

3.2.2Regional Events

It was decided that a Regional Event would be conducted in Bangladesh in November as a preparation towards the Evaluation Conclave in 2013. COE members from Bangladesh, namely, Bhabatosh Nath and Khairul Islam took responsibility for organising the regional events.

3.2.3Networking with Other Institutions and Groups Involved in Capacity Building

COE has formally endorsed and recognised EvalPartners by taking membeship in the Management Committee and International Advisory Group. Ms. Shubh Kumar-Range from COE has joined the Management Committee while Mr. N. Raghunathan from CMS has joined the International Advisory Group of EvalPartners. EvalPartners is an international initiative to enhance the capacities of civil society organisations to influence policy makers, public opinion and other key stakeholders so that public policies are based on evidence, and incorporate considerations of equity and effectiveness. It is a joint initiative of International Organisation for Cooperation in Evaluation (IOCE), UNICEF and several other major organisations. It basically works in three major areas:

  • Technical capacity of members of evaluation networks;
  • Institutional capacity of evaluation networks;
  • The roles evaluation networks play and can play at the country or region level (e.g. influencing policy, hosting online courses, etc);

Marco Segone, Co-Chair from EvalParnters interacted with COE members in CMS office in Delhi. Following were the key discussion points:

  1. To better understand the objectives and activities of EvalPartners;
  2. To share the result areas and activities of COE and progress of work;
  3. To understand the expectations of EvalPartners from COE in terms of collaboration/ partnership?
  4. What are the various options/ possibilities for engagement?
  5. How can we contribute to each other’s activities and result areas?
  6. Possible action points for taking the engagement forward;

There were several communication between COE and EvalPartners on key events and issues and COE was invited to participate in the first international meeting of the Management Group. COE requested the following support from EvalPartners:

  • Link to other evaluation societies for networking;
  • Technological support on hosting webinars;
  • Support to Evaluation Conclave 2013;

3.3Networking and Advocacy

In terms of networking and advocacy, COE has initiated collaboration with CLEAR, American Evaluation Association (AEA), Evaluation Capacity Development Group (ECDG), Malaysian Evaluation Society(MES)and African Evaluation Association.

CLEAR is a global initiative, whose goal is to contribute to strengthening the monitoring and evaluation (M&E) and performance management capacity of countries and their governments to achieve development outcomes. John Floretta from CLEAR met and interacted with COE members in Delhi. Following was the agenda of the meeting:

  • Share about CLEAR, its structure, results and plan for 2012;
  • Discuss about individuals, organisations and networks related to M&E in the region, who could be possible partners for CLEAR;
  • Explore areas of collaboration with COE;

Capacity building and knowledge sharing were identified as areas for collaboration between COE and CLEAR.

COE contacted AEA for collaboration and support specifically in relation to the 2013 Evaluation Conclave in Nepal. The request was acknowledged and accepted by AEA as a part of their overall effort to expand international collaboration. COE members are in touch with AEA on this.