ARMENIA
SOCIAL INVESTMENT FUND II PROJECT
MID-TERM MISSION REPORT
JANUARY 2004
1
Table of Contents
Page no.
Introduction...... 1
PART I:DEVELOPMENT IMPACT REVIEW...... 2
Coverage of ASIF II Operations...... 2
Community Infrastructure Projects...... 4
Community Participation...... 5
Enhancement of Social Capital...... 6
Capacity Building...... 8
Capacity Building for Implementing Agencies and Local Contractors...... 9
Local Government and School Training...... 10
PART II:ASIF II POVERTY TARGETING STRATEGY AND REGIONAL
ALLOCATION REVIEW...... 15
Poverty Profile...... 15
Targeting Strategy...... 15
Regional Allocations and Actual Disbursements...... 16
Disbursements in Relation to Population and Poverty Scores...... 17
Impact on Poverty Reduction...... 18
Poverty Monitoring...... 19
PART III:MICROPROJECT CYCLE REVIEW...... 20
Promotion...... 20
Appraisal...... 23
Follow-Up...... 25
PART IV:COST EFFECTIVENESS REVIEW...... 27
Approach to the Cost-Effectiveness Analysis...... 27
ASIF II Microprojects and Development Strategy...... 27
Cost-Effectiveness of Microprojects...... 28
The ASIF Methodology in Microproject Analysis...... 29
Cost-Effectiveness Study...... 30
PART V:ADMINISTRATIVE, ACCOUNTING AND
REPORTING PROCEDURES...... 30
Financial Management Review...... 31
Project Disbursement Performance...... 31
The Project Accounting System and the Project’s Financial Accounts...... 38
Government Counterpart Contribution...... 39
Status of Matching Fund Arrangements under the Project...... 40
Status of Co-Financing Arrangements under the DFID Trust Fund...... 44
Financial Reporting Activities of ASIF...... 44
The Management Information System...... 45
Financial Audits...... 45
PSR Ratings...... 46
Action Plan...... 46
Procurement Review...... 47
Procurement Plan Review...... 47
Capacity of the ASIF Procurement Staff...... 49
ASIF General Reporting Arrangements...... 49
ASIF Staff Training...... 51
PART VI:A PROPOSED FOLLOW-UP PROJECT...... 51
ANNEXES
Annex 1List of ASIFII Microprojects visited during Supervision Missions.....52
Annex 2ASIF II Social Capital Assessment...... 54
Annex 3Summary of ASIF II Local Level Training Activities...... 59
Annex 4Terms of Reference for ASIF II Cost-Effectiveness Study...... 60
Annex 5ASIF II Disbursement Table...... 67
Annex 6ASIF II Staff Training Activities...... 68
1
INTRODUCTION
The Mid-term Review of the Armenia Social Investment Fund III (ASIF III) Principal Project conducted from April 26 to May 6, 2010, was carried out with the purpose of assessing overall project performance to-date, the current issues and future direction of the project. Team members of the mission included: Caroline Mascarell, Mission Leader, Hilarian Codippily, Social Development Economist, Susanna Hayrapetyan, Senior Health Specialist (ECSHD), Ekaterina Arsenyeva, Financial Management Specialist (ECC10), Alexander Astvatsatryan, Procurement Specialist, and Arman Vatyan, Financial Management Specialist. Mr. Ashot Kirakosyan, Executive Director of the ASIF II Project, and his team, actively participated in and contributed to the Mid-term review.
The Mid-term Review report consists of five parts covering in-depth reviews and a sixth part regarding the Government’s request for a proposed ASIF III Project, which are set out below. These reviews were based on documentation available in project and other related files in the ASIF, and project reports and studies, including, but not limited to, procurement and financial management assessments, quality of works reviews, financial audit reports, project progress reports, Beneficiary Assessments, and the recent Social Capital Assessment. Specific attention during these reviews was given to issues raised during project supervision missions, as well as those raised in the quality of works reviews, in the Beneficiary Assessments and in the recent Social Capital Assessment. The reviews were complemented by field visits to selected communities to assess management and supervision arrangements, as well as related training activities of community investment projects in the field.
Part I:Development Impact Review: Assessment of the development impact of the project regarding the improvement of basic social and economic infrastructure, short-term employment opportunities, community outreach, and local capacity building.
Part II: Poverty Targeting Strategy and Regional Allocation Review: Assessment of the poverty targeting strategy and the regional allocation of funds, focusing in particular on the extent to which actual expenditures by regions are in line with the regional allocations. The assessment also covers the development/poverty alleviation impact.
Part III: Microproject Cycle Review: Assessment of the effectiveness of ASIF II activities at all stages of the project cycle covering targeting, transparency and accountability, internal project monitoring systems, quality of works, technical assistance and training, and compliance with the project’s operating criteria and procedures.
Part IV:Cost Effectiveness Review: Assessment covering: (i) the main contributions of the ASIF II Project within the broader economic context of the Country Assistance Strategy; (ii) the extent to which simple cost-effectiveness analysis on community microprojects has been carried out, based on criteria such as cost per beneficiary, the size of microprojects, the physical content of the work, unit cost indicators, and the use of least-cost analysis; and (iii) the cost-effectiveness in the use of funds, based on unit costs derived from construction work carried out by comparator agencies and other sources.
Part V: Administrative, Accounting and Reporting Procedures Review: Assessment of project disbursements, the project accounting system and financial accounts, the status of the government counterpart contributions, matching fund arrangements under the project, cofinancing arrangements under the DFID Trust Fund, procurement and reporting activities, the management information system (MIS), financial audits, staffing issues, and compliance with ASIF II Credit Agreement.
Part VI: A Proposed Follow-Up Project. An introduction to the proposed ASIF III Project highlighting the priorities assigned by Government to a possible follow-up ASIF III Project, and some additional components that should be included in such a project.
What follows are the findings of the review, outlining the main issues identified, recommendations made for addressing these issues, and arrangements reached on next steps. The findings were prepared jointly by all mission members and with the excellent support, participation and cooperation received from staff of the ASIF. The complete list of microprojects visited during supervision missions, including the Mid-term mission, is included in Annex 1.
PART I: DEVELOPMENT IMPACT REVIEW
The review consists of an assessment of the development impact of the project regarding the improvement of basic social and economic infrastructure, short-term employment opportunities, community outreach, and local capacity building. The focus of the assessment covers: (i) demand orientation of the project; (ii) community role and participation; (iii) effectiveness of coordination, management, and information dissemination activities; (iv) appropriateness of technical standards; and (iv) the existence of institutional mechanisms to sustain community involvement in decisions about microproject design, implementation, operation and maintenance, and evaluation.
Coverage of ASIF II Operations
The ASIF II Project began its implementation activities in October 2000 with the objective of assisting the Government of Armenia in its continuing endeavor to improve living standards of the lower groups among the Armenian population and strengthen institutions at the local level. Within a period of three years, the ASIF has made significant progress in extending its coverage with the aim of improving the quality and availability of community infrastructure in poor communities throughout Armenia, including the provision of school heating systems and school furniture. During the first two years of implementation, the ASIF focused their program in the earthquake zone in support of the Government’s Earthquake Recovery Program. By the end of December 2002, the ASIF had successfully supported the Government’s program responding to priority community infrastructure needs in poor communities throughout the earthquake zone. The ASIF then focused its activities on other regions, targeting a wide range of communities throughout Armenia. As of December 2003, the ASIF targeted communities in nine regions covering Aragacotn, Gegarqunick, Kotaik, Lori, Shirak, Syunik, Tavush, Vajoc Dzor, and Yerevan. The ASIF received 946 proposals, of which 188 microprojects had either been completed or were under implementation.
There are currently 83 MPs under implementation, of which 54 are for school rehabilitation or construction. As of December 2003, a total of US$13.3 million (IDA: US$9.6 Million, Sponsors: US$1.6 million, DFID: US$1.1 million, Communities: US$911,000, and Government: US$75,000) has been disbursed against microprojects. The average cost of MPs is estimated at US$57,000. The average cost per beneficiary is US$19. In addition, a total of US$1 million was disbursed against goods for microprojects, consisting mainly of school furniture. Tables 1 and 2 below provide a summary of infrastructure works projects by typology covering completed and ongoing projects. See also Table 5 for cost per project and cost per beneficiary.
Table 1: ASIF II - Completed Microprojects by Typology
Type / Number / % / Value (US$) / % / Beneficiaries / %Schools / 62 / 59.05 / 3,795,908.00 / 63.65 / 90,819 / 28.14
Special Schools / 4 / 3.81 / 454,707.00 / 7.62 / 3,239 / 1.00
Water Supply / 22 / 20.95 / 888,285.00 / 14.90 / 44,171 / 13.69
Irrigation / 5 / 4.76 / 160,145.00 / 2.69 / 30,193 / 9.36
Polyclinics / 4 / 3.81 / 216,518.00 / 3.63 / 62,001 / 19.21
Community Centers / 2 / 1.90 / 72,383.00 / 1.21 / 29,983 / 9.29
Health Centers / 1 / 0.95 / 29,279.00 / 0.49 / 24,600 / 7.62
Orphanages / 1 / 0.95 / 93,980.00 / 1.58 / 153 / 0.05
Waste Disposal / 4 / 3.81 / 252,193.00 / 4.23 / 37,580 / 11.64
TOTAL / 105 / 100.00 / 5,963,398.00 / 100.00 / 322,739 / 100.00
Source: ASIF Office, MIS
Table 2: ASIF II – Ongoing Microprojects by Typology
Type / Number / % / Value (US$) / % / Beneficiaries / %Schools / 54 / 65.06 / 3,340,033.00 / 62.00 / 101,954 / 57.31
Special Schools / 6 / 7.23 / 440,139.00 / 8.17 / 13,040 / 7.33
Kindergarten / 1 / 1.20 / 23,130.00 / 0.43 / 1,540 / 0.87
Water Supply / 14 / 16.87 / 1,009,159.00 / 18.73 / 39,512 / 22.21
Irrigation / 1 / 1.20 / 29,935.00 / 0.56 / 1,017 / 0.57
Community Centers / 3 / 3.61 / 229,745.00 / 4.26 / 2,593 / 1.46
Health Centers / 3 / 3.61 / 247,773.00 / 4.60 / 16,901 / 9.50
Waste Disposal / 1 / 1.20 / 66,956.00 / 1.24 / 1,331 / 0.75
TOTAL / 83 / 100.00 / 5,386,870.00 / 100.00 / 177,888 / 100.00
Source: ASIF Office, MIS
Community Infrastructure Projects
Microprojects: The community infrastructure projects carried out under the ASIF II Project have generated a visible development impact in terms of social and economic benefits to the poor communities in all regions of Armenia. Of the 105 completed MPs, the majority comprised school rehabilitation or construction (62), followed by water supply (22), and irrigation (5), benefiting 322,700 community members and creating nearly 40,000 man days of short-term employment. In addition, school heating systems were constructed or repaired in 19 schools as part of school projects. Stand-alone school heating systems were also constructed in 15 schools. These projects responded to the communities’ priority needs and were carried out through self-help interventions promoted by the ASIF. Residents in these communities are now enjoying rehabilitated and warmer schools during winter for their children, potable water in their homes, rehabilitated specialized schools for orphanages, handicap children, art students, and rehabilitated sports facilities. They expressed satisfaction in seeing projects being implemented in their communities that yield tangible, visible results which are designed for the common good of the community. These projects also represent hope for the future of children in poor remote rural communities and in the regions devastated by the earthquake in 1988.
School Furniture: Since the start of project effectiveness, a total of 392 schools (rehabilitated and newly constructed) received school furniture. This comprised 22,059 desks and 44,118 chairs, amounting to a total value of US$1.06 million.
Impact of Community Infrastructure Projects
As documented in previous supervision missions and confirmed in the Beneficiary Assessments carried out, as well as in the recent Social Capital Assessment, microprojects implemented under the ASIF II Project improved the quality of essential social and economic infrastructure and services in beneficiary communities throughout Armenia. The benefits of the ASIF microprojects range from improved access to potable and irrigation water to improvement in class attendance and school enrollment, hygiene and health, and the psychological well-being of communities. The ASIF program created opportunities for effective decentralized systems, building upon the interactive partnership between the public and the government. This activity involved redefining the roles and responsibilities of municipal authorities and citizens in the delivery of municipal services, as well as disseminating accurate and accessible information to the public. The participatory process promoted under the ASIF consisted of gradually preparing governments and communities in the fundamentals of participation, project management, and accountability.
The Decentralized Approach to the ASIF: The recent Beneficiary Assessment (BA) and Social Capital Assessment were instrumental in highlighting the positive impact of the ASIF II Project in terms of supporting the Government’s decentralization program at the local level, and in particular, strengthening local level institutions and empowering communities. More importantly, the BA revealed that community infrastructure microprojects served as an effective framework for promoting the institutionally strengthened local governments, mobilized community members, and created linkages with their constituents under the ASIF microprojects, which enhanced interpersonal trust and strengthened intra-community bonds and cooperation. Both evidence from the BA and from observations made during field visits highlighted that the ASIF II Project provides a clear example of how a decentralized approach has contributed significantly to meeting its key objective of improving the living standards of the poor, through improved basic social services sustained by capacity building.
Community Participation
Since the project became effective, a total of 1,133 community meetings were organized by the ASIF. There are currently a total of 5,227 community members who have participated or are participating as members of Implementing Agencies, of whom 1,829 are women and 2,188 are local government officials. The majority of community members took part in community meetings to identify and prioritize local problems and to select members of the Implementing Agencies (IAs). During microproject implementation, many community residents provided paid and/or unpaid labor, and volunteered support to the IAs. The in-cash contributions by the communities as of end-October 2003 is estimated at US$911,000. The communities have also contributed a total of US$83,000 in kind and US$14,000 in labor. The key participatory features of the ASIF program are open community meetings and elected Implementing Agencies as part of every microproject, and the effective mechanism of mandatory community contributions adopted under the ASIF I Project and maintained under the ASIF II.
With regard to contractor participation, as of December 2003, a total of 151 bid conferences were organized by the ASIF benefiting 1,006 contractors, and a total of 1,001 bid packages were received. A total of 110 contractors participated in ASIF works projects and an estimated number of 47 contractors participated in more than one ASIF project.
Effectiveness of ASIF Participatory Framework: Experience under the ASIF has shown that the active engagement of community members under the project’s participatory framework, developed and tested under the ASIF I and later refined under the ASIF II, has yielded positive results. Over the last eight years, citizens in these communities gradually came together and participated in ASIF activities, establishing close working contacts with ASIF staff from promotion to appraisal, to implementation. More importantly, these communities have gradually developed trust in the ASIF program. The combination of experience gained under ASIF I, together with existing levels of social capital, has significantly contributed to the effective participation of community members in the design, implementation and monitoring of projects under ASIF II, which respond to the common good of the community.
Community members involved in ASIF projects gradually became aware of the benefits of their engagement in microprojects. A number of communities from Vanadsor, Yerevan, Meghrashen, and Hartavan who participated in ASIF I projects were successful in obtaining funding from ASIF II to support their second round of priority projects. These communities were confident of their new skills in applying for another ASIF project and for carrying out successfully the implementation of a microproject. Furthermore, improved public facilities in Armenian communities created a strong incentive for community members to become more actively engaged in the maintenance of the rehabilitated facilities. As noted in the recent Beneficiary Assessment, “microprojects were crucial in further boosting participatory spirit and civic identity in the communities targeted.” The ASIF Project successfully built from and strengthened existing levels of social capital, which played an important role in the capacity of local communities to initiate and effectively manage microprojects. The prior endowments of social capital in communities, such as the presence of strong, trusted and effective leaders were key in determining the success of beneficiary communities in obtaining microproject funds from the ASIF and in effectively managing microprojects.
Other Benefits of Community Participation in ASIF Projects: The recent BA highlights that many of the beneficiaries who were not directly involved in ASIF microprojects revealed interest in the process, progress and outcomes of the microprojects being carried out in their community. Community members actively took part in the monitoring of microprojects, periodically inquiring about their status by conversing with members of the IA, contractors and ASIF staff. The BA also highlighted an interesting observation that ASIF microprojects in some communities helped lessen the level of tension in a community, as a result of providing equal access to a public facility. In Meghrashen, for instance, the ASIF microproject provided all residents with access to irrigation water. A village member of the beneficiary community who was interviewed expressed the positive impact that the microproject had on the community when he stated “the impact of the irrigation project was good, we received produce this year. And when people’s well-being improves, they become more open-hearted.”
Enhancement of Social Capital
The recent Social Capital Assessment of the ASIF II Project focused on the approach of the project to enhancing social capital. As mentioned in the Concept Note of the assessment developed in May 2003, the evaluation looked at the effectiveness of the integrated development framework (i.e. project structure, methods and techniques, and service delivery) under the project to enhance both bridging and bonding forms of social capital. During the supervision mission of June 2003, meetings were held with ASIF staff, the Chief of the Household Department of Statistics, and a number of donor and other agencies to discuss technical issues related to the social capital assessment and sampling issues, as well as administrative issues for the preparation of the combined beneficiary/institutional assessments, as part of the social capital review. Annex 2 provides details on the assessment covering objectives, key areas of analysis, methodology, sampling and selection of sites.
The findings of the assessment confirm the following key working hypotheses:
- The institutional framework of ASIF II – consisting of institutional arrangements, participatory methods, service delivery, and capacity building - - served as an effective catalyst for institutional development and social capital formation.
- The institutional linkages strengthened under the framework of the project increased the efficiency of local communities and government institutions in service delivery and created effective social capital.
The recent Beneficiary Assessment, carried out as part of the Social Capital Assessment, highlights some key findings: