Managing for Development Results:

SHIPDET and COP –MfDR Annual Meeting,

Shanghai, PeoplesRepublic of China

26th October to 3rd November 2007

Results-based Monitoring and Evaluation System In Sri Lanka: Lessons, Experiences and theWay Forward

October, 2007

V.Sivagnanasothy

Director General

Department of Foreign Aid and Budget Monitoring

Ministry of Plan Implementation


Executive Summary

Developing Countries and development partners have long been concerned about effectiveness of development resources and recognize the value of Monitoring and Evaluation (M&E) to understand what interventions work well, which do not and why. Moreover, many countries are pursuing results orientation in their development initiatives by building more effective Monitoring and Evaluation Systems. The Paris Declaration also emphasizes on Managing for Development Results (MfDR) which implies harmonization of results reporting by establishment country-based monitoring and evaluation systems for reporting on the effectiveness of policy execution and aid effectiveness. As part of its broader efforts to institutionalize Managing for Development Results (MfDR), the Government of Sri Lanka has taken specific steps to strengthen Results-based Monitoring and Evaluation (RBME) System at the national level through the Ministry of Plan Implementation (MPI) and its Monitoring and Evaluation Arm – the Department of Foreign Aid and Budget Monitoring (D/FABM).

A Senior Cabinet Level Minister has been appointed as the Minister of Plan Implementation and the Results Based M&E system has received top-level political support in the Government of Sri Lanka (GOSL). President reviews the progress of projects, programes, sector performance and institutions on a quarterly basis and this forum serves as a guiding and trouble shooting forum with top level political commitment.

Institutionalization of Monitoring and Evaluation (M&E) means creation of a monitoring and evaluation system with policy, legal and institutional arrangements to produce monitoring and evaluation information which are demand-driven and judged valuable by key stakeholders in planning, budgeting and policy formulation exercise. When M&E is institutionalized it serves as an integral part of the development policy/programme cycle to improve performance accountability and to provide effective feedback to improve planning, budgeting and policy making to achieve development effectiveness.

Under the current M&E arrangements, self Monitoring and Evaluation takes place at the project level under the leadership of Project Director; at the Line Ministry level under chairmanship of the Secretary of the relevant sectoral Ministry and at the national level under the leadership of H.E. the President. The national M&E system is guided, directed and overseen by the Ministry of Plan Implementation (MPI) and its Department of Foreign Aid and Budget Monitoring (D/FABM).

One of the noteworthy and significant aspect in the M&E system in Sri Lanka is that comprehensive web-based National e-Project Monitoring System (ePMS) that has been established by the Department of Foreign Aid and Budget Monitoring of the Ministry of Plan Implementation which captures implementation progress as well as ‘results’ of all key development projects and programmes and provides policy makers and senior officials with on-line and real time access to progress information. The interesting feature of the system is that it generates ‘early warnings’ and assists in trouble shooting of problem projects and projects ‘behind schedule’. Increasingly data is being captured at source with the idea of establishing paperless monitoring capabilities in the government. Moreover, the MPI also monitors progressof all capital expenditure of line ministries and projects over Rs 50 Million andthese reports are submitted to Cabinet of Ministers on a quarterly basis.

On a selective basis large programes are evaluated by the Department of Foreign Aid and Budget Monitoring of the Ministry of Plan Implementation. GOSL Evaluation System,while recognizing the objectivity in evaluation, emphasizes more on lessons learning and feedback rather than accountability. As in-depth evaluations are costly, the Ministry of Plan Implementation undertakes‘project reviews’ which are less costly and less time consuming as a substitute arrangement for evaluation. On a selective basis, considering factors such as cost and size of the project, relevance to policy level, replicability and evaluability the Department of Foreign Aid and Budget Monitoring undertakes in-depth evaluations. Quality of evaluations needs to be raised and sectoral Ministries still need evaluation capacity development support.The Department of Foreign Aid and Budget Monitoring is developing an Evaluation Information System (EIS) to ensure evaluation lessons are, widely disseminated andintegrated into the planning,budgeting and policymaking process. The EIS based library of evaluation findings will be made available on a web-site to support and influence planning, budgeting and policy formulation decisions.

Government of Sri Lanka (GOSL) has introduced a Results-based Monitoring and Evaluation System (RBME) with the technical support of the UNDP to track performance of ministries and institutions using output and outcome indicators thus making public officials accountable for ‘results’ rather than ‘efforts’. The Hon Minister of Plan Implementation and the Secretary Ministry of Plan Implementation has taken a lead role as powerful champions of RBM. Under the RBM initiative each ministry is encouraged to develop a clear mission and comprehensive performance measures. This comprehensive Performance Measurement System is being piloted with 4 key line Ministries. The experiences of the North American RBM Models operated in States such as Oregon, Minnesota and Virginia were taken into consideration in developing a localized model. Under the RBM framework, emphasis of monitoring is on ‘results’ rather than ‘efforts’ taken by Ministries. The MPI is to establish a web-based RBM platform in the National Operations Room (NOR) - the information arm of the MPI- to maintain Agency Results Framework and Score Card for each line Ministry. The score card is to keep track of the progress of line ministries under its core thrust areas using key performance indicators (KPIs). The Agency Score Cards will appear in the computer screen to enable the assessment of the performance of the line ministries. The Ministry of Plan Implementation has taken the following steps to Institutionalize RBM government-wide.

  • Established a Core Group on MfDR to guide, direct and steer MfDR initiatives
  • Developed a government wide MfDR Action Plan
  • Established a mechanism to review and report MfDR implementation progress.
  • Developed Agency Results Framework for line Ministries to institutionalize MfDR
  • Agency Scorecard for line Ministries to report on results using key performance indicators (KPIs)
  • Develop Results Framework for Mega Development Projects
  • CapacityBuilding, Advocacy, awareness and sensitization initiatives
  • Establishment of an MfDR web-platform in the NOR.

The Government of Sri Lanka has many parallel performance management initiatives such as performance budgeting, performance audits, strategic planning and staff performance evaluation systems. The Millennium Development Goals (MDGs) are part of the National Development Framework and are being adopted at the country level as the basis for tracking country outcomes and monitored annually. Also MDG targets and indicators are being localized at sub-national level which helps to establish RBM at provincial and district level. The Department of Census and Statistics (DCS) has been strengthened to collect and feed such development related statistical information through Household Income and Expenditure Surveys and Demographic Health Survey.The Dev Info system is being used to track MDG achievements by the DCS.

There is also extensive use of performance information in the budget preparation. The performance information helps to rationalize resource allocation. There is a need to establish strong feedback links between evaluation and policy making, planning and budgeting process.The Department of National Planning has made mandatory use of Results Frameworks and Logic Models in the project concept documents and project submission formats. Also ex-post evaluation lessons are incorporated in the project concept documents and project submission forms. As such from strategic planning through implementation to completion and beyond, the dialogue on results takes place.

Outcome/output based performance indicators and targets have been incorporated into the budgeting process. The line Ministries Annual Implementation Plan (AIPs) sets out the key deliverable, operational intentions, and serves as a basis for reporting progress. AIPs which are tabled in the cabinet also emphasize on the importance of setting realistic expectations for results by engaging stakeholder participationand ensure transparency in performance reporting. AIPs as agency plan need to be tied to Employee Performance Plan. This need to be further strengthened through Performance Contract arrangements and is being discussed for consideration.

01.Background

The Government of Sri Lanka (GOSL) fully recognizes the growing international consensus that monitoring and evaluation[1] is an essential aspect ofgood governance to improve development efficiency and effectiveness, transparency, accountability and informeddecision- making. In the recent past, globally monitoring and evaluation expanded and diversified in many contexts with many uses such as decision-making, organizational learning, knowledge base, programme improvement, policy development, impact/outcome assessment, improved service delivery, accountability, performance audit, empowerment and even transformation. Ambitious government systems with multiple stakeholders needs tend to achieve most of these desired uses. A good Monitoring and Evaluationsystem should go beyond institutional boundaries to cover national, sectoral,programme and project levelto ensure results orientation in government.

The Ministry of Plan Implementation has introduced the Results based Management and reporting system with performance indicators to track development results of the line ministries and their programmes. Logical Framework Analysis (LFA) and Results Framework is being increasingly used in planning and M&E arrangements within government. Line Ministries are required to justify their budgets with well defined output/ outcome indicators. TheDepartment of National Budget and Treasury has revised their budget circulars to focus on ‘results’ to institutionalize performance budgeting systems. The Government’sthree year Medium Term Expenditure Framework (MTEF) incorporates outcome based key performance indicators to justify Public Expenditure and helps to allocate resources rationally on a results oriented manner. All these developments clearly indicates that the concept of “Managing for Results” set-out in the Paris Declaration is being institutionalized and moving forward in Sri Lanka.

02.The Government’s M&E Strategy

There exists a strong link between Ministry of Plan Implementation (MPI), Ministry of Finance and Planning (MOFP) and President’s Office and they work very closely with each other on M&E strategy. The Ministry of Plan Implementation is mandated with the responsibility for monitoring and evaluation of all government policies, programmes, projects and institutions. The Department of Foreign Aid and Budget Monitoring is the key functional arm of the Ministry of Plan Implementation to undertake the M&E functions. The Cabinet made a formal decision and directed the MPI to monitor all projects over Rs 50 million executed by the line ministries and submit quarterly progress report to the Cabinet. Major achievements on M&E strategy include the following:

  • Electronic Project Monitoring System (ePMS):A Distinctive Feature in Sri Lanka–One of the noteworthy and significant aspect is that ahome-grown, user-friendly,national, Web-based electronic on-line Project Monitoring System (ePMS) has been established in the Ministry of Plan Implementation (MPI) to track the implementation progress (financial/physical) and results of all development projects and programmes. The system provides access to project information on a donor-wise, sector-wise and ministry-wise basis. The system uses an early warning (traffic lights) system that enables ‘problems projects’ to be separated from others. The system identifies bottlenecks, delays, issues and constraints in the implementation of projects and any additional needs of the executing agency. It includes results monitoring (using LFA), monitoring loan covenants, cash flow, Reimbursable Foreign Aid Claims, Procurement progress and major issues with pictorial proof. Flash Reports on problem projects helps in troubleshooting exercise and submitted to Cabinet of Ministers on a quarterly basis. The ePMS system is being upgraded to capture feedback from beneficiaries and citizens. Notable features of the system are;

-Tracks financial and physical progress.

-Harmonization around Results Reporting through Results Framework

-Home–grown, country owned, country driven system

-Builds paperless monitoring capability in the Public Sector

-Pictorial Proof to demonstrate progress at ground level

-Data in Captured at Source hence reduces transaction costs

-Red Alerts on Problem Projects and Projects behind schedule

A recently completed Donor evaluation mission rated the ePMS as a success story in terms of its comprehensive coverage, periodical updating, and use of information for troubleshooting. However, the current low level utilization of the system by sector Ministries and Ministry of Finance and Planning indicates an unexploited opportunity. This is being addressed. Also it is necessary to enhance ‘data capture at source’. The system allows for better overall coordination and helps to address execution problems. Progress of all projects over Rs 50 million and progress of capital budget of line ministries are submitted to Cabinet of Ministers on a quarterly basis. Also the report to the Cabinet identifies progress, major issues and project status to separate on schedule projects from behind schedule once and halted once.

  • Government-wide Performance Measurement System and Score Cards:

Governmentof Sri Lanka (GOSL) has introduced a Results-based Monitoring and Evaluation System (RBME) with the technical support of the UNDP to track performance of ministries and institutions using output and outcome indicators.This is a new management style that holds Ministries/ Departments and public officials accountable for results rather than ‘efforts’. RBM when operation ensures line-of-sight which implies everyone in the organization understands the strategic vision and mission irrespective of their level and position in the organization. .

With RBM, a comprehensive Performance Measurement System is being piloted with 4 key line Ministries (M/Education, M/Health, M/Agriculture, M/Highways). An RBM Core Group has been established to drive this initiative with the leadership of Secretary, MPI. The experiences of the North American RBM Models operated in States such as Oregon, Minnesota and Virginia were taken into consideration in developing a localized model. Under the RBM framework emphasis of monitoring is on ‘results’ rather than ‘efforts’ taken by Ministries.

The MPI is to establish a web-based RBM platform in the National Operations Room (NOR)- the information arm of the MPI- to maintain Agency Results Framework and Score Card/Report cardfor each line Ministry. The centerpiece of the government’s results based management system is the ‘agency results framework’ and ‘agency score card’. The customized Score Cards/Report cards will appear in the computer screen to enable the assessment of the performance of the line ministries.This “Dashboard” serves as an ‘early warning’ signal to alert when there is slippage in the achievement of targeted outcomes. Availability of trained M&E personnel is a key constraint. Also incentives need to be in place to reward success. The RBM when fully operational government-wide helps to improve continuous learning culture.

The Ministry of Plan Implementation has taken the following steps to Institutionalize RBM government-wide.

  • Established a Core Group on MfDR to guide, direct and steer MfDR initiatives
  • Developed a government wide MfDR Action Plan
  • Established a mechanism to review and report MfDR implementation progress.
  • Developed Agency Results Framework for line Ministries to institutionalize MfDR
  • Agency Scorecard for line Ministries to report on results using key performance indicators (KPIs)
  • Develop Results Framework for Mega Development Projects
  • CapacityBuilding, Advocacy and awareness and sensitization initiatives
  • Establishment of an MfDR web-platform in the NOR.

In the past, the progress monitoring of ministries were focused heavily on financial progress (budget utilization) and physical progress (activity monitoring) with very little emphasis achievement of outcomes, results and policy objectives. However, implementation issues ad inter-agency coordination issues were addressed through the High level monitoring meetings held by the President. MPI on a quarterly basis submits to the Cabinet of Ministers the progress on capital budget of all line Ministries and development projects over Rs 50 million. In the recent past management emphasized more on ‘synthesis type reporting’ rather than the traditional way of producing large volume of reports that resulted in ‘information overload’.

The RBM approach encourages Ministries and Departments to understand the ‘results chain’ and establish logical linkages between planned inputs, expected activities/outputs and envisaged outcomes. The Results Based Management (RBM) initiative is to be complemented with Performance Agreements to ensure ministries and institutions manage priorities in a results oriented manner accountable for the targets they set. While MPI is introducing RBM to Ministries and Department, the Department of Public Enterprises of the MOFP has introduced RBM to State Owned Enterprises (SOEs) and Statutory Bodies. Advocacy and awareness creation on efforts on RBM are in progress.

  • Outcome-based National Economic Performance:

Under the traditional national economic performance reporting system high economic growth, low inflation and unemployment were regarded as indicators for healthy economic climate and believed to result in prosperity for citizens. However, citizens are increasingly concerned about their quality of life measured in terms of quality of education, health care, safety from crimes, clean environment (safe drinking water and sanitation) etc. Hence traditional economic based measurement systems miserably failed to address such quality of life issues and outcomes. The GOSL has established a localized MDG results reporting system for which data collections support is extended by the Department of Census and Statistics and is being widely used within the government.