VI Annual Conference of the Latin America and Caribbean Regional Network on Monitoring and Evaluation
25-27 August 2010, Mexico City
Improving the Performance of Sub-National Governments and their provision of information capacity:
The Public Management Rapid Assessments and Action Plansfor Sub-Nationals Governments in Colombia[1][2]
Summary
Monitoring and Evaluation Systems require strong public administrative management frameworks, systems and practices, in order to ensure the creation, collection and administration of reliable information regarding the provision of monitored and evaluated policies and services. Lack of them contributes to the generation of poor or insufficient information that prevent a reliable monitoring and evaluation of policies and services. The “Rapid Assessments and Action Plans Methodology” was designed and implemented in order to contribute to the creation or improvement of these management systems at sub-national level of government, as well to facilitate the adequate implementation of Colombian mechanisms aimed to monitor and evaluate the performance of sub-national governments and their delivery of services. Its pilot implementation in selected Colombian municipalities has demonstrated important impacts in the improvement of public management and delivery of services, and equally important for M&E purposes, in the generation of reliable information on public expenditure management.
1.Background and Colombia’s Country Context
In Colombia, many services important for social and economic development are the responsibility of sub-national governments. In 2008, the collectively executed expenditures were worth at least 7.7 percent of GDP[3]. Municipalities and departments raise 2.4 percent of GDP in tax revenues (equivalent to almost one fifth of the tax revenues of the combined public sector). The difference is covered by fiscal transfers (3.9 percent of GDP) through the “Sistema General de Participaciones” and by other revenue sources, including non-tax revenues and royalties or “regalías”. Given the size of resources managed at the local level, the country has an interest that they are managed efficiently. Having effective sub-national governments is also critical to achieving the service delivery targets that have been set for the years 2010 and 2011.
Colombia’s National Development Plan 2003-2006 mandated the development of a sustainable framework for the evaluation of the decentralization process and its impact on territorial development. The Plan called for “the design and application of a comprehensive evaluation system of both the decentralization process and performance of territorial entities, that can help assess the extent to which: a) goals established in the local development plans have been accomplished; b) resources have been use efficiently; c) managerial results have been attained, and; d) legal requirements have been fulfilled. It is expected that Departments will produce a performance report of their municipalities that will be compiled by the Department of National Planning (DNP), and that results will be shared broadly with the citizenry.”The DNPmoved ahead and decided to develop a more comprehensive instrument to evaluate the performance of sub-national governments. In May 2005, the DNP published the “Metodologia para la Medicion y Analisis del Desempeno Municipal.”The DNP’s methodology assesses five components of sub-national performance: (i) Effectiveness; (ii) Efficiency; (iii) Legal Requirements; (iv) Management Capacity, and; (v) Exogenous Variables.[4] Annex 1 provides a brief description of the methodology and each of the components assessed by it. Later, in January 2008, the Government approved Decree 28 in order to improve and strengthen its strategy for the supervision, control and monitoring of expenditures executed by sub-national governments with resources of the transfer system (“Sistema General de Participaciones - SGP”). This Decree establishes a set of mechanisms aimed to supervise the adequate use of transfers as well the fulfillment of service delivery targets. The Decree also allows the Central Government to apply to sub-national governments preventive and corrective measures aimed to guarantee the Decree objectives. One key measure is the agreement of Performance Plans (Planes de Desempeno) which allow the Central Government to commit sub-national governments for the implementation of measures aimed to eliminate the causes of poor service delivery, including the improvement of administrative management tools, systems and practices. The Fiscal Support Direction (Direccion de Apoyo Fiscal-DAF) at the Minister of Finance and Public Credit is responsible for the implementation of this norm.
The “Public Management Rapid Assessments and Action Plans” (RAAP) methodology was developedby a World Bank team to contribute to the implementation of the new Colombian sub-national framework for the monitoring and control of the transfer system’s resources.The RAAPs are not a monitoring and evaluation system as such. Instead, theyareused to help sub-national governments to improve the capacity of their administration,including efficiency of public management of expenditures, revenues and services, as wellthe creation of basic management tools aimed to provide reliable and on-time information for monitoring and evaluation purposes.
Two pilots using the RAAP methodology have been carried out in Cartagena and Barranquilla, Colombia between 2009 and 2010.Their impact in the administrative capacities of the municipalities is described later in this paper.
2.RAAP Methodology: a problem-based approach offering feasible short term improvements
TheRAAPs evaluate five keysub-national government’s management areas (Fiscal management and sustainability, Financial, Tax, Human Resources and Procurement management). However, they are customized to the specific needs of each sub-national government and might include other areas (Planning, Assets, Legal Defense management) as well specific service delivery management arrangements. The tool (i) identifies the main weaknesses of each of these areas, particularly with a view to improving efficiency; (ii) assesses the impact on the fiscal situation, service delivery and overall management; and (iii) proposes a set of managerial and operational reforms. TheRAAPs contribute to identify shortcomings and constraints preventing the efficient delivery of services due to poor management tools, systems and capacities.
Table 1: Comparison of the DNP and RAAP Scope (customized to the Case of Barranquilla)
Public Management Area / DNP’s Metodologia para la Medicion y Analisis del Desempeno Municipal scope / Rapid Assessments and Action Plans MethodologyScope
Human Resources Management / Senior Staff Stability/Rotation
Professionalization of the unit / Analysis of personnel characteristics (by unit and temporary / non professional), turnover and distribution of functions as well as remuneration policies
Comprehensive analysis of the mechanisms of recruitment, hiring, promotion, removal, training and performance evaluation.
Financial Management / Compliance with legal requirements in the incorporation of resources in the budget. / Comprehensive analysis of the operation of the planning, programming and budget execution.
Tax Administration / Comprehensive analysis of revenue and assets management, including coverage of the tax base, tax collection policies and rebates for the taxpayer.
Information Technology / Access to computers and level of automatization of the processes / Analysis of information systems used to support in the areas of civil service management, financial, tax, procurement.
Public Procurement / Management of contracts, tendering and inventories. / Comprehensive analysis of procurement, personnel, procedures, resource allocation process, procurement planning, transparency, corporate purchasing policies, systems claim.
Internal Control / Internal Control Standard Model / Comprehensive analysis of the functioning of the budgetary control systems and management.
The RAAPs start by examining whether each of the management areas are delivering some accepted outcomes of a well-functioning public sector. They characterize the desired outcomes, and then offer some suggested areas in which shortcomings in achieving these might be seen. They then set out the institutional and management arrangements that are widely seen as contributing to these outcomes, before offering some areas which might identify weaknesses in those arrangements and hence account for any failures to deliver the outcomes. The RAAPs use information that is currently available and can be easily collected. The methodology is less ambitious than PEFA, CFAA and CPAR in terms of depth of analysis. However, its purpose is to have a quick and broad understanding of the way that key areas of public management are fulfilling their objectives at sub-national level of government.
Once the shortcomings that prevent the adequate operation of the management arrangements and the reach of desired outcomes are identified, an Action Plan will propose a set of reforms that will contribute to the creation or improvement of those institutional arrangements necessary to achieve the desired outcomes. The reform measures will have some common characteristics: They can be decided by the sub-national authorities without any kind of intervention from the Central Government; the measures will consist in managerial reforms (organizational, allocation of resources, processes and systems) to improve operations; the cost of their implementation will be largely covered by the benefits they will generate; their implementation require short term actions.
3.Piloting the RAAPs in Barranquilla and Cartagena
The Cartagena pilot was finalized in June 2009 while the Barranquilla assessment was completed in mid 2010. The coverage of the RAAPswas agreed with the selected sub-national governments and the Central Government[5]. Based on the information gathered as part of the exercise, “Management Action Plans” were prepared for the two municipalities recommending the adoption of politically and technically feasible measures whose implementation would provide quick management, fiscal and service delivery gains within a short timescale.
In Cartagena, the action plan included proposals to improve financial management, tax administration, procurement and human resources management. The RAAP presented proposals to improvethe control of health expenditures while guaranteeing the health service provision. These measures together with the creation of a pension fund were expected to ensure sustainable public spending and would contribute to free resources to finance the foreseen investments for the Government’s strategy “ Por una Sola Cartagena”. The positive fiscal impact largely exceeded the costs of its execution. The concrete outcomes that can be associated with theRAAP Action Plan include (i) tax administration improved as information systems and cadastres were strengthened,tax policies revised and collection incresed; (ii) more efficient tools and databases for procurement were implemented and new procurement methods were applied representing fiscal savings; (iii) the human resources functions were enhanced by modernizing the HR records, streamlining the HR processes and revising the organization andmandate of the “Direccion de Talento Humano”; (iv) the health expenditure system was restructured and significant fiscal savings were achieved while reaching to the universal coverage of services; and (v) reliable information on public expenditures and revenues are available. More detailed information about the Cartagena pilot ispresented in Annex 2.
In Barranquillathe RAAPwas focused onfinancial management, tax administration and human resource management, in order to improve the management of public expenditures and the fiscal situation. The concrete outcomes following the implementation of the reform agenda and recommendations include: (i) the fiscal situation has dramatically improved; (ii) significant increased in tax collection;(iii) financial management and human resources reorganization has allowed the District to generate reliable information and reallocate its human resources more effectively; (iv) procurement and legal defense management measures allow the municipality to generate important fiscal savings; and (v) reliable information on public expenditures and revenues are available.More detailsabout the Barranquilla experience are included as Annex 3 of this paper.
- Lessons Learnt for Wider Application
In Latin America, the number of municipalities and their responsibility in public expenditure and delivery of services have increased significantly in the last two decades. One key shortcoming for increased responsibilities is their low and heterogeneous institutional capacities. Different tools are currently available to assess these shortcomings and propose actions for improvement. After completing a world-wide review of management, performance, and improvement tools for local governments, the team foundthat none of them provide feasible short-term solutions to improve public management capacities, overcome political economy constraints for their implementation and generate a rapid improvement in the generation of reliable information.
A number of contextual factors are important to consider when comparing different approaches of management at the local level. There are significant inter- and intra-country differences, not only with regards to institutional capacities. More importantly, it is the different intergovernmental systems and decentralization models which determine to large extent how, what, and who is evaluating local governments. The Rapid Assessments methodology can be customized to the national and local context where applied. Although the methodology at present covers every aspect of public management or administrative areas, not necessarily all of them require to be assessed.
Based on the Barranquilla and Cartagena pilots, the RAAP characteristics ensured the commitment of local authorities for their implementation. The Cartagena Action Plan, for example, highlighted that specific actions aimed at improving tax administration lead to important increases in collection of revenues. This was an important incentive for authorities to undertake the recommended plan.
Both pilots also confirm that rapid improvement of management areas through gradual measures can accelerate the execution of public expenditures and investments, improve the quality of them and increased the quality and coverage of public services.
References
Acosta, O. and Bird, R. (2003) “The Dilemma of Decentralization in Colombia”, available at
Departamento Nacional de Planeación (2005), “Metodología para la medición y análisis del desempeño municipal” available at
Departamento Nacional de Planeación (2008), “Desempeño Fiscal de los Departamentos y Municipios – Vigencia 2007” available at
Departamento Nacional de Planeación (2008), “Evaluación del desempeño integral de los Municipios – Vigencia 2007” available at
Departamento Nacional de Planeación (2008), “Técnicas para el análisis de la gestión económica y financiera de las entidades territoriales - 2009” available at
Dumas Victor (2010): Intergovernmental Performance Information Monitoring Systems. Colombia’s Comprehensive Performance Measurement System for Municipalities. World Bank.
Echavarría, J., Rentería, C. and Steiner, R. (2003) “Descentralización y salvamentos (bail outs) en Colombia” en Coyuntura Social available at ftp://ftp.fedesarrollo.org.co/pub/r_social/2003/diciembre/Juan_Jose_Echavarria.pdf
Gaviria, Jean Philippe (2003) “Evaluación del proceso de descentralización en Colombia” in Economía y Desarrollo, Vol. No.2, Number 1.
Mosqueira Edgardo et al (2009): Methodology for Public Management Rapid Assessments and Action Plans for Sub-National Governments, Public Sector Management Unit, Latin America and the Caribbean, World Bank.
Olivera, M., Gómez, R. and Velasco, M. (2009) “Building a Results-oriented Management and Budgeting System at the Sub-national level – Medellin as a Case Study”, World Bank Internal Evaluation Group ECD Working Paper Series, forthcoming in 2009.
Restrepo, Dario (2002) “El futuro de la descentralización el Colombia” available at
Villareal, Julio (2007) “Colombia y el Sistema de Monitoreo y Evaluación, SINERGIA” paper supported by the World Bank and Centro Latinoamericano de Administración para el Desarrollo (CLAD) available at
World Bank, June 2009. “Plan de Acción Rápida para la Mejora de la Gestión Pública de la Municipalidad de Cartagena de Indias”
World Bank, June 2009. “Plan de Acción Rápida para la Mejora de la Gestión Pública de la Municipalidad de Barranquilla” (DRAFT)
World Bank (2008) “Colombia Decentralization: Options and Incentives for Efficiency – Volume I: Main Report"
World Bank. 2005a. “Colombia Public Expenditure Review.” Report No.25163-CO, Washington, D.C.
World Bank. 2005b. “Country Assistance Progress Report for the Republic of Colombia.” Report No. 32999-CO, Washington, D.C.
World Bank. 2007. “Colombia 2006-2010: A Window of Opportunity.” Colombia Policy Notes 2007, Washington, D.C.
World Bank. 2005. “Country Procurement Assessment Review.” Washington, D.
Annex 2: Case –Cartagena
The main challenge of the Municipal Government of Cartagena that took office at the beginning of 2008 consisted of implementing the District’s Development Plan 2008-2011 “Por una Sola Cartagena”. The plan aims at reducing the gap between the“two Cartagenas”. While part of the municipality is modern and developed with access to services, other parts are characterized by a high poverty incidence rate and limited access to public services. With the objective of improving conditions for the population of fewer resources, the Plan proposes to increase the District’s public investment by approximately 40% in 2011, compared to public investment in 2005. However, these objectives faced two obstacles. On the one hand, the municipality’s management and operation presented weaknesses that reduced the institutional capacities needed to implement the Plan. On the other hand, the fiscal situation of the District required the adoption of strengthening measures to generate the necessary fiscal space to finance the Plan.[6]
Related toPublic Expenditure Execution the Action Plan proposed measures to improve financial management, tax administration, procurement, and human resource management. The RAAP also presented proposals to better control health expenditures and guarantee health service provision. Regarding revenue side, the RAAP madesuggestions to increase tax collection. These measures together with the creation of a Pension Fund would ensure sustainable public spending and would contribute to free resources to finance the foreseen investments for “ Por una Sola Cartagena”. The financing of the fund would come from balance resources, assets of the District, and a medium term loan. The positive fiscal impact largely exceeds the costs of its execution. The specific characteristics of Cartagena’s organization and processes, the weight and impact of each management area in the sectors and the type of information available and reliable were valuable inputs to adapt the questionnaires for the RAAP. Cartagena’s RAAP was completed in June 2009, and the progress presented in this document is up to May 2010.
In the field ofTax AdministrationCartagena has advanced significantly compared to the initially assessed situation. In regards to institutional progress the Alcaldía strengthened its Tax Division and created special task groups. Also, the design of procedures and strategies for tax collection specifically, and strengthening the information systems that support tax administration and easier online access for tax payers in general, have been important. In 2009, the Cadastre of the District was updated (the last was done in 2003. Tax policies were also revised particularly the ones regarding exonerations.