February 25, 2014
Draft version
Page 1
Expenditure Analysis Supported by BOOST:
Standard Tables, and Selected Tools and Techniques
Guidance Note to PER Teams
Table of Contents
Foreword
Section 1. The Application of Efficiency and Effectiveness Measures in Public Expenditure Analysis.
Section 1.1. Expenditure Analysis supported by BOOST–Getting the Analytical Dimensions and the Standard Tables Identified.
Section 1.1.1. Identification of BOOST Standard Tables on Expenditure (descriptive stats)
Section 1.1.2. Identification of BOOST Tables for further Analytical Purposes.
Section 1.1.3. Framework on Expenditure Analysis using Standard Tables.
Section 2. Descriptive Statistics – Profile, Trends and Composition Analysis of Expenditures
Section 2.1. Descriptive Statistics in PERs.
Section 2.2. Profile Analysis.
Section 2.3. Trend Analysis.
Section 2.4. Budget Composition Analysis.
Section 2.5. Descriptive Statistical Analysis --- Examples from selected PERs.
Section 2.5.1. Peru PER
Section 2.5.2. Uganda PER
Section 2.5.3. Armenia PER
Section 2.5.4. llustrations of BOOST standard tables and charts [Work in progress]
Section 2.6. Definitions and Data Sources
Section 3. Selected Analytical Dimensions using BOOST Databases– Efficiency, Effectiveness and Equity analysis of expenditures.
Section 3.1. Effectiveness
Section 3.2. Equity
Section 3.3. Efficiency
Section 3.3.1. Technical Efficiency
References
Annex 1: Institutional Coverage in BOOST
Annex 2: Devolved Local Government Entities versus Deconcentrated Local Administration Bodies
Annex 3: BOOST Core Variable Definitions Aligned With Internationally Agreed Standards
Appendix 4: BOOST Standard and Quality Check Table
Foreword
The search for improvements in public expenditure efficiency, effectiveness and equity is at the core of PERs at the Bank. This purpose of this note is to improve the efficiency of PER work, by providing guidance to teams on a selected number of techniques, tools and standard data tables related to the preparation of a PER.
The note was prepared in the context of a Bank-wide review of PER – the Public Expenditure Review Stocktaking and Guidance. The Review intends to issue a number of specific guidance notes, covering in more details also some of the techniques and tools mentioned in this paper. As an example, a Guidance Note is being prepared on the DEA and SFA models, to present the various applications in details, including the related data requirements, and to provide hands-on guidance on how to conduct analysis.
The BOOST Technical Advisory Group provided comments on earlier drafts, as well as contributions were received from… Overall guidance was provided by…
The draft paper is presented to the BOOST Technical Advisory Group at its meeting on March 4, 2014
The TAG is asked to:
= review and endorse the proposed Standard Tables on BOOST (Sections2-3)
= provide overall guidance and comments on the draft paper, in particularly on:
- The overall contents and directions of the paper
- The relevance of the selected tools and techniques. Any others?
- Standard tables – should be paper be generalized, to focus on standard tables and standard data in PERs, rather than the current, narrower perspective on standard tables using BOOST data?
- Country cases. Would there be other examples of PERs to bring into the paper?
- Views on dissemination and out reach
- Any other views on how to complete the draft paper? Would TAG members be interested in joining this work, including preparing country cases?
Further to the discussion at the TAG meeting, PRMPS will continue drafting (in particularly Section 3 needs more work) and will insert comments from TAG members. A completed draft version will be circulated to the TAG, for a final review. Final version of the paper is planned to be issued in May.
Section 1. The Application of Efficiency and Effectiveness Measures in Public Expenditure Analysis.
The application of concepts on efficiency, effectiveness and equity often prove difficult in PERs for a number of reasons. These include:
- Difficulties in measuring efficiency and effectiveness – defining output in public administration or service delivery still poses problems; the various outputs may contribute to several policy objectives at one and the same time; and the costs of producing any given output are difficult to identify, since a clear understanding of the cost functions is absent and/or costs only partially captured in the accounting systems.
- While progress has been made in developing measurement techniques, often good quality data is lacking to apply these techniques.
- In some cases the various terms may have a different meaning when applied in different contexts, as well as exogenous factors vary depending on the level of analysis. As an example, in an analysis of technical, allocative and scale efficiency on education, the wage setting framework is seen as an exogenous factor, while in an analysis of the overall public expenditure levels and composition, the wage setting framework would be considered a variable to use, to address inefficiencies. The “allocative efficiency” term would also have been used differently at this level of analysis, as compared to an analysis on education, (or of a specific school for that matter).
- The application of different levels of aggregation of analysis, as often determined by the poor availability and quality of data, may imply an uneven understanding across countries of efficiency issues, including level of policy makers control to make changes. A high level of aggregation of analysis may conceal inefficiencies, while on the other hand, a granular sector-related analysis could succeed in unbundling the various drivers behind inefficiencies, including identifying those which may be in the government’ immediate control to change.
- Contextualization. PERs on MIC, LIC or FCS will vary in the application of efficiency and effectiveness measures, in great part due to variances in data availability and quality. There is a need for customization and careful attention to setting the scope of the analysis, the tools and techniques to be applied, and the data availability to be expected. [1]
- These issues illustrate the importance of correctly defining and understanding the scope of an efficiency, effectiveness or equity analysis, including selecting the appropriate techniques and data sources. It is equally important to situate the PER in the context of the overall expenditure policy framework for the country as such, including to capture how the analysis relate to the various ‘levels of analysis’. This point is illustrated in Figure 1 below.
Figure 1: Public Expenditure Analysis – Analytical Questions and Selected Techniques
Figure 1 illustrates the nested structure expenditure analysis, including application of the core measurements on efficiency and effectiveness. As an example, looking bottom-up, an effectiveness assessment on a specific sector, including a discussion of determinants, will remain incomplete or constrained to the extent that the efficiency review is missing or incomplete. Moving further up, a review of public expenditure matters on public sector, including improvement in public expenditure management efforts, usually require the efficiency and effectiveness reviews already completed at sector, program/project or facility unit-levels. It follows thereof that the complexity of analytical design in the PER and the related data requirements increases, as the analysis moves from one level to the next. Much more institutional context and qualitative data are required when going from efficiency to effectiveness or to the analysis of aspects of the overall expenditure policy. The preparation of a PER needs to take these aspects into account in order to ensure relevance and impact of the exercise.[2]
In this perspective, the purpose of the paper is to assist PER teams in clarifying scope and comprehensiveness of the PER, by using standard tables, and tools and techniques well adapted to the analytical questions at the various levels of analysis as outlined in Figure 1. The paper will come back to this issue in details in Sections 2 and 3.
The paper is organized as follows: Within the overall context as outlined in Figure 1, the remainder part of Section 1 identifies the analytical questions and the related tools, techniques and data, to support presentation and analysis PERs on two levels: descriptive- and analytics statistics. Section 2 then further presents the descriptive statistics, by identifying the standard tables and the various analytical questions on ‘profile’, ‘trends’ and ‘composition’ presentations, while Section 3 presents and discuss similar issues on analytics statistics – ‘efficiency, effectiveness and equity’.
Please note that while the paper mainly defines tables and analytical questions in the context of BOOST data, the intention of the paper is nonetheless to provide guidance to PER teams on tools, techniques and data in general.
Section 1.1. Expenditure Analysis supported by BOOST – Getting the Analytical Dimensions and the Standard Tables Identified.
The construction of BOOST datasets is being guided by a set of BOOST standards, which outline the optimal institutional coverage and data structure, as well as some procedural steps to ensure high data quality (see Annex 4- BOOST Standards and Quality Checks). The identification of BOOST standard tables is thus a logical next step in ensuring quality support to PERs and the relevance of BOOST databases. While the main purpose of BOOST standard tables is to improve efficiency of the PER preparation, the standard tables also supports a continued high quality of BOOST data. With the tables defined ex ante and independent of the specific PER, the tables guide the PER teams in defining their data queries, but the tables also give indications to the BOOST assembly team on the desired structure and coverage of the BOOST. In a broader and medium-term perspective, the standard tables may lead to better opportunities for cross-country expenditure comparisons.
Section 1.1.1. Identification of BOOST Standard Tables on Expenditure (descriptive stats).
To date there are 21 BOOSTs delivered and each BOOST database supports in varying degrees the preparation of standard tables. The databases can be found on the iTeam site. Once the design of the standard tables has been approved, such tables on the 21 completed databases will be posted on the iTeam site.
The standard tables are based on the different cuts of the three main dimensions of expenditure analysis or the core BOOST database variables (see Figure 2). These are the Administrative variable which provides information about which spending unit incurred the expense; the Economic type variable which provide information about the category or type of expense incurred; and the Function/sector variables which provide information about the sector or purpose for which an expense was incurred. In addition to these core variables, it also includes the expenditure type (recurrent, personnel, capital, non-personnel) and the financing source variables.
Figure 2: Dimensions of a Basic BOOST Expenditure Database
The numbers of analytical tables (standard and customized tables) that can be produced vary from database to database. Ideally one should be able to produce most of the standard tables as outlined in Section 2 from any given basic BOOST expenditure database. Within the basic expenditure database, the number of customized tables one can produce depends on the granularity of the data and the number of custom variables included in the BOOST database. In some databases the granularity of the data is such that it allows the user to track expenditures at the point of service delivery - districts, hospitals, schools, etc. In others, pro-poor expenditure data are already coded within the database enabling poverty related analysis.
Section 1.1.2. Identification of BOOST Tables for further Analytical Purposes.
The breadth and depth of analysis that can be done with the basic BOOST expenditure database can be further enhanced in three ways. First, customized modules such as Pay-roll module can supplement the basic expenditure database to answer questions like number of staff paid by program (see example on Mauritius in Table 6) or a Capital expenditure module can be prepared to investigate the dramatic decline in capital expenditure in a country (see example on Moldova in Table 5). Second, a BOOST expenditure database can also be supplemented with socio-economic data such as population to figure out per capita spending on education etc. Third, the BOOST dataset may be supplemented by specific sector input- and performance data, in cases where an in-depth analysis of a sector is desired. Most of the technical efficiency, effectiveness and (to some extent) equity analysis require output and/or outcome data, and for which reasons, the BOOST expenditure data set needs to be supplemented with this data. Currently Education and health sector BOOSTs are routinely prepared in support of PERs (see example on Serbia in Table 6).
Figure 3: Integrated Policy Analysis Platform
Section 1.1.3.Framework on Expenditure Analysis using Standard Tables.
The standard tables in this note are organized on the basis of their application on the most commonly used analytical techniques in PERs. These techniques are profile analysis, trend analysis, budget composition analysis, efficiency analysis, effectiveness analysis and equity analysis. These techniques are further grouped into two main perspectives: descriptive (profile, trend, and composition), and analytical (efficiency, effectiveness and equity). Table 1 summarizes the presentation of expenditure analysis, by purpose, tools and techniques and data, as discussed in Section 1.1.
TABLE 1: FRAMEWORK FOR EXPENDITURE ANALYSIS USING BOOST
Analytical Purpose / Tools & technique / Data requirementDescriptive stats / Profile analysis / Profile analysis is used to lay out the expenditure landscape of the country at a given point in time. It is also used to gauge how a country performs relative to its peers. These questions in turn will lead to how is the trend of issues. /
- Presenting expenditure and/or components of expenditure in levels or as a share of GDP.
- Comparing/benchmarking using peer countries.
- Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. Including information on the financing source and expenditure type is important.
- Minimum of 1-year (most current year) data is required. If presenting the average overtime makes more analytical sense, then will require data for more than one year.
Trend analysis / Trend analysis is used to gauging whether a country significantly worse (or better) than “expected,” or that improvements are slow in coming. Would the country reach its target and how quickly.
These observations then provoke deeper exploration into the causes of problems. /
- Presenting time series of total expenditure and/or components of expenditure growth rate or in levels or as a share of GDP.
- Comparison/Benchmarking with peer countries.
- Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. Including information on the financing source and expenditure type is important.
- Minimum of 5-10 years data is required.
Budget Composition analysis / The Budget composition analysis help identify the government’s priority sectors and activities. This motivates efficiency, effectiveness and equity questions. /
- Presenting expenditure as a share of total expenditure by sector and/or by economic type.
- Comparison/Benchmarking with peer countries.
- Basic BOOST database containing expenditure data by functional, economic, or administrative classification of the budget or any combination of each. In addition, information on the financing source and expenditure type can be included.
- Minimum of 1-year depending on the type of analysis done: Profile (minimum 1-year) and trend (5-10yrs).
Analytical stats / Allocative efficiency / Allocative Efficiency analysis provides information on whether a country or sector is achieving optimal mix of inputs. /
- Benchmarking with comparator countries and international norms.
- Budget deviation analysis.
Technical efficiency / Technical efficiency gives insight into the efficiency with which inputs are converted into outputs. /
- DEA analysis
- Four quadrant analysis
- Bi/multivariate regression residuals analysis
Effectiveness / The analysis of effectiveness is about the relationships between inputs, outputs andoutcomes. / Basic BOOST database and data ib output/outcome indicators
Equity / Equity analysis is done to see whether public money reaches those with the greatest need. /
- Presenting geographic distribution of expenditure
- Presenting expenditure and demographic data
- Presenting outcome data and demographic data
Section 2. Descriptive Statistics– Profile,Trends and Composition Analysis of Expenditures.
Section 2.1. Descriptive Statistics in PERs.
Descriptive expenditure analyses, as the name implies, are descriptive, giving the first account of the expenditure landscape of a given country which will lead to a declaration of the issues.This in turn will lead to deeper examination into the causes of the issues. They do not provide diagnosis or explanation. Basically they answer broadly one of the core PER question “where does the money go?” Often in PERs the descriptive expenditure analysis used are profile, trend, and composition analysis. Profile analysis captures expenditure at a point in time; trend measures progress over time while composition shows government priority areas presented as a share of total expenditure.
Usually in profile analysis expenditure or components of expenditure data are presented in levels (in local currency unit) or as a share of GDP. In trend analysis, they are presented as levels or as growth rate, or as shares of GDP. For composition analysis, on the other handthey are presented as a share of total expenditure to show the governments’ priority sectors or areas.
Another way these three analytical methods are presented is by using country comparisons or benchmarking as a tool/technique. In most casesthey show profile or trend analysis for a given country as compared to other countries in the same region, income level or ethnic mix or even with countries to which the country aspires to. Here it is very important to make sure that comparisons are done apples to apples. There needs to be consistency in economic classification and institutional coverage in order to compare the expenditure performance of two countries. Please refer to Annex1. Finally, it is worth noting that these comparisons or benchmarking techniques are not substitutes for deeper analysis but rather a step towards further framing the question of performance to be further investigated using efficiency, effectiveness and equity analysis which will be covered in Section 3.