PEMPAL BCOP Program and Performance Working Group Workshop and 12th OECD CESEE SBO meeting in Ljubljana, Slovenia

EVENT REPORT

Distribution: PEMPAL website, report to be distributed within participants' governments/institutions by participants if needed to assist with finalizing the OECD 2016 Performance Budgeting Survey.

Process: The report was prepared by Naida Carsimamovic Vukotic from the PEMPAL Budget Community of Practice Resource Team, based on discussions from the BCOP’s Program and Performance Budgeting Working Group workshop. The report was also shared with the OECD to ensure proper interpretation of the discussions and OECD’s guidance related to the 2016 OECD Performance Budgeting Surveys. The report was shared with all event participants-participants of the OECD Performance Budget Survey from PEMPAL countries.

1. Background

Event Objectives and Participants

On June 28-29, 2016, members of PEMPAL Budget Community of Practice (BCOP) gathered in Ljubljana, Slovenia, to attend the workshop of the BCOP’s Program and Performance Budgeting Working Group on the OECD Performance Budgeting survey and to participate in the 12th OECD Central, Eastern and South-Eastern European (CESEE) Senior Budget Officers meeting.

BCOP representatives came from Ministries of Finance of 14 BCOP countries: Armenia, Azerbaijan, Belarus, Bosnia and Herzegovina, Bulgaria, Croatia, Georgia, Kyrgyz Republic, Moldova, Russian Federation, Serbia, Turkey, Ukraine, and Uzbekistan. A representative from Slovakia also attended as a self-payer in light of interest in survey participation and potential future donor support to PEMPAL.

The BCOP resource team members Deanna Aubrey and Naida Čaršimamović Vukotić supported the technical organization with virtual support by Maya Gusarova and logical and administrative support by Ksenia Galantova and Kristina Zaituna of the PEMPAL Secretariat (World Bank Moscow office).

The objectives of the workshop of the BCOP’s Program and Performance Budgeting Working Group were to i) discuss the preliminary results of the OECD Performance Budgeting Survey and to clarify any issues related to completing the first draft of the Survey and ii) discuss future plans for this recently established Working Group.

The objective of the BCOP members’ attendance and contribution to the 12th OECD CESEE Senior Budget Officers meeting was to use the opportunity for PEMPAL BCOP members to share information and benchmark reforms with a wider representation of Ministries of Finance in the CESEE region.

Agenda Overview

The Agenda of the workshop of the BCOP’s Program and Performance Budgeting Working Group is provided in Annex 1. It included presentation by OECD on the survey structure, objectives, concepts and terminology. Common issues and questions with data interpretation (available at followed by Tour de table during which Survey questionnaire was examined and participants asked specific questions and raised issues they have experienced from entering their preliminary responses to the online survey. The workshop was closed with the presentation by the Working Group’s Leader Nikolay Begchin on the future planning of the new Working Group on Program Budgeting. BCOP Executive Committee meeting was also held after the Workshop (minutes provided separately).

The remainder of this Report provides details on the discussion and outcomes of the workshop of the BCOP’s Program and Performance Budgeting Working Group, while the Agenda and all presentations from the 12th OECD CESEE Senior Budget Officers meeting (including the presentations of the PEMPAL members, facilitated through PEMPAL) are available at PEMPAL BCOP is increasingly having larger roles in these OECD CESEE SBO meetings, with significant inputs to the SBO agenda made by Russian Federation, Kyrgyz Republic and Croatia.

2. Overview of Program and Performance Budgeting Working Group’s Workshop Outcomes

On Monday, June 27th, 2016 in Ljubljana, Slovenia, the BCOP Program and Performance Budgeting Working Group (led by Nicolay Begchin of the Ministry of Finance, Russian Federation) held a workshop on the 2016 OECD Performance Budgeting Survey, as the first meeting for this new sub-group established in early 2016 at the BCOP annual plenary meeting in Minsk, Belarus given the ongoing priority of program budgeting reforms in PEMPAL countries. Participants are currently in the process of completing the OECD Performance Budgeting survey (a precondition to joining the group), which will provide baseline data for the proposed activities of the group over the coming year.

Ronald Downes, Deputy Head, Budgeting & Public Expenditures Division, Public Governance & Territorial Development Directorate, OECD presented the objectives and structure of the survey, initial survey findings from the OECD countries, and some general reflections from OECD on the administration of the 2016 Survey for the PEMPAL countries. After this introductory presentations, participants were invited to raise any issues encountered in completion of the survey preliminary response. As a result of this extensive discussions, the following additional explanation for selected questions of the Survey were provided by the OECD:

For questions 7-11, the following additional guidelines on what qualifies as a standard performance budgeting framework were provided by the OECD:

Respondents should answer that their country has a standard performance budgeting framework even in the case where performance budgeting is only presentational (i.e. given for information purposes only) and when the budget is not officially adopted by programs. As long as there is a standard practice that requires generating performance information regularly within budget requests, the answer should be positive.

For question 12, the following additional clarifications on national performance framework were provided by the OECD:

A national performance framework does not have to be related to budget. A country has a national performance framework when the Prime Minister, the Cabinet or President have some articulation of objectives (e.g. a national strategy) at high outcome levels. It must have high level specific goals that are quite comprehensive, quantifiable, and linked to specific indicators. Opposite to a national performance framework is when there is no high-level national performance framework or some ad hoc and unrelated indicators that are not framed within a comprehensive national strategic document.

For questions 13, 16, and 17, the following additional clarifications on key national indicators (KNIs) were given:

KNIs are connected to the national performance framework, as KNIs are a group of high-level selected indicators for the government’s overall strategic objectives.

Overall, KNIs are subjective as each government chooses their own KNIs, but KNI hallmarks are that they are high-level, strategic, measurable, and that they have continuity in time (as opposed to being tied to political cycles). Overall, KNIs represent a point of reference of how a country is making its citizens’ lives better and they are ideally also comparable to international benchmarks. Governments often choose some high-level internationally used indicators as a part of their own KNIs, such as UN Sustainable Development Goals, PISA indicators, Global Competitiveness Index, World Bank’s Doing Business indicators, EU’s Europe 2020 indicators etc. In general, there should be a relatively small number of selected KNIs (usually around a dozen).

For countries that have KNIs, the underlying logics is that national strategic systems (i.e. national performance framework) and performance budgeting system are connected through KNIs. In these cases, CBA usually plays a role within both of these systems.

However, theoretically, a country could have KNIs without having a national performance framework, and in those cases Question 16 should be responded positively (and question 18 subsequently responded based on the assumption that KNIs exists well).

In general, if there are parts of responsibilities/activities covered by the OECD survey that is performed by or shared with a government institution that is outside of the CBA (e.g. Ministry of Economy), Ministry of Finance should ask those institutions to provide information on the relevant parts of Survey, so that the survey is answered comprehensively by each country.

For question 14, the answer should be positive only in the case when the sole responsibility of such a unit is performance budgeting. In the cases that such a unit may exist but within an institution other than CBA (e.g. in Ministry of Economy), respondents should select other and then insert a comment with an explanation of how responsibilities are divided between Ministry of Finance and such a unit.

For questions 18 and 19, respondents should only take into account quantitative performance indicators.

For question 19, satisfaction/trust refers to user satisfaction with the quality or public services or at higher level overall citizens’ trust in government, while inequality/inequity refers to any type of inequality and inequity, such as income, gender, marginalized groups etc.

For question 21, term “activity” is included to cover the cases in which spending units/budget users do not quantify their outputs and outcomes within indicators but rather only provide their activities (tasks), such as providing training etc.

For question 24, respondents should answer positively even if the performance targets are not officially adopted by the government.

For questions 28-33, OECD Survey’s goal is to collect information on professional views of the respondents, understanding that the answers may in some cases be based on opinions rather than actual evidence.

For questions 40-49 (survey section on Evaluation), evaluation is defined as a more focused, systematic assessment or appraisal of a project, program or policy using performance information. Within the OECD countries, countries mostly use ex-post evaluations, while good practice is to also have ex-ante evaluation. In practice, ex-ante evaluations are done for capital budgeting only, while they are rare for other types of expenditures.

For question 45 and 51, respondents should round up their answers to the closes range shown (e.g. round up to 80% if the answer is 75%)

For question 47, respondents should provide answers even if their country does not have evaluations, from the perspective of what are the reasons for lack of evaluations.

For questions 50-60 (survey section on Spending Reviews), what is meant by spending review is a review that is more systematic than just ongoing reviews done on regular basis by the Ministry of Finance. Rather than only providing information of what spending should be cut, spending reviews look more analytically on what is going on in that specific sector. In addition, spending reviews involve independent external experts and spending reviews represent a structured attempt to examine baseline spending and identify savings.

In general, within the boxes available for comments at the end of each survey section, respondents are asked to write comments and explanations on any of the questions in that sections for which comments space is not available.

It was agreed that the final survey responses are submitted by 1 September, 2016 and OECD will present the results, including benchmarking of practices with OECD countries who completed the survey in the first half of 2017, at the next annual plenary meeting of all BCOP members planned for Kyrgyz Republic in April 2017.

The workshop was closed with the presentation by the Working Group’s Leader Nikolay Begchin on the future planning of the new Working Group on Program Budgeting. He presented the following objectives of the Working Group:

  1. to identify key trends in program budgeting implementation and spending reviews.
  2. to learn from specific PEMPAL and international country examples in these areas.
  3. to participate in OECD survey of performance budgeting to provide baseline data on status of reforms and to identify good practices.

Finally, he presented the tentative Draft Working Group Action plan. The Action Plan, as updated based on the discussion of the Executive Committee after the workshop is as follows:

1. / Participating in the OECD performance budgeting survey / March 2016 – February 2017
2. / Working Group Meeting in the context of the 12th CESEE SBO meeting in Slovenia / June, 2016
3. / Program Budgeting Working Group meeting to discuss selected advance country cases + back to back meeting with the OECD Performance Network / November, 2016
4. / Russia’s experience in implementing program budgeting (Video conference VC) / Spring, 2017
5. / Plenary discussions on trends and challenges of program and performance budgeting across PEMPAL countries identified in the course of the OECD survey and the presentation of the findings of the World Bank survey of countries implementing program budgeting / April, 2017, as part of the BCOP plenary meeting

ANNEX: Workshop on the 2016 OECD Performance Budgeting Survey – Agenda

Date: Monday, June 27, 2016

Time: 09:30-12.30

Place: Ljubljana, Slovenia

Venue: Best Western Slon Hotel

Time / Activity
09.30-09.45 / Opening by
  • OECD
  • PEMPAL

09.45-10:15 / Presentation by OECD of the survey structure, objectives and explanation of concepts and terminology. Common issues and questions with data interpretation
10.15-10.30 / Coffee
10:30-12:00 / Questions and Answers
Tour de table
  • Participants should come prepared with any questions and issues they have experienced from entering their preliminary responses to the online survey

12.00-12.30 / Future planning of the new Working Group on Program Budgeting – Nikolay Begchin, Ministry of Finance of the Russian Federation
12:30-14:00 / Lunch
14:00-16:00 / BCOP Executive Committee meeting (Committee members only – observers welcome)