Training Program on
Social Policy Reform in Transition Economies
(SPRITE)
Status Report No. 3
April 2000
______
The World Bank Institute
Universalia
WBI Evaluation Studies
Number ES00-50
World Bank Institute
The World Bank
Washington, D.C.
Copyright © 2000
The International Bank for Reconstruction
and Development/The World Bank
1818 H Street, N.W.
Washington, D.C. 20433, U.S.A.
The World Bank enjoys copyright under protocol 2 of the Universal Copyright Convention. This material may nonetheless be copied for research, educational, or scholarly purposes only in the member countries of The World Bank. Material in this series is subject to revision. The findings, interpretations, and conclusions expressed in this document are entirely those of the author(s) and should not be attributed in any manner to The World Bank, to its affiliated organizations, or the members of its Board of Directors or the countries they represent. If this is reproduced or translated, WBI would appreciate a copy.
April 2000 SPRITE
Executive Summary
Introduction
In February 1999, the World Bank Institute (WBI) of the World Bank - formerly the Economic Development Institute (EDI) - contracted Universalia to monitor the Social Policy Reform in Transition Economies (SPRITE) Training Program from July 1998 to 2000. The goal of the monitoring project is to help ensure that the objectives of the program remain on track and the desired outputs, outcomes and impacts are achieved.
Monitoring is important for it is a process that continually asks the questions[1]: “Where do we stand?" “What are we moving towards?” and “What corrections may be necessary to achieve our goals?” While one could focus on many different aspects of the program cycle, the three broad issues at the core of this assignment are[2]:
· The extent to which the rationale and logic of the SPRITE program remains sound and sensible;
· The degree to which the SPRITE program achieves its objectives at the individual, organizational and policy levels; and
· The extent to which SPRITE has managed for results, in terms of development and management, at the individual, organizational and policy level.[3]
This is the second annual report produced by Universalia and it covers the period: April 1, 1999 to March 31, 2000. The report is organized into five sections. Section 1 reviews the methodology used. Section 2 describes the rationale and logic for the program. Section 3 presents the main objectives, the current monitoring framework, the data on the outputs, outcomes and impacts of the various activities, and discusses limitations. Section 4 discusses the program results, and Section 5 concludes the report with recommendations for future directions.
Methodology
The report is based on primary sources of qualitative and quantitative data. A major data source for this reporting period was an electronic status report (Appendix I) – a report the monitors designed with the WBI to garner a stronger understanding of the outcomes and impacts of the SPRITE training. The report was forwarded electronically to the country team leaders to obtain their feedback and to test the viability of using the Internet as a primary means for communicating. The period of reference for the 1999 status report was aligned with the World Bank’s fiscal year July 1 - June 30, rather than the year-end of major donors such as CIDA (April 1 – March 31).
Nine of the 13 countries submitted their report on time (September 30, 1999) and this data was analyzed and included in the November 1999 Monitoring Report. The missing reports - from Kazakhstan, the Kyrgyz Republic, Mongolia and Uzbekistan - were eventually received by the SPRITE coordinator by the end of November 1999, and this report reflects the information from all reports. Appendix II contains a consolidated view of the status reports.
A second, valuable data source was obtained from 16, one to two hour in-depth interviews, with SPRITE team managers or a team designate, and with the Director of SPRITE and the Team Leader from the World Bank (WB) who were attending the Labor Markets in Transition Economies workshop held October 10 to 13, 1999 in Dublin, Ireland. Twelve interviews were conducted using two Russian interpreters, three were conducted in English, and one was a bilingual English/French interview.
A third data source was telephone communications, email and faxed correspondence with the WBI SPRITE Management Team in the WB headquarters in Washington and in the WB Moscow office, which acts a focal point for the neighboring countries.
Finally, the bulk of the quantitative data was derived from the SPS and Ireland participant questionnaires, the SPS and Ireland participant registration forms, and the SPS team leader summary reports.
Monitoring Results for April 1998 - March 2000
Sample of Outputs during the April 1999 - March 2000 Period
· 45 seminars planned, 34 delivered, 11 cancelled
· 12 out of 13 countries offered SPS
· 34 seminar reports, however, not all were received by the monitors
· 1-10 day sector workshop and study tour on labor market reforms held in Ireland with 28 delegates from 12 countries.
· 1 coordination meeting organized by the World Bank, held in Kiev, Ukraine.
· Vast array of print training materials and training tools made available to participants in the SPS’s and sector workshop
· Publishing of academic articles, books, and newspaper articles (see complete list in the Status Report Summary)
The results gathered during this reporting period indicate that the program is moving forward in meeting its objectives and delivering training, but that the pace has decreased a little. Data indicates that the number of planned to actual SPS seminars dropped 18 % during this reporting period (from 94% in 1998-99 to 76% in 1999-2000). Nonetheless, self-report data shows that SPS has contributed to triggering initiatives that impacted legislative processes such as preparing policy statements, drafting legislation and amending laws. Most importantly, the program has reportedly now achieved ‘brand recognition’ and the programs offered by SPRITE are deemed to be valid, credible, and current. This is illustrated by: 1) the demand for SPRITE material and requests for further research and training by a growing number of individuals that, in turn, disseminate their knowledge within their organization; and 2) the fact that teams are able to raise funds to co-sponsor SPRITE training. The monitors are unable to report on cost-effectiveness of the program, as this is not part of the monitoring mandate.
The majority of the outcome and impact data collected thus far is derived from self-reports submitted by the country teams. The interviews conducted by the monitoring team in Ireland (October 1999) provided the first opportunity to validate the self-reports. Insights gathered during interviews generally substantiated the information the monitors were receiving and shed light on several factors affecting the success of the program including the program having name recognition, the use of training materials in multiple outlets, and the ability for training people to write proposals for policy change and draft legislative documents. While early evidence is positive, further validation, with a broad range of stakeholders, is required to determine the full impact of SPRITE. To this end, a proposal for an impact study for fiscal year 2000/01 will be forwarded to WBI according to the dates stated in section 5.1.
Sample of Outcomes at the Individual, Organizational and Policy Levels during the April 1999-March 2000 Period
· Participants have increased their knowledge and understanding of reform issues in seven topic areas
· Contacts and networking opportunities with colleagues in other institutions were strengthened
· Evidence that organization’s support for SPRITE appears to be a reality for 27% of the seminars attendees are return participants, thus indicating the employer found value in sending their employee(s) to the SPS seminars or workshops
· Participants report sharing, in their workplace, the knowledge, tools and technologies obtained from the SPRITE training to assist colleagues, staff, students, and decision-makers
· SPRITE participants report they use their new knowledge to push for changes, write draft legislation, conduct research and help change at varying levels of the government and the administration
· Senior government officials can draw on the knowledge they gain or their employees can to improve policies and laws
Thus far, SPRITE had the greatest impact at the individual level that is consistent with the program’s underlying logic. Because the program targets people of influence and decision-making authority, there is mounting evidence of the social impact of SPRITE in terms of people reporting that their SPRITE training gave them tools to work in ways that educate others and influence the way people think. The impact at the organization level, on the other hand, is always more difficult to measure. However, as emphasized by respondents during the interviews, that organizations support and pay for participants to send their employees to SPRITE training and that SPRITE material is in demand, is a strong indication of the interest in the program. At the societal level, the fact that a number of countries report that SPRITE influenced the legislative change process is a clear indicator of the potential for the training delivered by SPRITE to impact social changes, policies and processes.
Sample of Impacts at the Organizational and Policy Levels during the April 1999 - March 2000 Period
· SPRITE participants extend their reach into their work environment, use their knowledge to develop draft discussions or legislations, train others in new ideas, and encourage and facilitate dialogue.
· Organizations turn on to SPRITE for training and the quality of its educational material.
· Organizations express interest in continuing to support training and some are engaged in cosponsoring activities with the World Bank for the organization of SPRITE seminars and other activities.
· Interviews show that the program has passed the stage of brand recognition and is building credibility at the national level.
· The great majority of SPRITE team managers estimate that seminars affect policy discussions and activities in the beneficiary countries. This includes the organization of parliamentarian committees to discuss social policy reforms.
· Yearly status reports and interviews show that the drafting or reform of new laws, bills and policies as well as their implementation are taking place in several FSU countries.
One of the positive factors of SPRITE consistently mentioned by people interviewed was that the social policy seminars create a neutral, open, critically needed forum for people from various sectors to discuss issues and share ideas. As more than one person stated, a large part of social policy change is influencing the way people think, and SPRITE encourages a form of intellectual exchange among people who might otherwise never have the opportunity to speak or learn with each other. Another key success factor of the program was recruiting high-level government officials to be on the SPRITE team – this facilitates more rapid action and support in the government.
Yet, the ability for SPRITE to build a sustainable capacity for social policy training within the beneficiary countries will remain a challenge. Teams are aware of their responsibilities to develop their capacity for local co-sponsoring, but most highlighted the difficulty in moving in that direction. Although 50% local–50% donors co-funding is already happening in countries such as Russia and the Kyrgyz Republics, the 30% WB–70% local is viewed as difficult to reach. The prospect for increasing local participation without decreasing the amount of activities may appear unrealistic, especially in countries suffering political and economic instability.
Conclusion
The program has existed for just over six years and at this stage in monitoring, there is sufficient evidence to demonstrate that SPRITE is having a positive impact in the participating countries. Based on the data collected during two years of monitoring, Universalia can confidently state that the SPRITE rational, its underlying assumptions, and overall logic remain sound. To varying degrees, the program objectives are met in each of the beneficiary countries and he influence of the social policy training is seen at the organizational, individual and societal levels, most significantly the latter two. The in-depth interviews conducted in Ireland where representatives from 12 of the 13 countries were gathered with colleagues to participate in a training of the trainer’s workshop and study tour, provided the opportunity to challenge the self-report data and, in general, the latter appears to be validated. The real validation, however, will be realized during the impact study that is tentatively scheduled for this year.
Acronyms
CIDA / The Canadian International Development Agency
FSU / Former Soviet Union
LFA / Logical Framework of Analysis
NGO / Non Governmental Organization
SEA / Skill Enhancement Activities
SPKNI / The Social Policy Knowledge Network Initiative
SPRITE / The Social Policy Reform in Transition Economies
SPS / Social Policy Seminars
TOT / Training of Trainers
WB / The World Bank
WBI / The World Bank Institute
WBIES / The World Bank Institute Evaluation and Scholarship Unit
Contents
1. Introduction 1
1.1 Methodology 1
2. Rational and Logic of Sprite 3
3. The SPRITE Training Program and Objectives 5
3.1 SPRITE Objectives 5
3.2 SPKNI – Social Policy Network Initiative 6
3.3 The Monitoring Framework 7
4. Program Activity Results 9
4.1 Outputs 9
4.1.1 Social Policy Seminars 9
4.1.2 Skill Enhancement Activities 13
4.1.3 Publications, Dissemination and Outreach 14
4.2 Outcomes 14
4.2.1 Social Policy Seminars 15
4.2.2 Skill Enhancement Activities 16
4.2.3 Publications, Dissemination, Research and Outreach 18
4.3 Impacts 19
5. Managing for Results: Suggestions for Future Directions 29
5.1 Next Steps 31
Exhibits
Exhibit 3.1 Topic Areas for SPRITE Training 6
Exhibit 3.2 The Monitoring Framework 8
Exhibit 4.1 Social Policy Topic Areas: Planned Offerings to Actual Delivered 9
Exhibit 4.2 Social Policy Seminars: 01 April 1999 to 31 March 2000 10
Exhibit 4.3 Aggregate Age of SPS Participants 12
Exhibit 4.4 SPS Participants per Country 13
Exhibit 4.5 Length of Social Policy Seminar 13
Exhibit 4.6 New Information 15
Exhibit 4.7 Feedback on Sector Workshop and Study Tour on Labor Markets in Transition Economies 16
Exhibit 4.8 Outcomes 19
Exhibit 4.9 Social Policy Seminars 20