Evaluation of 2 projects implemented by Umanotera and its’ partners
in Slovenia with the CEE Trust financial support
EVALUATION REPORT
Content:
I. Methodology
II. Background information on the evaluated projects
III. Evaluation findings
IV. Conclusions and recommendations
V. Annexes
March 2007
Lidia Kuczmierowska LK Konsulting
e-mail:
I. Methodology
The methodology of data collection was developed on a basis of the Evaluation Project submitted to the Trust for Civil Society in January 2007. It embraces a combination of documents’ studies, in-depth individual and group interviews. Due to the structure of the program, and consequently - to the size and structure of a researched population, the evaluation tends more towards a qualitative survey.
1. Analysis of the secondary source data
The first source of information about evaluated projects was various documents that may be grouped into the following categories:
-proposals submitted to the Trust,
-interim reports on the projects’ progress,
-reports on the overall situation of civil society and in particular the NGO sector in Slovenia,
-correspondence between the Trust and Slovenian organizations, in this Umanotera, setting main terms and conditions of the projects supported by the Trust.
There were also visited websites of the program Dobra Druzba, Umanotera, Dodogovor and other key implementing organizations.
The list of the main analyzed documents includes:
-Proposal of Programme for Slovenia 2004-2006 (the first proposal submitted by a consortium),
-Strategic Programme for Strengthening of Civil Society in Slovenia. Outline of a Three Year Plan for Implementation of the Trust in Slovenia (submitted by Umanotera in June 2004),
-Quality System for Slovenian NGOs (the proposal submitted by the Center for Non-profit Management and Legal Information Center PIC),
-2nd Strategic Programme for Strengthening of Civil Society in Slovenia. Outline of a Two Year Plan for Implementation of the Trust in Slovenia (submitted by Umanotera in March 2005 – rejected),
-3 quarterly reports (2004) submitted by Umanotera,
-Annual Report of Programme for Strengthening of Civil Society in Slovenia for 2004 (submitted by Umanotera in March 2005),
-Financial Report of Programme for Strengthening of Civil Society in Slovenia for 2004 (submitted by Umanotera in January 2005),
-Mid-term Report to Trust for Civil Society in CEE. Period January – June 2005 (submitted in July 2005),
-Programme for Strengthening of Civil Society in Slovenia. Annual Report for 2005 to the Trust for Civil Society in CEE (submitted in March 2006),
-Programme for Strengthening of Civil Society in Slovenia for 2005. Financial Report to the Trust for Civil Society in CEE (submitted in January 2006),
-Mid-Term Report of Programme for Strengthening of Civil Society in Slovenia. Period January-June 2006 (submitted in September 2006),
-Programme for Strengthening of Civil Society in Slovenia for 2006. Financial Report to the Trust for Civil Society in CEE (submitted in February 2007),
-Correspondence between the Trust and Slovenian NGOs regarding submitted proposals and selection of the Trust partner from the period 2003-2004,
-National Report: Third Sector in Slovenia, S. Hvalic, J. Ramovs, K. Ramovs (2002),
-NGO Sustainability Index (Reports for Slovenia 2003, 2004, 2005),
-Slovenia, report by Brian J. Pozun (2004),
-After 15 Years of Independence and a New Regime (Civil Society Index for Slovenia), M. Verbajs, PIC (January 2006),
-Report. Analysis of Donors for the Slovene NGO Sector, Umanotera (2006).
2. Individual and group interviews
The interactive part of evaluation began with an information meeting with the program grantees. Its’ main purpose was to inform all future participants of the evaluation about the scope and methodology of evaluation process. 17 people participated in the meeting.
Interviews with key stakeholders of the program were conducted during two visits to Slovenia (1st visit on February 28 - March 2, 2007; 2nd visit on March 11 – March 15, 2007). Combinations of evaluation questions asked during interviews depended on the role and the level of engagement of the interviewed. For each interview (or a group of interviews) a separate list of questions was developed, however the interview scenarios were often modified in order to make them more appropriate to a particular context of a meeting. At the beginning of the first visit there was held an orientation meeting with the Program Manager, Vida Ogorelic-Wagner aimed at complementing and clarifying information about evaluated projects and their evolution in time.
During two evaluation visits to Slovenia19 following interviews were carried on:
- 2 in-depth interviews with Vida Ogorelic Wagner, the director of Umanotera (the Program Manager),
- 1 interview with Gaja Brecelj, Umanotera(the Program Assistant) – a review of grant giving schemes’ documents,
- 1 interview with Alena Lipavec (the former Program Manager),
- 1 group interview with representatives of the Advisory Board,
- 12 individual (or in pairs) interviews with the program’s grantees (in this 2 outside of Ljubljana), altogether 60% of grantees were met and interviewed (the list of interviewed grantees is attached in the annex 1),
- 1 interview with Igor Medjugorac, Stereo, PR and communication consultant to Umanotera,
- 1 interview with Irma Meznaric, Ministry of Public Administration.
One long informal interview was also held with Tomaz Bole from ZUS (Association of Foundations) on the way to two community foundations that were met during the second evaluation visit to Slovenia. It provided a lot of valuable insights into the program and overall past and present situation of non-profit organizations in Slovenia.
In addition, at the early stage of a data collecting process there was conducted an interview with Jacek Wojnarowski, the former Executive Director of the Trust.
3. Stakeholders’ survey
Due to a differentiated character of various stakeholders’ involvement in the program three versions of questionnaires were developed. Questionnaires were sent to 29 persons, out of which 9 responded. A low response rate and – what is more important – too small sizes of survey groupsto use in a reasonable way quantitative research methods made this component of evaluation of a little use and importance. The questionnaire survey was anonymous.
II. Background information on the evaluated projects
The subject of evaluation were 2 projects implemented in Sloveniaby the Slovenian Foundation for Sustainable Development Umanotera with the Trust for Civil Society financial support:
1)Overall Program Management and NGOs in Elections 2004 (SLO-01-04/amount of the grant: 244,200 USD/duration: April 1, 2004 – March 31, 2007 – extended to the end of April 2007);
2)Strategic Program of Strengthening of Civil Society in Slovenia (SLO-02-04/amount of the grant: 680,000 USD/duration: September 1, 2004 – March 31, 2007 – extended to end of April 2007).
The projects were carried on within the second stage of the Trust involvement in Slovenia, after theimplementation of four smaller projects in the period 2002-2003. Following a half-year preparation process and several rounds of consultations with a group of Slovenian organizations, the Trust decided to establish a partnership with Umanotera, a foundation set up in 1994 and active in the field of sustainable development
Although formally Umanotera has been implementing 2 projects under two separate grant agreements, in practice they form one program named “Dobra Druzba” (good society, good company).
The structure of the program Dobra Druzba responds to three strategic goals of the Trust for Civil Society in CEE:
-to contribute to a supportive legal, fiscal and political environment in which civil society can flourish;
-to strengthen the non-profit sector through capacity building;
-to enhance the long-term financial sustainability of non-profit organizations.
In each of these three pillars various activities were planned. The main program line and objectives have been maintained, although some of components evolved in time and finally took a different shape than it was planned in the proposal. All changes – prior to their implementation - were accepted by the Trust.
The current program structure is presented in a diagram on the next page.
1
Program Dobra Druzba 2004-2007
Objective 1:Supportive Environment for Civil Society
Program components:
- NGOs in Elections 2004
- Civil Dialogue
(support to the Initiative)
- Small Grants Scheme
- Promotion of NGO Success Stories
- Mirror to the Government
CapacityBuilding to Strengthen the Sector
Program components:
- Mechanism for supporting NGO participation in EU networks and federations (small grant schemes)
- NGO portal (virtual handbooks)
- Quality Schemes for NGOs
- Handbook and seminars on governance
Financial Sustainability
Program components:
- Developing Local Financial Sources and Promotion of Philanthropy
- Capacity building of community foundations
- Grants for community foundations
Overall Program Management: on-going management, incl. financial management, grantmaking, PR and information dissemination, supervision of program components implemented by partner organizations, cooperation with the Advisory Board
1
All program components are briefly presented below.
Objective 1: Supportive Environment for Civil Society
NGOs in Elections 2004
Project aimed at raising awareness of politicians regarding civil society issues in the pre-election time. One of the first resultswas a report presenting an overview of the current situation of the non-profit sector. Thenin June 2004 a questionnaireexamining the level of inclusion of civil society issues into political programs was sent to 38 parties, out of which 11 responded. The meeting with all parties’ representatives planned for August did not take place, because none of parties responded to the invitation. Finally, 8 separate meetings with parties representatives were held and as Umanotera described them in the annual report, they “were more of presentation nature and no conclusions could be drawn”. Work with media did not bring expected results neither – although information packages were sent to all major media, only two articles based on this material were published.
Mirror to the Government
Originally planned as a second stage of the NGOs in Elections, the project aimed atmonitoring the government’s commitments towards the third sector declared during the election campaign. Since such commitments were not formulated, the initialideawas developed intothe concept of monitoring public participation in drafting of legislation and policy documents. The direct stimulus for such redefinition of the project’s goal was an amendment to the Governmental Procedural Act, adopted in March 2006 that put on various public bodies an obligation to report on consultation processes. Set of indicators was developed and more than 40 drafts of law were analyzed in this regard. Currently the final report is being prepared.
Budget allocation for NGOs in Elections and Mirror to the Government: 74,200 USD *).
Support to the Initiative for the Future of NGOs
The Initiative (Pobuda)is a group of representatives of various organizations that has been working on the improvement of relations between NGOs and the government since 2002. Within the project two forms of support were offered to the Initiative: technical assistance (which in fact was rather an administrative assistance) and a small grants fund for expert assistance. The Initiative’s task was to develop a proposal of an agreement that would have regulatedcooperation between NGOs and the government and an action plan to implement the agreement. Consulted with a wider NGO community, the agreement would have been negotiated with the government by a special group chosen from the Initiative’s members. Additionally, the Initiative planned its’ formalization (defining terms of membership, communication and decision making procedures, etc.). The document was prepared in the fall of 2004. It covered 4 key issues: civil dialogue, legal environment, financing of NGOs, employment in NGOs. In one of the reports it was defined as “extremely ambitious”. Four consultation meetings with NGOs organized in different regions of Slovenia were poorly attended, so the factual contribution of organizations to the agreement was much beyond expectations. In order to disseminate information about the work of the Initiative, the negotiation group decided to set up a special web page. Opinions and comments presented on the web page for many seemed to be controversial, but the group decided it was a proper approach at that moment. In April 2005 the negotiation group was invited by the Ministry of Public Administration. The meeting was a failure: the Ministry questioned the transparency and legitimacy of consultation and negotiation processes, while NGOs’ representatives in the protest to not honoring the previous agreements (made by the previous government) left the meeting. Following the collapse of the negotiation a serious internal problems became more visible (the leadership changed twice, many members left). The Advisory Board of the program Dobra Druzba decided that due to the weakness of the Initiative other ways of supporting civil dialogue should be applied in 2006.
Support to Civil Dialogue
Within the new project aiming at supporting civil dialogue 3 events were organized:
-a meeting of key NGO representatives focused on an assessment of the current situation in this regard,
-a meeting concentrated on analyzing possible NGOs’ involvement in the preparation of the National Development Plan,
-a conference “Let Us Use the Potential of NGOs” organized in cooperation with the Ministry of Public Administration, with Jeremy Rifkin as a keynote speaker; at the occasion of the conference a special newspaper presenting the Slovenian non-profit sector was published.
Small Grants Scheme
With the purpose of supporting NGOs’ involvement in consultations of the National Development Plan, as well as other strategic documents and a tax reform, Umanotera organized a small grants scheme. The first round that took place in February 2006 did not bring good quality projects – only one grant was awarded. The second round organized 3 months later was more successful (partially thanks to a workshop on grant applications writing that was delivered to potential grantees prior to the opening of the call for proposals) – 8 grants to 7 organizations were awarded.
Budget allocation for all components related to Civil Dialogue: 138,442 USD*).
Promotion of NGO Success Stories
The main idea of the project was to promote the non-profit sector through publicizing the best cases of NGO activity, identified within an open competition. In the first edition, held in 2005, organizations submitted 35 proposals of “success stories”, 6 of them were selected and awarded prize money or otherawards sponsored by companies. The second competition was organized in the beginning of 2006 – this time 39 applications were received and again 6 best examples were awarded. In both editions award ceremonies were held at the Ljubljana castle, and the awarded success stories were presented in consequent issues of “ONA”, a supplement to one of the most popular newspaper “Delo”.
Budget allocation: 93,471 USD*).
Objective 2: CapacityBuilding to Strengthen the Sector
Initially, the capacity building component was to be based on a project supported through a grant received from the Leonardo da Vinci program. However, a proposal submitted to Leonardo was in 2005 rejected and thus the original concept of planned activities had to be significantly changed.
Handbook and seminars on governance
In the beginning of 2006 a Slovene translation of a handbook on NGO governance by Marilyn Wyatt was published and three 2-day seminars on this topic for NGO leaders were organized.
Mechanism for supporting NGO participation in EU networks and federations (small grants scheme)
The main component of the project were two financial mechanisms (grants and short term loans) supporting NGOs’ participation in various EU networks and federations and in international events. The level of interest was very low – by mid of 2006 only 2 grants and 3 loans were awarded.
NGO portal (virtual handbooks)
The main result of this program module will be 3 on-line handbooks addressing employment, accounting and registration aspects of NGO activity. The original idea of producing handbooks has been developed, so the final output will be a portal for NGOs with other relevant information. The project will be completed by the end of March.
Quality Schemes for NGOs
This is the newest component of the program (the proposal was submitted in fall 2006) that addresses the issue of quality in NGOs. A special quality system for NGOs will be developed, then after a pilot phase and all necessary adjustments, the new model of setting and assessing quality standards for NGOs will be promoted further. The project is rather in its initial stage.
Total budget allocation for CapacityBuilding component: 42,500 USD*).
Objective 3: Financial Sustainability
Developing Local Financial Sources and Promotion of Philanthropy
The main goal of this project was twofold: to introduce new, transparent and effective grant giving mechanism that would support local NGOs and to promote philanthropy. Planned activities included: an analysis of various funding sources and mechanisms available in Slovenia and other countries, selecting within an open competition procedure candidates for community foundations, setting up community foundations, providing them financial and non-financial (through capacity building) support.
In the beginning of 2005 a consortium of 6 organizations responsible for the implementation of the community foundation component was formed. Few months later they visited Ekopolis in Slovakia to get knowledge and better understanding of functioning of community foundations; there was also organized another study trip to EuropeanFoundationCenter in Brussels. In addition one of the consortium members prepared a study on setting up community foundations in Slovenia. In September 2005 there were organized eight regional workshops presenting an idea of community foundations and a conference in which Slovak speakers participated.
As a result of the first call for proposal eight organizations as candidates for community foundations were selected. In December they participated in a study trip to Slovakia, where they visited four community foundations and Ekopolis. Next month they were offered a training program on strategic planning, project management, promotion and fundraising. In 2006 the consortium organized a second call for proposals to choose these organizations out of eight that would be receiving further support as community foundations (although the initial plan was to establish and support 8-10 foundations). Four of them were selected and received start-up grants in the amount of $15,000. In 2007 they received also challenge grants on a proportional matching basis.
It is important to highlight here that the model of a community foundation applied in the program does not put on foundations a requirement to build their own endowments. The maintenance of a grant giving role is secured by a condition (written in foundations’ by-laws) that 70% of their revenue must be allocated for grants distribution.