Evaluation report
"Building Environmental Citizenship to Support Transboundary Pollution Reduction in the Danube:
A Pilot Project in Hungary and Slovenia".
March 31, 2002
Pavol Zilincik
Content:
Introduction
Project Concept and Design
Project Implementation
Project Results
Findings and Lessons Learned
Recommendations
Attachments: Terms of Reference
List of Documentation Reviewed
List of Persons Interviewed
Agenda of the Evaluation Mission
Acronyms and Terms
CEE Central and Eastern Europe
CBI Confidential Business Information
EU European Union
GEF Global Environment Facility
ICPDR International Commission for the Protection of the Danube River
MoE Ministry of Environment
NGO Non-Governmental Organization
NYU New York University School of Law
PIT Project Implementation Team
REC Regional Environmental Center for Central and Eastern Europe
RFF Resources for the Future
SAP Strategic Action Program
UNDP United Nations Development Programme
Introduction
This is the final evaluation of the Project "Building Environmental Citizenship to Support Transboundary Pollution Reduction in the Danube: A Pilot Project in Hungary and Slovenia". The objective of this Project was to enable Hungary and Slovenia to operationalize and institutionalize public access to environmental information and public participation measures in support of reducing transboundary pollution from the discharge of nutrients and toxins into the Danube River. At the "global" environmental level, the Project was to demonstrate how these measures can help Central and Eastern European countries to achieve the important global environmental goals of the Danube SAP, the Aarhus Convention, and the Danube River Protection Convention signed by Hungary, Slovenia and other countries of the Danube River basin. The activities conducted in Hungary and Slovenia were to serve as a pilot for future efforts to operationalize public involvement in support of pollution reduction in the Danube in other countries in the Danube River basin. The project was designed for the time period of February 15, 2000 – August 15, 2001, later extended until March 31, 2002.
The project was designed and implemented jointly by
- the Regional Environmental Center for Central and Eastern Europe, based in Szentendre, Hungary (REC),
- New York University School of Law (NYU), in New York, and
- Resources for the Future (RFF), in Washington, D.C.
The three organizations cooperated in the project development and execution as co-equal partners in Project Implementation Team. They agreed to select the REC for the role of Executing Agency. RFF and NYU worked, technically, as subcontractors fulfilling also vast majority of technical assistance tasks in the project.
Country experts including Environmental Law Experts from Hungary and Slovenia and an EU Law Expert were selected to serve as consultants to the Project and ensure proper consideration of the national legislation as well as EU legislation during the course of the project.
This evaluation is based on review of all relevant Project documents and materials produced; interviews with Project leaders and key Project participants; and meetings with GEF and UNDP staff concerned with the Project. From February 4, 2002, until March 1, 2002 the evaluator conducted a mission to Hungary and Slovenia to interview REC, Hungarian and Slovenian environmental law expert consultants to the Project, Project participants and beneficiaries. From March 4-8, 2002, the evaluator visited NYU and RFF and discussed Project details with UNDP and GEF SEC staff involved in the Project.
With the invaluable assistance of REC, the evaluator prepared the agenda for individual missions. The criteria guiding the selection of interviewees were 1) level of their involvement/participation in the Project, 2) relevance of their position to the Project evaluation, 3) professional as well as cross-sectoral diversity. During the missions in Slovenia, Hungary and the USA 29 interviews have been conducted.
Apart from interviews with persons relevant to the Project, the evaluation report is grounded in evaluation of the Project related documents, including preparatory papers, minutes and agendas of the workshops, numerous communication between members of Project implementation team, consultations with country experts, project outputs including publications produced, project reports, study tour materials, technical assistance papers, case studies, etc.
Based on these materials and interviews, the evaluator prepared this final evaluation report which, in 6 sections, assesses the relevance, efficiency, effectiveness, impact and sustainability of the project. The first section evaluates the concept and design of the project as were planned. Project implementation follows including evaluation of concrete activities, their strengths and obstacles faced. The description of results both specific and general can be found in the fourth section followed by findings and lessons learned. The report ends by providing recommendations for future projects.
Project Concept and Design
Description of the Problem
The Pilot Project intended to help reduce the discharge of toxics and nutrients into the Danube through establishing conditions for better public access to environmental, especially water-related information. The Project aimed to determine the main barriers and obstacles in access to information and to assist in building necessary capacities in Hungary and Slovenia for adequate legal, institutional, social and practical infrastructure. The Project reflected how countries with developed access to information mechanisms, that enhance public access to environmental information, promote through avariety of mechanisms, more effective environmental protection.
The Project illustrates that the various causes and effects of barriers in public access to environmental information differ in both pilot countries. The primary problem in Slovenia is inadequate legislation. Incomplete and vague legal provisions, lack of implementing regulations as well as poor institutional arrangements for providing public information are key barriers of public access to basic environmental information in Slovenia. For example, access to an already publicized EIA announcement is denied and emission data on water discharges are entirely inaccessible. According to interviewees, denial of request for information are first met with statements that it is unclear whether the respective information is public or not. If an applicant proves that the information should be public, barriers arise in that it is not clear under what procedure the agency should deal with request. Lack of "legal interest" or broadly interpreted “business confidentiality” often prevent access to important environmental information, without an effective remedy.
In the case of Hungary, the problem rests more in the area of practical arrangements and implementation of relevant legislation. Lack of knowledge about applicable procedural and substantive provisions of laws often causes problems in access to information. As described in the needs assessment, "... people have to know the sources [for information] well, they have to make some special efforts to cope with the bureaucracy and those have the best chances of gaining access to information who have some direct personal connections to the authorities possessing the given information". The current legislative environment further obstructs this situation, since laws enacted in the 60’s and 70’s are incompatible with laws passed in the 90’s. Mutually conflicting legal provisions once again result in barriers in access to information important for effective public participation and environmental protection.
Description and evaluation of the Project concept and design
Design of the Project assumed assessment of needs and identification of barriers described in the Needs Assessment. Consecutive Capacity building workshops (with practical case studies), technical assistance, a study tour for key officials and NGO representatives, and the drafting of relevant legislation and policy recommendations were activities expected to enable the pilot countries to operationalize and institutionalize public access to environmental information. From this, replicable elements were to be identified and together with project materials and lessons learned would become background materials for replicability of the Project in other countries. This would ultimately enhance public access to environmental information in other Danube basin countries, and ultimately decrease pollution of the Danube River.
It is important to evaluate whether the concept and design of the Project as set were able to achieve the goal "to enable Hungary and Slovenia to operationalize and institutionalize public access to environmental and water related information". With this in mind, the aim of the Project appears rather ambitious. It is the opinion of evaluator that to achieve this aim a substantial segment of relevant public officials in Hungary and Slovenia would have to be trained, or at least a comprehensive system of capacity building for this target group would have to be outlined. Additionally, in the case of Slovenia, a legal framework for access to information should be formed. These tasks are obviously outside of the scope of this project and require amuch longer time period with significantly different demands on capacities.
The Project design can be considered as agood initial phase of "enabling" the pilot countries to make the flow of governmental information to the public easier.
As described below, main obstacles with implementation of the Project appeared in Slovenia. Several interviewees pointed out that it would have been appropriate to involve more high-level partners from the Slovenian government in earlier phases of Project design. It is not clear exactly how the Project design would look like under such circumstances, but their earlier involvement might have been beneficial to the Project implementation.
Evaluation of the design of Project activities
Needs assessment - Identification of barriers
A detailed analysis of needs and barriers at the beginning of the Project was undoubtedly a beneficial initial step. Moreover, the needs assessment phase was conducted in a very professional manner. REC contacted excellent local experts, their needs assessment was grounded, in-depth analysis of legislation and extensive research of the practical functioning of information flows. The draft Needs Assessment was extensively consulted with all project partners (REC, NYU and RFF) as well as key stakeholders in the respective countries, which finally resulted in very systematic and target oriented recommendations for Project implementation.
It would be interesting to compare the evaluated Project with a project in which the design of the project is based on an already completed needs assessment.
Perhaps some interesting ideas from the Hungarian needs assessment (i.e. the "local initiatives priority"), which were not addressed in the Project due to an already designed structure, might have been tested, since these ideas are fully compatible with the GEF mission of testing new problem solving approaches.
Scheduling the Needs assessment before the Project implementation phase rather than before the Project designing phase didn’t have fundamentally negative effect to the Project for the following reasons:
- high quality of the Needs assessment provided an excellent base for implementation activities
- Project implementation basically reflected needs assessment and responded to the most important identified conclusions
- Project implementation was flexible and adequately responded to modifications
Case studies and Workshops
The structure of three joint regional workshops was changed at the outset of the implementation phase to two regional workshops and two sets of country workshops in Hungary and Slovenia. Participants of the kick off meeting decided that this structure would reflect specific in-country issues better and promoted changing the workshop structure. The evaluator also considers this change to be very beneficial to the Project.
The design of the workshops based on practical examples, with active involvement of the key stakeholders, is clearly an excellent technique for achieving the goals defined in the Project. High quality workshops, based on the specific needs of participants, lead to a number of benefits; both direct (discussion on controversial issues, joint outline of suitable outcomes/solutions) and indirect (intensified cross-sectoral communication and confidence). However, interviewees have repeatedly stressed further education, targeted to countryside and regional offices as an important need. Thus it may be concluded that this area of focus provides broad opportunities for follow up activities, both in the pilot countries as well as in other countries of the region. Future projects, should also comprise a model system in their design for meeting this need, clearly furthering project goals.
Technical assistance
Observing the experience of the countries with well developed, mature systems of access to information, where the "growing pains" have been completed, is very important. Such an example provides motivation for those officials who are open to information access, and concrete arguments for those officials who are too restrictive. For NGOs, examples of the benefits of access to information bring both a base for argumentation and a means for motivation to implement positive changes in their countries. Transfer of experience from the more advanced countries is especially significant to countries in transition.
Defining content and the scale of technical assistance has not been an easy task, since quantitative parameters of input from foreign experts as well as capacities needed for implementation were not clearly determined. When technical assistance (in this case, technical assistance includes phases of the Project with extensive and time consuming input from foreign subcontractors, e.g. preparation of and commenting on project materials) represents a noticeable share of resources, as was the case with this Project, it is crucial to have a clear concept of the needs for such assistance, planned activities related to the assistance and, consequently, resource and capacity requirements. The Project document defined the activities of subcontractors too broadly, and specification in the Terms of Reference (referred to in Project documentation) did not present a sufficiently clear picture of the specific subject and scale of activities of non-local subcontractors either. According to the Terms of Reference several activities are assigned to more experts, but their share and scale of capacities needed for implementation of a particular activity are not indicated. Budget categories were not justified by the estimated volume of work needed to achieve the goals.
Study tour
A study tour is a marvelous capacity building activity for properly selected participants. It provides a unique type of experience, which is difficult to obtain, if at all, in any other way. There are three crucial tasks for this type of Project activity: suitable selection of participants, a well prepared agenda, and the best possible transfer of experience from participants to the institutions which they represent. The project outlined a clear concept of experience which participants would obtain on the study tour, and correctly matched the agenda with the specific needs of participants. It was less specific about the process of suitable selection of participants and providing for the best application of obtained experience of participants in their home countries.