Evaluation of the Cooperation Program for Central and Eastern Europe

Division of Environment and Development

Evaluation of the Cooperation Programme for Central and Eastern Europe

Final Report of the Evaluation Group

The Research Council of Norway

P.O.Box 2700 St. Hanshaugen

N-0131 OSLO

Phone: +47 22 03 70 00

Fax: +47 22 03 70 01

Internett:

X.400: S=bibliotek;PRMD=forskningsradet;ADMD=telemax;C=no;

Homepage: http://www.forskningsradet.no/

Homepage of the Co-Operation Programme with Central- and Eastern Europe:

http://www.program.forskningsradet.no/easterneurope/

Print: GCS

Copies printed: 800

Oslo, March 2001

ISBN 82-12- 01564-5

Evaluators’ Preface

In August 2000 the Research Council of Norway commissioned the present working group to evaluate the research component of ‘The Cooperation Programme with Central- and Eastern Europe. Research and Higher Education.’ We were to perform our evaluation in accordance with the Terms of Reference (see appendix). Other than having to comply with the dimensions of the program included in the Terms of Reference, the working group was given a free hand in choosing approaches and methodologies in the evaluation process. We decided on an approach that sees the evaluation as a forward looking and two-way learning process between the evaluators and the evaluated who, to the greatest extent possible, should be involved in the evaluation process. This approach was subsequently implemented by a written survey (questionnaire) distributed to all project participants, interviews with project leaders and feed-back from project leaders on the first drafts of relevant sections of the present report.

While the final responsibility for the evaluation obviously rests with the evaluation group, we are indebted to the project participants for their positive responses and very grateful for the active and constructive reactions and comments received from the project leaders. It would have been impossible to make the present report without this positive dialogue. We also gratefully acknowledge the professional advice received on specific research topics in the evaluation from Professors Peter Gundelach and Michael Skou Andersen. Associate Professor Lars Johannsen contributed with his insights into institution building in post-communist countries and Internet based research resources. Ole Hersted Hansen assisted with the technical construction of the questionnaire. The group also benefited from initial advice from The Danish Institute for Studies in Research and Research Policy. Else Løvdal transformed the rather different styles of the working group members into a consistent report. Last, but certainly not least, we want to thank the Research Council of Norway for entrusting this exiting task to us and for the support provided during the working process.

Aarhus, March 2001

Gunna Christiansen Per Strand

Magnus Gulbrandsen Ole Nørgaard

(secretary) (Chairman)

Summary

The Norwegian Coorperation Program with Central and Eastern Europe is based on the assumption that scientific collaboration is an excellent vehicle for transfering knowledge and supporting institution building in post communist countries. Transfer of scientific knowledge and institution building are seen as ways to solve knowledge and information problems, thus contributing to the political and economic transition. The overall judgment of the evaluation group is that the research part of the Collaboration Program for Central and Eastern Europe has been a major success. Most program objectives have been met:

· In general, very good research has been produced, and most of the projects have had a positive impact on the transformation processes in the Eastern countries and have met urgent social, medical and environmental concerns. If the relationships created and strengthened by these projects are maintained, the results may spill over into other areas such as business and politics, providing Norway with a competitive edge in dealing with Eastern Europe.

· There has been a significant transfer of knowledge from Norway to the East, and to a lesser extent, from the East to Norway. Many of the projects are sustainable because they have created professional networks with great potential for future interaction.

· The most successful projects have initiated processes of institution building, although some have focused specifically on transfering knowledge and skills to individuals.

· Overall cost-efficiency has been high – an impressive number of publications have been produced, equipment has been transferred to the East, many courses and workshops have been arranged, and substantial large-scale empirical work carried out.

· Most projects have been characterized by professional equality or decreasing inequality (also given the constraints of a Norwegian managed and financed program), particularly regarding research implementation, credit for result and access to data.

The program has been well managed by the staff of the Research Council of Norway, and their efforts have received positive comments from all project managers, who mention their professionalism, active interest, and adminitrative support.

The Norwegian Cooperation Program with Central and Eastern Europe also avoided most, although not all, of the hazards that have beset similar programs in the past. There were only isolated cases of Eastern researchers complaining that they had not been granted academic credit due for their contribution to joint research projects. Nor did the Norwegian program, similar to other Western programs, fully recognize the financial constraints that hamper the contributions of Eastern partners. Finally, the Norwegian program has also encountered the well-known dilemma in institution building between strategies prioritizing reform of existing institutions versus construction of new institutions. The Norwegian program has in fact experienced successes and failures in the pursuit of both strategies. Thus, we find no “best” collaboration strategy – successful projects have been aimed at individuals, existing institutions and new institutions, emphasizing the need for prudence when choosing individual and institutional partners in the East.

An important lesson learned is that pracatially all collaborations initially showed a variety of asymmetries between Eastern and Norwegian partners, rendering some goals (especially design/financial/managerial equality) and combinations of goals (especially institution building versus good research) difficult to reach. Institution building requires a very long-term perspective, and the current projects spanning two to five years can only be seen as one step of a process that requires mutual and continued commitment from all those involved.

Project assessments

We found a number of beneficial effects of having had previous collaboration with the same partners, regardless of scientific area. Scientific motivation was greater in projects founded on existing networks, the transfer of knowledge across national borders was more balanced, and equality between the partners was better, also on financial and project management issues.

There are major differences between the three primary focus areas addressed by the program. In the social sciences, we found initial professional asymmetries in disciplines that were banned or underdeveloped under communism. The most successful Norwegian project leaders met this challenge by integrating courses and other scientific training elements in their projects. In addition, the social scientists often faced difficult initial conditions due to weak professional identities, sub-standard training, unfamiliarity with contemporary theory and methodology, and the strong ideologization of many institutions. Conditions that made it very difficult to combine institution building and scientific quality. Project managers who bypassed local institutions and handpicked their collaborators have generally made better research than those who, in loyalty to the program objectives, were assigned sub-standard or unmotivated researchers by Eastern institutions. All social science investigations address important social problems in the Eastern countries, and some of the projects include end-users in various government institutions.

In the projects within the medical sciences were based on already existing partnerships, and the output in terms of publications and training exceeds, especially for one group, what might reasonably have been expected during such a short project period. It is expected that international publications from the two other projects will appear later on. When the projects were initiated, Eastern partners were relatively weak in terms of technology, scientific thinking and analytical methods. Reciprocal visits (including formal training) and transfer of equipment have been very useful, although the scientific work was somewhat delayed.

In the environmental sciences, there were no initial differences in scientific standards between Eastern and Western scientists, but rather varying strengths and perspectives that proved to be complementary in the work process. The synergy between these two research cultures may explain why projects in environmental sciences on average performed better than those in the other two fields. The combination of the Eastern tradition – more specialized and stronger in basic research and methodology – and the Western tradition – stronger in cross-disciplinary and applied perspectives – was extremely important for achieving such superior scientific results.

Recommendations

The evaluation group’s main recommendations are:

· It is recommended that program be continued because its major objectives – institution building in the East and creating sustainable networks of scientific cooperation – are long-term processes. A termination of the program render many investments futile and send negative signals to Eastern partners.

· If the ambition is to produce high quality scientific research we recommend a continued focus on environmental sciences, as there are excellent opportunities for professional synergies due to the high quality of Russian research and the quality of scientific manpower in the East. The projects target issues of great interest in both Norway and the East.

· If the ambition is to assist Eastern countries in narrowing knowledge and information gaps, further concentration on health sciences and social sciences is recommended. Both fields also meet urgent social and political concerns in the transition countries.

· It is recommended that future programs provide incentives for including training elements and joint conferences in the Eastern countries.

· It is recommended that future programs include additional incentives to increase the funding of Eastern collaborators, be they institutions or individuals.

· In future selection processes we recommend that an applicant’s total project portfolio, including participation in other programs, be taken into consideration.

· If short-term equality and two-way, balanced transfer of knowledge is desired, we recommend to focus on projects based on already established partnerships.

· It is recommended that a written or electronic “guide” about administrative/project management issues be produced for the benefits of future project managers.

Table of contents

Evaluators’ Preface 3

Summary 5

Project assessments 6

Recommendations 7

Part One

Chapter 1 13

Introduction 13

References: 16

Part Two

Chapter 2 19

Research collaboration, knowledge and institutional reform 19

References 22

Chapter 3 23

Individuals as conduits of knowledge and information 23

References. 26

Chapter 4 27

Institutions as vehicles for reform 27

References. 28

Part Three

Chapter 5 31

Statistical background Feil! Bokmerke er ikke definert.

Evaluated projects Feil! Bokmerke er ikke definert.

Chapter 6 34

Success and failure in knowledge transfer and institution building – the views and experiences of the participants 34

Project implementation 35

Research collaboration 37

Institution building 40

Relation to other programs and social relevance 42

Project and program management 43

Equality 44

Sustainability, lessons learned, and suggensions for improvement 45

Chapter 7 48

Project evaluations 48

Social sciences 49

Summary evaluation of social science projects. 75

Environmental sciences 78

Summary evaluation of the environmental science projects 90

Medical sciences 92

Summary evaluation of medical science projects 104

Chapter 8 105

Lessons Learned 105

Chapter 9 109

Recommendations 109

Fundamental recommendations: Implementation and justifications 110

Project selection criteria and similar considerations 111

Running the projects and the program 112

Appendix A: Questionnaire 114

Appendix B: List of publications from the 13 evaluated projects 129

Appendix C 151

Part One

Introducing the program and the evaluation

‘The quality of any academic programme can be measured in a number of ways: by considering its fitness for the purpose for which is designed; or by considering the fitness – that is to say, the suitability and appropriateness – of that purpose. (Alderman et. al, 1996, p. 15).

Chapter 1

Introduction

In August 2000 the Norwegian Research Council commissioned a working group comprising Professors Gunna Christensen, Århus University, Professor Ole Nørgaard (Chairman), Århus University, Director Per Strand, Norwegian Radiation Protection Authority, and Researcher Magnus Gulbrandsen (secretary), Norwegian Institute for Studies in Research and Higher Education (NIFU) to evaluate the Norwegian Program for Research and higher education within the cooperation program for Central and Eastern Europe running from 1997-2001. The working group was entrusted with the part of the program dealing with research collaboration, while cooperation within higher education is the ubject of a separate evaluation (FAFO, 2000). The overall objective of the evaluation was to examine the experiences, strategies and performance quality of research collaboration with former communist countries in Central and Eastern Europe, including countries of the former Soviet Union (henceforth the East European Countries). On the basis of this assessment the working group has been asked by the Research Council to comment on the merits of the program and to propose possible methods for improving strategies and priorities.

The Norwegian government program for higher education and research cooperation with the reformed countries in Central and Eastern Europe under the Cooperation Program for Central and Eastern Europe (running from 1997-2001) is a continuation of the 1992-96 Action Program for Eastern Europe. Thematically the phase one program focused on democracy building and environmental action in all countries in Central and Eastern Europe and primarily focused on establishing contacts and building broadly based networks. The present phase two programs are based on an agreement between the Royal Norwegian Ministry of Foreign Affairs and the Norwegian University Council and the Norwegian Research Council. It is funded by the Ministry of Foreign Affairs, which has granted a total of NKR 85 million, or NKR 17 million per year. A Program Committee with overall responsibility for the implementation of the program selects concrete projects to be funded. The committee consists of six persons. The University Council elects three and the Research Council three. The higher education and research institution managing a project is responsible for its successful conclusion. In contrast to the first program, the present program is both thematically and geographically specific:

· Environmental protection and environmental technology, with special emphasis on areas in Northern Russia

· Social Sciences in the Baltic States

· Economics and administration in Russia and the Baltic States, with special emphasis on Russia

The Russian part of the Barents Region including:

· Medicine and Health

· Language and culture

· Further development of the Norwegian Pomor University Center

The program is based on the rationale that knowledge is a precondition for institutional change and, in a wider sense, for development towards democracy and a sustainable market economy. In the East European context, this assumption implies that the transfer of knowledge from Norway to the Eastern partners is seen as a vehicle for institutional learning and restructuring in the East European countries. In particular, it is assumed that research collaboration provides an important vehicle for long-term multi-strand relations and hence for institutional learning and change. While recognizing the initial asymmetry between institutions and researchers, the program is also based on the principles of equality and reciprocity where the Norwegian side is expected to gain access to new data and insights that may become an incentive for innovative research and benefit Norwegian interests in other areas. It is, however, also recognized that there is a latent contradiction between the developmental goals, the asymmetric financial (in certain fields also professional) relationship and the principles of equality and reciprocity.