- 1 -
Fifth Conference on Evaluation of the Structural Funds
Budapest 26/27 June 2003
Challenges for Evaluation in an enlarged Europe
Paper presented for Conference Workshop 2
Comparative element to the ongoing mid-term evaluations
of the objective 2 Programmes in the
East of Scotland, Nordrhein-Westfalen in Germany and Western Scotland
Ulrike Kugler1, Carolyn Sawers1, Ruth Downes2, Regina Trenkler-Fraser2,
1Ulrike Kugler - Objective 2 Secretariat (managed by agiplan ProjectManagement) in Nordrhein-Westfalen (NRW) – Haroldstr. 4, 40190 Düsseldorf, or - and
Carolyn Sawers - Western Scotland Programme (managed by SEP - Scottish European Partnership Ltd); or for the East of Scotland
Both represent the regions engaging the mid term evaluations
2Ruth Downes – European Policies Research Centre (EPRC)
Carrying out parts of the evaluation work in NRW and
Regina Trenkler-Fraser – EKOS ,
Carrying out the evaluation of the horizontal themes and the benchmarking
Content
1Introduction
2Context - Background of regional co-operation between Scotland and NRW
2(a)The Partnership
2(b)The Regions
2(c)The Programmes
2(d)Integration of Horizontal Themes
3Comparative Elements and Benchmarking study
3(a)Evaluation approaches - General Concept
3(b)Methodology
3(c)The results of the first workshop
3(d)Proposed structure of the Benchmarking Study and questions to be answered
1Introduction
Partnership connections and co-operation have been developed between NRW and Scotland for a while. The co-operation and the benchmarking of certain aspects in the ongoing midterm evaluation for objective 2 programmes of Nordrhein-Westfalen (NRW), West and East Scotland are built on those experiences. This new co-operation process shall contribute to respective understanding and knowledge-transfer in the context of EU structural funds and here in particular in evaluation and the use of benchmarking as an approach in evaluation. The central objective of this evaluation- and benchmarking- process is a discussion and reflexion on programme experiences in different national and regional contexts. The engaged managing authorities , regional secretariats and evaluators believe, that the resulting experiences can contribute significantly to a better understanding of complex problems and improve the ongoing communication and support of the programme managing organisations.
The focus of the partnership is the following: There are three evaluations - one in each region - carried out independent from each other. But the ongoing process and the main results of each evaluation will be compared during and at the end of evaluation work. The benchmarking is integrated into this process. It goes beyond a general comparison and considers the implementation and relevance of horizontal themes in the Objective 2 programmes in depth.
The paper will address good practice in terms of programme evaluation at different stages of the programming life cycle, benchmarking and cross-region co-operation projects. The evaluation’s methodology has a number of interesting and innovative elements including:
- Bringing an element of comparative evaluation to the mid-term evaluation process
- Evaluation of the horizontal themes focussed on the practical process of mainstreaming
- Co-operation and joint working: this project demonstrates co-operation between public administrations in regions, between private sector evaluators and between the public and private sectors generally.
This Paper is a combined effort of the regional secretariats and the evaluators of the three programmes. It describes the evaluation process from the beginning until June 2003. Therefore it can be regarded as a working paper – not aiming to present final results. It gives opportunity for the regions and evaluators to discuss the initiated process in a broader context. For this it will be helpful to all being involved in the process to get new ideas where to do better, where to ask other questions.
2Context – Background of regional co-operation between Scotland and NRW
The following parts describe the origins of the partnership between the regions. They show the basis that has been built up and that enables the regions with all their commonalities and differences to try out a combined evaluation approach. Necessary for the understanding of the co-operation and the evaluation approach is also a basic knowledge of the regional and programme context. As it will be pointed out later, the integration of horizontal themes within the programmes needs to be highlighted.
2(a)The partnership
The startingpoint of Scottish-Nordrhein-Westfalian partnership was in the year 2001. The project has been chaired and co-ordinated by Scottish Enterprise under the auspices of the Scottish Executive and the Ministerium für Wirtschaft und Arbeit des Landes Nordrhein-Westfalen (MWA). Also involved were experts from different organizations in both regions.
The partners agreed that an examination into some of the economic development issues surrounding common problems of land reclamation and environmental renewal would be mutuall beneficial.
This collaborative work resulted in a document "Partners in Development", supported also by officials from the respective Ministries.
The initial concept developed into a project proposal for collaborative work examining four specific themes, with additional consideration of the part played in the economic development process by the EU-Structural Funds. The report, to some extent, tests some of the assumptions we tend to make about the way others do their business; are systems in Nordrhein-Westfalen more geared to long term thinking than in Scotland, and if so, can Scotland learn from that? Are partnership structures in Scotland more fully developed than in Nordrhein-Westfalen, and if so, what can, or should, be transferred?
The result was a project which demonstrated partnership in action, and underlined the benefits of the regions of Europe working together. The diversity of regions within the Member States, even where they face similar problems, leads to different approaches and different solutions. Work such as this allows the regions to learn from each other and grow together.
The project was a comparative, but concise, discussion of four areas of economic development activity:
- Land Reclamation and Environmental Renewal
- Clusters / ‘Competence Fields’
- Sustainable Development and Environmental Technology Industries
- Entrepreneurialism and Sustainable New Business Start-Ups.
- Use and management of EU-Structural Funds
So the project was much more about seeking positive outcomes for the participating regions and organisations. In seeking to understand each others’ systems, the authors of the relevant chapters have noted, and described, differences where that is helpful in terms of contextual understanding. They have quickly moved on, however, to identify very specific areas of policy and practice which can be influenced by joint learning, or where further work is called for. The project team has identified a number of outcomes of value:
- Joint learning, both in exploring systems in the partner region, and in re-examining systems in the home region
- Opportunities for further collaborative investigation
- Opportunities for influencing policy development at home
- The opportunity to leave a legacy which may be of value to candidate countries developing policies in similar fields.
Due to the opinion of all project partners an abundance of knowledge was gathered by the first partnership approach in 2001. Both regions signalised lively interest to continue co-operation and with that going beyond description of situation. A new approach is ment to lead to concrete projects and to regard more dynamic aspects.
In addition to the study Partners in Development the contact network IQ-Net[1] has to be mentioned as the initial point of collaborative work. Since 1996, IQ-Net has offered to partner regions of Objective 1 and 2 areas research and exchange on structural fund topics- including evaluation.
In context of the IQ-Net meeting in December 2001 in Grobbendonk/Belgium the three regions agreed to intensify the cooperation between Scotland and NRW. Result of this discussion was , that the midterm evaluations of objective 2 Programmes offered a good chance to start-up a new co-operation project.
Why choosing the mid term evaluations?:
-All three regions have the same regulations in timetable, procedure and methodologies to be fulfilled in evaluation. In comparison to other political aspects the conditions for setting up a regional benchmark are advantageous.
-Evaluations will give chance to get deeper into strategies and implementation of structural policy of the regions, to assess the effectiveness of funding and to re-think the own practice compared to each other region.
-The implementation of the Scottish and the NRW structural funds programmes are regarded as good practice in terms of the EU-Commission. The quality of the three programmes is said to be high.
-The size of eligible areas, the ERDF funding and -with restrictions- the economic problems/challenges are comparable.
Further on, in February 2003 a co-operation agreement between Scotland and Nordrhein-Westfalen was signed by ministers of both regions in Düsseldorf. The agreement on a high political level covers several areas EU-Policies, Research and Development and industrial reconversion. It is a political commitment,
2(b)The Regions compared[2]
The following part is giving a brief overview on the regional context – showing the eligible area, population, contribution of the structural funds (objective 2), the main strengths and weaknesses, the main similarities and differences are additionally pointed out.
Objective 2 area North Rhine-Westfalia
Total population of Objective 2 and phasing out areas / 4 millionTotal Structural Fund contribution for 2000-06 programme / 970 mill. euro
ERDF contribution / 823 mill. euro
ESF contribution / 147 mill. euro
ERDF per Capita / 206 TEuro
Objective 2 area Scotland
West of Scotland / East of ScotlandTotal population of Objective 2 and phasing out areas / 2.3 million / 550,000
Total Structural Fund contribution for 2000-06 programme / 483 mill. euro / 250 mill. euro
ERDF contribution / 419 mill. euro / 250 mill. euro
ESF contribution / 64 mill. euro / -
ERDF per Capita / 210 TEuro / 455 TEuro
Key regional strengths and weaknesses – NRW
Strengthslargest market area in Germany with certain areas of untapped market potential for regional firms
central European location with good access to key transport access and with high international accessibility
comprehensive network of technology and incubator units
good supply of business sites, many in inner-city locations
strong network of universities and technical colleges, research institutes and higher education bodies
high number of graduates
good pool of qualified labour
financially strong large firms and high concentration of HQs in or near the assisted area
significant improvement in environmental situation and wide-ranging skills in environmental protection and renewable energies
experience with innovative approaches to regional development / Weaknesses
Growth rate and investment quota significantly below federal and Land average leading to increasing disparities in terms of GDP and per capita income
Below average business productivity and competitiveness
considerable lack of job and above average unemployment
low share of SMEs and craft firms
no growing network or cooperation structures (except in mining sector)
below average rate of new firm formation, including in technology sectors
poor rate of R&D activities and lack of regional innovation strategies
below average growth of service sector
qualification deficits in ‘new’ sectors
particularly poor labour market prospects for women given traditional focus on ‘male-dominated’ sectors
poor image of Ruhr hinders service and tourist sector and the attraction of new technologies
congested road system
environmental damage from industrial past
KEY COMMONALITIES
Low or underdeveloped business and growth rate
Skills shortages, particularly in new or growth sections (particularly NRW and West of Scotland)
Strong research/university presence
Key regional strengths and weaknesses – Scotland
West of ScotlandStrengths
Significant presence of growth sectors and strategic sites
Excellent research base
Wide range of visitor attractions and conference facilities
The region's natural environment
Well-established infrastructure
Strong partnership approach
Available match funding via Social Inclusion Partnerships
Experience in intermediate labour market initiatives
Weaknesses
Low rate of new firm formation (particularly among women) and low business performance
Gaps in ICT infrastructure and lack of relevant strategy
High number of vacant/derelict sites
High levels of multiple deprivation in concentrated areas
Low level of educational attainment
Strong vertical gender segregation
Poor transport infrastructure
Private sector attitudes towards employing previously unemployed
Weak interface between research and business sectors / East of Scotland
Strengths
Large/diverse manufacturing base
Attracting overseas investment
Well established tourism product
Research facilities/R&D presence
International business outlook
High quality environment
Well-qualified workforce
Weaknesses
Underdeveloped business base
Number of disadvantaged communities
Over-represented in sectors in transition
Fewer businesses in growth sectors
Low levels of business investment in R&D
Large areas of vacant/derelict land
Skills shortages in key areas
Towns struggling to adjust
Shortage of suitable sites and premises
KEY CONTRASTS
NRW Objective 2 area considerably larger in population terms and with stronger heavy industrial tradition
Environmental conditions in Scottish regions relatively better than in NRW
Higher profile of disadvantaged/deprived communities and areas in Scottish programmes
2(c)The programmes – main objectives and structure
The following part presents the main strategic priorities of each programme, the structure of measures and implementation.
Strategic priorities - NRW
Core strategic aims
“Creation of new and securing of existing jobs, particularly in SMEs, through improving the competitiveness of the region”
Strategic objectives
Increasing investment activity including the creation of new firms
Developing and strengthening regional competencies/clusters
Improving the infrastructure framework conditions
Horizontal aims
Long-term and environmentally friendly development
Creation of equal opportunities for men and women in the labour market
Integration of disadvantage groups in the economic renewal process
Innovation orientation and optimal preparations for the information society.
Strategic priorities – Scotland
West of ScotlandCore strategic aim:
“strengthen the capacity to create and maintain sustainable economic growth that leads to greater economic and social cohesion, that protects and enhances the environment, and results in increased prosperity, jobs and an improved quality of life for residents in Western Scotland”
Strategic objectives:
to enhance the competitiveness and innovative capacity of the Western Scotland economy in order to improve growth
to develop greater economic and social cohesion and reduce the disparities within the region
to promote equal opportunities
to protect and enhance the environment / East of Scotland
Core strategic aim:
“To promote sustainable economic development in the East of Scotland which is founded on the key principles of enterprise, learning and social justice”
Strategic objectives:
build a strong regional and community capability in order to leave a durable legacy beyond the end of the Plan period
assist and encourage the development of a more competitive, dynamic and innovative business base
embed an inclusive and collaborative approach to economic and community development throughout the region
work towards an equality of opportunity for all people in the region
improve the environmental performance of the enterprises, communities and organisations in the region
meet the challenges of economic and social change through fostering entrepreneurship and lifelong learning
Strategic vision for the region:
“a dynamic, innovative and competitive regional economy where people and businesses are well equipped to face the challenges of new and emerging European and international markets”
Strategic development drivers and themes:
sustainable development (equal opportunities, economic and social inclusion and environmental enhancement)
enterprise learning (business creation and growth, social entrepreneurship, community enterprise)
learning and innovation (life-long learning, programme and project innovation)
building regional capacity (hard infrastructure – spatial priorities & strategic sites – and soft infrastructure – innovation in strategic development partnerships and financial engineering)
Priorities and measures – NRW Objective 2 programme
Priority 1: Business and start-up finance1.1Investment grants for commercial investment
1.2Loan capital
1.3Funds for business start-ups by university graduates
1.4Start-up premium for SMEs
1.5Wage subsidies for unemployment (ESF)
Priority 2: Innovation and development of competencies/clusters
2.1Technology and innovation
2.2Start-up initiative
2.3Business development initiative for SMEs
2.4Integrated environmental technologies/ environmental protection
2.5Information and communication technologies
2.6Tourism, culture and leisure industries
2.7Consumer and business related services
2.8Renewable/future energies
2.9Regional development strategies and interregional cooperation
2.10Training, consulting and coaching of unemployed (ESF) / Priority 3: Innovation related infrastructure development
3.1Premises for manufacturing and service industries
3.2Greening of industrial sites and emission-reducing infrastructures
3.3Technology and training infrastructure
3.4Logistical services and infrastructure
3.5Integrated employment and infrastructure support (ESF)
Priority 4: Support for selected target groups
4.1Vocational training for young people
4.2Integrated development of urban problem areas (ERDF + ESF)
4.3Integrated development of rural areas
4.4Improvement of labour market opportunities for women
KEY COMMONALITIES
Emphasis on technology and innovation
Overall focus through strategic aims, priorities and measures on competitiveness
Focus on SME development
Inclusion of new forms of finance (eg. risk and loan capita)
Development of strategic sites, locations and infrastructure
Priorities and measures – Scottish Objective 2 programmes
West of ScotlandPriority 1: Develop competitiveness and innovative capacity of region’s SMEs
1.1Enhance access to finance for SMEs
1.2Enhance SME services to develop competitive and innovative business base
1.3Develop a competitive workforce
Priority 2: To develop the region as a competitive location
2.1Develop the region’s competitive locations
2.2Develop SME facilities to support competitive sectors and clusters outside competitive locations
2.3Marketing and promoting the region as a competitive location
Priority 3: Increase the economic and social cohesion of the region
3.1Community area regeneration
3.2Routes to opportunities / East of Scotland
Priority 1: Strategic economic development
1.1SME creation and development
1.2Access to risk capital
1.3Technology and knowledge transfer
Priority 2: Strategic locations and sectors
2.1Strategic locations and sectors (revenue)
2.2Strategic locations and sectors (capital)
Priority 3: Community economic development
3.1Community engagement and capacity building
3.2CED implementation – spatial targeting
3.3CED implementation – thematic activity
KEY CONTRASTS