ex-ante evaluation
rop nuts ii south-east
for the programming period 2007 - 2013
(Summary)This project was co-financed by the European Union
6 December 2006
Introduction
This report is to summarizethe main results, conclusions and recommendations from the final ex-ante evaluation of the Regional Operational Programme NUTS II South-East for the programming period 2007-2013.
The ex-ante evaluation of ROP NUTS II South-East was elaborated on the basis of a call appeal from the South-Moravian Region and the Vysočina Region, and on the basis of a subsequently concluded contract for work dated 10 April 2006 and 3 October 2006, respectively.
The ex-ante evalutation proceeded in parallel with the ROP creation and its starting points linked up with outputs from the ex-ante evaluation of the National Development Plan and the National Strategic Reference Framework (hereinafter the “NSRR“). Concurrently with the ex-ante evaluation, there was also an assessment of the impact of the operational plan on environment (SEA), which is an integral part of the preliminary assessment of strategic programme documents.
The structure of the final ex-ante evaluation of ROP NUTS II South-East is as follows:
Chapter 2 describes the approach to the evaluation project, specifying the process of assessment and the chosen methodology. The methodology of evaluation dwells on working documents of the European Commission, viz. “Working Paper on Ex-Ante Evaluation“ and “Indicators for Monitoring and Evaluation: A Practical Guide“. Thus, all basic principles emphasized by the European Commission were applied in the assessment. Main methods used for the ex-ante evaluation included e.g. comparative analysis, secondary analysis of administrative and statistical data, analysis of intervention logic, SWOT analysis, problem analysis and participation methods.
Chapter 3 presents results from the ex-ante evaluation in six sub-chapters. The first sub-chapter brings an evaluation of the analytical part of ROP, the second one deals with the assessment of whether the chosen strategy is justifiable, generally consistent and coherent with externalities, the third one brings a judgment on the relevance of the direction of priority axes. The fourth sub-chapter presents an assessment of the adjustment of indicators, quantification of objectives and anticipated impacts, and the fifth sub-chapter deals with the provision. The sixth sub-chapter is an evaluation of the draft implementation system.
Chapter 4 of the ex-ante evaluation of ROP NUTS II South-East brings conclusions and recommendations of the evaluator, following out from the performed evaluation.
The team of evaluators (Lucie Bučinová, Mariana Nachtigallová, Marie Kaufmann, Martin Krištof and Pavel Machánek) closely collaborated with the authors of ROP NUTS II South-East, with the Regional Councils of the South-Moravian region and the Vysočina region, and with other relevant partners in the region. The number of workshops held at various levels during the evaluation was above 20, which shows that a source for the ex-ante evaluation was not only the analysis of submitted documents but also results from the controlled group discussion and from individual consultations.
Progress in the period from February - November 2006
The final elaboration of the ex-ante evaluation of ROP NUTS II South-East was preceded by interim evaluations and consultations to individual versions dated from February to October 2006 with the currently evaluated version being dated 22 November 2006.
Reminders and comments to versions from February–May 2006, especially those of methodological character, were available to the authors of ROP only to a limited extent. An essential progress occurred after the evaluation of a consolidated version of the document ROP NUTS II South-East that was submitted to the government of the CzechRepublic and acknowledged by it. In the month of July, the ex-ante evaluator prepared a general ex-ante evaluation to that version, which identified gross imperfections in the operational programme thatwould impede – in the evaluator’s opinion – the programme acceptance by the European Commission.
Based on this evaluation report the reading of ROP NUTS II South-East was completely reworked and the recommendations of the ex-ante evaluator of both methodological and technical character were respected in drawing up a new version of ROP.
- The analytical part experienced essential changes. The current version is more straightforward, more concise and interlinked with the strategic part of the programme. The chapter in which the experience with the implementation of the programmes of aid in the region South-East is analyzed has been extended and added necessary conclusions.
- The external coherence of the strategy has been made better arranged as recommended in the ex-ante evaluation by reducing the description of individual strategic documents to a necessary minimum and through the focus on a clear definition of mutual links including the illustration of their intensity. Another important improvement was a greater consideration of complementarity and possible synergic effects with other programmes.
- Priority axes of the operational programme have become a logical climax of the line socio-economic analysis – SWOT analysis – strategies, which sntrengthened the internal consistence of the operational programme.
- Adjustment of indicators, quantification of objectives and anticipated impacts experienced an important move forward in the completion of the system of indicators which now includes relevant indicators linked to NSRR and in fact all mandatory, referred to as “core" indicators.
- Implementation and monitoring reflects the coming to existence of new implementation structures in the regions (Regional Council of the Cohesion Region), and a beneficial influence of the modified methodology for the quantification of indicators, extended description of groundworks collection for monitoring and precised periodicity of the collection.
Results and conclusions of the ex-ante evaluation
Analytical part
The socio-economic analysis was shortened and is now better arranged, with more logical interlinks, containing only the areas that are relevant for the evaluated programme ROP NUTS II South-East. It clearly shows what problems will be solved by ROP NUTS II South-East.
The analytical part deals only with areas that are relevant for the subsequent strategy of the programme. It has been added a sub-chapter of regional disparities, which helps to interlink the analytical part with the strategic part. A more exact identification of regional disparities would have been feasible if a clear and concise formulation of essential differences within the region was presented in a separate sub-chapter, i.e. their separation from the analysis of urban and rural spaces.
The socio-economic analysis shows an unambiguous need to invest into the regional road pattern, to enhance the accessibility of some districts by public transport service, to support tourism in order to exploit the existing developmental potential in this industry, to stabilize settlement of rural areas and to resolve problems in towns – main carriers of the region’s growth and development.
There is no discrepancy between the situation analysis and the SWOT analysis. The situation analysis includes developmental trends and the fields of developmental potential, and the suggested interventions reflect the developmental trends.
The current form of the SWOT analysis puts together fundamental findings and conclusions of the situation analysis and represents a bridge connecting the analytical part with the strategic one. In line with the extent of the socio-economic analysis the authors reduced the SWOT analysis so that it reflects only the areas that are relevant with respect to the strategies and interventions of ROP NUTS II South-East. The logical line between thesituation analysis, disparities, SWOT analysis and the programme strategy is strong and there are no “gaps“ in it.
Strategy
The defined strategy of the programme document ROP NUTS II South-East is relevant with respect to requirements identified in the analytical part of the document. The strategy and proposed interventions based on it directly reflect main problems of the region defined in the analytical part.
The programme strategy includes the principles of equal opportunities, sustainable development and well-balanced regional growth. Horizontal themes are projected into the adjustment of indicators.
The strategy exhibits the required external coherence with strategic documents at the European, national and regional level and expects a significant complementarity with the thematic operational programmes. The implementation of activities drafted within the framework of ROP with relatively limited financial resources cannot by itself meet the defined global objective. This is why the ROP activities are complemented with priorities and interventions proposed within the framework of individual thematic operational programmes. It can be assumed that the fulfilment of the global objective will exceed the financial possibilities and the time horizon of the programme.
Specific objectives extend upon the global objective and it can be claimed that the hierarchy and linkage of the global objective with the individual specific objectives have been retained at the programme level. The individual priority axes are not isolated but rather mutually interlinked and creating a consistent entity.
The internal consistence of the programme and the integral orientation of all priority axes towards the achievement of the global objective are assured by the hierarchic interlinking of the objectives which are defined not only at the programme level but also at the level of priority axes.
Priorities
Priority axis Transport accessibility is of prominent cross-sectional character and contributes to the fulfilment of all three specific programme objectives. The financial allocation for this priority axis amounts to 51% of total allocation. As compared with the other priority axes, a considerable emphasis in terms of resources is put particularly to this axis. The amunt of this allocation is in line with the identified needs and is underlaid with actual absorption capcity. The character of interventions within the framework of this priority asix is well balanced with respect to sustainability.
Priority axis Development of sustainable tourism concerns the whole territory of the region and the allocation for its realization amounts to 19% of available financial resources. The possibility of cross-financing is not likely to be used for this priority axis. Nevertheless, the issue of tourism closely related to the level of human capital, and it is therefore presumable that the use of the cross-financing might contribute to the achievement of set-up objectives. In the evaluator’s opinion, the overlapping with IOP in the field of refurbishment and use of cultural monuments has been resolved in a not entirely satisfactory way.
Priority axis Sustainable development of towns and rural area can be understood as a key one in the context of ROP NUTS II South-East. In the Region’s territory two urbanization centres were identified with populations above 50 000 (towns of Brno and Jihlava) and 9 seats whose size and importance is lower but which can be still characterized as centres of regional significance. The remaining part of the Region with 60% of total Region’s population is a rural space which can be characterized by pronounced regional disparities. From this point of view the inclusion of urban issues together with problems of rural areas within the framework of a single priority axis appears unsuitable to the evaluator. This priority axis has been allocated 26.6% of total financial resources.
Indicators and the quantification of objectives
The system of indicators is in line with the National Code Book of Indicators and to a greater part consistent with NSRR indicators, both by means of indicators at a programme level and by means of those at the level of priority axes. The coverage of all priority axes with the indicators of output, outcome and impact is well balanced with a greater part of indicators reaching beyond the framework of the National Code Book of Indicators.
The quantification of objectives was made by using the impact and outcome indicators both at a level of global- and specific programme objectives, and at a level of the specific objectives of priority axes. The quantified objectives mirror in fact the financial allocation, only in some indicators their levels can be considered to have been adjusted in a relatively conservative way. The coherence of indicators with NSRR is sufficient both at a level of the global objective and at a level of the respective relevant strategic objectives. Satisfactory is also the scale of indicators to measure the programme impact on the number and quality of jobs and on the achievement of the objectives of the Lisbon strategy.
Implementation
The chapter contains all information required by EC legislation, Council Regulation (EC) 1083/2006 in particular.
The managing authority for ROP NUTS II South-East is the Regional Council of the Cohesion Region South-East. The Committee of the Regional Council of the Cohesion Region South-East is fully functional which can be deduced from the active participation of its members in the proper elaboration of ROP NUTS II South-East. Executive body of the Regional Council is the Regional Council Office which performs all activities connected with the function of the managing authority.
The Regional Council Office will establish “territorial“ workplaces in the concerned two regions, which can be considered a very positive and responsive step towards the applicants for the payment of project expenses. The reading of ROP NUTSII South-East does not specify the individual processes of programme implementation and the responsibilities of concrete participants; nevertheless, regarding the gradual building of capacities at the Regional Council Office and with respect to the gradual creation of managing and implementary documentation it can be expected that a detailed description of the processes will be included in the implementation document and consequently in the Operational Manual.
It can be generally concluded, however, that the diction of the chapter on implementation shows a certain non-conceptuality and methodological disunity in the adjustment of the implementation mechanism, which apparently results from the above mentioned gradual building of the managing authority, and from the absence of primary methodologies and manuals for management. The fact can be seen especially in the disproportional extent and elaboration of individual partial chapters.
Recommendations from the ex-ante evaluation
Socio-economic analysis and SWOT analysis
- To consider a separation of the identification of regional disparities from the analysis of rural and urban spaces. A separate sub-chapter on regional disparities and their clear and concise definitions will reinforce the bridging of the analytical and strategic parts of the document.
Internal consistence of the strategy
- To consider to single out the urban issue in a separate priority axis which would include the aid directed both to urbanized centres (Brno and Jihlava) and to the other nine defined centres of regional significance. It is recommended that the intervention is provided on a basis of the Integrated programme of town development also within the framework of the aid to regional centres because the absence of an integrated approach may result in the realization of a range of isolated projects without a sufficient impact and without synergic effects on development of concrete territory.
External coherence of the strategy
- To consider the use of cross-financing in Priority axis Development of sustainable tourism. The issue of tourism closely relates to the level of human potential and it can be assumed therefore that the use of cross-financing could contribute to the achievement of the set-up objectives.
Indicators and the quantification of objectives
- As to the adjustment of the system of indicators it would be considered useful to add the allocation of indicatorsto individual areas of support. Another recommendation is to complement the definition and description of indicators in such a way that their content is entirely unambiguous, possibly also their scope (e.g. impact indicator of Priority axis 1 – Environmental impacts in affected areas); in some indicators expressed in per cent (increase/decrease) it is advised that the initial values are presented also in respective units for a possible comparison.
- In order to achieve the set-up goals of ROP NUTS II South-East it will be necessary to adjust an objective system for the assessment of the impact of implemented measures including the adjustment of criteria for project assessment that will contribute to the fulfilment of complex developmental programmes in the whole Region.
Financing
- To rectify the discordance in Chapter 5:Financial Provision where the reading claims that the national financing will be provided from the state budget, from the budgets of regions, budgets of municipalities and from private sources. The indicative financial table mentions only the national public sources without providing a closer specification of these financial resources and it does not calculate upon the private sources.
Implementation and monitoring
- One of critical moments in the current system of SF implementation is the method of selection and assessment of projects, which has not been emphasized in the analysis of experience. In adjusting a detailed system for the assessment and selection of projects within a framework of the implementation document it is necessary to take into account experience of the current period and to attempt at an adjustment of such a system that would not evaluate only the formal part of submitted projects but which would be capable of assessing also their quality and impacts.
- It would be useful to specify details about the subsequent control of the fulfilment of monitoring indicators and about the impact of their possible non-observance, namely in cases in which the value of these indicators will affect the project evaluation.
6/12/2006 Ex-ante evaluation ROP NUTS II South-East – summary /page1