Report of the European Commission
on the implementation of Council Recommendation 98/561/EC of 24 September 1998
on European cooperation in quality assurance in higher education
Draft 26 April 2003
- Introduction
On the 24th of September 1998, the Council of Ministers adopted the Recommendation on European cooperation in quality assurance in higher education (cf. annex 3). The Recommendation calls upon the Member States to support or establish quality assurance systems and to encourage higher education institutions and competent authorities to cooperate and exchange experience. It also asks the Commission to support such cooperation and to report on the implementation of the objectives of the Recommendation at European and Member State level.
The Council Recommendation of 1998 was to a large degree the fruit of the European Pilot Projects organised by the Commission in the nineties. The Recommendation laid the foundation for the creation of the ENQA Network, the European network for Quality Assurance in Higher Education and its growing membership. Other political developments have also given new momentum to the quality activities, notably the Bologna Process and the Lisbon Objectives.
In 1999, the Bologna Declaration was signed, calling for a restructuring of European higher education in order to raise its competitiveness and attractiveness for our own citizens and for students and scholars from around the word. Quality is the key component of the Bologna reform package. The Ministers called for the development of comparable criteria and methodologies for quality assurance. In their follow-up conference in Prague 2001, Ministers asked for scenarios on quality assurance and accreditation.
In Lisbon in March 2000, the European Council agreed on the strategic objective to make the EU by 2010 "the most competitive and dynamic knowledge-based economy in the world". It also asked the Ministers of Education to prepare a report on the objectives of educational systems. At the European Council held in Stockholm in March 2001, a report on the future objectives was presented, pointing out three strategic objectives, the very first one being the improvement of quality and effectiveness of education and training systems in Europe. In Barcelona in 2002, the European Council has set the objective of making the EU education and training systems a "world quality reference" by 2010.
The crucial importance of enhancing quality of education was also underlined in the Commission Communications on the role of the universities in the Europe of knowledge[1] and on Investment[2] and Benchmarking[3].
There are several basic reasons underlining the importance of quality assurance: concern about excellence in universities, issues of public expenditure and accountability, the expansion of most higher education systems, the need to meet labour market requirements, the promotion of mobility and the recognition of degrees. The overall aim is to develop a ‘quality culture’ in all higher education systems, already existing in some European countries, but needing support in others. This quality culture should contribute to the appeal of a future European higher education area.
Since the 1998 Council Recommendation, considerable progress has been made in the development of quality assurance systems in the various Member States, as well as in cooperation activities and networking at European level.
The present report provides a brief overview of progress made in Member States and in cooperation activities at European level. It also points out a few options for future development of quality assurance in higher education in Europe. It places the European developments in a global context. Definitions of fundamental concepts are given in annex 1.
The report is based in part on the ENQA Survey ‘Quality Procedures in European Higher Education’[4], which provides a more detailed analysis of the quality assurance operations carried out by member agencies.
- Establishing quality assurance systems: the creation of quality assurance agencies
The Recommendation calls upon the Member States to support and, where necessary, establish transparent quality assurance systems with the following aims:
- “to safeguard the quality of higher education within the specific economic, social and cultural context of their countries while taking due account of the European dimension and of a rapidly changing world,
- to encourage and help higher education institutions to use appropriate measures, particularly quality assurance, as a means of improving the quality of teaching and learning and also training for research, another important part of their task,
- to stimulate mutual exchanges of information on quality and quality assurance at Community and world level and to encourage cooperation between higher education institutions in this area.”
The answer of most Member States and other European countries has been the creation of one or more quality assurance agencies, aiming at quality improvement through evaluation (cf. annex 2). To a lesser extent have the higher education institutions been encouraged to set up their own internal quality assurance mechanisms. Agencies and universities have been stimulated or have taken the initiative to exchange information and to engage in networking at regional, European and global level (cf. section 5). The present section concentrates on the position of the agencies.
Some agencies were pre-existing, but most were established following the adoption of the Recommendation. Their numbers are still increasing. They have been set up with the aim of carrying out external evaluations of higher education institutions or programmes on national or regional level. Despite differences in size and scope they function according to similar features.
The existence of a binary system with a university and a ‘non-university’ sector, has lead in some countries to the creation of two quality assurance agencies. Other countries have chosen for a star model with one central accreditation agency overseeing a series of smaller evaluation or accreditation agencies.
European quality assurance agencies see as their main objectives: quality improvement, accountability and transparency and linked to these objectives, they consider as their three main functions: quality improvement, dissemination of information and, in a growing number of cases, accreditation. The information function contributes indirectly to the recognition of diplomas and periods of study. The recent cooperation between ENQA and the ENIC and NARIC networks is aimed at reinforcing this link. Some quality assurance agencies also have the authority to recognise and license providers of higher education.
In the Council Recommendation, the issue of autonomy and independence is connected to the choice of procedures and methods. Autonomy and independence in terms of governing structures are likely to contribute to the effectiveness of quality assurance procedures and the acceptance of their results. On the other hand, agencies should operate in close contact with academia and take due account of the needs of society, public and private sponsors, students, parents and the labour market.
In practice, governing structures vary according to the national or regional context. In most cases public authorities set the rules for the functioning of the agencies. They are represented in the board of a number of agencies and provide, directly or indirectly, most of the funding. Higher education institutions or rectors’ conferences are the second source of (indirect public) funding for the agencies. In two-thirds of cases, higher education institutions are represented in the boards and in one third of cases the students. In half the cases, labour market parties are represented in the board of the agency.
Next to the national or regional agencies there are a great number of private professional accrediting organisations (e.g. for engineers, medical doctors, accountants). They evaluate and accredit degree programmes related to one or more professions. They existed long before the Council Recommendation and their modus operandi is not influenced by it. There are, however, similarities and synergies worth exploring. This is also true for international accrediting agencies or bodies such as EQUIS (business studies accreditation) and the U.S. based ABET (engineering accreditation).
- Evaluation types
Eight evaluation types can be distinguished: Subject evaluation, programme evaluation, institutional evaluation, programme accreditation, institutional accreditation, institutional audit (of internal quality assurance mechanisms), subject benchmarking and programme benchmarking (cf. the definitions in annex 1). Most in use was and is programme evaluation, but programme accreditation comes close and institutional audit is the third most popular type. More recent is the interest for programme or subject benchmarking, not based on minimum- or average-, but on best practice-, i.e. excellence criteria.
Programme evaluation and private professional accreditation are most used in the ‘non-university sector’ of colleges and polytechnics, probably because of the strong link with certain professions.
Accreditation builds on the same methodological elements as the other types of evaluation, but differs from the other procedures in that judgement is provided according to predefined standards to decide whether a given subject, programme, institution or theme meets the necessary level. The accreditation process is usually seen as a dual process, whereby one body of the agency evaluates and makes an assessment according to pre-defined standards, and another body (e.g. accreditation commission) takes the final decision whether to approve the programme or not. Agencies themselves can also be the object of accreditation procedures.
There is a clear trend towards more variety in evaluation types and most agencies use more than one evaluation type on a regular basis.
- Criteria and methodologies
The Council recommended that systems of quality assurance should be based on the following features, listed in the Annex to the Recommendation:
-"Autonomy and/or independence, in terms of the relevant structures in each Member State, of the body responsible for quality assurance (as regards procedures and methods).
- Criteria closely linked to the aims assigned to each institution in relation to the needs of society and of the labour market.
- Quality assurance procedures should generally consist of an internal, self-examination component and an external component based on appraisal by external experts. The internal element of self-examination should aim to involve all the relevant players, especially teaching staff and, where appropriate, administrators in charge of academic and professional guidance, as well as students. The external element should be a process of cooperation, consultation and advice between independent experts from outside and players from within the institution.
- In the light of the objectives and criteria used in the quality assurance procedure and with reference to the structures of higher education in the Member States, professional associations, social partners and alumni could be included in the expert groups.
- The participation of foreign experts in the procedures would be desirable in order to encourage exchange of experience acquired in other countries.
- Reports on quality assurance procedures and their outcome should be published in a form appropriate to each Member State and should provide a source of good reference material for partners and for the general public."
By and large the agencies work according to these features, although the application may differ according to the national or institutional context. Below follows a short analysis of their implementation.
4.1Criteria
The Recommendation links quality assurance criteria closely to the aims assigned to each institution (fitness for purpose) in relation to the needs of society and of the labour market. The ENQA survey demonstrates that there is a clear shift away from evaluation against the stated goals of the institution, towards the use of more objective external criteria and standards in evaluation and accreditation exercises (fitness of purpose). These may be minimum criteria, average criteria or more demanding best practice i.e. excellence criteria.
4.2Self-evaluation
Self-evaluation is included in most of the evaluations, but in only two third of accreditation schemes. The self-evaluation group consists in most cases of management and teaching staff. Graduates rarely participate. Participation of administrative staff and students varies. Students are more involved in evaluation than in accreditation exercises. The information obtained through self-evaluation is often completed with statistical data or supplementary surveys.
4.3Site-visits by expert group
The site-visit is a standard element of the evaluation process. The visit consists of interviews, a tour of the facilities, a final meeting with the management and the examination of documentary evidence. Classroom observations are only used in a quarter of cases.
4.4Composition of the expert groups
All Agencies use external experts and very often international experts are included in the expert panel, often from neighbouring countries sharing the same language. Experts from the world of employment are used in less than half of cases, professional associations, students and graduates even less.
4.5Reporting and follow-up
Reports are published in almost all cases of evaluation, but sometimes not in connection with accreditation. It is common practice to consult the evaluated institutions before the reports are published. In three quarter of cases, institutions are held responsible for follow-up on the recommendations; in half of cases they share this responsibility with agencies and government.
5.Cooperation activities
The Council Recommendation calls upon Member States to support a series of cooperation activities in order to stimulate mutual exchanges of information on quality and quality assurance at Community and world level. It calls for cooperation between higher education institutions, cooperation between authorities responsible for quality assessment or quality assurance (agencies) and networking. The Commission is invited to encourage this cooperation, working closely with the Member States and making use of existing programmes.
As a result Member States and quality assurance agencies have been engaged in bilateral, regional, European and some global cooperation activities. University networks and associations have also set up activities related to quality assurance.
5.1Bilateral and regional cooperation
- The Governments of Flanders (Belgium) and the Netherlands have decided to integrate their quality assurance activities into one joint accreditation system.
- Twelve European countries[5] have decided to engage in the ‘Joint Quality Initiative’, an informal network cooperating on issues of quality assurance and accreditation of bachelor and master programmes in Europe. As a first result, the Joint Quality Initiative has produced short overviews of expected learning outcomes (competences) of bachelor and master programmes in Europe, the so-called ‘Dublin Descriptors’, which could be used in quality assurance exercises.
- The Network of Central and Eastern European Quality Assurance Agencies in Higher Education (CEE Network) has 18 member agencies[6]. Its main aim is to act as a clearing house for information and to harmonise quality assurance activities, as well as to represent the region in other international arenas.
- The Nordic countries Denmark, Finland, Iceland, Norway and Sweden closely cooperate on issues of common interest within the framework of the European Network for Quality assurance in Higher Education (ENQA).
5.2The European Network for Quality Assurance in Higher Education (ENQA)
ENQA was created as a direct result of the 1998 Council Recommendation and can therefore be considered both as its most concrete outcomes at European level and as a starting point and key actor for future developments. ENQA has been established to promote European co-operation in the field of quality assessment and quality assurance between all actors involved in the quality assurance process. ENQA is funded by its members and by the Commission through the Socrates programme.
Membership of the Network is open to quality assurance agencies, public authorities responsible for quality assurance in higher education and associations of higher education institutions in the Member States of the European Union, acceding and candidate countries. Agencies have to fulfil certain criteria as regards independence and experience in quality assurance. At present there are 34 full member agencies and some 10 candidates. The European University Association EUA, the National Unions of Students in Europe (ESIB) and the European Commission are currently represented in the ENQA steering group.
The main activities of ENQA are exchange and dissemination of information and experience, agency staff training, exchange of experts, workshops, studies and advice to public authorities. ENQA has engaged itself in close cooperation with the ENIC and NARIC networks of credential evaluators in order to explore how better information on quality can speed up the recognition of diplomas and periods of study, in particular as regards joint degrees and transnational education. ENQA is also the coordinator of the Commission supported Transnational European Evaluation Pilot Project (TEEP 2002) in which five departments in five different countries in three subject areas (History, Physics and Veterinary Science) are being evaluated against sets of common criteria.
ENQA’s main focus is to be a professional organisation assisting in the development needs of its growing agency membership. In April 2003 ENQA will publish the results of a survey mapping and analysing the quality assurance activities carried out by member agencies. On the basis of this analysis and further reflection, the ENQA General Assembly will consider in September 2003, and decide upon in May 2004, the future role and organisation of the network.
5.3University networks
The Council Recommendation calls for support to higher education institutions that wish to cooperate in the field of quality assurance on a transnational basis. Several initiatives have been taken in this respect or gained new momentum thanks to the Recommendation and also because of Bologna process peer pressure.
- The European University Association EUA has been active for a number of years carrying out the transnational EUA Institutional Evaluation Programme in a deliberate attempt to make the institution as a whole responsible for continuous quality assurance and enhancement. EUA is also engaged in the Commission supported ‘Quality culture’ project, in which groups of universities help each other to introduce internal quality assurance mechanisms, improve their quality levels and being better prepared for external evaluations. They focus on areas such as ‘research management’, ‘teaching and learning’ and ‘implementing Bologna reforms’.
- Universities cooperating in the large Socrates-Erasmus Thematic Networks (e.g. physics, history and teacher training) have worked since 1996 on curriculum and quality related issues, resulting in 2000 in the pilot project ‘Tuning Educational structures in Europe’. Some 135 universities are participating in this project, coordinated by the universities of Deusto (Spain) and Groningen (The Netherlands). The project, now entering into its second phase (2003-2004) aims at identifying generic and subject specific competences (learning outcomes) for first and second cycle studies in nine different subject areas. These competences could provide ‘reference points’ for curriculum development, quality assurance and evaluation exercises.