School self-evaluation: What we are learning from other countries

John MacBeath

School self-evaluation is now seen as a matter of priority in most economically advanced countries of the world. It flows from a shared concern for quality assurance and effectiveness, fuelled by international comparison which rank countries on a range of common indicators. For governments who invest in OECD and UNESCO surveys, pupil performance in what are seen as ‘key’ areas of skill and knowledge acquisition carry high political stakes. This is the international policy context for self- evaluation which is driven by three primary ‘logics’.

1An economic logic: the costs of training, administration, conduct and follow-up of external evaluation are too high, and may not offer value-for-money

2An accountability logic in which schools render an account to government, parents in return for the investment and public trust placed in teachers and school leaders

3A school improvement logic which holds that the process of reflection, dialogue and concern for evidence is the motor of better schools

These three are not discrete in their expression but can easily become the prevailing or driving motive. When they do get out of balance the quality of learning and teaching suffers. Requiring schools to be ‘self-inspecting’ (that is, assuming the role external inspection) may have economic benefits but may divert attention and energy from the core work of the classroom. Over-emphasis on accountability, as shown by evidence from many countries of the world[1], reveals an attrition of professional engagement and vitality of teaching. School improvement, while the most compelling of the three ‘logics’ will falter without accountability and attention to the attendant time and opportunity costs.

There are important lessons to be learned from school self-evaluation around the world, countries striving to reconcile the three prevailing logics and to find the balance between external and internal evaluation. Examining current models within European countries the Standing International Conference of Central and General Inspectors of Education (SICI) identify three typologies:

Proportional: in which inspection takes the school’ own data as its starting point. The better the self-evaluation the less intensive the inspection. The Netherlands, Scotland, Portugal, Flanders, the Czech Republic, Ireland and England are described as falling under this rubric.

Ideal: in which the inspectorates report on the quality of self-evaluation and point to improvements needed. Northern Ireland, Austria and France are in this category.

Supporting, in which the role of inspectors is to provide support for schools in carrying out self-evaluation more effectively. Denmark and some German Lander fall into this category.

Within these clusters of countries there are, however, distinctive differences in the nature of the process, the high stakes implications, support and pressure, the flexibility of frameworks and criteria, extent of the dialogue and the involvement of teachers in developing frameworks, criteria and procedures.

Ireland provides an example of a process which involved wide consultation with schools. A pilot in 35 schools, published in 1999 (DES) described it as ‘consultative evolutionary evaluation’ and ‘a developmental model to serve the trinitarian purposes of school improvement, school development and school effectiveness.’ Great care was taken to couch the process in cultural and contextual language and to involve teachers’ unions, parent associations in amending the model. Illustrative of this was the eventual agreement to describe the central in indicator as quality of learning and teaching rather than teaching and learning – an important shift in focus from what teachers do to how pupils learn. In May 2003 Whole School Evaluation (WSE) was reincarnated as Looking at Our School, An aid to Self Evaluation in Schools – a model in which external evaluation focuses primarily on the schools’ own approach to self-evaluation. The model does, however, have two central weaknesses. One is the lack of reference to pupils and parents as participants in the process. The second is the lack of support at national level for schools to use achievement evidence in either formative or comparative benchmarking sense.

In Scotland the process of development and refinement involving teachers and other bodies has extended over a fourteen year period. Critical to its widespread acceptance by schools has been its progressive growth over that period in which teachers have been involved, in piloting, designing instruments, modifying and slimming down the indicator set, and above all making the approach less prescriptive and more customisable by schools. While there are schools which use How Good is Our School? (HGIOS) mechanistically and with little sense of ownership, there are others schools who see it as a reference point, as a basis for dialogue among staff and pupils, and sometimes parents.

As part of our whole strategy for raising achievement, we decided to look at ‘How Good is our School’s as a way of getting into the whole dialogue with staff and pupils on areas an effective school we should be concerned with. We looked at what hinders learning, what makes effective learning, what makes effective teaching and we worked through that with our own teachers to begin with, then decided it was time to talk to our pupils about it because we were only getting half of the picture. We set up small focus groups in each year group. Now children, youngsters were asked to join - they had the opportunity to volunteer to join so approximately 50 pupils from each year group came along to these focus groups that were led by staff. And they talked about things that hinder learning, things that help learning, things that hinder teaching and help teaching - that was really enlightening. They were then asked to feed that information back to their registration class so, we had representatives from every class in the school.

(St. Kentigern’s High School quoted in MacBeath, forthcoming)

This illustrates self-evaluation being used imaginatively, formatively and with ownership by teachers rather than as a formulaic exercise. A similar kind of ownership is exemplified by this extract from an Austrian Primary school. The national context is one in which inspection and self-evaluation are in a process of development but allow latitude for schools to be imaginative and in control of the process. .

The self-evaluation activities were focused directly on pupils and teachers. …….. Highly individualised learning plans, including a range of lesson materials and activities, were designed to meet individual pupil needs across the spectrum from the special educational needs to the very gifted. The acknowledgement of different learning styles and pace was very much in evidence. More recently, the school has focussed on the development of skills and learning strategies to equip pupils for successful transfer and attendance at secondary school. In promoting self-directed learning, the school had engaged in a range of activities. The QPR project provided support for these activities. A set of quality indicators for pupils and teachers were developed within the school for self-directed learning. A survey on homework and pupil leisure behaviour was conducted and the outcomes presented at a parents’ meeting that was documented in photographs and video. This provided a point of focus and development for school management, teachers and parents.

(Quoted in Antoine et. al, 2001)

Implicit in this example are both purpose and audience. The school’s focus in on telling its story about what matters, conveying it through a range of visual as well as written media. With external support from a critical friend or a project team, as in the cases above, schools have been both enthusiastic and imaginative in self-evaluation. Similar examples can be found in English schools where there has been support either from local authority critical friends or in the context of development projects..

While these examples from individual schools suggest a clear focus on improvement they sit within countries where there is a greater or lesser emphasis on accountability. In England OFSTED dropped its ‘improvement through inspection’ strapline as it moved more candidly towards an accountability purpose. Yet with self-confidence and support at individual school level schools in England an in many other countries can and do pursue an improvement agenda. When there is a sense of ownership and latitude for schools they may go down their own diverging paths in the knowledge that they can render an account when asked for, with due attention to evidence and ‘rigour’. This is explicit in some national policies and appears to be what is implied in the new OFSTED framework, although the proposed critical friend appointment, the negotiation of targets and the sharper focus of inspection do not sit comfortably within that more flexible and collaborative approach. What we have learned from fifteen years of work on self-evaluation is that schools need to believe that they have the space, the authority and the goodwill to pursue an improvement agenda within an accountability framework.

The 101 schools from 18 countries who participated in the E.C. Project Evaluating Quality in European Schools all worked within different accountability contexts from Iceland in the north to Greece in the south. In all of these schools self-evaluation was located at the improvement end of the spectrum. The Irish schools involved, like schools in many of the other E.C. countries, still make reference to the European Project as much preferable to other models, as containing the essential ingredients which made it engaging and empowering[2]. This was also the view of the European Commission, national policy makers in the 18 countries and the 101 schools, 98 of which asked to be included in a further stage of the project. The success of that model is explained by seven seminal qualities

  • The central involvement of key stakeholders (teachers, pupils and parents) in the process
  • The identification of what matters most to teachers and school leaders in evaluating school quality and effectiveness
  • The support and challenge of critical friends chosen by, or in consultation with the schools
  • The dialogue which flowed from the differing viewpoints and the press for supporting evidence
  • The repertoire of tools for use by teachers
  • The simplicity and accessibility of the framework
  • The focus on learning and support for teaching

These principles seem to be continually observed in the breach, and important lessons are there to be learned from North American legislations which lie at the at the accountability end of the spectrum. The high stakes No Child Left Behind policy under George Bush is a salutary lesson as to the collateral damage that can follow from too single-minded a pursuit of accountability[3]. With rigid and oppressive legislation the vitality that we have seen in school-based self-evaluation is stifled. Trends in Canadian provinces in recent years towards harder-edged accountability further illustrate the constraints which serve to inhibit, rather than promote, school self-evaluation.

In British Columbia the School Accreditation Program was recently discontinued,[i] and replaced in 2002 by the Accountability Framework, which includes school district accountability contracts, school planning councils, and, a school district review process, large scale assessment and reports to account for system and institutional performance.[ii] In Ontario the Education Quality and Accountability Office (EQAO) has a mandate to ensure greater accountability and contribute to the enhancement of the quality of education in Ontario ‘through assessments and reviews based on objective, reliable and relevant information, and the timely public release of that information along with recommendations for system improvement.’[iii] In Quebec the Ministère de l’Éducation du Québec (MEQ) publishes Education Indicators, whose purpose is ‘to ensure accountability by providing specific information on the resources allocated to education, the various activities pursued by the education system and the results obtained.’[iv] The annual publication of results on uniform Ministry examinations consist of raw data,[v] with no adjustment scores for intake differences, socio-economic characteristics, or value-added measures.

Within this regime in Quebec a cluster of schools, with support from McGill University, embarked on a project entitled Schools Speaking to Stakeholders. Itpursued two aims: first, to collect, analyse and disseminate information about each of the schools involved in the project; and second, to create a flexible framework that any school could use to communicate significant information about itself to its major stakeholders. In the first phase of the project (18 months), each school completed a school case report and communicated the results of the exercise to its stakeholders by means of a school profile. The second phase (2 years) consisted of a loose coupling of new and old project schools organised around a developing school performance framework which led to a guidebook for use by schools more generally undertaking self-evaluation of their performance. This process resulted in a “starter kit” – a first attempt at providing schools with the basics to get started on a journey of discovery about school self-evaluation.. More important than any of the outputs produced by this project was the demonstration that school teams, with the support of appropriate critical friends and other resources, could engage in a meaningful way in the evaluation of their school and communicate the results to their stakeholders in an accessible form. Although providing a vivid demonstration of the energy and enthusiasm that can be engendered by bottom-up developments such as these the model was never generalised, and was starved of resources and support from the Ministry.

As the Canadian Schools Speaking to Stakeholders project demonstrates accountability is not an anathema to teachers but entered into positively when seen as an integral part of school-based self-evaluation and professional development. This can be seen in the response to government of the Canadian Teachers’ Federation (CTF) which recognises accountability as a priority policy issue and sets outsome of the criteria for a professional model of accountability:

  • Accountability involves accepting responsibility for actions, reporting on those actions and working to improve performance.
  • Accountability must reflect the multiple goals of public education and the diverse nature of students, schools and communities.
  • Educational accountability is a responsibility shared by all those involved in public education.
  • Accountability should be focussed on supporting and enhancing student learning.
  • Quality classroom-based assessment must be the central feature of educational accountability.
  • Teachers are responsible for possessing a current subject and pedagogical knowledge base, using this knowledge base to make decisions in the best interest of students, explaining these decisions about student learning to parents and the public and working to improve their practice.
  • Parents have a right to clear, comprehensive and timely information about their child’s progress.
  • The public has a right to know how well the system is achieving its goals.

As this shows, accountability may be construed in quite different terms from the top-down version, as shown in this. Example from Rhode Island’s School Accountability for Learning and Teaching (SALT). It describes itself as ‘practitioner’ or ‘professional’ accountability because its focus is on teachers’ practice evaluated by teachers. The review team is composed of practising Rhode Island teachers together with a parent, an administrator and a member of university staff. The team spends four to five days in the school, writes a report which is negotiated with the school, the process of which can be lengthy but is highly valued in teasing out evidence and the basis for judgements made. The team then draws up ‘a compact for learning’ the purposes of which are to ensure that school staff have the capacity to implement improvement.

Practitioner accountability was also the rationale for the School Change through Inquiry Project (SCIP) in Chicago. The external review process was designed to help schools become reflective inquiring communities, and to remove the threat which accountability may imply. An important facet of the review was to set it in a continuous context rather than as a one-off snapshot at a given moment.

There are important lessons to be learned from the many school-based projects which thrived within top down accountability-led systems, and from those that have been stifled and starved of support, The three key principles are set out in Gladwell’s book The Tipping Point in which he studies the epidemiology of change. Evidence suggests that for good ideas and breakthrough practice to survive there need to be:

1The vital few: the innovative people with the vision, energy and enthusiasm to take an idea forward

2Stickability: people or structures which endorse the project and help it ‘stick’

3Conditions for growth: a culture which promotes ideas and practices :