Target setting made easy
Francis Marslen-Wilson
Paper presented at the British Educational Research Association Annual Conference
(September 11-14 1997: University of York)
Abstract
The vast majority of Local Education Authorities receive examination data each year from the National Consortium for Exam Results, in electronic form. LEAs can receive a matched data file containing both GCSE and A and AS level results for all year 13 candidates. This data can be effectivelyused to provide a value-added evaluation of pupil and school performance and to enable schools and LEAs to set targets for future cohorts of pupils.This workshop will demonstrate how a combination of two statistical methods - value-added regression analysis and pupil-referenced subjectdifference analysis can be used to set targets for individual pupils, teaching groups and schools, and how schools and LEAs can collaborate effectivelyto raise standards.
1. TARGET SETTING VERSUS BENCHMARKING
Schools and Education Officers will be familiar with the words "target setting" and "benchmarking", but it is important to understand the difference between the two. "Benchmarking" refers to school-level measures of performance, comparing the school with either itself in previous years, with national, regional or "school clusters". Schools can, for example, compare their overall performance on a variety of measures with schools with similar levels of Free School Meals. The average for the cluster, the higher or lower interquartile points, or the highest performing school in the cluster can be compared with your school's performance. A benchmark is contextual information to assist schools in evaluating their performance. However, it is important to recognise that benchmarks are not targets.
Real target setting for pupil performance should be based on pupil-level data, and compares individual pupil performance at different points in time. A school's targets for next year may be lower than last year's performance, if the ability range of the pupils is lower; a school may achieve lower "raw" results but have improved their overall performance. Real target setting is also effective at achieving improvement where it really matters - with individual pupils. Benchmarking, on the other hand, is unfocused and can become empty exhortation to do better .
Real target setting provides targets for individual pupils, which are discussed with the pupils, and reviewed with pupils on a termly basis, as part of a well structured pastoral monitoring system.
2.THE NATIONAL CONSORTIUM FOR EXAM RESULTS
Almost all LEAs in England are members of NCER. Member authorities are familiar with two analytical techniques which have been developed by the consortium:
- Matching GCSE to A Level Performance. Two options are available for members -an annual study which links GCSE and A Level performance using median values. Graphs are produced at national and local level showing the relationship. Note that the median line is not a straight line. In addition, NCER provides a matched data disc which allows LEAs to perform regression analysis. Some LEAs do this using a straight line relationship, others favour a quadratic regression equation. All these approaches provide a relationship between GCSE and A level performance which can be expressed mathematically.
- GCSE subject differences (pupil-referenced) analysis. This analysis compares pupils' performance in individual subjects with their overall average performance. For England as a whole, and at LEA level, this analysis shows differences between subject performance; for example, pupils tend to perform better in English by 0.27 points; pupils tend to perform worse than their own average by -0.28 points in maths.
This year, NCER will provide, at national level, for all member LEAs
-an equation for the relationship between average GCSE score and average A Level score
-a subject differences analysis of A Level performance.
These two elements form the basis of a straightforward and robust approach to target setting.
3. METHODOLOGY
By combining a prediction of pupils' expected average A Level score with the residuals for each subject from subject-difference analysis, grades for individual subjects for each pupil can be predicted.
The purpose of the approach is to provide schools with:
-grade bands into which pupils can be placed, which will be used as part of the pastoral monitoring of pupils throughout their A level courses
-predictions of actual grades for each subject, which can be converted into scores, and aggregated into departmental targets and whole school targets.
-evaluation of school and departmental performance to assist with school improvement strategies.
4. THE WORKSHOP
The methodology will be explained in detail at the workshop. It is hoped that discussion will focus on:
- The usefulness of the approach at school level;
- Ways of varying the targets for individual schools in line with their expectations;
- The importance of involving school staff and pupils in progress monitoring;
- Computer-based approaches to producing the required target setting within schools or for schools.
For people attending the workshop, a copy of the full paper describing the approach will be made available on the day. Others wishing to obtain a copy of the paper should contact:
Tonnie Richmond
Information and Planning
Bradford Education
Flockton House
Flockton Road
Bradford BD7 4RY
01274-751728
T.Richmond/Francis Marslen-Wilson/Bradford LEA