Central Bedfordshire pilot project on Comparative Judgement (CJ)

Aims

The project is intended to evaluate CJ as an alternative to external moderation and assess its potential to support schools in improving outcomes in writing.

CJ is of particular interest in those year groups where there is no national exemplification as it provides a means of benchmarking children’s work against a nationally standardised sample. As part of our work to support schools in improving Key Stage 2 outcomes and diminishing the difference for children from disadvantaged groups, Central Bedfordshire Council (CBC) is funding a pilot project to explore the potential of CJ to support school improvement, using Year 4 writing as the focus. Schools taking part will receive training and a report ranking the performance of their Year 4 writers and their school in comparison with a national sample. There will be no charge to participating schools.

How CJ works

Nomoremarking.com is the company whose software enables CJ to be undertaken on a national scale and who have been invited to discuss CJ with the DfE who are exploring its potential. Work from participating schools is uploaded and pooled. The work is then distributed electronically for judging. Most of the work that a school judges will be that of its own pupils, but 20% will be from other schools.

Teachers are presented with two pieces of writing, and are asked to decide which of the two is better. This is a quick process as it is a subjective judgement, rather than marking against a set of criteria. Each piece of work will be judged in different pairings and by many teachers.

Once enough judgements have been made a scale of writing from the worst to the best is built. Judgements are spread amongst teachers, so schools can involve all the teaching staff in assessing writing. Checks are made of the validity of each teacher’s judgements, so decisions made by someone whose judgement is markedly out-of-line with other judges would be discounted.

Teachers are never asked to compare their own pupils’ work with that of pupils from other schools. This means that your teachers are responsible for determining the order of pupils in your own school, but the absolute standard is determined objectively by teachers in other schools anonymously judging your pupils’ work.

Our project will form part of a national ‘sharing standards’ session which means that schools taking part will be able not only to compare their own pupils’ writing, but also the writing of pupils in other schools. The process is intended to indicate with high reliability what standards your pupils are working at.

What your school will need to do

1. Prepare your systems

Upload your school data to to produce bar coded answer sheets which you print out and distribute to pupils.

2. Record Writing on Bar Coded Sheets and upload

Each pupil will receive one or more bar coded answer sheets to write on.The writing task will be quite open, but will need to be done in conditions that are independent – this is in line with guidelines on externally moderated writing, eg not over-scaffolded.This must be done by 26 January.

3. Spend 1 hour doing an in-school judging session

Set up a within school judging session. The session should involve all teachers familiar with Key Stage 2 writing. The judging is done online, and can usually be completed in a 1 hour after school session.This must be done in the w/c 5 February.

As soon as you have finished judging you will be able to see your own internal measures of pupils’ writing ability – ie the ranking of your own pupils. Once all schools have finished a report is created for your school which will position your school within the national samples and indicate for each pupil where their performance ranks in relation to the national standard

Commitment to participate

In addition to the above logistical requirements, in order to help CBC evaluate the potential of CJ as a means of supporting schools to improve outcomes, we would ask you to:

  • Have at least one representative at a twilight training session in January, ie before judging, to introduce the process and ensure all schools know exactly what they need to do and how to exploit the potential of the judging process itself to explore opportunities to improve writing standards
  • Have at least one representative at a half day workshop once reports are received to explore ways of using the school’s own results and national comparisons to facilitate school improvement and work within clusters. National data will be published on 9 March, and the workshop will be in w/c 19 March.
  • Take part in a brief interview on your school’s experience and views on the benefits and drawbacks of the approach.

There will also be an opportunity to take part in a second judging process in the summer. The timings here will be more flexible as this will not be part of a national process. However, ‘anchor’ work from other schools will be included to enable standardisation. Taking part in this second session will enable a school to explore children’s progress in writing since the initial session and offers a chance to work with other schools taking part to explore relative progress of different groups.

To evaluate the potential of CJ to support improvement work across schools, CBC will fund the costs of participation in the project, both fees payable to nomoremarking.com and the costs of running the training events outlined above. CBC would receive a copy of the reports provided to schools taking part in the pilot project and would share anonymised results with schools taking part and subsequently within locality clusters and other groups with a role in school-to-school support.

To confirm your understanding of your school’s commitment, please sign and return this agreement by Wednesday 6 December 2017.

School ______

Headteacher’s signature ______

Date ______

Return by email to