Guidance on possible preliminary data analysis of your 2013 KS2 results

The following pieces of data can easily be extracted from the SIMS AM7 report ‘HCC KS1 2009 to KS2 2013 Tests’, which is available from the SITSS website using the shortcut:

1. Headline Figures – Floor Standards

On the first sheet of the report (called ‘Progress Summary’) you can see the cohort level summary figures. The four highlighted figures relate to this year’s floor standard i.e.

• 60% of pupils achieving Level 4+ in all three of reading, writing and maths
• proportion of pupils making 2 levels of progress in reading at least in line with national median (we have taken this to be 92%, although it could change this year)
• proportion of pupils making 2 levels of progress in writing at least in line with national median (we have taken this to be 92%, although it could change this year)
• proportion of pupils making 2 levels of progress in maths at least in line with national median (we have taken this to be 90%, although it could change this year)

Are you above/below each of these?

(If below all 4, then the school is below the floor standard.)

If below on any of these, why do you think this happened this year?

Do these figures suggest any key issues for future development?

(If below on the attainment measure, which area/s contributed most to this outcome – reading, writing or maths?)

2. Expected Progress (using the Transition Matrices)

The other tables on the Progress Summary sheet show progress from KS1 to KS2 – overall and broken down into prior attainment bands. (These are compared against last year’s national figures.)

Compare the proportions making and exceeding expected progress against the national figures (in each of reading, writing and maths).

Are there any subjects where the overall school % figure is below the national?

Are there any subjects where the overall school % figure is significantly above the national?

Look at each prior attainment band (i.e. children who were at Level 1 at KS1, Level 2c at KS1 etc) – are there any groups where proportions are below or significantly above national figures?

Ofsted descriptor for ‘good’ states that proportions of pupils making or exceeding expected progress, from all starting points, ‘compare favourably’ to the national (in line with or better).

Ofsted descriptor for ‘outstanding’ states that proportions of pupils making or exceeding expected progress, from all starting points, are high compared to the national (ie significantly better).

Would you therefore describe progress in reading, writing and maths as good or outstanding?

(If not good, then it must at best be Requires Improvement.)

3. Attainment and Progress by pupil groups

Use the Detailed Summary sheet, which shows for different pupil groups:

• % achieving Level 4+, Level 4b+ or Level 5+ in each area (RWM)
• % achieving Level 4+ and Level 4b+ in all areas (RWM)*
• % achieving Level 4+ and 5+ in the Grammar, Punctuation and Spelling test
• APS in each area
• % making Expected Progress from KS1 (RWM)

* This year the national data will include the proportion of pupils achieving a ‘good level 4’ in all areas, i.e. 4b+ in reading and maths and 4+ in writing.

As a starting point, focus on the APS column (R/W/M combined) - how does this compare with the national figure? (2012 national APS was 28.2)

(A score that is 1 point higher than national might be sig+ in RAISE – depending on cohort size.

A difference of 2 points is very likely to be statistically significant.)

Compare the APS for reading, writing and maths separately against last year’s national figures.

(Last year, reading APS=28.8; writing APS=27.3; maths APS=28.4)

Would you describe these figures as in line with average, significantly above or significantly below?

Explore the attainment of different pupil groups.

Are there any particularly pleasing results? (compare with national figures in RAISEonline)

To what can you attribute these successes?

Does the attainment of any of these groups cause concern?

Why might this be the case this year?

What action points might you consider for further development?

Look at important gaps, e.g. between boys and girls, between FSM and non-FSM.

Compared to last year’s data, are these gaps closing?

If there are any particular groups that your school has been focusing on (e.g. boys, FSM etc) does the data indicate that your focus has had impact?

Having explored the different pupil groups’ Average Points Scores, it would also be worth looking at any significant difference in terms of:

• proportions getting L4+ in reading, writing and maths
• proportions getting L4b+ in reading and maths and L4+ in writing
• proportions making expected progress in each area

4. Further Analysis of Groups including Custom groups (Attainment)

Using the Pupil Data sheet, you can apply filters to look at more precise sub-groups, e.g. if you wanted to explore attainment of boys on Free School Meals, apply a gender filter (=Male) and a FSM filter (=Yes).

To explore a custom-defined group, use the Custom filter column. For example, if you want to focus on pupils that were part of an intervention group, or received one=to-one tuition etc, define these by entering a letter or symbol into the Custom column, then filter it.

Note the attainment of any particularly significant groups within your school.

Use the Pupil Data (Fine Levels) sheet. (This uses reading and maths test scores and finely grades them, as per the RAISEonline Value Added model.)

What is the overall APS increase figure (final pink column, top figure)?

If national progress this year is in line with national progress last year, then we can interpret an APS increase of 13 as equivalent to Value Added score = 100; APS increase of 14 equivalent to VA = 101 etc (i.e. add 87 to the APS increase to find VA score).

Explore the APS increase figures by subject and for different pupil groups by applying the filters in this report. Note any groups where APS increase is particularly high (over 14) or particularly low (under 12).

6. Progress – individual pupils

It can be useful to explore the progress of individuals and consider those cases where pupils made exceptionally good progress and exceptionally poor progress. What explanations might there be and what lessons can be learnt for the future?

One approach for doing this would be to look at the increase in individual pupils’ point scores from KS1 to KS2, as above.

Think particularly about any pupils making 10 points progress or less across the Key Stage – what were the issues for that child? What strategies did you put in place?

Any pupils making more than 14 points progress? How was this achieved?

7.Conclusions

Thinking about all of these areas:

• headline attainment figures (including trend from previous years)
• differences in attainment and progress between significant groups of learners

and using the current Ofsted guidance, how would you self-evaluate your school’s achievement grade?

What are the key areas for development?

What might your key actions be to try to improve these areas?

Further analysis of KS2 pupil data:

Question level analysis (QLA) can be carried out in RAISEonline to identify key areas of the curriculum where pupils underperformed, e.g. maths topics, Assessment Focuses in reading etc.

These can be useful in thinking about the teaching throughout the school, particularly if these are part of a trend, not merely a one-off picture relevant only to that cohort.

NB QLA can also be carried out on QCA optional tests, and it might prove a more beneficial use of time to do this for current Y5, to analyse gaps that could be addressed during Y6, than to analyse the data for the pupils who are leaving your school.

Ben Fuller, Assessment & Analysis Adviser, Hertfordshire Children’s Services