Grading Correction

Following the meeting in London on 15th July I have performed an iteration on the live data. I have included only games from players who met the following criteria

1.  The player had a published SP grade in 2005

2.  Only SP games played between 1/6/05 and 31/5/06 against players in the sample group would count

3.  The player had a minimum of 5 games in the sample

4.  The players games yielded a score between 15% and 85%

5.  Steps 4 and 5 were repeated until all criteria were met

6.  All players were assigned an arbitrary initial grade of 150

The sample produced 6302 players who played 61375 games between them.

The calculation process followed the ECF system with 2 exceptions. Firstly, juniors did not receive a junior supplement. Secondly, the 40 point rule was not applied as it is not appropriate for this kind of iteration.

It is clear from the first 6 graphs that the number of games played does not affect the reported result. It also appears that the results closely follow a predictable trend between current ECF 70 and 200. Outside of that range the differential from the trend appears to increase, probably due to the smaller sample beyond those limits.

What is clear beyond doubt is that the current grades have become stretched. This is demonstrable in the iteration where lower graded players have performed significantly better relative to their existing grade than higher graded players. This means that lower graded players are currently under-graded and the lower the grade, the worse the problem.

The Required Fix

I propose a dual action to correct current grades. Firstly an adjustment that assumes 217 as the pivotal point where grades above that threshold grades are accurate relative to each other. 217 has been selected because above that grade the conversion formula of ELO = ECF*8+600 still produces an accurate approximation. To then make all sub 217 grades accurate relative to both each other and those above 217 we need to apply the following adjustment

where G1 is the corrected grade and G0 the 2006 grade

It may be useful to understand what this means in real terms. Below is a table showing a summary of the adjusted grades.

2006 Grade / Corrected
Grade
50 / 87.5
60 / 95.3
70 / 103.0
80 / 110.8
90 / 118.5
100 / 126.3
110 / 134.1
120 / 141.8
130 / 149.6
140 / 157.3
150 / 165.1
160 / 172.8
170 / 180.6
180 / 188.3
190 / 196.1
200 / 203.8
210 / 211.6

FIDE Conversion

If we apply the correction showed above then the conversion formula

ELO = ECF*8+600

produces a good approximation for players above 150 (ie above current ECF130). Below that level the approximation is not so good, but this is perhaps explained by the surprisingly high ELO ratings of players below ECF130 which on average is 1913.

Ongoing Calculations

Having corrected the current grades, the second action is to ensure that such a problem did not occur again. The definitive cause(s) of the drift is currently unknown. It may be the result of certain “tweaks” such as junior adjustments, 40 point rule, treatment of ungraded players etc or it may be that some process that used to be done caused the problem and that the grades are now steady state as this process is no longer performed. What is certain though is that because higher graded players are more active than lower graded players, this has a deflationary effect that cannot be prevented. We should therefore, prior to publication of a new list, perform a check similar to that performed by me here to ensure that, on average, players are performing to expectations and apply a small correction annually where necessary.

We should identify and eliminate (where possible) statistically flawed techniques within the current calculus. Two that can and should be dealt with immediately are :-

·  Ungraded Players

It is clearly incongruous to suggest that a player who plays a minimum of 1 game but less than 9 games has played enough for his grading to be used in the calculation of others but has not played enough for his grade to be published. I propose that the only sensible way forward is to ignore games played against ungraded players. I further suggest that quarterly supplementary lists should be published containing only new grades for previously ungraded players so that the games of these players can be counted as soon as we have sufficient data to justify a grade.

·  Juniors

Of 927 juniors I have identified in the 2006 list, 265 increased their grade (compared to 2005) by more than 10 points of whom 100 increased by 20 points or more. This suggest that for many the junior increment is insufficient. I have therefore investigated the iterated grades by age compared to all iterated grades. It appears that, according to the iteration, the grades of many junior players appear to be inaccurate, some too high but the majority too low. The results would be even more pronounced if the junior supplement were added to the iterated grades.

This could be due to rising numbers of junior only tournaments where the output is not verified by the participation of established, statically graded players. It seems to me that junior players are likely to be having a serious deflationary effect upon the list as a whole and the best solution appears to be to treat them as new players each year, as we are doing now with players with negative grades.

I believe the steps mentioned above go a long way to correcting and maintaining an accurate grading system for the future.

Sean Hewitt

Leicester

20th July 2006