8
BRITISH EDUCATION RESEARCH ASSOCIATION
ANNUAL CONFERENCE
UNIVERSITY OF EXETER
12-14 SEPTEMBER 2002
EXPLORING A POSSIBLE LINK BETWEEN SECONDARY SCHOOL COMPOSITION AND EDUCATIONAL OUTCOMES
Edwin Smith, Institute of Education, University of Warwick
DRAFT ONLY: NOT TO BE REPRODUCED WITHOUT THE AUTHOR’S PERMISSION
Correspondence: Edwin Smith, Institute of education, University of Warwick, Coventry CV4 7AL ()
EXPLORING A POSSIBLE LINK BETWEEN SECONDARY SCHOOL COMPOSITION AND EDUCATIONAL OUTCOMES
Edwin Smith, University of Warwick Institute of Education
Abstract
The effects of school pupil population composition on educational outcomes are well documented. They parallel composition effects on health variations and delinquency in communities. In schools, one possible connection is teacher attitudes towards their classes - their motivation, perceptions of pupils, job satisfaction, etc. In a multi-site case study of two schools in challenging circumstances and two others in more favourable situations all in the same LEA, all the teachers were asked to complete visual analogue attitude items grounded in the researcher’s participant observation With a 67% response rate (N = 194), analysis of variance indicated significant differences among teachers in the four schools, with those in less favourable circumstances showing less positive attitudes. The differences correlate with postcode-census estimates of disadvantage including Jarman Under Privileged Area scores, but teacher workload did not relate to either teacher attitude or type of school.
The paper focuses upon one of several domains of organisational and management processes which may mediate the school composition effect on educational outcomes. Others include school ethos as measured by pupils’ responses to visual analogue items, time spent by senior and other staff on different professional activities like ‘firefighting’, long term planning, professional development, discipline, and providing additional help and support to pupils.
Acknowledgements
I was introduced to visual analogue scales by Tirjinder Singh Gidda, one of the many students at Churchfields High School, who over many years generously allowed me to learn from them as well as being one of their teachers. He and others continued to supply me with opportunities to learn after they left the school, and it was during a discussion of a project in his BDS course that he explained the scales to me. Without his unfailing willingness to share, I would not have known about them.
I am grateful, too, to Dr Sean Neill and Dr Daniel Mujis in the Warwick University Institute of Education for their comments on preliminary drafts. Any errors remain my responsibility. Professor Brian Jarman of Imperial College, University of London kindly supplied the data-set of his index for the area covered in this study
The staff and students of five schools in a Midlands LEA have been exceptionally forbearing and generous in giving time to help with the research.
1, Context
This paper provides an account of an attempt to measure teacher attitudes towards their classes in four secondary schools in a Midlands LEA in England. Two of the schools can be understood as having a higher mean SES in their pupil populations than the other two, whether the measure is the percentage of pupils taking free school meals or various indexes of deprivation which are based on decennial census data. It is part of a study of the effects of school composition (or pupil mix) on processes in the schools (Thrupp 1999).
2. Measures of SES
The ‘postcode to enumeration district’ (PC2ED) database provided by the Office of National Statistics and available through Manchester Information and Associated Services (MIMAS) enables an individual 6-character postcode to be matched with its corresponding census Enumeration District, each of which, contains typically fourteen residential addresses. The indexes available in the PC2ED database include: Department of the Environment Index of Local Conditions, Townsend Index, Carstairs Index and Jarman Index of Under-Privileged Areas (UPA) (Jarman 1983). Of these, only the Townsend scores are at Enumeration District level. For the other three, the scores are at electoral ward level, and so provide data which are not sensitive enough for use in estimating the mean SES of a school.
The Townsend scores were calculated using the PC2ED database. Using the 1991 census (the 2001 data not being available at the time) limited the currency of the information, particularly as in the past ten years there has been significant housing development in some parts of the catchment areas of the schools, particularly one of the schools, and the economic fortunes, affecting unemployment (particularly) and overcrowding, car ownership, and owner-occupier status of households. These are the four variables which compose the Townsend Index. However, it can be argued that the effects of social background are cumulative for the individual over possibly more than ten years, and that there may be a time lag in the effects of neighbourhood change on the local culture. This should not be overstated, and it remains true that the elapsed time between the 1991 census and this study may constitute a threat (albeit minor) to its validity.
Since a six-figure post code identifies typically about 14 residential addresses there is a probability that any one of those fourteen households will be typical of the post code set as a whole and a smaller but real probability that it will be typical of households in the corresponding enumeration District (on average about 150 households). Thus, as Gibson (2000) demonstrates, in the case of a large secondary school, careful use of post code - enumeration district linkages provides considerably more valid measures of SES than percentages of free school meals, in educational research the most commonly used measure of disadvantage. The area covered by the LEA in question may be untypical: only 40% of the post codes have fewer than 20 dwellings. (Addresspoint 2001). This may constitute a further threat to the validity of SES measures using post code and census data for the area concerned.
Furthermore, in England, as distinct from Scotland, post code areas do not fit precisely into enumeration districts. One post code area can overlap into as many as six enumeration districts. The PC2ED directory addresses this overlap by providing ‘part postcode units’ (PPUs). Thus for each post code up to six values for the Townsend index may be displayed, and that poses a dilemma because there is not a way of knowing which of those values is the appropriate one for a given household. Gibson (2000) has devised an algorithm which overcomes this difficulty by weighting the probability that a given address lies in each of the PPUs according to the distribution of the fourteen or so addresses which form the post code area among the six PPUs. (private correspondence). The algorithm is not widely available, and it is custom and practice to use the unweighted arithmetical mean of all the PPU indexes to estimate the index for a given postcode. It was necessary to resort to this device in the present study and it added a further element of error of measurement into the use of the Townsend index.
The PC2ED directory does not provide ED-level figures for the Jarman index (Jarman 1983), but it was possible to obtain from Professor Jarman the entire Jarman dataset for the area covered by the LEA in question as calculated at the Imperial College School of Medicine where the Index was developed. The PPU issue was resolved there by a ‘best fit’ model, using the PPU in which the majority of addresses lie as the ‘true’ value. (private correspondence). This is more accurate than taking unweighted arithmetical means, but less accurate than the Gibson algorithm. Another benefit of the Jarman dataset is that it includes values by postcode for each of the eight constituent variables separately so that individual variables like the proportion of lone parents and low social class and unemployment can be considered in isolation from less relevant ones like the proportion of lone pensioners and children aged under five. The weightings applied to the variables in the Jarman index reflect their scores in an extensive survey of general medical practitioners in England for the degree to which they increase their workload or contribute to pressure of work While it cannot be argued that the relative weightings would be the same in schools, there are self-evident parallels, even only in terms of the findings of the ESRC Health Variations Programme (Graham 2000). The profiles of the four schools are in Appendix 1
Both the Townsend and Jarman indices carry inherent errors of measurement. They are probabilistic rather than definite, the calculation introduces errors, and their source data are over ten years old. However, in a comparative study, It is safe to assume that for a population of about 4,300 pupils, most of the errors will be randomly distributed or will cancel out. However, the small number on roll (560) in Ardell may limit the validity of that assumption.
Arithmetical mean scores, even with standard deviations for the various indices are of less value than the range and distribution so profiles showing the estimated indices at pupil level in a school were constructed (Appendix 1). The visual comparison is striking. Nonetheless, it was necessary to use a single measure for analysis of variance, and for this purpose the arithmetical mean limits was used
Using the school-level measures of SES (Jarman index) is an ecological approach, and correlating them with individual teachers’ attitudes is susceptible to an ecological fallacy.(Robinson 1950, Fitz-Gibbon (1996, Goldstein 1998) However, correlating SES at individual pupil level with individual teacher attitudes would be demonstrably fallacious because the nature of the Jarman index (and other indices based on census and post code) is probabilistic and it does not provide individual SES measures. An individual’s Jarman index merely represents the conditions in the post code area where (s)he lives. A high index for ethnic minority or unemployed does not mean that the individual is of ethnic minority origin or unemployed - only that there is a high incidence of those factors in the fourteen or so households (Gibson 2000) in their post code area - or rather such an incidence was recorded over ten years ago in the 1991 census. Thus the Jarman index is properly regarded as a valid ecological factor, and it is not fallacious to test it as an independent variable. In fact it would be equally an ecological fallacy to assume that all the past, present and future effects of background factors affecting pupils’ school achievement can be reduced to individual characteristics measured at a point in time. It would be analogous with assuming that all the effects of soil type, temperature, sunlight, etc., that affect a plant community can be captured in the characteristics of the plants at a given time.
3 Teacher attitudes
The teacher questionnaire (Appendix 2) was piloted with several groups of teachers attending CPD courses in the Institute. All the teachers in the four schools were asked to complete the duly modified questionnaire. The response rate across the four schools was 66 percent. This provides a 95% confidence limit overall, but for only three of the four individual schools since the response rate varied from school to school. Both questionnaires used horizontal 10-centimetre visual analogue scales in preference to more conventional Likert scales. Although visual analogue scores appear not to be reported in the social science journals, they are commonly used in medicine, particularly to measure patients’ experience of pain, and also sometimes to measure attitudes. The validity and reliability of such scales in educational research are the subject of another paper in this conference (Smith E F in Session 3: Research Methods) The provision of a ten-centimetre line with end-point verbal anchors enables respondents to mark any point on the continuum. It can more safely be assumed that the result is an interval-level measurement, and if the lower anchor point represents a zero, then the measure can even be considered as a ratio-level one.
In the questionnaire developed for this study (Appendix 2), 10-centimetre lines were used and they were arranged horizontally rather than vertically, mainly because that format fitted most easily onto A4 size paper - an important consideration in view of the numbers of questions employing them - 24. Ten centimetres is the minimum length for acceptable levels of reliability (Reville et al 1976). The teacher questionnaires also sought factual information about role and workload in order to triangulate with teacher diaries.
Results
Factor analysis with varimax rotation yielded five factors (Appendix 3) Factor 1 projects a positive teachers perception of their classes, for example pupils are well organised, have high self esteem, thrive on challenging work, have mature relationships, can be trusted,, etc. Teachers in the two higher SES schools responded more favourably than their colleagues in the lower SES schools. (Fig 1)
Fig 1 Teachers’ positive perceptions of their classes (Ardell and Denham are lower SES schools than Bulton and Carlton)
In the case of negative teacher perceptions of their classes (high expectations lead to negative pupil reaction, pupils need threats to make them work, teacher feels like giving up, etc.), the reverse pattern emerges from the data (Fig 2).
Fig 2 Teachers’ negative perceptions of their classes
Factor 3 relates principally to teacher motivation and their attitudes to their own work (e.g. feeling highly motivated to prepare, looking forward to marking pupils’ work, looking forward with pleasure to the next lesson with a class, etc.). here, the pattern of responses shows less between-school difference, with Ardell ( a lower SES school) being not very different from Bulton, a high SES school. This may indicate that Ardell teachers, despite having less positive perceptions of their classes are as motivated towards them as their colleagues in the higher SES schools