Geoff Pugh and Shqiponje Telhaj

Attainment effects of school enmeshment with external communities: community policy, church/religious influence, and TIMSS-R mathematics scores in Flemish secondary schools

Geoff Pugh and Shqiponje Telhaj

(Centre for Economics and Business Education,

Staffordshire University Business School)

Paper presented at the European Conference on Educational Research, University of Hamburg, 17-20 September 2003

Abstract

This paper analyses school enmeshment effects, which we define as attainment effects arising from either the voluntary or structural position of schools in external networks. We use a unique dataset on Flemish secondary school students from the 1999 repeat of the Third International Mathematics and Science Study to estimate an educational production function. Our estimates of enmeshment effects suggest attainment benefits

1. when school principals take responsibility for community relations, and
2. when schools are influenced by certain groups in the wider community; in particular, by faith communities rather than by trade unions or business groups.

These enmeshment effects are consistent with the literature on social capital as well as with recent developments in the economics of identity, while the second group of results contributes to the literature on the attainment effects of faith schools (in particular, of Catholic schooling). In addition, other results indicate possibilities for improving attainment in all schools.

JEL Classification: I21

Key words:

Educational attainment; social capital; identity; Catholic schools; TIMSS.

Corresponding author:

Geoff Pugh,

Staffordshire University Business School,

Blackheath Lane,

Stafford,

ST18 0AD,

UK.

Word count: Text: 8,358 (inclusive); Appendices 1 and 2: 2,060

Acknowledgements

We are grateful to the Education Department of the Ministry of the Flemish Community for sharing their valuable augmentation of the TIMMS-R data for Flanders. For helpful comments, we also thank Peter Davies, Nick Adnett and participants at the European Conference on Educational Research (Hamburg).

1.Introduction

This paper uses a unique dataset on Flemish secondary school students to analyse the determinants of academic attainment. From this platform, we investigate the particular attainment effects of:

1. school policy on community relations; and
2. church/religious influence on schools, which is our main topic of interest.

We refer to these as enmeshment effects. We propose the concept of enmeshment to describe the situating of schools in external networks of institutional and cultural influences. From schools’ perspective, enmeshment may be either voluntaristic (i.e., resulting from freely-chosen policy) and/or structural (i.e., “built in”, or determined by forms of governance and ownership). From within the school community, school policy may deliberately promote enmeshment with the external community. Alternatively, institutional forms of governance and/or ownership may themselves enmesh the school community with stakeholders in the external community. By enmeshment effects, we refer to the potential of differing types and degrees of enmeshment to influence academic attainment. Of course, enmeshment effects may have greater, and arguably more important, effects on other types of schooling output. However, these are beyond the scope of this paper.

The concept of enmeshment effects and, hence, the empirical investigation reported in this paper, are motivated by

1. a large and diffuse literature on social capital,
2. a smaller but focussed literature on the “Catholic school effect”, and
3. recent developments by Akerlof and Kranton on “the economics of identity”.

There is overlap between these research agendas; for example, theoretical and empirical work by James Colman is prominent in research on social capital and Catholic schooling and is also much cited by Akerlof and Kranton (2002). We now demonstrate that all three suggest similar hypotheses on enmeshment effects.

Social capital embraces trust, norms and networks in communities and organisations (Pollit, 2001). It is social, not only because it is a resource available to everyone in the group, but also because it is created by the quantity and quality of connections between individuals in a group. And it is capital, because it amplifies human effectiveness. Social capital in its various forms lowers the costs of cooperating and so promotes cooperation, because group cohesiveness and pressure are sustained by trust and norms rather than by costly monitoring, control and enforcement. Accordingly, economists have referred to social capital as “excess cooperation” (i.e., excess relative to what could be predicted by assuming rational self interest) (Paldam, 2000), while sociologists have referred to it as “social glue”. Social capital improves efficiency by facilitating coordinated actions and can have particularly beneficial effects in complex activities such as innovation, and during change, when difficulties in monitoring and incipient loss of control can make trust more important, particularly in reducing uncertainty. Putnam’s definition of social capital emphasises ‘networks’ (cited Paldam, 2000, pp.651): ‘Networks of civic engagement … represent intense horizontal interaction … the denser such networks in a community, the more likely its citizens will be able to cooperate for mutual benefit.’ Social networks are both promoted by trust and social norms and, potentially, are the means for their production. Granovetter (1973) makes the case that “weak” ties are better than “strong” ties. Strong ties are characteristic of, for example, mafia groups, and some business networks, whose norms are anti-social and who are not externally trustworthy. Strong ties tend to be exclusive and aim at redistribution in favour of the group. Conversely, weak ties tend to be non-exclusive and increase cooperative activity.

These themes suggest that social capital of the kind generated by weak ties may complement material resources and other inputs into educational production. Groups identified in the literature as promoting weak ties include Guides/Scouts, PTAs, and broad-based churches. In particular, Coleman (1988) argues both that social capital is important in the creation of human capital and that broad-based churches can endow schools with social capital in the forms of community relationships, norms and sanctions. In turn, these enhance cooperation between teachers, students and their families with respect to homework, behaviour, and attainment. Accordingly, the social capital literature suggests

1. that broadly-based churches are a source not only of norms and values that promote trust but also of networks of weak ties that promote cooperation and, hence, exert a generally beneficial effect in communities and organisations; and
2. that the enmeshing of schools with broadly-based churches or other faith communities, and/or with other community stakeholders – as long as their values, norms and networks promote cooperation - may exert a beneficial effect on attainment.

Together, these suggestions lead us to investigate the possibility of attainment effects arising

1. from schools’ relationships with broad-based churches/religious groups and other external stakeholders, and
2. from schools’ community relations policies.

These enmeshment effects are much narrower and focussed than the broad and diffuse concept of social capital. However, structural and voluntary enmeshment effects may be considered as examples of social capital effects and are sufficiently precise to be measured by our data.

There is an extensive literature on the effects of governance and ownership on educational outcomes (for a survey, see Rentoul et al., 2000). However, there are few empirical studies, and little consensus on the differential effects of the various governance reforms and ownership patterns. One exception is the mainly US literature that finds a positive “Catholic school effect” on student outcomes (Evans and Schwab, 1995; Neal, 1997 and 1998; Sander, 1997).

In many European countries, the role of the Catholic Church in schooling has long been contentious. In the UK, recent emphasis on faith schools by the government, together with initiatives from the Church of England to open more such schools, has created a widespread debate. Claims and counter-claims about the attainment effects of faith schools, while not the most important source of disagreement, are nonetheless a prominent part of the debate. However, in Europe - in contrast to the US - quantitative evidence on the attainment effects of Catholic and other faith schools is scarce: in the UK only one paper has analysed the attainment effects of faith schools (Schagen and Schagen, 2002), while ‘in continental Europe there has not been a great deal of research in this area’ (Arthur, 2002). Accordingly, one purpose of this paper is to contribute to evidence-based policy on faith schools.

Our focus on enmeshment effects is informed not only by the social capital literature and the Catholic schooling literature but also by theoretical developments in the economics of identity and their recent application to the economics of education. Akerlof and Kranton (2002) survey evidence from sociology, anthropology and education on the attainment effects of cooperation between students, teachers, and parents. They then proceed from the positive effects identified in this non-economic literature to develop a theoretical explanation of why ‘the establishment of community affects educational attainment’ (p.1191). Their model incorporates the sociological concept of self-image or identity, with the consequence that students’ attainment depends not only on the resources used but also upon ‘student formation of academic identity’ (p.1168). This reflects ‘a basic concern of education scholars – the dependence of students’ achievement on the gap between their own self-images and the person the school intends them to be’ (pp.1197-98). Accordingly, developing community – and, hence, consensus over ideals - reduces the negative impact of social differences on attainment. In particular, by ‘narrowing the gap between school and home’ (p.1191) more students can be induced to identify with the school (p.1187) and so enabled to ‘fit in’ (p.1174). In turn, ‘as students identify more with the school, the school can promote an ideal that is more amenable to marketable skills’ (p.1188). In discussing the evidence for their model, Akerlof and Kranton (2002) explain the success of Catholic schools by their ability to ‘establish such a school community where students accept the school’s goals and ideals’ (p.1191).

Theoretical hints from the social capital literature, empirical results from the “Catholic school effect” literature, and the economics of identity all suggest potential enmeshment effects. These can be stated as two hypotheses:

1. that faith schools associated with broad based churches and faith communities “add value”, in the sense that their students have higher levels of attainment than similar students in secular schools; and
2. that schools with a policy of promoting community links “adds value”, in the sense that their students have higher levels of attainment than similar students in schools with no such policy.

As well as investigating these hypotheses separately, we also investigate whether these structural and voluntaristic enmeshing effects may have complementary effects on attainment.

Unfortunately, our data does not enable us to test a possible alternative to our first hypothesis; namely: that positive attainment effects are caused by the “religious human capital” of parents and pupils (e.g., the possibility that faith inclines either students to work harder or/and parents to make their children work harder) rather than by faith schools as such. (For the concept of religious human capital, see Iannaccone, 1998.) In principle, we could test this alternative hypothesis by examining – other factors held constant - the performance of pupils with varying religious backgrounds and degrees of conviction in schools with varying degrees of church/religious influence. However, in common with other studies on this topic, we cannot test this alternative hypothesis, because the student questionnaires do not include questions about the religious faith of either students or their parents (see Section 2, below).

This paper analyses the Flemish data from the 1999 repeat of the Third International Mathematics and Science Study (TIMSS-R; the data is discussed in Section 2 below). Two reasons for using Flemish data are explained in Section 2: namely, the inclusion of variables on both schools’ community policy and the influence on schools from church/religious groups; and the unique augmentation by the Education Department of the Ministry of the Flemish Community of the TIMSS-R data with an intelligence score. Toma (1996) gives further reasons for analysing Flemish data: ‘Overall, Belgium is a particularly interesting country for measuring the production of education.’ Belgium combines full government funding with minimal political control of schools. Hence, in Flanders, 70 percent of secondary students attend private schools, which is one of the world’s largest percentages of private enrolments. The high degree of school autonomy is reflected in huge variation in school governance. Four organisational forms receive full taxpayer funding: schools controlled by central, provincial and local governments; and free schools. Most of the free schools are operated by the Roman Catholic Church (75 percent of the Belgian population is Catholic); with a few having Protestant or Jewish affiliation, and a few being secular private or “alternative” schools. They are regulated with respect to the subjects taught and the language of instruction, but not with respect to either teaching methods or the religious basis of teaching. Consequently, Flemish data is unusually appropriate for analysing school effects.

The rest of the paper proceeds as follows. Section 2 discusses the data and the imputation of missing data. Section 3 outlines our model, an educational production function, and explains our method of estimation. Section 4 presents our empirical results; in particular, our results on schools’ community relations policy and the attainment effects of church/religious affiliation. Section 5 reports alternative specifications to investigate the robustness of our results. Section 6 concludes. Detailed discussion of the data, data imputation and estimation has been excluded for reasons for space: full discussion of these issues, together with more detail on the results, is included in a Working Paper that is available on request.

2.The data

In 1995, the Third International Mathematics and Science Study (TIMSS) tested large representative samples of students from 39 countries in mathematics and science at both primary and secondary school levels. Following Wöβmann (2003), we analyse the determinants of student’s mathematics scores. We use the international proficiency scores, because these are recommended for both international and within country comparisons. The extensive TIMSS documentation (Gonzalez, 2001; TIMSS, 2003) explains the procedures designed to prevent bias, to ensure comparability in school and student sampling, and also to assure quality in test design, data collection, scoring procedures and analysis.

TIMSS contains many educational variables in addition to the achievement scores of individual students. A wide range of data from both classroom and school levels was obtained from questionnaires completed by both teachers and heads of schools. Students who were tested also completed a questionnaire concerning their attitudes towards mathematics, classroom activities, home background, and out of school activities. We focus on Flemish students enrolled in Grade 8 (mean age 14.05 years) in order to measure secondary school-level effects on student achievement.

One limitation of TIMSS data is that it is generated by a single cross-section study. This means that it is not well suited to deal with the potential endogeneity problems that beset the estimation of resource effects – in particular, class size - on student attainment. However, this is not a problem for studies in which resource variables are included as control variables but which are designed to answer other questions (Krueger, 2003). In particular, TIMSS data is well suited to the estimation of family background and institutional effects, which are exogenous to individual students’ educational attainment (Wöβmann, 2003, discusses these issues at length).

There are two main reasons why we have conducted this study using Flemish data.

1. In 1998-1999, TIMSS-Repeat (TIMSS-R) again assessed eighth grade students in both mathematics and science, which in Flanders included a representative sample of 5259 students in 135 schools. The Flemish data in TIMSS-R is unique, because the Education Department of the Ministry of the Flemish Community augmented the TIMSS questionnaires by administering at the same time a measure of students’ ability (or “intelligence score”), which not only provides a valuable control variable but enables us to investigate the possibility of omitted variable bias in our results.
2. In addition to students’ attainment scores, and student, teacher and school background variables, school principals in the Flemish sample provided a wealth of school-level information, including their assessment of the strength of various sources of influence on the school’s curriculum, which include church/religious groups, the business community and teacher unions. These variables are central in the present study, but were not implemented by the TIMSS authorities in many other countries.

In survey research, missing data is endemic. TIMSS is no exception. For a particular student, only one missing variable means that we lose all the other data for that student. Hence, because we use a large number of variables, even low percentages of missing values compound into a very high proportion of missing cases. For the model reported in Table 2, the number of students with a full set of observations on all variables is 1,567 from a sample of 5,259. If the subjects with complete data are unrepresentative of the entire sample, then such a reduction in sample size would bias the estimates and make them less efficient. Accordingly, we impute the missing values. Given that in the Flemish TIMSS data the percentage of missing values is usually around 5 percent with few above 10 percent, Smits et al. (2002) and Schaffer (2003) suggest that there should not be much difference between the outcomes of different methods of imputation. We adopt the approach of Wöβmann (2003), who uses deterministic regression imputation to impute missing TIMSS data. We dropped cases for which non-imputed data was available for fewer than 35 variables, which excluded 101 students from the original sample of 5,360 students.

3.The model: the education production function

The education production function treats schools like any other economic activity. An “output” (academic attainment measured by test scores) is produced by combining a range of “inputs” (from the student and the school) according to an educational “technology” (the process of teaching and learning). The form of our education production function follows Wöβmann (2003):