December 1996
EDUCATIONAL VOUCHERS:
EFFECTIVENESS, CHOICE, AND COSTS
Henry M. Levin
Stanford University
December 1996 Draft—For Comments
Prepared for presentation at Annual Meetings of the American Economics Association, New Orleans, January 4, 1997. The author is a Visiting Scholar at The Russell Sage Foundation, 199697 and the David Jacks Professor of Higher Education and Economics, Stanford University.
Abstract:
Most of the policy discussion on the effects of educational vouchers has been premised on theoretical or ideological positions rather than evidence. In recent years a substantial amount of empirical evidence has accumulated on achievement differences between public and private schools; on who chooses and its probable impact on educational equity; and on the relative costs of public and private schools and on a voucher system. The purpose of this paper is to provide a summary of that evidence. (1) Present results among numerous studies suggest little or no difference in student achievement between public and private schools for a given student, but some evidence of higher rates of graduation, college attendance, and college graduation for Catholic high school students. (2) Evidence is consistent that educational choice leads to greater socioeconomic (SES) segregation of students as the more advantaged have historically been the families most likely to take advantage of choice. Families tend to choose schools where other students are from higher SES backgrounds, resulting in probable effects of rising inequality in achievement because of “peer” and other effects associated with high SES schools and the withdrawal of such students from schools with lower SES students. (3) Despite assertions that costs of private schools are considerably lower than those of public schools, a comparison of costs in the Milwaukee Public Schools with private, voucher schools suggests that the costs of comparable services at public and private schools appear to be similar. But, the shift from the existing system of providing public education through school districts to a market system of educational vouchers would require considerable additional public resources for a supportive infrastructure that would provide an effective system of choice and competition. In particular, costs would rise because of public subsidies for existing private school students, recordkeeping for each voucher student, school monitoring, school accreditation, student transportation, information systems on alternatives, and adjudication of disputes. Preliminary estimates suggest an excess public cost on the order of $75 billion per year nationally, about 25 percent of present spending, about $ 1,500 per year per student.
EDUCATIONAL VOUCHERS: EFFECTIVENESS, CHOICE, AND COSTS
INTRODUCTION
Since Milton Friedman proposed his original voucher plan some four decades ago (1955) with a wider dissemination in his important book on Capitalism and Freedom (1962), the idea has taken on more and more credibility. Frustration with public schools in the inner cities has been a particularly important reason for emerging support of vouchers. Yet, both advocates and detractors tend to argue more from theoretical and ideological grounds than empirical ones on the consequences of vouchers. The purpose of this article is to consider empirical evidence concerning three issues on which there have been strong views expressed in the policy arena. (1) Will vouchers improve student achievement? (2) Who will choose and what are the educational consequences? (3) What is the evidence on comparative costs of public versus private schools and on the costs of a voucher system?
It is only fair in addressing these types of issues that I clarify where I stand on vouchers. Almost thirty years ago (Levin 1968) I argued that the situation of inner city students is so dire that we ought to be willing to design good experiments with vouchers or vouchertype mechanisms to ascertain their effects on both individual and societal outcomes. In subsequent publications (e.g. Levin 1980, 1987) I have argued that the specific design of a voucher system with respect to finance, regulation, and information will be crucial in determining specific outcomes rather than leaving the discussion at a generic and abstract level, a point that is also stressed by Moe (1995) and Murnane (1986). More recently (Levin 1991) I have suggested that the private benefits of vouchers are likely to be positive relative to the present system in terms of satisfying narrow consumer preferences, but the social consequences will be worse because of greater inequality and the further deterioration of a common educational experience as social goals of schooling are sacrificed to consumer sovereignty. In what follows I will not take a stand on vouchers as much as try to read the present evidence on the three issues set out above.
DIFFERENCES IN ACHIEVEMENT BETWEEN PUBLIC AND PRIVATE SCHOOLS
To a large degree the arguments for educational vouchers have been premised on whether they will improve the educational achievement of students, particularly students from poverty backgrounds and innercities where school results are considered to be particularly woeful. Because student achievement is considered to be a universal goal of schools, it has become the sine qua non for evaluating school reform. We should begin this section by explaining how limited this focus is in the context of market choice. The rationale for market choice in education is to give families the freedom to pursue their own educational preferences. For some families academic achievement will be the prime goal; for others it will be a school environment that is safe and supportive; for others yet it will be a quest for educational reinforcement of religious or philosophic values. Although most families may have some concern for the academic dimension, it may not be the prime dimension and may even be overwhelmed completely by other school and family goals as systematic studies have shown (Echols and Willms 1995). For example, in evangelical Christian schools it appears that preparation for the Kingdom of God far outweighs concerns about academic achievement (Peshkin 1986). So, comparing the effectiveness of schools only on student achievement is not fully consistent with measuring the impact of vouchers on educational outcomes where families may choose schools according to many criteria. And, even as a measure of social outcome, achievement tests are a limited and highly imperfect sample of the range and depth of knowledge and skills, values, attitudes, and other behaviors that we expect schools to inculcate in the young (Inkeles 1966).
Comparisons of Public/Private Student Achievement
Nevertheless, the primary focus in comparing public and private schools—even in the absence of vouchers—has been to ascertain whether either sector has an advantage in achievement, net of differences attributable to differences in student characteristics. It should be noted that controls for selfselection are problematic in that even when controlling for race and indicators of social class of students, families who choose private schools and make a financial effort to pay for them are likely to be more educationally motivated than those who do not. Therefore, we would expect students from such families to have higher achievement than similar students who do not make the efforts to switch from a public to a private school. Whether one can control statistically for this selfselection effect is problematic (Witte 1992).
The first major study by Coleman et al. (1982) compared a crosssection of 10th grade students in public and private (mainly Catholic) schools, controlling for race and socioeconomic background. They found that students in private schools had slightly higher achievement, from .12 to .29 standard deviation units, depending upon the test. But, their results were criticized as overstatements of the private school effects because of inadequate controls for selection bias and other problems in the statistical design (Goldberger and Cain 1982). Purported adjustments for some of these problems reduced considerably or eliminated the private school advantages (Willms 1983).
Longitudinal results based on sophomoretosenior changes found smaller private school advantages, from a range of no difference to .1 of a standard deviation in achievement (Alexander and Pallas 1985; Haertel et al.., 1987; Hoffer et al., 1985; Willms, 1985). This effect is statistically significant, but small, amounting to only about 10 points or less on the Scholastic Aptitude Test for college admissions, a trivial advantage. Further, it means that the achievement overlap between the two sectors is so great that 46 percent or more of public school students have higher achievement than the average private school student who is statistically similar (Levin 1987:634). Using earnings equations for 1976 data (the achievement data were collected in 1980), such an achievement advantage translated into earnings gains of less than 5 cents an hour for high school graduates some 4 years after graduation and about 1 day less unemployment a year among a cohort that experienced about 50 days of annual unemployment.
More recent statistical studies have also found no differences in achievement or only minimal differences. The most sophisticated studies from a modeling and statistical perspective are those of Goldhaber (1996) and Gamoran (1996). Goldhaber uses the (NELS 88) data set and finds no difference in achievement between comparable students in public and private schools (Goldhaber 1996). Gamoran’s use of the same data set with a different statistical technique, hierarchical linear modeling, also finds no achievement difference or a very slight private school advantage, depending upon which statistical formulation is used. In the few cases where differences are found in favor of private schools, the advantage is not even as large as the trivial differences cited earlier.
Even when differences are found in such publicprivate achievement studies, they are often questionable. For example, Sander (1996) found no difference in achievement between public and Catholic schools for students who attended Catholic schools for 17 years, but an advantage only for those who had attended Catholic schools for 8 years. Not only is it puzzling that the putative Catholic school advantage takes eight years to “bloom” with nary a hint of a bud in the earlier years, but even this result is questionable because it is not based upon an equivalent public school comparison group. When restricting the finding only to those who have attended Catholic school continuously for eight years, it is necessary to compare achievement with an equally stable public school group of students who have not been mobile. School stability has been found to be an important correlate of school success in the general literature (Rumberger and Larson 1996). But no attempt was made to compare the achievement of students with 8 years of Catholic school (presumably most attending the same school) with a comparable group of public school students who attended the same public school. An appropriate comparison would have been to compare students with the same stable attendance patterns between the two sectors to net out school effects.
Another recent study that models existing data sets to estimate effects of vouchers, Hoxby (1996), finds that a statistical proxy for private school subsidies was associated with both greater school competition and both higher private and public school achievement, findings that reinforce the textbook version of competition. However, as it turns out her model is based upon arbitrary assumptions which, when relaxed, can yield exactly the opposite conclusions (Kane 1996). That is, the results are not robust under a range of plausible assumptions on model construction and interpretation.
The Milwaukee Experiment
Of course, none of the publicprivate comparisons can be as instructive as the direct evaluation of a voucher intervention. There are a handful of vouchertype mechanisms funded by private sources, but none has been subjected to a careful evaluation of achievement effects (Moe 1995a). The only attempt to assess directly the impact of vouchers on student achievement has been the Milwaukee Voucher experiment. That experiment allowed students from families with incomes no more than 1.75 times the poverty line to attend private nonsectarian schools in Milwaukee with public funds. The numbers of participants were limited to no more than 1 percent of Milwaukee Public School enrollments except for the fifth year of the program when the limit was raised to 1.5 percent. Some seven schools participated initially, rising to 12 in the last two years. September enrollments in the private school program rose from 341 in 199091 to 830 in the 199495 school year, considerably below the maximum number eligible to participate which varied from 931 in the first year of the program to 1450 in the fifth year. Attrition rates from yeartoyear were considerable, varying from 46 percent in the first year to 28 percent in the fifth year, so relatively few students participated for three years or more (Witte, Sterr and Thorn 1996:Table 1)..
Witte and his associates compared student achievement and found no systematic differences between voucher students in private schools and statistically similar students in the Milwaukee Public Schools. Recently their findings were challenged by Jay P. Greene, Paul E. Peterson, and Jiangtao Du (1996). These authors argued that since oversubscribed schools had to randomly choose students from their applicant pool, these conditions “allowed for a natural randomized experiment.” They then compared students who had been chosen to participate with those in the applicant pool who had not been chosen. In short, they found that private school voucher students in their first two years had achievement levels that were not different from nonaccepted applicants who were in the Milwaukee Public Schools. However, they found that voucher students in the third and fourth years of participation scored higher than the general pool of nonselected students. They concluded that “Students benefit in measurable ways from the choice experience only after participating in the program for three or more years (Greene, Peterson, and Du, 1996:13).”
Although it can be argued that the students who entered the voucher schools were equivalent for comparative purposes to the nonselected students, it cannot be argued that third and fourth year students were equivalent to the control group of nonselected students. In fact, attrition rates were approximately 30 percent annually. Attrition students had lower test scores than those who continued to participate in the voucher schools (Witte et al., 1994, Table 1.8, p. 23). That is, consistent with the general literature on school mobility, in that students who persisted in the same school were superior to those who moved back to the Milwaukee Public Schools. The Peterson et al . analysis, then, compares the stable group who persisted for three or four years in the same school (superior in achievement to those who did not persist) with all nonselected students. The persistent voucher students were a superior subset, not a random subset, of the original applicant pool. Therefore it is invalid to compare them with the original nonchosen group and to conclude that the higher achievement scores of voucher students in their third and fourth year were due to a schooling effect. In fact, it is probable that Witte has overstated the comparative achievement of the choice students in the third and fourth years by not providing a statistical control for mobility to compare only with public school students who had been in the same school for three to four years. A more elaborate analysis of this comparison using instrumental variables and Heckman adjustments for selfselection finds no difference in achievement, and the “effect” that Peterson, Greene, and Du report is created by a dramatic loss in both numbers of students and achievement scores of remaining students in the control group rather than a comparative rise in achievement of those in choice schools (Witte 1996).
Differences on Other Outcomes
My own reading of the body of studies comparing student achievement in public and private schools is that there is no difference for equivalent students or that differences are trivial. In other respects there may be a private school advantage for some groups. Although Sanders (1996) found no difference in achievement between public and private schools for Hispanics and AfricanAmericans with statistical controls for family background, Neal (1995) found that urban minority students attending Catholic secondary schools have considerably higher graduation rates than comparable public school students and higher college graduation rates. He attributes these results to the particularly poor public schools that are available to this group of students. It should be noted that Bryk and Lee (1988) found that students in Catholic high schools are more likely to be assigned to an academic track than in public high schools. Neal estimated that the greater educational attainments of Catholic School minority students in urban settings lead to earnings that are eight percent higher than for comparable students in urban public schools. Evans and Schwab (1993) also found greater high school graduation rates and college enrollment rates for Catholic school students. Finally, as expected, parental satisfaction with the schools their students are attending seems to be consistently higher for parents of students enrolled in private schools relative to prior public schools attended (Witte et al. 1995; Moe 1995a).