COST EFFICIENCY IN THE UNIVERSITY. A DEPARTMENTAL EVALUATION MODEL

Víctor M. Giménez García ()

José Luis Martínez Parra ()

Departamento de Economía de la Empresa

Universitat Autònoma de Barcelona

08193 Bellaterra (Barcelona)

Tel. 93 581 12 09

Fax. 93 581 25 55

ABSTRACT

This work presents a model for the analysis of cost efficiency within the framework of data envelopment analysis models. It calculates the cost excess separating a unit of production from its optimal or frontier levels, at the same time as decomposing these excesses into four explanatory factors: a) technical inefficiency, which depends on the quality of the factors consumed, the type of organisation and also on the behaviour of the human factor; b) the composition of the variable factors, with their impact on potential cost savings; c) the availability of the fixed factors along with their level of utilisation; and finally d) the scale or size of the unit of production. The empirical application is carried out on the departments of the Autonomous University of Barcelona. The results show that department costs could be reduced on average by more than 11% in the long term.

JEL Classification: C61, H52, I21

Key words: Cost efficiency, higher education, Data Envelopment Analysis, university quality

1. INTRODUCTION

In recent years evaluating and improving efficiency has taken on particular importance both in private and public organisations. But it is in these latter organisations where it is particularly relevant due to constraints on their administration or their lack of resources, and in view of the rationalising trend in public expenditure currently underway in the majority of developed countries.

Spanish universities, like other public organisations, have not been unaffected by this situation. In particular, the environment in which they operate has changed markedly since the Organic Law on Universities in 2001 brought reform to the sector. Improving quality has become a priority in the new legal framework, which, combined with the budgetary constraints already faced by the universities for some years, have presented universities with a new and more complex framework in which to operate. They will henceforth require new administrative instruments capable of providing institutions with tools for allocating scarce economic resources more efficiently between departments. Likewise, these instruments should also prove useful in providing university administrators with unequivocal criteria for evaluating and subsequently improving the performance of the operational units – the departments.

In this current work we propose an instrument of departmental evaluation in terms of costs, which can be used to determine improvement targets both in costs and levels of production and quality. With this purpose, we present a new model of cost efficiency based on the Data Envelopment Analysis (DEA) methodology initially developed by Charnes, Cooper and Rhodes (1978). The specific objectives of this model are to determine: a) the overall deviation in costs of each department under analysis, to be understood as the difference between the observed cost and the optimal long-term cost assuming an optimal scale; b) the factors into which this deviation can be decomposed, distinguishing between technical inefficiency, the incorrect allocation of factors, the level of utilisation of the available fixed factors or the appropriateness of the scale adopted; c) targets for the different inputs and outputs that would lead to cost efficiency in the departments; and d) which inefficiencies can be corrected in the short term, and which in the long term.

The complexity of the productive process in universities is due to the fact that they combine diverse resources to produce numerous products and services, many of which are unquantifiable in monetary terms. The traditional use of a series of indicators to evaluate the university sector is inappropriate since it collects partial and isolated measures, ignoring the myriad of interconnections between the resources employed and the large number of outputs of the system (Kells, 1991). Some representative studies using such indicators to evaluate universities are those of Ball and Halwachi (1987), Jarratt (1985) or Cutt, Trotter and Lee (1993).

One of the main advantages of the frontier models is their ability to evaluate the overall efficiency of a group of units from the inputs consumed and the outputs produced. This is the reason they are particularly appropriate to apply to sectors with complex productive processes, such as universities, where there is a lack of information about output prices indicating the direction in which to evaluate the performance of each decision making unit (DMU). Under this perspective, frontier models offer a valuable and objective tool for evaluating the public sector, and one that is supported by an extensive literature (see, for example, Grosskopf, Margaritis and Valdmanis, 1995; Ray, 1991; Pedraja-Chaparro and Salinas-Jiménez, 1996; Drake and Simper, 2003; Bifulco and Bretschneider, 2001).

However, merely estimating technical efficiency as a criterion of evaluation and control is generally not enough in the public sector. It is not sufficient even in a sector where administrators have traditionally neglected the conventional monetary evaluation criteria of the market economy, such as profit and profitability. The fact that it is the public sector does not necessarily mean that economic criteria should not or cannot be employed. In this sense, governments, regardless of their political colour, are increasingly administering the public sector more efficiently, effectively and economically. Their aim is to cut public spending without this leading to losses in services or poorer quality. The use of monetary-based criteria in the public sector generally implies an analysis from the cost perspective, since there are usually no selling prices, or these are public. The increasing demand for control and efficiency in the administration of monetary resources makes one doubt whether technical efficiency is the most appropriate evaluation criterion. In fact it is not, as is well known: behaviours that are economically undesirable can be considered technically efficient, such as, for example, not minimising costs. Thus, in this work we present a model of cost efficiency rather than one based exclusively on measuring technical efficiency.

DEA models offer a powerful methodology for university evaluation that is supported in various studies in the literature, although generally with different aims to ours. We should mention among these numerous works, that of Beasley (1990) where 52 chemistry and physics departments of British universities are analysed for the academic year 1986-87. The author uses a measure of research quality based on four levels of research excellence. In subsequent research Beasley (1995), using the same inputs and outputs as in the earlier work, proposes a model of teaching and research efficiency without losing a global perspective of the productive process, which is a characteristic of frontier models. Athanassopoulos and Shale (1997) analyse the efficiency of British university institutions in terms of costs and outcomes, using performance indicators for the academic year 1992-93. Another interesting example is the research of Sarrico, Hogan, Dyson and Athanassopoulos (1997), where they develop a model aimed at guiding students’ university choices, as an alternative to the official ratings list. Johnes and Johnes (1993) analyse research efficiency for the period 1984-88 in 36 economics departments in British universities using the original DEA model of Charnes, Cooper and Rhodes (1978) in ratio form. One of the main advances of this study is its use of hierarchical cluster analysis to study the stability of the results obtained according to various specifications. Madden, Savage and Kemp (1997) analyse the effects of the reform of higher education in Australia in the late 1980s, using data from the years 1987 and 1991 – i.e., before and after the reform. Ahn (1987) evaluates the efficiency of higher education in a group of US universities. The productive process defined in this work consists in the use of capital and personnel (inputs) to produce two outputs: teaching and research. Among the works based on dynamic analysis subject, we should remark the one made by Glass, Mckillop and O’Rourke (1998), in which they analyse the in productivity change occurred in British universities in the period 1989-92 using the Malmquist index. Apart from the inputs and outputs selected, they include a measure of research quality. Finally, there is a group of more recent research combining two quantitative techniques for decision making: DEA and multi-criteria decision making (MCDM). Among these, we might mention: Caballero, Galache, Gómez, Molina and Torrico (2001; 2003); Post and Spronk (1999); Li and Reeves (1999); Korhonen (2000) or Korhonen, Tainio and Wallenius (2001).

The rest of this article is organised as follows: in the second section we describe the evaluation model in which this work is focused. The characteristics of the sample and the variables description are presented in Section 3. After this, in Section 4, we present the most relevant results we have obtained in our application of the model to the departments of the Universidad Autonoma de Barcelona (UAB). Finally, we show our most significant conclusions.

2. MODEL

As we said in the previous section, the main goal of the model is to determine the difference between the observed total costs of a department “k” and the total frontier costs that should be achieved if costs were minimized at long term and the optimal scale performance was also achieved. We will call this difference, total cost deviation (TCV). Consider the case of k=1,...,K university departments. The productive process of a department k is characterised by the production of a group of i=1,...,I outputs yk=(yk,1,...,yk,I) with quality levels in q=1,...,Q dimensions Qk,q=(Qk,1,...,Qk,Q) starting from f=1,...,F costs adjustable in the long term Ck,f=(Ck,1,..,Ck,F,) and v=1,...,V costs adjustable in the short term Ck,v=(Ck,1,...,Ck,V). Consequently, the observed total costs of a department k for a level of production () and of quality (Qk,q), is represented by

The model decomposes the total cost inefficiency (TCVk) of department k into four factors: technical inefficiency (TEVk), allocative inefficiency (AVk), fixed factors utilisation (FCVk) and scale (SVk). This is expressed as follows:

[1]

where:

[2]

is the total cost that the DMU k should achieve if it was technically efficient; is the variable cost associated with the technical efficiency;

[3]

is the total cost that the DMU k should achieve if it was efficient at the short term – i.e., given the observed level of fixed costs; is the level of associated variable costs;

[4]

is the total cost that the DMU k should achieve if it were efficient in the long term – i.e., if it was able to change the input mix, fixed as well as variable, but maintaining a similar scale (assuming variable returns to scale); and are the associated levels of variable and fixed costs, respectively;

[5]

is the cost that the DMU k should achieve if it was at long term cost efficient and able to adapt its scale to the optimal size; and are the associated levels of variable and fixed costs, respectively.

The scale deviation (SVk) shows the excess in costs due to differences between the average cost of the activity that minimises the costs globally and the frontier value relative to the level of production of the DMU k. This deviation only includes effects of scale and does not assume any type of inefficiency in the utilisation of the factors, either fixed or variable. The SVk mathematical expression is:

[6]

The deviation in fixed factorsutilisation (FCVk) represents the deviation in total costs explained by differences between the requirement for fixed factors, assuming a situation of long-term equilibrium (optimal scale, given the level of activity) and the minimum short-term cost:

[7]

The total costs deviation due to inefficiencies in the composition of the factors employed is measured by the allocative deviation (AVk). In other words, it captures a non-optimal package of factors considering their prices. This inefficiency may be explained by different reasons, as incorrect decisions about the mix, the fact that public organisations may have limited margin to minimise costs owing to their lack of financial resources for taking decisions aimed to correct an inappropriate factors mix. Their expression is:

[8]

Finally, the technical inefficiency deviation (TEVk) measures the increase in total costs caused by an excessive consumption of factors. This may occur in situations of management incompetence, errors of organisation or lack of incentives, which can be explained partially by the lack of competition, according to Leibenstein (1966) and his X-efficiency theory. It is given by:

[9]

Calculating the above deviations requires first knowing the intermediate optimal total costs, denoted TC. For each one, we use an ad hoc DEA model. is calculated by solving the following linear program for each department:

[10]

The linear program [10] is very similar to the usual formulation of DEA programs for calculating technical efficiency. The objective is to determine the maximum radial reduction of the variable costs maintaining the observed levels of production and quality. Quality has been introduced as an output (Olesen and Petersen, 1995; Adler and Berechman, 2001), assuming strong disposability since its reduction is assumed to be free (Dismuke and Sena, 2001). This restriction is applied in order to carry out an evaluation assuming variable returns to scale (VRS) to eliminate the potential scale effect, ensuring that departments of similar size are being compared.

Once the optimal value is calculated, the cost associated with the technical efficiency is easily obtained by replacing the observed variable costs in the total costs function with

To determine the short-term frontier cost , the following DEA model is used:

[11]

The main difference with respect to [10] in this case is that we determine the levels of the variable costs () that minimise the short-term total costs of each department. The assumption of VRS is maintained in this case.

The frontier level of the long-term costs without scale changes is obtained from the following program:

[12]

The main difference compared to [11] is that we determine the levels of both the variable costs and the fixed costs that minimise the long-term total costs.

Finally, the minimum total cost , assuming a technological environment with constant returns to scale (CRS), is given by:

[13]

3. SAMPLE AND VARIABLES

The empirical application of the model described above was carried out on departments of the Autonomous University of Barcelona (UAB). The UAB consists of a total of 46 departments, although we had complete information for only 42 of these. The data obtained corresponded to the period 1996-98. We chose a period of more than one year essentially because scientific research requires a development process lasting more than one year. Specifically, we decided to use a three-year period because this is the duration of most of the projects of more than one year supported by the different institutions and organisms that fund most public research in Spain. A period of more than one year also ensures that potential anomalies in the data of one single year will not distort the results.

The DEA methodology is not appropriate for analysing heterogeneous units. In our case, it is notable that those departments with a high level of experimental research require bigger and more specialised installations to carry out their normal research and teaching work, implying more financial resources for investment and maintenance than the other less experimental departments. This aspect has been considered in other studies focusing on Spanish universities, although with different objectives from ours, such as those of Caballero, Galache, Gómez, Molina and Torrico (2003). For that reason we will use the degree of experimental activity as homogenisation criterion. In this sense, the UAB assigns each department an experimental level that is used as a criterion for allocating funds between departments and for deciding on the cost to the students of each credit at registration. This level distinguishes between departments of a high level and those of a low level of experimental activity. In Table 1 we can see that the operational costs of experimental departments in the UAB are two times bigger than the less experimental ones, because of laboratories and specialised equipment for teaching and research required for the former. Consequently, and to ensure a fair comparison, we have followed the procedure proposed in Banker and Morey (1986). Specifically, we evaluated the two types of department together to obtain efficiency indices for the departments of low experimental activity, while the experimental departments were evaluated in isolation to obtain their efficiency indexes.

After reviewing the literature on nonparametric frontier models applied to higher education, we concluded that there is no agreement with regards what inputs and outputs to use to evaluate efficiency in university departments. However, there is a common denominator in the majority of the studies we have reviewed: the departments’ main activities are teaching and research. We have choose a double approach about this two items; first a quantitative approach (production ) and second a qualitative approach because we believe that both views should be considered as one if you are trying to obtain a reliable image of reality.

In order to choose the outputs, we presented a list of the most used outputs in the literature to a group of university administrators. After several interviews guided by the authors, and taking into account the available information, the following outputs were finally selected as the most representatives of the university productive process in the UAB:

  • Articles in national and international journals (O1). This variable measures the total number of articles published in prestigious scientific publications that are significantly placed in the international citation lists for each field.
  • Books and book chapters (O2). Unfortunately, the database we had available did not allow us to treat these two variables separately. The administrators of the database justified grouping the two variables together by arguing that in Spain has traditionally been accorded that both, books and book chapters, are less prestigious than articles published in prestigious scientific journals. That’s why it was more practical not to distinguish between publishing an entire book or just a chapter.
  • Teaching load (O3). This variable measures the teaching load that the departments have according to the several courses they lead. The university publishes this information expressed as full-time equivalent lecturers (FTE)[1].

The first two outputs (the one’s concerning to scientific production ) analyze all the production made during the 1996-98 period because we believe that account only a single year production would be an error considering that some times the maturation process of an investigation takes more than a year. Specifically, we decided to use a three-year study period because this is the usual duration of the projects funded by public administration in Spain.