Evaluating Air Navigation Service Efficiency of European Airports utilizing DEA

Steffen Hoffmann, Andreas Dellnitz, Andreas Kleine,
FernUniversität in Hagen (University of Hagen), Hagen, Germany

Rainer Kölle, EUROCONTROL, Brussels, Belgium

Abstract

The rising air traffic volume in Europe, and beyond, is demanding. Accordingly, the need for safe but also efficient air traffic management asks for approaches to evaluate service productions by more than just univariate measures. Data Envelopment Analysis (DEA) is a non-parametric method to assess the efficiency of organizations and processes, so-called Decision-Making Units (DMUs).The major advantage of DEA is to reduce multivariate data to a single key performance indicator (KPI). However, this KPI is just one element to analyze the (economic) situation of a DMU, as shown in the contribution at hand. While almost all DEA applications in the field of air transport refer to airports or airlines as complete units, we concentrate on the efficiency measurement of air navigation services (ANS), especially thearrival and departure phases.The set of DMUs comprises 32 major European airports. Due to a considerable amount of data for potential DEA studies of ANS, this contribution discusses the process of data-selection in a first step; the Performance Review Unit of EUROCONTROL provided all data. In a second step, graphical projections of the DEA results are determined by applying the multivariate method of Multidimensional Scaling (MDS). Finally, possible interpretations of underlying latent variables are given. Thus, the proposed approach provides new insights: for air navigation service providers as well as for authorities, i.e. for regulatory purposes. With regard to a liberalized market for air navigation services in Europe, the overall project goal is to develop a powerful tool for ANS regulation.

Introduction

Data Envelopment Analysis (DEA) is a non-parametric method for evaluating the efficiency of organizations and processes. Classical DEA useslinear programming, and the solution of respective optimization problems is straightforward. Since the DEA debut in 1978, many applications in different sectors have emerged, especially in the field of regulation, i.e. for power grids. Reducing multivariate data to a single key performance indicator (KPI) is one of the most eminent advantages of DEA. Beyond that, DEA results offer much more information about a single Decision-Making Unit (DMU) or the whole set of DMUs. This information then might helpto determine similarities or dissimilarities between all DMUs.

A multivariate method – called Multidimensional Scaling (MDS) – is a tool for generating low-dimensional projections of high-dimensional data. Such low-dimensional data can be used to identify common features and differences in ANS. In this contribution, therefore, MDS will be applied to assessDEA results of Air Navigation Services (ANS) in the EUROCONTROL area and, eventually, to detect similarities or dissimilarities between all DMUs, i.e. all Air Navigation Service Providers (ANSPs).By visualizing the projected data, one can explore such relationships intuitively as well as analytically. This might enable the analyst to discover valuable knowledge for decision support.

The motivation for assessing ANSPs originates in the demanding riseof air traffic volume in Europe and beyond. Moreover, the need for safe but also efficient air traffic management (ATM) asks for approaches to evaluate the service production by more than just univariate measures, as most conventional KPIs.At hand and for the first stage of the overall project, we focus on the arrival and departure phases. Furthermore, the set of DMUs comprises ANS at 32 major European airports. For this purpose, the Performance Review Unit of EUROCONTROL provides all data. Visualizing the results and proposing a first intuitional method of analysis canhelp a DEA non-professional to explore the considerable amount of data. Therefore, the proposed approach delivers new insights: for air navigation service providers as well as for authorities, i.e. regulatory purposes. With regard to a liberalized market for air navigation services in Europe,it is noteworthy that our overall project goal is to develop a powerful tool for ANS regulation.

The paper is organized as follows: After this introductionand as key aspect of second section, we discuss the dataset.The third section presentsthe technical principles of the applied methods, namely DEA and MDS. Afterwards, section four is dedicated to the ANSP evaluation, including a debate about some properties and possible latent variables. Finally, the paper concludes with a short summary and provides an outlook on future prospects.

Dataset

Literature Review

Efficiency evaluation, especially by utilizing DEA, is a common practice in aviation. Besides assessing airlines and air-transportation, overall airport-efficiency measurement is one major concern here. While the former evaluations primary focus on particular (business-)processes[1], this is – to some extend – unclear for the latterone. Due to being conglomerates of variousprocesses, the scope of efficiency evaluations of airports spreads widely, and hence, the relevant data set, which should be used, is rather ambiguous. Fasone and Zapata-Aguirre confirm this by reviewing 15 years of business performance measuring in the airport context [2]. Their literature review shows a quite heterogeneousselection of inputs and outputs. For example, different parts of airports, especially processes of airside and landside, are mixed-up.In contrast, the study of Bilotkach et al. [3] covers cost-efficiency benchmarking of European ANSPs. Again, rather than focusing on one part of ANS, the studied data involves enroute movements as well as airport movements. The resultis valuable knowledge about the development of productivity regarding costs of ANS, while it is hard to derive recommended actions for deciders.Moreover, as the rising air traffic volume in Europe, and beyond is growing fast, the need for new approaches to evaluate and analyze specific service productions is quite obvious. Thus, it is subject of our first project stage to focus on special processes of ANS such as the capability of ANSPs at airports. Although it is clear, that each ANSP has to face different local issues, which may induce inefficiency.Therefore, the emphasis of this contribution will be on learning about whatmake ANS efficientand also what circumstances encourage or which of them tend to hamper. However, the process of efficiency evaluation of ANS by DEA is just an initial step within the journey towards an ANS regulation.

Selecting Data

The dataset ofthis contribution includes the ANS of 32 major European airports. With progress of the project, we plan to extend the list by yet missing pertinent ones. Table 1 depicts the set of airports at hand. For detailed data, please refer to [4,5]. Beside ICAO-code for identification purposes, the list comprises the number of usually serviceable runways for takeoff and landing. The runway data is used for categorizingthe European Airports. A categorization of airports is necessary because– in principle – more runways allow for more flexibility in handling incoming and outgoing traffic. In the future,we will check also for other classification methods or categorization approaches.

Here, we actually focus on arrival and departure phases of ANS at the aforementioned airports. While both phases are independent of each other, we propose to analyze them independently. As will be explained later in the methodologicalsection, we observe production processes. This means in short, an ANSP transforms inputs into outputs. Beside shared technical requirements like being cardinal numbers, the main reason for assignment as input or output originates in an economic point of view:Concisely, less input related to the same output is ratedbetter, whereas and vice versa, more output related to the same amount of input is better as well. Regarding the selection of data within this exemplification, we confine ourselves to this definition; for details on requirements of properties, please refer to [6,7].

Table 1. Evaluated Airports

ICAO
Code / Airport /
location / runways / cat
arr. / dpt.
EBBR / Bruessels / 6 / 6 / 1
EDDB / Berlin-International / 4 / 4 / 2
EDDF / Frankfurt/Main / 6 / 5 / 1
EDDH / Hamburg / 4 / 4 / 2
EDDK / Cologne/Bonn / 6 / 6 / 1
EDDL / Dusseldorf / 4 / 4 / 2
EDDM / Munich / 4 / 4 / 2
EDDP / Leipzig / 4 / 4 / 2
EDDT / Berlin-Tegel / 4 / 4 / 2
EDDV / Hannover / 3 / 3 / 2
EFHK / Helsinki-Vantaa / 6 / 6 / 1
EGGW / London/Luton / 2 / 2 / 2
EGLC / London/City / 2 / 2 / 2
EGLL / London/Heathrow / 4 / 4 / 2
EHAM / Amsterdam/Schiphol / 10 / 10 / 1
EPWA / Warszawa Chopin / 4 / 4 / 2
ESSA / Stockholm-Arlanda / 6 / 6 / 1
GCLP / Gran Canaria / 4 / 4 / 2
GCTS / Tenerife Sur / 2 / 2 / 2
GCXO / Tenerife Norte / 2 / 2 / 2
LEAL / Alicante / 2 / 2 / 2
LEBL / Barcelona / 5 / 5 / 1
LEMD / Madrid / 4 / 4 / 2
LEPA / Palma de Mallorca / 4 / 4 / 2
LIMC / Milano Malpensa / 4 / 4 / 2
LIML / Milano Linate / 3 / 4 / 2
LIPZ / Venice / 4 / 4 / 2
LIRF / Rome Fiumicino / 7 / 8 / 1
LOWW / Vienna-Schwechat / 4 / 4 / 2
LPPR / Porto / 2 / 2 / 2
LSGG / Geneva / 2 / 2 / 2
LSZH / Zurich / 4 / 5 / 2

Furthermore, we determine relevant inputs and outputs for two separated processes: an arrival scenario and a departure scenario. On the subject of the arrival scenario (ARR), we select two inputs: corehours ultimate IFR arrival capacity (CAPAA, flights) and annual service volume (ASV, flights) as well as three outputs: airport arrival ATFM delays (AAADLY, minutes), arrival sequencing metering area additional time (ASMAT, minutes) and total IFR arrivals (IFRARR, flights). The departure scenario (DPT) contains also two inputs and three outputs: core hours ultimate IFR departure capacity (CAPAD, flights) and annual service volume (ASV, flights) on the input side and taxi out additional time (TXOAT, minutes), ATC pre-departure delays (APDDLY, minutes) and total IFR departures (IFRDPT, flights) on the output side.CAPAA and CAPAD are limitations predefined by regulatory authorities, e.g. defined for limiting local noise pollution. As both are potentials, they count for input. Likewise, ASV is a potential, too. The ASV depends much on technical and infrastructural capacities. Thus, we select ASV for the input side as well – and in this case for both scenarios, while there is no differentiation by the database. As the outputs AAADLY, ASMAT, TXOAT and APDDLY are delays, this is contradictory to the aforementioned requirement for output selection “the more, the better”.However, while delays are an outcome of the production process and thus, in any case not an input, we have to handle this special case of so-called bad orundesirable outputs in an even special way. We solve the conflict by setting an upper bound for each delay and every airport. Accordingly, the number of flights multiplied with the upper bound defines a worst case or target. This bound should not be exceeded at the respective airports. Subsequently, we subtract the actual delay from the bound and use the modified bad output then as a normal behaving output: the less the difference from the target, the more the output and hence, the better. To determine the upper bound, several approaches may apply. Here, we use the mean value of the delay per flightin multiple periods, i.e. due to availability of data up to four periods. This follows the assumption to reduce delays in amedium-term. Just in case where the converse holds true, the lower bound is a non-archimedean ϵ – a small number near zero. Regarding the output AAADLY, we consider only delays[1], which are controllable by the local ANSP. The last in line are the outputs IFRARR and IFRDPT. As both are an outcome from the production process of ANS itself, they count for the output side.

Methodology

Data Envelopment Analysis

Production Process

Data Envelopment Analysis (DEA) is a non-parametric method for measuring the efficiency of a Decision-Making Unit (DMU) relative to a given set of DMUs. The method was introduced by Charnes, Cooper and Rodes in 1978 [7]. The work of Charneset al. bases on the fundamental research of Farell, which goes back to tshe late 1950th [8]. Regarding the efficiency of a DMU Farell distinguished technical fromallocative efficiency. The difference between both is worth to note: Technical efficiencydetermines the capability of a DMU to maximize the output from a preexisting amountof input. Allocative efficiency considers the ability to do this in an optimal proportionwith regard to prices. A DMU can be a firm as well as a non-profit organization, adepartment, a machine or a process – and in the present case, an ANSP at an airport. This is due to the fact, that DEA can handle multidimensional data – physical as well as financial numbers [7]. However, it is required that theDMUs are similar to each other with respect tosuch data. Thus, DMUs – here ASNPs – consume the same setof inputs and transform it to the same set of outputs, whereas the amounts may differ.The transformation process of each ANSPis unknown in detail. Differences in efficiency are caused by individual process skills. Hence, the transformation follows a black-box principle [7] as depicted in Figure 1 for the “Arrival-“ and in Figure 2 for the “Departure-Scenario”. Subsequently, we will denote an ANSP synonymously to the airport where it operates, regardless the parent company the ANSP belongs to.

Figure 1. Black-box Principle “Arrival”

Figure 2: Black-box Principle “Departure”

In general an efficiency score (θ) is determinedby the proportion of output per input:

(1.1)

Most classical ratio systems limit this to a single input/output case. To extend theapproach to the case of multiple inputs and/or outputs it is common to introduceweights (virtual prices) vifor each input xi (i=1,…,m) and urfor each output yr (r=1,…,s).Thus, each input and output is specifically weighted – so-called virtual input andvirtual output [9]:

(1.2)

However, it poses a challenge to select appropriate weighting vectors u and v, especially when e.g. market prices are unknown. DEA meets the challenge by computingthe weights, applying linear programming. Therefore, the efficiency score (here: θo) forthe observed ANSPo is maximized while evaluating it against all ANSPs j (j=1,…,n) as shown within the next section.

Basic Model

In this paper, we restrict the exemplification to the basic model implying constant returns to scale (CRS). This equates to the CCR-model proposed by and named after Charnes, Cooper and Rhodes [7]. CRS means that output always increases by the same proportion of increased input. Implementing model(1.2) in DEA would lead into a computationally intensive fractional programming problem. Thus, it is common practice utilizing the Charnes-Cooper Transformation to convert it into a linear problem [10]:

(1.3)

(1.4)

(1.5)

(1.6)

ANSPswithan optimal θo=1 are called efficient under CRS, otherwise inefficient.The output valuation may never be greater than the input valuation; this is what constraint(1.4) is about. (1.5) is a result of the aforementioned linearization process. As can be seen in constraint (1.6) weightsu and vmustbe greater than the non-archimedean infinitesimal ϵ– a smallnumbernearby 0. First, this is for the reason that weights should not be computed to 0 forrealistic assumptions [11]. Second, an important halo effect is that ANSPs with an optimal θo=1 are not only technical efficient but also Pareto-efficient. Hence, there does not remain any input excess or output shortfall for efficient ANSPs.

Categorization

As mentioned before, we allow categorization to consider advantages or disadvantages within classes of airports, however, the ANSPs in operation.In particular, we want to handle hierarchical categorization [12] of airports. Assume we observe nANSPs of ccategories, whereas each ANSPs j (j=1,…,n) belongs to a category Ca (a=1,…,c). If Ca is in advantage to a category Ca+1, than it holds the hierarchical and transitive order

(1.7)

The underlying intention is to evaluate an airporto only toairports of the same or disadvantaged categories, but not to advantaged ones. In example, c=3 categories of ANSPs.Following straightrelation (1.7), an ANSPs of category 2 has to evaluate toANSPs of categories 2 and 3 but not to 1,as illustrated via (1.8).

(1.8)

Suppose nowγj equals to the category Cb of airport j(b{1,…,c}), then the index set Noof airport o consists of all ANSPs jthat hold true forγj≥γo. Consequently, we update constraint (1.4) of model (CCRm) to:

(1.9)

Please note, when referencing (CCRm) afterwards, it always means (1.4) is replaced by (1.9).

In this contribution, we categorize ANSPs by the amount of runways that are in general available for take-off and landing at the airport. As can be seen in Table 1, we partition the set of airports into 2 hierarchical categories:

  • Category (cat) 1: more than 4 runways for take-off or landing.
  • Category (cat) 2: less or equal to 4 runways for take-off and landing.

Thus, e.g. an airport operating 5 runways for take-off but only 3 for landing belongs to category (cat.) 1.

Efficiency Configurations

One of the big advantages of DEA is to reduce the complex and multivariate dataset to asingle number – the aforementioned efficiency score. This allows us to distinguish efficientand inefficient ANSPs. Regarding inefficient ANSPs, the decider also gets informationabout the amount of inefficiency relative to the peer-group of a specific ANSP. Thus, the decider knows how much the ANSP has to improve overall – that means foran input oriented view, how much to reduce each input. This seems hard to handleas knowledge about individual strengths and weaknesses would be a much prettierstarting point for making sufficient decisions. For that purpose, we decomposethe efficiency score for inefficient as well as efficient ANSPs. This is done to some extendin the style of Cooper et al., p.25 [9].

The approach bases on the optimal weights (multipliers) and for outputsand inputs of themodel (CCRm).Combiningeach multiplierwith either the respective input or output, we can determine the contribution of each input andoutput to the efficiency score θo, in other words the relative importance of each individualinput and output. From a decision-makers point of view, this depicts evidence on inputs entailing weakness and outputs causing strength. As each impact factor isexpressed as a percentage, it is also easyto interpret them. Displaying all impact factorsfor each observed ANSPoasa so-called efficiency configuration (EC) of o, we obtain anindividual and comparable profile for every ANSP. Asan individual profile is relative to the other profiles, similarities and dissimilarities can be identified. Hence, after determining the impact factors and ECs we visualize the field ofANSP for an exploratory analysis as shown within the next few sections.