Julian Todd

By e-mail:

request-29570-


Tom Worsley

Divisional Manager

Network Analysis and Modelling Division

Department for Transport

Great Minster House

76 Marsham Street

LONDON SW1P 4DR

Our ref: E0006237

Web Site:

Date:18 May 2010

Dear Mr Todd

UK Air Passenger Demand and CO2 Forecasting Model: Request for Internal Review of Response to Information Request E0006237

I am replying to your e-mail of 22 April to Richard Clarkson which requested an internal review of the decision set out in his letter of the same date. I was asked as a member of the Senior Civil Service who was not connected with the original decision to conduct this internal review. I am also answering in this letter the additional questions you asked of Richard Clarkson.

I have now carried out the review of your request. The request was, in my opinion, appropriately considered under the EIR Regulations (rather than under the Freedom of Information Act).

  1. I have considered the public interest test as applied by Richard Clarkson and under which he decided to withhold the information you requested under Regulation 12(4)(b) of the EIRs on the grounds that the request for the information was unreasonable.
  1. I have subsequently carried out my own public interest test. I have reviewed your request for more information about the programme language or languages used in the models which are used to derive the Department’s forecasts. I note that Richard Clarkson provided you with information about the language used by one of the models as an example. I have decided that there are no grounds for withholding information about the languages and software platforms used in the other models that constitute the modelling suite. I have set out this information in Annex B to this letter and apologise that it was not provided in full when you first requested it.
  1. I have reviewed your request to supply you with a copy of the modelling software in the actual state it is. I have considered the costs of this option, which would require deleting from the code all proprietary software, andassessed whether the public interest would be served by providing you with this software without any back-up. Estimates for the cost of this work (which DfT would have to commission from its consultants) are well in excess of the FOIA £600 appropriate cost limit. Although this request falls to be handled under the EIR rather than FOIA, I consider that the fact the FOIA cost limit would be exceeded supports the application of the EIR reg. 12(4)(b) (manifestly unreasonable) exception.
  2. I have also considered the process whereby at each stage as the model is run, the user intervenes to provide inputs, to check and review the outputs of any new test and to reach a decision on whether those stages of the modelling which are based on iterations have converged to an acceptable degree. I have also considered the level of expert knowledge required to inform many of the inputs to the model, for example in the modelling of terminal passengers.

In carrying out my review I have taken into account the presumption in favour of disclosure that I am required to bear in mind when considering application of any EIR exception, in this case regulation 12(4)(b). However, in this case my review leads me to conclude that release of the software might be in the public interest only if you were to be provided with the support to enable you to carry out well informed model runs as described in Richard Clarkson’s letter of 22 April. Without such support I am of the opinion that the public interest would not be served by any use of the model that might be made in these circumstances and that happened to deliver some form of output. And I have reviewed the costs that the public purse would incur in providing this support, which would in my view also require provision for external review and checking of the outputs, and conclude that these costs would outweigh the public interest served by release of the software.

In reaching this decision I have also been mindful of the benefit, in terms of furthering public participation in the debate on air passenger forecasting, of the Department’s policy on access to our air passenger demand and CO2 forecasting suite. As Richard Clarkson explained, many members of the public with an interest in understanding the issues and furthering the debate have made use of this opportunity. Our offer to provide these facilities on the terms outlined in Richard Clarkson’s letter remains open. This consideration has been material to my decision.

I have set out in Annex A below the factors I have taken into account in reaching this decision.

You asked Richard Clarkson for some additional information in your e-mail. I have agreed with Richard that I should include in this letter his response to your questions rather than his sending a separate response. You asked him for more information to determine whether there was any independent verification of the models or their parameters. You wanted to know whether the "independent peer reviewer" or the "external appraisal expert" was also employed by the DfT, and to what depth they reviewed the modelling.

Richard Clarkson’s response of 7th April confirmed that the technical working group overseeing the development of these models includes an independent peer reviewer and an external appraisal expert. I can assure you that neither the independent peer reviewer nor the external appraisal expert is a DfT employee. The independent peer reviewer is provided access to all aspects of the model in discharging his responsibility for determining whether the Department’s aviation forecasting and appraisal methods are fit for the purpose of providing input to support the development of aviation policy.

You also asked Richard Clarkson about the procedures for updating those economic variables, forecasts for which are updated on a frequent basis by government and independent agencies. You pointed out that it was routine practice to rerun forecasts every time new data become available and asked Richard Clarkson to confirm that there was no procedure for rerunning these models and updating the conclusions when input parameters that were impacted by, say, an unanticipated global recession vary.

The original response explained that the Department regularly updates and develops the models between publication dates. However, it further explained that, given that they are long-term forecasts, it would not be cost effective to publish them on a more frequent basis. Any published forecast requires staff and other resources to ensure agreement on and, where appropriate, peer review of the assumptions being used in the model and in checking the model outputs and in producing a technical report describing the methods and assumptions underpinning the forecasts.

The Department is currently in the midst of an extensive programme of model development. The next generation of demand and CO2 forecasts will be published in due course, the exact timing to be subject to Ministerial decision.

If you are not content with the outcome of this internal review, you have the right to apply directly to the Information Commissioner for a decision. The Information Commissioner can be contacted at the following address:

Information Commissioner’s Office
Wycliffe House
Water Lane
Wilmslow
Cheshire
SK9 5AF

Yours sincerely

Tom Worsley

Head of Rail Network Analysis and Modelling Division

Annex A: Public Interest Test – summary of review of decision against releasing model software

  1. Potential benefit of releasing model for use by the applicant:
  2. Better informed debate
  3. Possibility of further verification of the model
  1. Costs of releasing model in the event of DfT providing no documentation, user support or other guidance being provided:
  2. Cleaning the code to remove all proprietary coding
  3. Taking off Scott Wilson plc’s computers and writing on to disks for transmission to applicant

Estimates for the cost of this work (which DfT would have to commission from its consultants) are well in excess of the FOIA £600 appropriate cost limit. (Although this request falls to be handled under the EIR rather than FOIA, I consider that the fact the FOIA cost limit would be exceeded supports the application of the EIR reg. 12(4)(b) (manifestly unreasonable) exception.)

  1. Additional benefits of releasing the model without documentation, user support, guidance or independent checking and with copyright material removed:
  2. It is not certain that the model would run successfully in this format without further modification where the copyright code has been deleted.
  3. No reliance could be placed on the outputs
  4. The outputs from the model could not be compared with outputs from the DfT model as the modification to replace deleted copyright code would result in different models.
  1. Costs of releasing model with documentation, guidance, user support and model checking provided by the Department:
  2. As in Richard Clarkson’s letter of 22 April, plus
  3. Some re-writing of the code to protect proprietary code
  4. Providing independent checking of model results, as is done for all published DfT work.
  1. Additional benefits and costs from release of model compared with the access to the model offered by DfT:
  1. There is unlikely to be a significant benefit from having the model installed on the applicant’s own computer compared with DfT undertaking runs as specified by the applicant, as has been done previously for many individuals with an interest in the forecasts. There is a risk of confusing the debate rather than furthering it because the interventions required to run the full model need to be assessed and they each rely on expert judgement. The DfT can draw on relevant expertise to help users form a judgement on these interventions.
  2. The modelling suite has been scrutinised by the independent peer reviewer of the Impact Assessment for the third runway. The reviewer was authorised to commission runs of the model and this work has confirmed that the model is fit for purpose. In my view there is therefore no case for spending public funds to facilitate a further examination of the model.

Annex B: Software

Model Stage / Description / Software

1

/ National Air Passenger Demand Model / Unconstrained national demand forecasts / Excel

2

/ National Air Passenger Demand Model / Validate

3

/ “Distgrowth” / Apply regional growth adjustments. / Excel VBA

3a

/ NAAM / Airport surface access and intra-district skims / SATURN, MINITRAMP or EMME/2 for skims; then bespoke FORTRAN77 then Excel VBA interfaces

4

/ Unconstrained Airport Allocation / Initial NAPAlM run to check the full unconstrained demand in 2030 is allocated to airports. / Excel VBA and C++

5

/ Constrained Airport Allocation (NAPAlM) / Allocate passengers to airports, forecast ATMs by seat band and shadow costs, annually 2004-2030. / Excel VBA and C++

6

/ Constrained Airport Allocation (NAPAlM) / Run the output processor (Orange.xls).
Validate allocation results. / Excel VBA

7

/ Constrained Airport Allocation (NAPAlM) / Re-run NAPAlM if there are stability problems arising in (6) / Excel VBA

8

/ Fleet Mix Model / Convert NAPAlM seat-banded ATM output into specific aircraft types (annually 2004-2030) for CO2 model. / Excel VBA

9

/ CO2 Forecasting Model / Project annual fuel burn and CO2 by individual route and individual aircraft type 2004-2030 / Excel VBA

10

/ Air Passenger Duty (APD) Revenue / Calculate government revenue stream 2004-2030 from Route level NAPAlM outputs. / Excel

11

/ Surface Access CO2 Calculation / Calculate mode share of airport access, and apply Webtag surface transport fuel burn rates to district to airport NAPAlM outputs. / Excel

12

/ Transport User Benefits Model / Calculation of monetised net benefits (“SCAB”); extension of forecasting to 2080. / Mainly Excel (but Excel VBA data interface)

1