Adjusting for Optimism Bias in Regeneration Projects and Programmes

General Guidance Note

regeneration

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Guidance Note

March 2007

Department for Communities and Local Government: London

On 5th May 2006 the responsibilities of the Office of the Deputy Prime Minister (ODPM) transferred to the Department for

Communities and Local Government.

Department for Communities and Local Government

Eland House Bressenden Place London SW1E 5DU

Telephone: 020 7944 4400

Website: www.communities.gov.uk

Copyright in the typographical arrangement rests with the Crown.

This publication, excluding logos, may be reproduced free of charge in any format or medium for research, private study or for internal circulation within an organisation. This is subject to it being reproduced accurately and not used in a misleading context. The material must be acknowledged as Crown Copyright and the title of the publication specified.

Any other use of the contents of this publication would require a copyright licence. Please apply for a Click-Use Licence for core material at www.opsi.gov.uk/click-use/system/online/pLogin.asp or by writing to the Office of Public Sector Information, Information Policy Team, St Clements House, 2-16 Colegate, Norwich NR3 1BQ.

Fax: 01603 723000 or email: .

If you require this publication in an alternative format please email

Communities and Local Government Publications

PO Box 236

Wetherby

West Yorkshire

LS23 7NB

Tel: 0870 1226 236

Fax: 0870 1226 237

Textphone: 0870 1207 405

Email:

or online via the Communities and Local Government website: www.communities.gov.uk

March 2007

Product Code 06HC04376

CONTENTS

Chapter 1Purpose4

Chapter 2Introduction5

Chapter 3Why does optimism bias arise?6

Chapter 4Optimism bias evidence base7

Chapter 5When to consider optimism bias8

Chapter 6Using numerical indicators in project decision-making10

Chapter 7Link with financial approvals11

Appendices

1 Calculating optimism bias on works duration and capital costs12

2 Sensitivities for the assessment of job-related outputs18

3 References and sources of further guidance19

Chapter 1.Purpose

1.1The purpose of this Guidance Note is to help regeneration agency senior managers and project staff understand (i) what optimism bias is, (ii) how to assess it in regeneration projects and (iii) to provide information about the techniques that can be used to assess the sensitivity of numerical indicators to optimism bias. The Appendices provide appropriate uplift tables for both costs and duration and explain how optimism bias should be applied to projects.

1.2The Guidance Note considers:

optimism bias;

optimism bias evidence base;

when to consider optimism bias;

proportionality;

financial indicators; and

financial approvals.

Chapter 2.Introduction

2.1HM Treasury’s Green Book (2003) introduced the concept of optimism bias and required that it should be taken into account in project appraisal and evaluation. This requirement is picked up in the Communities and Local Government’s 3Rs1 and the DTI’s Single Programme Appraisal Guidance to RDAs.

2.2It is recognised that risk and optimism bias are closely linked and that good risk management can reduce optimism bias. However, as guidance on project risk identification, assessment and management is provided elsewhere2 it is not covered in this Guidance Note.

2.3Optimism bias is the term used to describe the demonstrated, systematic tendency for project appraisers to be overly optimistic about project costs, duration and benefits

(outputs and receipts/income). In other words, it is the systematic tendency to view things in an overly positive light. It can arise in relation to any aspect of a project but it particularly applies to:

costs (capital and revenue);

works’ duration; and

benefits delivery (outputs and outcomes).

2.4To redress this tendency HM Treasury’s Supplementary Guidance on Optimism Bias

(http://www.hm-treasury.gov.uk/media/885/68/GreenBook_optimism_bias.pdf) recommends that project appraisers should make explicit adjustments to the estimates of project costs, benefits, and duration based on empirical data to inform project decisions. Adjusting for optimism bias should help agencies to answer key questions such as:

By how much can we allow benefits to fall short of expectations if the proposal is to remain worthwhile? How likely is this to happen?

How much can operating costs increase if the proposal is to remain worthwhile? How likely is this to happen?

What will be the impact on benefits if operating costs are constrained?

1 Assessing the Impacts of Spatial Interventions – Regeneration, Renewal & Regional Development – The 3Rs Guidance

2 HM Treasury Green Book Ch5 and Annex 4; Orange Book and Risk website http://www.hm-treasury.gov.uk/documents/

public_spending_and_services/risk/pss_risk_index.cfm; OGC Successful Delivery Toolkit – Management of Risk

Chapter 3.Why does optimism bias arise?

3.1Projects are inherently risky and the risks increase with the size and complexity of the project. Studies have found that there are deep-seated causes of optimism bias and that they can be broadly split into two categories: technical and institutional.

Technical

Risks and uncertainties associated with forecasting costs, income etc.

Changes in project scope.

Poor project management.

Institutional

The desire to see projects happen.

Institutional pressures.

The decision-making process, etc.

3.2In one influential study, the authors argued that “political-institutional factors have in the past created a climate where only a few actors have a direct interest in avoiding optimism bias.”3 However, it is the norm in projects, rather than the exception, that unplanned events do occur and experienced project managers should consider the effect of these when appraising projects.

3.3The challenge is therefore to:

make explicit allowance for optimism bias in appraisal in a proportional and cost effective way;

consider whether a project represents value for money once an allowance for optimism bias is included; and

be aware of and work to reduce the causes of both technical and institutional optimism bias.

3 DfT (2004) Procedures for Dealing with Optimism Bias in Transport Planning, Bent Flyvbjerg in association with COWI.

Chapter 4.Optimism bias evidence base

4.1This Guidance Note is based on research commissioned by the Office of the Deputy Prime Minister (ODPM)4. The research examined 119 regeneration projects and programmes undertaken by the Regional Development Agencies, English Partnerships and other regeneration bodies. It also considered wider-scale evaluation studies and involved a series of practitioner consultations and workshops.

4.2The Guidance Note provides tailored estimates of optimism bias uplifts for project costs and duration for regeneration projects and programmes. The Guidance Note also provides advice on accounting for optimism bias when assessing jobs created outputs.

4 The responsibilities of the Office of the Deputy Prime Minister were taken over by the Department for Communities and Local

Government on 5 May 2006.

Chapter 5.When to consider optimism bias

Project appraisal

5.2In a two-stage appraisal process, it is important to be aware that optimism bias may be greatest at the initial appraisal stage before the project costs, timescales and benefits have been fully worked up.

5.3At the initial project assessment stage the agency’s focus is primarily with the project’s strategic fit and practicality more than the preliminary forecast estimates of costs, duration and outputs. Few projects are rejected simply on cost/benefit grounds at this stage. At this stage it may be useful to apply the Upper Bound allowances for optimism bias (see Appendix 1 Table 1) to understand whether a project still appears acceptable when these are applied. The appraiser should be clear what steps are to be taken during the full appraisal to mitigate the risk of optimism bias, for example market research, reviews of business plans, independent assessment of capital costs, etc.

5.4At the full project appraisal stage, optimism bias should be formally considered in respect of costs, duration and benefits. Appraisals should consider whether:

a project represents value for money for the Agency once allowance for optimism bias is included; and

if the inclusion of an allowance for optimism bias affects the relative performance of options.

5.5If after the application of optimism bias the project appears poor value for money, then approval should be withheld or given on a qualified basis, eg requiring further research, costing and risk management.

Monitoring

5.6Once the agency has committed funds to the project it should monitor its delivery using the optimism bias factors to guide data gathering and risk management. Thus if the project had to come back to the agency decision makers for a further approval they would expect to see evidence that the most likely cost estimate had been adjusted to reflect this developing experience.

Evaluation

5.7Post completion, when the project is evaluated, the evaluators should compare the actual and forecast outturn costs/receipts, timings, outputs and outcomes to enhance the empirical evidence base of optimism bias factors to inform future appraisals.

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Proportionality

5.8The application of the optimism bias approach via comparators, risk analysis and specific uplifts can become rather complex. It should therefore be applied proportionately, eg the DTI’s Single Programme Appraisal Guidance says that:

“… in designing their systems to meet SPAG requirements RDAs should build in proportionality. For large complex and high risk projects including those outwith their financial delegations RDAs should show that they have given full consideration to all the key requirements … but that … for smaller simpler less risky projects RDAs still need to have addressed all the appraisal requirements but in less detail”.

Chapter 6.Using numerical indicators in project decision-making

6.1The tools outlined in the rest of the Guidance Note relate to the numerical and financial elements of project proposals/business plans and appraisals. It is therefore important to understand which numerical indicators are important when making a decision about whether a project should be supported. These can include:

For large projects where the costs and benefits have all been monetised – is the net present value of the project greater than zero?

For many regeneration and economic development projects – is the cost per unit reasonable when compared to benchmarks?

For grant-funded projects – is the project financially self-sustaining, does it break even year-on-year without further public funding?

6.2Having established which financial indicators are most important to the decision to support the project, analysis can be undertaken on how sensitive these are to change.

Chapter 7.Link with financial approvals

7.1The project’s financial costs should include a sum for reasonable contingencies but should not include the optimism bias mark-up. The purpose of considering optimism bias costs is to assess the value for money of the project. Adding them onto the project financial costs would risk a disincentive effect in effectively managing the project delivery.

7.2Optimism bias and the sensitivity analysis information should be provided for decision makers to inform them of the consequences should these risk factors occur and the implications for the project’s value for money. Decisions as to whether the project is value for money and should be financially supported by the agency can then be made in the knowledge of these factors. Agencies should approve project financial costs excluding the optimism bias costs.

7.3If changes are required to the contracted costs, duration or output of the project during its delivery these should be managed using the agency’s project change process. Agencies should review requests for further funding or other contract changes against the optimism bias and switching values whereby the project would still offer value for money when it was originally approved. When these projects complete then the agency can review the actual optimism bias that transpired, incorporate it into its analysis and use it to update its local optimism bias look up tables.

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Appendix 1. Calculating optimism bias on works duration and capital costs

This Guidance Note provides bespoke advice for those senior managers and project staff engaged in delivering regeneration projects. Regeneration projects here are defined as any capital and/or revenue project funded by the English RDAs under the Single Programme, the London Development Agency or English Partnerships for the purposes of regeneration, renewal, regional development and developing the New Towns and Sustainable Communities Plan Growth Areas.

The Treasury’s Green Book Supplementary Guidance (http://www.hm-treasury.gov.uk/ media/885/68/GreenBook_optimism_bias.pdf ) on optimism bias contains evidence from a study undertaken by Mott MacDonald which researched optimism bias on large capital and engineering projects. Mott MacDonald developed a methodology for calculating optimism bias on capital costs and works duration, and provided tables of suggested uplifts to be applied to duration and cost estimates.

This Guidance Note provides tailored advice which is particularly relevant to regeneration projects and programmes.

There are five key stages in the methodological approach to calculate optimism bias. The stages are:

1. Decide on the type of project.

2. Start with the upper bound.

3. Consider whether the optimism bias factor can be reduced.

4. Apply the optimism bias factor.

5. Review the optimism bias adjustment.

The process is outlined in Figure 1, and summarised below.

Stage 1 – Decide on the type of project.

There are three categories of regeneration projects identified in this Guidance Note. They are:

i.Physically-based regeneration projects – this includes public realm (town centres, parks etc), site development (site acquisition, remediation, preparation, infrastructure works, landscaping), property (industrial, commercial, retail, community facilities), equipment (plant machinery) etc.

ii.Housing and residential developments.

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Figure 1: Calculating Optimism Bias

iii.Revenue-based interventions (enterprise, business support, inward investment, innovation, marketing).

This Guidance should be used for all three types of intervention, except large projects involving public sector spending of over £40m. For projects over £40m, agencies should use the standard uplift tables outlined in the Treasury’s Green Book Supplementary Guidance.

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Where an agency is considering an intervention based on non-standard civil engineering, outsourcing and equipment development, the agency should use the standard uplift tables outlined in the Treasury’s Green Book Supplementary Guidance. Such projects are not normally undertaken by regeneration bodies and are beyond the scope of this Guidance.

Civil engineering projects are defined as projects which involve the construction of facilities, in addition to buildings, requiring special design considerations due to space constraints or unusual output specifications.

Outsourcing projects are defined as projects which are concerned with the provision of hard and soft facilities management, eg ICT services, facilities projects or maintenance projects.

Equipment development projects are defined as projects that are concerned with the provision of equipment and/or development of software and systems, i.e. manufactured equipment, Information and Communications equipment (ICT) or leading edge projects.

Stage 2 – Start with the upper bound.

Once you have identified the type of project and programme involved, select the appropriate upper bound value from Table 1, the starting value for calculating optimism bias. Note that for revenue-based interventions, cost uplifts should not be applied. For physical regeneration projects and housing/residential interventions, uplifts should be applied to total project costs.

Table 1: Optimism bias adjustment ranges

Works durationProject costs

UpperLowerUpperLower

Physical regeneration17%2%11%2% Housing and residentia 20% 2% 12% 2% Revenue-based interventions 17% 0% Na Na

Stage 3 – Consider whether the optimism bias factor can be reduced. The research underpinning this Guidance Note identified specific factors that contribute towards optimism bias for each category of regeneration project. Table 2 sets out the influence of different contributory factors in terms of cost and duration values for the three project types. Each contributory factor is in turn split into a number of sub-factors, the importance of which varies between project types.

Practitioners should reduce the upper bound optimism bias factors established in stage 2 according to the extent to which the contributory factors have been managed using the factors in Tables 2 and 3. Optimism bias should be reduced in proportion to the amount that each factor has been mitigated.

Adjusting for Optimism Bias in Regeneration Projects and Programmes

Table 2: Weighting of contributory factors to optimism bias – costs

Contributory factorsStandard physicaHousing andRevenue-based

regenerationResidentiainterventions Site specific /property issues 30% 30% Na Unforeseen site characteristics

Pattern of land ownership

Land acquisition Dated valuations Permissions.

Complex or innovative design

Procurement20%20%Na

Complexity of contract structure

Late contractor involvement in design

Risk of claims

Project management and approvals30%30%Na