Title of the Paper (16Pt Times New Roman, Bold, Centred)
Cost Uncertainty Assessment and Management: The Integrated Cost-Risk Analysis Model
Vaggelis BELLOS1, Vrassidas Leopoulos2, Michael Sfantsikopoulos3
Authors listed in alphabetical order
1Department of Mechanical Design and Control Systems, tel. +30 210 772 36 33, Fax: +30 210 772 35 99,
2 Department of Industrial Management and Operational Research, tel.: +30 210 772 35 85. Fax: +30 210 772 35 71,
3Department of Mechanical Design and Control Systems, tel. +30 210 772 36 81, Fax: +30 210 772 35 99,
National Technical University of Athens
Iroon Politechniou 9, Zografou Campus
157 80 Zografou / Athens
Greece
Abstract: - The paper presents a framework for cost uncertainty assessment and management for complex systems, by incorporating risk in the cost estimation process. The traditional approach of probabilistic treatment of cost does not enable cost estimators to obtain a clear view of the potential uncertainty factors, in order to adopt an appropriate strategy for their reduction. In order to capture, capitalise, store and make reusable the corporate knowledge concerning the uncertain factors a risk breakdown structure is proposed. The risks are identified, assessed and treated using risk management techniques.
Key-Words: - cost estimation, cost uncertainty, cost – risk, cost uncertainty assessment, risk management
1
1 Introduction
Aim of this paper is to present a framework for cost uncertainty assessment and management for complex systems, by incorporating risk in the cost estimation process. Several techniques have been developed in order to face these problems, indicating a continuous research effort towards a reliable and accurate cost estimation. A fundamental element of the cost estimation process is the concept of uncertainty. Basic factors contributing to cost uncertainty are: schedule deviations, unexpected events, inappropriate estimating models, misunderstanding of cost data, insufficient cost data, improper implementation of estimating techniques, changes in system structure and requirements. A classic approach for cost uncertainty assessment is the probabilistic treatment of cost. When statistical information about the “past occurred costs” is available, a probability value of occurrence on each potential cost value can be calculated. In the absence of relevant data, the cost estimator can express his “global feeling” about the uncertainty of the cost item, by estimating a probability value of occurrence and attaching it on each potential cost value. Various techniques have been proposed towards this direction nevertheless, a common understanding about the meaning of cost uncertainty and risk and their effective assessment and management does not exist. The main reason for this is that cost uncertainty remains a “black box”, the elements of which are only partially known. Estimators recognise the threat of potential cost increase, formulate their feeling in a probability distribution but they are not able to propose appropriate mitigation measures as they do not known what exactly has to be mitigated. In to capture, capitalise, store and make reusable the corporate knowledge concerning the uncertain factors a risk breakdown structure is proposed. The risks are identified, assessed and treated using risk management techniques.
The remainder of this paper is organised as follows: in Section 2 the various definitions and types of uncertainty are discussed along with the main methods and techniques for its assessment. In Section 3 the proposed methodology for cost-risk analysis is presented. In Section 4 the expected benefits and the conclusions of the current research are discussed.
2 Cost estimation under uncertainty
2.1 Uncertainty categorisation
Several perceptions exist in the literature concerning the definition of uncertainty. Rithcie and Marshall [1] define uncertainty as the opposite of certainty within a decision process. Certainty exists when a particular consequence can be unambiguously predicted to follow a given event. Uncertainty exists in situations where the estimator lacks complete knowledge, information or understanding concerning the decision and its possible consequences. This definition implies that uncertainty arises due to the gap between available and required information for an estimation or decision [2], concerning the estimated variables and their variability, the factors or conditions that may influence them and the structure and behaviour of the system under investigation [3], [4].
Two basic types of uncertainty are generally recognised [e.g. 1, 5]:
The aleatory (or stochastic) uncertainty or randomness, arising from a situation of pure chance. This type of uncertainty represents the variability of the parameters of a known population or sample.
The epistemic uncertainty, arising from incomplete knowledge or wrong judgement.
Under this distinction, uncertainty is considered to be objective when the estimator has perfect knowledge of all the outcomes that can be associated with a particular event and the variables’ values can either be calculated on a “a priori” basis or by applying statistical techniques on historical data. On the other hand, uncertainty is considered to be subjective when the level of knowledge concerning the possible outcomes or the variables’ values or the mechanisms that may affect them is incomplete and the estimation of the actual values is based on personal opinions (subjective probabilities).
A similar categorisation has been proposed by Kaplan [6], which separates the state of knowledge uncertainty from the population variability. Brown and Ulvila [7] distinguishes the outcome uncertainty, which concerns the inability of estimator to predict the exact outcome value and the estimation uncertainty, which can be reduced by using additional information.
Parry [8] refers to three types of uncertainty:
the parameter uncertainty that concerns the quantification of the estimated variables,
the modelling uncertainty that concerns the appropriateness of the estimation model and
The completeness uncertainty, which reflects the degree that the model captures all the phenomena related to the system or variable under investigation.
Wilkner [9] proposes the separation of uncertainty concerning observable events or conditions from the uncertainty concerning non-observable events or conditions. Nevertheless, he points out that all these categorisations can be useful only if they aim the improvement of the estimation model’s structure and the optimisation of the information management process.
2.2 Perception of uncertainty and risk
The terms “uncertainty” and “risk” are often confused and there is no clear and widely accepted distinction between them. The majority of authors makes a constructive beginning by recognising that there is a distinction between the two elements, but then proceeds to confuse the issue by using the two terms interchangeably, usually by subsuming uncertainty with the definition of risk. As an example Chapman and Ward [10] state that:
“…risk is the implications of the existence of significant uncertainty”, which means that there is not a distinction between the two. Chong [11] describes risk as the area of uncertainty surrounding an event and defines uncertainty as the source of risk. In the same direction Wideman [12] considers the terms “uncertainty”, “risk” and opportunity” as highly related. Specifically, he defines uncertainty as the set of all-possible outcomes or events, both favourable and unfavourable.
On the other hand, Raftery [13] recognises the distinction between “uncertainty” and “risk”. A risk exists in situations where the probability of occurrence of an event can be estimated by using statistical methods. Uncertainty describes the situations in which the estimation of the probability of occurrence of adverse events is not possible. This distinction, which is based on the use of objective or subjective probabilities, was adopted also by Broch and Mossin [14] and may lead to a distinction between repeatable (risks) and non-repeatable (uncertainty) events [15]. Ritchie and Marshall [1] and Chapman and Ward [16] find such a distinction, based on the estimation method, inappropriate as it does not reflect the real meaning of the terms.
Another wide – accepted approach bases the distinction on the identification of discrete events that may affect the estimated variable or system. As an example, Turner [17], states that there are two types of uncertainty in the real world:
First, the inherited uncertainty (business risk) of the system under discussion. That is, for instance the inability of a well-tuned machine to deliver a product characteristic of a certain value. No matter how precise the machine might be, it will deliver a product characteristic of the certain value +/- a percentage of deviation.
Second, there are the distinct events (insurable risks) that might affect the performance of the system. An example of such an event is the machine’s breakdown.
Similarly, Husby et al [2] separate parameter uncertainty from event uncertainty, which call risk, while Pierre Foussier [18] introduces another type of uncertainty, apart from risk (called “immaturity”) and inherited uncertainty (called “chance”), which is the imprecision. That is, the inability to accurately estimate the real value of the characteristic under consideration.
2.3 Approaches for uncertainty assessment
Nilsen and Aven [19] and Pate-Cornell [5] refer to three basic approaches for uncertainty assessment:
The classical statistical approach under which, the probability of an event (uncertainty) is defined as the relative frequency of event occurrences when the experiment, from which the event develops, is hypothetically repeated an infinite number of times. Hence, taking this approach, the analyst uses uncertainty analysis as a means for estimating true, yet unknown probabilities.
The classical statistical approach with uncertainty analysis [20], allows the uncertainty in the probabilistic input parameters to be expressed in terms of subjective probability distributions. The latter may be propagated in uncertainty measures of the final uncertainty indices of the model output. The distributions can be systematically updated when new information arises by the use of Bayes formula. This approach is also referred to as the probability of frequency approach [21]. In this framework the concept probability is used for the subjective probability while frequency is used for the objective, relative frequency based probability.
The predictive Bayesian approach, represents a framework for uncertainty analysis focussing on observable quantities. These quantities are further predicted using subjective [22]. Similar to this approach is the predictive, epistemic approach proposed by Appeland et al [23] which recognises the weaknesses of the classical probabilistic frameworks and develops a model based on subjective probabilities by using both historical information and experts judgement.
In an alternative approach, Mohamed and McCowan [24] state that the use of probability theory is not appropriate for the analysis and management of uncertainty as its primary source is the fuzziness of the estimated variables (epistemic uncertainty) and not the randomness (aleatory uncertainty). For this reason, they propose the use of possibility theory and fuzzy sets for the quantification and management of uncertainty. This approach is adopted and further extended by Ayyub and Chao [25], which propose a Fuzzy Stochastic analysis for cases with both aleatory and epistemic uncertainty.
2.4 Analysis of uncertainty in cost estimation
There are several methods for analysing the uncertainty in the cost estimating. These methods are usually called as cost-risk analysis methods, a term which is obviously affected by the confusion between risk and uncertainty. The selection of the appropriate technique depends on the type of uncertainty and the available data. For parametric or analog-based cost models and when statistical information about the “past occurred costs” is available, statistical analysis methods are used for the calculation of the uncertainty concerning the value of the independent parameter (e.g., weight) or the complexity factor. In the absence of relevant data, the cost estimator expresses his “global feeling” about the uncertainty of the cost item, by estimating a probability value of occurrence and attaching it on each potential cost value. Cost uncertainty is treated similarly for estimates based on the engineering bottom-up cost estimating techniques (e.g. subjectivity in estimating duration and materials requirements).
Pros and cons of the most common techniques for the assessment of cost uncertainty are shortly discussed in the following paragraphs:
2.4.1 Risk adjustment factor
This factor [26, 27, 28, 29] has a form of a percentage deviation of a baseline cost estimate. Experts usually estimate the potential variability of the cost value. The method has the advantage of simplicity and can be used in early phases of the life cycle when sufficient data are not available. On the other hand its major weaknesses are the unsystematic assessment of uncertainty, the possible biases, the “hidden” sources of uncertainty and the inability for detailed analysis and management of uncertainties.
2.4.2 Cost – risk analysis
The basic steps of which are [30, 31, 32, 33, 34]:
Definition of the product Work Breakdown Structure – WBS [35].
Definition and analysis of the correlation between cost items. Correlation contributes significantly to the uncertainty of the total system cost [36, 37, 38].
Estimation of the costs of each WBS element (single point estimate) and calculation of the baseline cost This single point estimate is either given by an expert or calculated using past historical data as the “most likely” value.
Assessment of uncertainty of individual cost items. A popular technique is the triple estimate, where the minimum, most likely and maximum value of the cost item is estimated [38]. Graham and Dechoretz [31] propose the establishment of general uncertainty categories such as technical, complexity schedule etc. and the independent assessment of their contribution to the cost item. Tummala and Burchett [39] base the uncertainty quantification process on the assessment of the discrete events (risks) that may influence the cost elements and they further define the appropriate probability distribution for each cost item. Usually the triangular distribution is selected to describe the potential outcomes of cost items. However, several authors have expressed their objection for the appropriateness of triangular distribution [37, 38]. Alternative probability distributions can be used like trapezoidal, beta, uniform, normal or log-normal [37].
Calculation of total uncertainty of cost estimate and cumulative probability distribution. The probability distributions of individual cost items are summed up on the basis of the WBS in order to derive the total uncertainty of the estimate, using analytical methods (classical probability theory and theorems)[37], simulation techniques [40] (e.g. Monte Carlo or Latin Hypercube) or heuristics [41], [42].
3 The Integrated Cost-Risk Analysis model
The developed cost – risk analysis model is based on the principle that:
“It is not possible to manage cost uncertainty if we are not able to fully understand WHAT is considered to be uncertain, WHY is considered to be uncertain and WHO is responsible for it”
The proposed method aims on a systematic reduction of the epistemic uncertainty through a structured and organised approach for capturing, capitalising, storing and making reusable the corporate knowledge regarding the uncertain factors. These factors are systematically identified, assessed and treated using risk management methods (Figure 1). The continuous effort for their control enriches the corporate memory and increases the ability of the enterprise to understand and manage the uncertainty.
Figure 1: Risk Management steps
Under the conventional approach, the cost estimator uses statistical information regarding the “past occurred costs” or expresses his “global feeling” about the uncertainty of a cost item by attaching probability values of occurrence on each potential cost value. The question that arises is what is exactly included in the probabilistic approach? What was in expert’s mind when he gave his judgement?
In order to enlighten this dark side of cost estimation, the method proposes two basic types of cost uncertainty:
Inherited Cost Uncertainty (ICU). It arises from inaccuracies inherited in the cost estimating process and it is a result of three incontestable, intrinsic factors, which are:
The variability of the system development processes
The subjectivity of the estimation and
The unknown factors (chance)
Cost Affecting Risks (CARs). They are potential, distinct events that may affect negatively (hazards) or positively (opportunities) the cost of the system under estimation. The definition of this type of uncertainty (event uncertainty) is considered to be crucial for the improvement of cost estimating process. The reason is that it is much easier for an enterprise to protect itself from distinct, identified events than from a “black-box” uncertainty depicted in a cumulative probability chart. CARs can be either internal – that may be controlled by the company - or external that concern market, customers, environment etc.
ICU is a non-avoidable, practically, uncertainty contained in all conventional cost estimating methods and directly responsible for the probabilistic nature of the resulting cost estimate. It does not give the opportunity to reduce the probability of cost overrun or increase the probability of cost improvement. On the contrary, CARs enable the costing engineer to become familiar with the reasons and mechanisms that can lead to a potential cost increase/reduction throughout the product life cycle and choose the appropriate mitigation actions.
The systematic identification and assessment of CARs and the enrichment of the corporate memory leads to the decrease of ICU and increases the controllability of the reducible cost uncertainty.
Conventional cost with its inherited uncertainty and evaluation of the cost affecting risks are considered as two independently assessed and estimated product cost components that in combination provide an Integrated Product Cost and Risk Estimate (IPCRE).
The proposed model is based on the Product – Process – Recourse (PPR) structure as it is briefly below:
3.1.1 Product – Process – Recourse (PPR) model
The PPR model is a product – oriented, family decomposition tree that organizes, defines, and displays the product to be produced as well as the processes to be applied and the resources (materials, machines, people, accounting, purchasing etc…) to be used toward achieving the specified result (Figure 2). It adopts the top-down approach of product decomposition (moving from higher levels down to lower and detailed descriptions), and gives the opportunity to the estimator to include all the parameters involved in product manufacturing process. The PPR model differs from the WBS as it is not only task based. It follows the Bill Of Materials (BOM) structure and allows cost estimator to allocate direct and indirect costs more efficiently. It gives also the possibility to define a WBS inside the product decomposition (e.g. by decomposing the processes) to cover the need for a more detailed activity representation.