Applied Information Economics
______
The AIE Reference Manual
Version 3.1
DRAFT
AIE Version 3.0
TABLE OF CONTENTS
Introduction to the AIE MethodPage 7
The AIE Procedure:
1Describe & Classify The IT investmentPage 11
2Clarify The Decision ModelPage 15
3Measure VariablesPage 23
4Conduct Value of Information Analysis (VIA)Page 35
5Conduct Risk & Return AnalysisPage 39
6Make RecommendationPage 41
Appendixes
- Custom DataPage 47
- AIE Assessment Project PlanningPage 49
- Workshop GuidelinesPage 57
- AIE Assessment Document Templates Page 59
- Intangibles ChecklistPage 71
- Spreadsheet Templates Page 75
- Equations ReferencePage 77
- Example Calibration TestsPage 79
- Presentation TemplatesPage 83
- GlossaryPage 111
- BibliographyPage 115
- AIE Feedback SurveysPage 117
1
AIE V3.0 Reference Manual : Introduction To The AIE Method
Introduction To The AIE Method
The AIE method (Applies Information Economics) is a set of tools, procedures and organizational structures to provide decision-makers with a quantitative evaluation of IT investments.
1
AIE V3.0 Reference Manual : Introduction To The AIE Method
General objectives and scope
The aim of the Applied Information Economics (AIE) is to improve the IT decision process by providing decision-makers with sound economic evaluation tools specifically designed for the features of IT investments. The AIE method consists of a set of tools, procedures and organizational structures that can be used to generate and supply this decisionsupport data.
Features of IT investments
An “IT investment” is a mobilization or acquisition of information technology resources (hardware and software) and manpower for a limited period to alter the way that part of the enterprise operates. The benefits expected from this change will not accrue immediately, but over a fairly lengthy period of time. IT investments can be characterized by the following features:
- IT investments represent a growing portion of companies' investments, and IT resources are most often used by companies to develop strategic opportunities.
- IT benefits are increasingly difficult to measure since the improvements in productivity generated by automating manual clerical tasks have generally already been realized, and any further benefits are now likely to be made in more subtle “intangible” areas such as communication, information management or expert systems.
- IT investments are risky. Examples abound of partial failure (schedule and cost overruns) and complete failure (project cancellation). These common risks are rarely considered in the decision-making process.
AIE Benefits
Using the AIE method can help quantify the risks and intangible benefits of IT investments. The primary benefits AIE offers are:
- The risks can be stated in a manner consistent with other financial and actuarial measures
- The benefits can be quantified no matter how intangible they may seem.
- The AIE method can also go beyond the simple “yes or no” recommendation for investments. AIE can also identify specific implementation considerations to optimize the investment. Some considerations will include methods for managing risks or means for accelerating the accrual of benefits.
- Finally, some of the methods of AIE are applied to the AIE method itself. AIE calculates the value of additional information about an investment so that data-gathering efforts for IT investment analysis are optimized.
A Practical Method
The AIE method was designed specifically with practical implementation in mind. For example, although certain stages involve sophisticated concepts and statistical techniques, the method itself can be implemented by filling out easytouse spreadsheets.
The method has also been designed to fit in squarely with the set of IT reporting activities for companies.
AIE will be used by a variety of people. The hands-on users will adopt the method for producing decision-support documents, while the indirect users, i.e. the senior management of the company (executive management, business unit managers and IT managers) will base their decisions on these documents.
Realism and Flexibility
It should be possible to apply the proposed method in all companies. The method is therefore flexible enough to be tailored to a variety of companies for a wide range of IT investments.
This flexibility is provided through the customizable pieces of AIE. Some sections of AIE are “fill-in-the-blank” procedures that would vary among organizations.
Uses Of The AIE Method Over Time
The AIE method can be applied several times during the life of a particular project - e.g. at the end of the feasibility study to decide whether or not to proceed with detailed studies, at the end of the detailed studies to start up development work or even during actual implementation as part of a project review to decide whether or not the project needs to be reorganized or even canceled. The AIE method can be used to assess the various options for a particular project and hence provide selection criteria.
The AIE method can be used in strategic planning to compare and sequence IT investment projects, to periodically review this sequence in light of possible external or internal changes and, if necessary, reassign priorities and resources.
AIE links in with strategic planning in one other way: the assessment takes account of how each individual project contributes to the strategic plan (alignment).
Scope
AIE cannot, on its own, guarantee to improve the project failure rate or senior management satisfaction. AIE is an investment evaluation tool not an IT-project management tool. AIE will assist with project success only by identifying good IT investments at the initial decision stage. Other project management factors need to be applied to improve the degree of satisfaction.
The AIE method deals specifically with uncertainty, providing a big picture of the project. AIE can assist the IT project
management effort by identifying the investment project parameters where uncertainty is most likely to jeopardize the project.
Key Principles
The key principles are:
in terms of assessment project organization, to separate the various roles to limit the effects of bias,
to clarify intangibles,
to express uncertainty in explicit, statistical terms,
to determine the factors where more extensive analysis is justified to reduce the associated uncertainty.
AIE Implementation
The AIE method is first introduced to an organization through an implementation project. This project will start with training of all AIE users within the organization. Then certain parts of AIE will be customized for the specific organization and plans for implementation will be developed. AIE will often be initiated with a pilot project as hands on experience for the new users of AIE and as a test of the new decision making process within the organization. A detailed discussion of the AIE implementation procedure is part of another document.
The Stages of AIE
Once AIE has been implemented each proposed IT investment will be evaluated according to the following AIE phases.
Describe & Classify
This stage classifies an IT investment according to size and type so that the appropriate level of analysis can be applied. For example, if an investment is small and is mandated by government then virtually no analysis is required to make the investment. On the other hand, if an investment is large as well as optional, then more extensive analysis is justified. The high-level costs, benefits and risks identified in this stage are also key input to the clarification stage of AIE.
Clarify
The aim of this stage is to transform all the benefit, risks and cost intangibles into tangibles, i.e. into parameters that can be associated with a unit of measurement. The ultimate output of this stage is to express the investment decision problem as a spreadsheet model. All of the “intangible” benefits, costs, and risks will be variables in a quantitative decision model.
Measure
This is initially the explicit statement of the uncertainty about a quantity in statistical terms. Subsequently, the uncertainty can be reduced through observation and analysis. Measurement should not be confused with the arbitrary generation of exact figures. Various techniques are proposed for obtaining these measures: standard estimation techniques, calibrated estimates, search for information in external databases, scientific observations.
Optimize
Optimization is the general term for choosing the best of a defined set of possible choices with a given amount of information. This stage comes in three parts:
- Conduct Value of Information analysis (optimizing the measurement process)
- Conduct Risk & Return Analysis (optimizing the decision for risk and return)
- Make Recommendations (identifying what factors to manage to optimize the implementation of the decision)
This stage uses a simulation to combine uncertainties for actual implementations of the different investment project variables and to provide an overview of the probability distribution for the return on investment (or other financial criteria). This profile can then be used to deduce the mean expected Return On Investment (ROI) and the risk of negative ROI.
Investment criteria specific to the company can then be used to approve or disapprove the investment and even to alter the initial definition of the project. At this stage, it is possible to go
beyond the single binary question (yes or no) and implement a sensitivity analysis technique to determine those variables for which it would be profitable to reduce uncertainty by expanding the search for information.
Overview of the Major AIE Tasks
*Steps 4, 5 & 6 are all part of “Optimize”
AIE Method Components
The AIE method takes the form of documents that provide a general, but detailed description of the models, concepts and procedures involved. These documents will, in particular, include a summary and a presentation for senior management. The AIE method comes with supporting documents and is backed up with training sessions, workshops and the associated training materials.
The AIE method also comes with various tools:
checklists for itemizing costs, benefits and risks,
document templates for producing the business case that will be submitted to the decision-makers,
spreadsheets for performing the calculations required by the method.
1
AIE V3.0 Reference Manual : 1. Describe & Classify The IT Investment
1. Describe & Classify The IT Investment
The objective is to provide a brief description of the missions of the investment project, the type (compliance, strategic or economic), the size, and a list of tangible and intangible elements of costs, benefits and risks. The classification of the IT investment is used to determine the necessary level of AIE analysis.
1
AIE V3.0 Reference Manual : 1. Describe & Classify The IT Investment
Summary Procedure Data
Responsibilities
SponsorEstimatorAuditor
Judge
Project Manager / Provide benefits and risks factors description
Respond to Questionair
Conduct Questionair and respond to some questions on it
Approve classification results
Develops AIE project plan if required
Time Required / 2 hours to 2 days
Prerequisites
/ - appropriate training- IS strategic plan
- customized AIE
Tools
/ - benefit and cost elements checklists- risk factors checklists
- Project Definition Template
Required/Optional
/ RequiredReference(s):
/ Appendix ADeliverable
/ 1-3 written pagesPurpose
Different types of IT investments require different levels of analysis. This section deals with the determination of analysis requirements based on a classification of the investment. There are three objectives for the Classification step of AIE:
- Provide a high-level desciription of the investment project that must be evaluated.
- Describe & Classify the investment by type and size.
- Agree on the plan for the remainder of the project based on the analysis requirements indicated by classification.
Approach
To classification is done by plotting the proposed investment on a "Classification Chart" (see chart on next page). Depending on what region of the chart the point plots to we may take one of 5 actions:
The investment is too small to even bother with classification, so make a judgement call.
Accept the investment based on classification results alone
Reject the investment based on classification results alone
Proceed with computing the risk/return analysis with an abbreviated report
Proceed with computing risk return analysis and generate a full report
The chart consists of 2 dimensions: the "Confidence Index" and "Estimated Investment Size". Each of these values are measured and the result shows what region of the chart the investment falls in and, consequently, which of the previously mentioned actions should be taken.
The Confidence Index
The Confidence Index (CI) is an initial assessment of how certain the decision maker is that the investment will have a positive value. But the decision maker is not actually directly involved in each assessment of the CI. Instead a model is developed that uses known characteristics of the proposed investment to estimate the CI and - some cases - user focus group or user survey responses. A short questionaire is filled out for the investment and the CI is computed from the responses.
Typical questions on the CI questionaire would be:
"Does the investment involve internally developed software or is it only purchased technology?"
"If it involves custom software development is it likely to be finished in 60 days or less?"
"Did the User Focus group give it a review of 'very useful' or 'critically needed'?"
Your organization's specific CI questionaire has been developed and is provided in the "Classification Worksheet" on your AIE diskette and an example is shown in Appendix A. This worksheet computes the range for the CI (conservative and optimistic) based on the responses to this questionaire.
The CI can be roughly interpreted as answering the question "What is the probability that this investment has a positive net value or is otherwise necessary?". This quantity is the vertical dimension on the Classification Chart. Note that if the investment is below a certain size, as indicated on the Classification Chart, then the CI is not required and a purely subjective judgement can be made.
Investment Size
The size of the investment is initially estimated subjectively or by whatever data is quickly available. Even very broad ranges are acceptable at this point (for example, $2M to $8M).
What qualifies as the "Investment size" must be defined by the decision makers of an organization. One example of the definition of investmen size might be "All costs incurred before first benefits are realized". For the specific definition of what is included in the investment size in your organization refer to the "Classification Worksheet" included on your AIE diskette.
Interpreting the Results
Once the CI and the investment size have each been given a range of values - that is, an upper bound and a lower bound - then together they form an elipse on the classification chart. The greater your uncertainty about the values the large the elipse will be.
Depending on what region the elipse falls in the decision maker can decide to accept or reject the investment or to proceed with additional analysis. The output is a simple one or two page report that includes the following:
- Objective of the investment
- Responses to the questionaire
- A classification chart
- Recommended action
See Appendix D or the MS Word file on the AIE diskette for this document template.
1
AIE V3.0 Reference Manual : 1. Describe & Classify The IT Investment
Example Classification Chart
(See Appendix A for the classification chart for your organization)
1
AIE V3.0 Reference Manual : 1. Describe & Classify The IT Investment
Activities
1.1.Write title, main purpose/mission statement,
1.2.Write list of benefits, cost elements and risk factors,
1.3.Fill out questionair
1.4.Compute confidence index
1.5.Plot the ellipse which represents the investment project on the classification chart
1.6.Plan the remainder of the project with a timeline and resources required
1.7.Validate the project definition document
The deliverable must contain each of the items listed in the subtask steps. Include a classification chart as a graphic. Use the templates shown in Appendix D (you should have the electronic files as well). Check your classification chart to be sure it is up to date with the one provided in Appendix Since part of this deliverable includes the plan for the remainder of the project, refer to Appendix B for project planning help. It will be helpful to include any high-level timelines and resource requirements as part of this deliverable.
1
AIE V3.0 Reference Manual : 2. Clarify The Decision Model
2. Clarify The Decision Model
Developing a quantitative decision model (in the form of a spreadsheet) of the benefit/cost/risk analysis, forces us to clarify many of the “softer” issues in the value of the proposed investment. In this section we resolve the “intangibles” and formulate a decision model.
1
AIE V3.0 Reference Manual : 2. Clarify The Decision Model
Summary Procedure Data
Responsibilities
SponsorEstimatorAuditor
Judge
Other:Facilitator
Financial Analyst / Workshop Participant
Workshop Participant
Optional Involvement
Optional Involvement
Facilitate the Workshop
Expert Review
Time Required / 1 to 5 Days
Prerequisites
/ 1. Project Description and ClassificationTools
/ Excel TemplatesRequired/Optional
/ RequiredReference(s):
/ App. C: Workshop GuidelinesApp. E: Intangibles Checklist
Deliverable
/ Excel Spreadsheet with Cost/Benefit/Risk Model (no numbers, yet); 1-2 written pagesPurpose
The objective of this task is to reduce all intangibles to unit-of-measure variables and to formulate a comprehensive spreadsheet-based model of the costs and benefits of the proposed system investment.
We cannot answer a question we do not entirely understand. However, our understanding is of a very limited kind when we are not able not able to express the problem quantitatively. Just like any scientific endeavor, we must express ourselves quantitatively in order to get a handle on the problem.
There are many tasks that depend on this task as input. We are developing a foundation during this task that will be critical to the rest of the analysis.
Approach
The Clarification stage is usually conducted in a series of workshops that apply specific facilitation tools. These tools will help the participants state the effects of the investment in a more tangible manner. Part of the approach involves focusing on the development of a spreadsheet based decision model as a very precise and unambiguous expression of the decision problem.
By focusing on the development of a specific formula for the decision we force ourselves to think of all factors in the decision as quantities. Most factors perceived to be “intangibles” will fade away and all we will be left with is a set of formulas in a spreadsheet.