Enterprise Modeling:
Improving the Analysis of Alternatives Process for
IT Solutions Government Contract Proposals
Proposal Final Report - 12/05/2011
Abstract
An Analysis of Alternatives (AoA) process for a proposal development is studied with an interest in increasing the time efficiency of the process, both in terms of the mean time duration of the process and the variability of that mean. The AoA process flow is defined in detail along with task categories included and simple, medium, and complex relative time durations of AoA sub-processes. Current process inefficiencies result in a need for a new system to decrease the mean time required for AoA by 33%, and the variability of that mean by 25%. Preliminary alternative solutions are determined based on requirements for the system and a detailed understanding of the AoA process. A simulation model of the AoA process is designed and tested preliminarily to determine the method for evaluating alternative solutions. It is found that the simulation can be used to determine the effect of a given alternative on the mean time duration of AoA.
Submitted to: Dr. Sherry
Submitted for: SYST 490 - Senior Design
Project Sponsors:
Martin Nordberg
Rob Oates
(Vangent, Inc)
Submitted by:
Saad El Beleidy
Peyman Jamshidi
Jared Kovacs
Gabriel Lewis
Table of Contents
1.0 Context
2.0 Stakeholder Analysis
2.1 Key Stakeholders
2.2 Primary Stakeholders
2.3 Stakeholder Goals
2.4 Stakeholder Conflict
3.0 Scope
4.0 Problem Statement
5.0 Statement of Need
6.0 Mission Requirements
7.0 Design Alternatives
7.1 Added Architect Alternative
7.2 Database Alternative
8.0 Simulation Design
8.1 Data
8.2 Preliminary Model
8.3 Preliminary Results
9.0 Project Management
References
1.0 Context
Government Contracting
Information Technology (IT) solutions government contracting companies are private commercial organizations that earn revenue through providing IT or IT-related solutions and services to meet needs within the government. A high level process by which needs are identified and solutions proposed and selected is described below in Figure 1.
The process begins when a need for a new system is identified in one of many government entities, such as government offices or field operators, who develop requirements based on their needs. These requirements are formed into a formal solicitation, such a Request for Proposal (RFP), by an acquisitions committee,
The acquisitions committee is responsible for acquiring new systems to meet government needs and ensuring an unbiased approach to evaluating proposed solutions, and so is required by law to be independent of the government entity voicing the need.
Once the formal solicitation has been made public, the contractor makes a bid decision--whether or not to pursue the solicitation by submitting a proposal to answer the need. If the contractor decides to pursue the solicitation, it develops a proposal via three roughly parallel processes: 1) Technical Solutions Development, where the technical solution is designed, 2) Proposal Writing, in which a proposal writing team, composed of both technical experts and writing experts, will craft the formal proposal, and finally 3) Budget and Management, which consists of the management of the Technical Solutions Development and Proposal Writing processes. The end result of this proposal development effort is the formal written proposal, which is submitted back to the acquisitions committee.
Any number of proposals may be submitted from various government contractors, so the acquisitions committee must evaluate all proposals and select the “best alternative” based on defined requirements. Once the selection is complete, the selected solution is provided to the government entity by the contractor.
(Figure 1: Level 1 Diagram, High Level Solicitation and Proposal Generation Process[1])
Sponsor: Civilian and National Security Division, Vangent Inc.
This study is sponsored by Vangent, Inc. and in particular Vangent’s Civilian and National Security (CNS) Division. Vangent Inc. is an government contractor based in the greater Washington, D.C. area specializing in consulting, information technology and management solutions as well as business process solutions and system integration services. Vangent employs about 7500 people across the country, and provides solutions to U.S. and international governments, as well as some private organizations such as educational institutions and other corporations. The contracts that Vangent wins and the associated solutions provided are usually quite large in value, ranging from several million dollars to over one hundred million dollars in value.
The CNS division within Vangent provides solutions to key customers such as the U.S. Department of State, Department of Education, and Department of Labor. It responds with proposals to between 15 and 25 solicitations per year. The proposals developed by the CNS division are highly complicated in nature, often incorporating multiple types of technology into one solution.
The environment in which the CNS division operates is highly competitive. Each proposal generated by the CNS division is estimated to have five to ten competing proposals submitted by other contractors in response to the same solicitation. It is expected that recent budget cuts within the government as well as future expected cuts will only increase the competitive nature of this environment. Efficiency is therefore key in all government contractor operations, especially in the process of developing proposals, as this is the crux of where revenue is made or lost.
The Technical Solution Development Process
An in-depth design process is undergone by the contractor to determine the technical solution to propose in response to the government entity’s need. The process by which the technical solution is developed in order to be proposed to the solicitor in response to the need is called, as previously mentioned, the “Technical Solution Development” process. This phase of proposal development is conducted in three steps: 1) Analysis of Alternatives (AoA), 2) Alternatives Analysis, and 3) Integration of Assets. In Analysis of Alternatives, the various possibilities for solutions the contractor could propose are identified and ranked, resulting in a “Ranked List of Alternatives” based on the solicitor’s requirements as documented in the solicitation, which is the primary input to this step. Alternatives Analysis consists of activities related to determining whether or not the previously determined alternatives actually meet the solicitor’s requirements, regardless of their rank in the analysis. The previous two steps may be repeated many times if there are many different technologies to consider in the solution. Once all technologies have been addressed, all assets, including technologies, to be included in the proposed solution are integrated into a coherent whole, resulting in the solution to be proposed.
(Figure 2: Level 2 Diagram, Technical Solution Development Process for Proposal Development)[2]
The Analysis of Alternatives (AoA) Process
The process by which alternatives solutions to propose are determined and evaluated is given by the project sponsors as Vangent’s Decision and Analysis Resolution process and is divided into four sub-processes: Define the Problem Domain, Define Evaluation Criteria, Explore Alternate Solutions, and Evaluate Solutions (Figure 3). The primary input for AoA is the solicitation from the solicitor, and the final output of AoA is the “ranked list of alternatives” mentioned previously. For Vangent’s CNS division, the “Solutions Architect”, a technical expert employee, is the primary (and usually the only) person conducting the AoA process, and will from here on be referred to as the performer of AoA.
(Figure 3: Level 3 Diagram, AoA Sub-processes: the Decision and Analysis Resolution)[3]
In the “Define the Problem Domain” process (seen in Figure 4), the requirements for alternate solutions are defined and rolled up into “key” requirements. These requirements are obtained first directly from the solicitation, but are also gained from a “technical reference model” which is also provided by the solicitor. This document contains requirements and recommendations focused on the technical functionality aspects of the government entity operations, including the solicitors past experience and, where relevant, preferred functionality or vendors of technology. Another input is customer knowledge gained through public Q&A sessions which the contractor may or may not choose to participate in, based on the subjective assessment of the Solutions Architect as to whether or not it is needed. Other factors in his/her decision include the risks of unintentionally sharing expertise through public questions, or revealing weaknesses to be exploited.
The requirements are divided into two categories: functional and non-functional. Functional requirements include “technical” and “environmental” requirements. “Technical” functional requirements are those dealing with the technical aspects of the systems functionality, such as performance, input/output, or interface requirements. “Environmental” functional requirements refer to those concerning the systems impact on its environment, including the working area and the system’s users. Non-functional requirements are divided into “technical”, “environmental”, “financial”, and “political” requirements. Non-functional “technical” requirements refer to those parts of the system’s technical aspects which are not functionality related, and “environmental” non-functional requirements refer to the same for environmental concerns. “Financial” requirements refer to cost constraints and other financially-oriented concerns of the system, and “political” requirements are drawn from the current political climate or past laws that place constraints on the system.
Once these requirements are defined, they are “rolled up” into “key” functional and non-functional requirements. This is done by a process of determining, based on the solution architect’s expert knowledge as well as the solicitations specifications, which requirements are more important and should be kept separate from the rest, and which can be aggregated, or “rolled” into a collection represented by a “key” requirement.
(Figure 4: Level 4, Define the Problem Domain)
The “Define Evaluation Criteria” sub-process (figure 5) includes those actions necessary to develop clear, measurable evaluation criteria from the previously defined requirements to be used in the evaluation of alternatives. Criteria are also defined from the solutions architect’s past experience. The criteria are divided into two parts: technical criteria, and business criteria. As can be imagined, technical criteria have to do with the technical aspects of the system, such as performance, related to functionality, which is measured by performance metrics. Business criteria have to do with less functional aspects of the system and more financial, cost-related aspects, in addition to other qualitatively defined aspects of the system. The evaluation criteria are documented as they are defined, and compiled into a coherent set of evaluation criteria.
(Figure 5: Level 4, Define Evaluation Criteria)
The third sub-process of AoA is “Explore Alternate Solutions” (Figure 6). In this sub-process, the Solutions Architect researches and uses whatever means available to identify reasonable candidate solutions for the proposed solution. The activities in this sub-process are obtaining an subject matter expert’s (SME) opinion, conducting personal market research in literature and Internet resources, conducting internal surveys to determine the capabilities of the company with regards to the technology at hand, and finally, using industry research organizations (IROs). IROs are organizations which conduct and maintain thorough market research in various technology areas, and then make their findings available for purchase. The solutions architect makes numerous run-time decisions on which exactly are necessary of the possible methods for determining alternatives. For example, if the proposal deals with a very simple and well known technology area, the solutions architect is less likely to consult a SME, but if the problem is complex and unfamiliar, he/she will almost certainly find and consult a SME. The alternatives are documented as they are identified and compiled into a coherent set of alternatives.
(Figure 6: Level 4, Explore Alternate Solutions)
The final sub-process of AoA is the evaluation of the alternate solutions, or “Evaluate Solutions” (Figure 7). In this phase, each alternate solution is evaluated against each evaluation criteria via five different methods of analysis to develop a ranking for each alternative. The methods are separated into qualitative analyses and quantitative analyses. The qualitative analyses are conducted either by a somewhat informal pro/con analysis or a formal Kepner-Tregoe analysis. The solutions architect makes a judgment call as to whether or not to conduct the pro/con analysis (for smaller, less complex solutions) or the Kepner Tregoe analysis (for larger and more complex solutions). Both of these analyses depend highly on the subjective assessment of the solutions architect, rather than a predefined scoring mechanism. Conversely, the quantitative analyses include a cost analysis based on the costs of the individual alternate solutions as well as a quantitative benefit analysis based on numerically measured aspects of the system, such as performance metrics. All these analyses are then wrapped up in a cost-benefit analysis to determine the final ranking of alternatives. The final output of the cost-benefit analysis and of the entire AoA process is the ranked list of alternatives.
(Figure 7: Level 4, Evaluate Solutions)
Final Output of AoA: The Ranked List of Alternatives
The final output and goal of the AoA process, the ranked list of alternatives, is a document, often including a table, showing the scoring of each alternate solution against each evaluation criteria. It may take several different forms based on the solutions architect’s preference, but will always include a description of alternatives considered and methods of analysis, with a ranked list of the alternatives. The table, if included, shows in matrix format the various alternatives weighed against the evaluation criteria and calculations of total scores, including the weights associated with each criterion if relevant. This is the final goal of the AoA process. Any system built to support the AoA process will be designed with this final goal in mind--to support the process of creating the ranked list of alternatives.
Task Categories in the AoA Process
Due to the fact that most of the AoA process is intellectual labor, there are multiple categories of tasks involved. These task categories reflect the nature of the work performed for specific tasks within the AoA sub-processes. From the categorization of tasks insight is gained into the cause of the mean time duration and variability of that mean time sub-processes and also into potential alternatives to consider. The task categories are 1) Labor intensive, 2) Decision Making, 3) Experience Recall, and 4) Networking (see Table 1).
“Labor Intensive” tasks are those emphasizing manual labor or work not very effortful but potentially highly time consuming. While the mean time duration for labor intensive may vary based on the size of the task, the variability of the mean remains low. The practical significance of this is that for labor intensive tasks, the solutions architect can depend on the task to take a certain amount of time. Labor intensive tasks in AoA include the requirements definition from solicitation and technical references models in the “Define Problem Domain” sub-process, the compilation of evaluation criteria in the “Define Evaluation Criteria” sub-process, conducting personal research for alternatives and compiling alternatives during the “Explore Alternate Solutions” sub-process, and finally the cost analysis in the “Evaluate Solutions” sub-process.
“Decision Making” tasks are those which require an expert to make a expertise-based decision or judgment call. These tasks are highly variable in their mean time duration because the number of aspects of the decision that the solutions architect must consider vary widely depending on the specific circumstances surrounding the decision task. Decision making tasks appear in the “rolling up” of requirements into key requirements in the “Define Problem Domain” sub-process, determining evaluation criteria from key requirements in the “Define Evaluation Criteria” sub-process, consulting subject matter experts, administering internal surveys, and utilizing industry research organizations within the “Explore Alternate Solutions” sub-process, and finally in the qualitative analyses during the “Evaluate Solutions” sub-process.
The “Experience Recall” task category involves those tasks in which the solutions architect relies on his or her own relevant past experience in order to make complete tasks and make subjective assessments in areas like researching, and qualitative aspects of AoA. Experience recall tasks appear in the “rolling up” of requirements into key requirements in the “Define Problem Domain” sub-process, developing evaluation criteria from lessons learned (i.e. past experience) in the “Define Evaluation Criteria” sub-process, and conducting manual research during the “Explore Alternate Solutions” sub-process.
The last task category is “Networking”, which refers to those tasks which are conducted through personal networking with contacts to accomplish tasks, such as gaining information during the research phase of AoA. The networking category occurs primarily in consulting with subject matter experts (SMEs) during the “Explore Alternate Solutions” sub-process of AoA.
(Table 1: Task Categories in AoA Sub-Processes)
Relative Time Duration of AoA Sub-Processes
Relative mean time durations for each sub-process of the AoA process have been determined as seen in Table 2. The difference in time durations are also captured for various proposal problem complexities, which are known to fluctuate at certain probabilities. These probabilities are shown in Table 3. The shortest mean duration sub-process is “Define Evaluation Criteria” with a relative duration of 10% of the whole AoA for a normal complexity AoA. The second shortest relative time duration is the “Define the Problem Domain” sub-process, with a 20% relative time duration for normal cases. “Explore Alternate Solutions” and “Evaluate Solutions” are larger, comprising 30% and 40% respectively of the entire AoA for normal cases. Relative time durations for complex and simple cases are shown in Table 2. The probabilities of the complex, medium, and simple AoA situations are shown in Table3, and vary for each sub-process. These, of course, effect the overall time duration of the AoA process.