FED (Feasibility Evidence Description)

Feasibility Evidence Description (FED)

A.Description of the FED

Sections A1-A3 of this Feasibility Evidence Description (FED) section of the Instructional ICM-Sw guidelines are intended to be read by CS577 students -- to help them understand what the FED is about and how to write their projects’ FED -- and are not intended to be included in the submitted FED.

1.Purpose

The purpose of the Feasibility Evidence Description (FED) is to provide or provide links to the evidence that the development project is feasible and to show that the other artifacts (OCD, SSRD, SSAD, LCP, and Prototypes) are consistent and compatible, in the sense that:

• Satisfaction of all the requirements specified in the SSRD will result in satisfaction of the Win-Win agreements made by the success-critical stakeholders, for all of the operational scenarios and goals (benefits, ROI, etc.) specified in the OCD.
• An implementation of the system whose architecture / design is specified in the SSAD will satisfy all product-related requirements specified in the SSRD and the Prototypes
• Execution of the project described in the LCP will:
• Result in the production of the system whose architecture / design is specified in the SSAD.
• Satisfy all project-related requirements specified in the SSRD, including the requirement ofcompletion within schedule and within budget

One of the main concepts of Incremental Commitment Model (ICM) is to show the evidence that the project is feasible to continue into the next phase. Hence, the FED is to ensure that the system developers have not just created a number of system definition elements, but have also demonstrated the compatibility and consistency of these elements in ensuring, with high probability, the feasibility of accomplishing the desired goals – both product-wise and project-wise. The FED will also encourage the development to regularly analyze the risk of the development project and prepare for risk mitigation and risk management plan.

That such a document is critical to project success is motivated by the fact that the ICM concurrent engineering approach involves different development team members writing different portions of the operational concept, requirements, architecture / design, and project management artifacts.[*]

Construction of the FED should alert developers to inconsistencies within and among project artifacts and to shortfalls in feasibility evidence. It will often not be possible to eliminate all of these shortfalls early in the life cycle; but they should be indentified as uncertainties and risks, and documented in Section 5 (Risk Management) of the FED.

2.FED Life Cycle Process

The Expected Benefits (Section 2.2) and Benefits Chain (Section 2.3) sections of the Operational Concept Description (OCD) should provide starting points for estimating the required client investments, the added value or cost savings resulting from the product’s use, and the resulting business-case return on investment. (ROI)

The VCR and FCR version of the FED will usually be incomplete, as key decisions usually remain to be made during the Foundations phase. Any shortfalls in the evidence for the feasibility with respect to the criteria above should be treated as risks, documented in FED Section 5, and explicit plans for mitigating the risks in FED Section 5. Shortfalls in subsequent versions of the FED should follow the same procedure.

3.Completion Criteria

This section describes the amount of information to be included and their levels of details required for each phase.

3.1Exploration Phase

• Identify risksand their mitigations gathering from the first few meetings with the client including for each risk, its probability of risk, size of loss, and its mitigation plans as included in FED Section 5.

3.2Valuation Phase

• Evidence of feasibility of development process with respect to cost, schedule, staffing, external dependencies
• Evidence of feasibility of at least one feasible architecture as defined in Section A1 (This should be done via appropriate combinations of analysis, measurement, prototyping, simulation, modeling, reference checking, etc.)
• Evidence of feasibility of a viable business case analysis for the system as defined
• Assurance that all major risks are either resolved or covered by a risk management plan in FED Section 5
• Assurance of project feasibility in terms of capability, level of service, and evolutionary requirements

3.3Foundations Phase

• Evidence of feasibility of development process as identified in Section A3.2
• Evidence of feasibility of a feasible architecture as defined in Section A1
• Evidence of feasibility of a viable business case analysis for the system as defined
• Assurance that all major risks are either resolved or covered by a risk management plan in FED Section 5
• Elaboration of Requirements traceability and architecture feasibility
• Assurance of project feasibility in terms of capability, level of service, and evolutionary requirements

3.4Development Phase

• Feasibility rationale for future increments beyond IOC, where applicable
• Validation of business case and Benefits Chain (OCD 2.3) assumptions

B.Sections of the FED Document

The (sub-) sections listed below describe the base format for the FED. For readability by faculty, TA’s, graders, and CS577b students, every submitted FED document (VCFED, FCFED, DCFED, and OC FED) should contain each of the indicated sections, with the title and section numbering unchanged. The text under each (sub-) heading describes what should be in the corresponding (sub-) section. (The texts in these (sub) -sections of the present document are to be read by CS577 students -- to help them prepare the contents of the corresponding (sub-) sections of their FED documents -- and are not intended to be included in the submitted FED’s.)

1.Introduction

1.1Purpose of the FED Document

Summarize any significant difference between the content of the FED and the Win-Win negotiated Agreements. Identify any major FED issues that have not yet been resolved.

1.2Status of the FED Document

Summarize the major changes of FED document comparing to the previous version or the major changes that might affect the project feasibility.

2.Business Case Analysis

In this section you analyze the software system’s return on investment (ROI): ROI= (Benefits-Costs)/Costs by using supporting data from subsections.

2.1Cost Analysis

Costs include actual client costs for system development, transition, operations, and maintenance. Development team costs are zero, but participation by non-developer stakeholders does cost (salary, overhead, etc.). Transition costs can include equipment purchase, facilities preparation, COTS licenses, training, conversion, and data preparation costs. Operation costs can include COTS licenses, supplies, system administration, and database administration costs. Maintenance costs can include hardware and software maintenance. The COCOMO II maintenance estimator can be helpful in producing the cost analysis.

Cost incurred should include one-time and recurring costs of personnel, hardware, software, etc.

2.1.1Personnel Costs

Personnel costs should be estimated in terms of effort. One – time effort includes development and transition effort by clients, users, etc; while recurring effort includes effort operational and maintenance effort.

When the project has zero budget, no hardware and software purchase, it does not mean that there is no cost. Cost can occur in terms of effort or time spent to the project. Table 1 shows example of personnel costs in terms of hours spent to the project. Basically, we do not count number of hours from student development team.

Table 1 shows the example of Personnel Cost calculation; you can use this as a reference or tailor it to fit your project.

Table 1: Example Personnel Costs of Volunteer Tracking System

Activities / Time Spent (Hours)
Development Period (24 weeks)
Valuation and Foundations Phases: Time Invested (CS577a, 12 weeks)
Client: Meeting via email, phone, and other channels [3 hrs/week * 12 weeks * people] / 72
Client Representatives: Meeting via email, phone, and other channels [2 hrs/week * 12 weeks * 2 people] / 48
Architecture Review Boards [1.5 hrs * 2 times * 2 people] / 6
Development and Operation Phases: Time Invested (CS577b, 12 weeks)
Client: Meeting via email, phone, and other channels [5 hrs/week * 12 weeks * 2 people] / 48
Maintainer: Meeting via email, phone, and other channels [8 hrs/week * 12 weeks * 2 people] / 192
Architecture Review Boards and Core Capability Drive-through session [1.5 hrs * 3 times * 2 people] / 9
Deployment of system in operation phase and training
- Installation & Deployment [5 hrs * 3 times * 2 people]
- Training & Support [5 hrs * 2 times * 2 people] / 50
Total / 425
Maintenance Period (1 year)
Maintenance [3 hr/week * 52 weeks] / 156
Total / 156

2.1.2Hardware and Software Costs

Table 2 and Table 3 shows example of cost that occurs in Volunteer Tracking System Project. Table 2 shows hardware and software cost required during the development period while Table 3 shows the cost required after the transition.

If your project does not acquire any new hardware or software, you should also provide supporting rationale such as using current server, or existing software.

Table 2: Hardware and Software Costs – Development

Type / Cost / Rationale
Hardware – Web Server / $1500 / A new machine is needed to act as a web server for the system.
Hardware – Web Hosting / $200/year / Although the CSC already has its own host, this new system requires additional cost based on the annually hosting fee. Starting from fall 2006, until the end of spring 2007, this is an amount needed to be spent.
Software – Adobe Dreamweaver CS3 / $399 / Used in developing the system and the team website.

Table 3: Hardware and Software Costs – Operation

Type / Cost / Rationale
Hardware – Web Server / $0 / Since the development machine can be used as a operation machine, no cost is required here.
Hardware – Web Hosting / $200/year / Although the CSC already has its own host, this new system still requires additional cost based on the annually hosting fee.

2.2Benefit Analysis

Possible benefits are expressed in financial terms compared to costs, such as increased sales and profits, or reduced operating costs.

Non-financial benefits and costs should also be included. For example, the reduced amount of man-hour from using the automated system as shown in Table 4.

The value added may also describe non-monetary improvements, which can be critical in customer support and satisfaction. Include the non-monetary benefits and its description such as:

• Increase in organizational reputation: organization’s website will be the main resource in the public relation activities
• Easier for sponsor to donate: with the online donation, sponsor will simply go to the website, which will provide convenience to the sponsor.

Table 4 shows example of benefit in terms of hours saved from using the developed application.

Table 4: Benefits of CaliforniaScienceCenter System

Current activities & resources used / % Reduce / Time Saved (Hours/Year)
Application data entry
- Volunteer coordinator
50 applications * 10 mins = 500 mins / 90% / 7.5
Time sheet data entry
- Volunteer coordinator
5 hrs * 52 weeks / 90% / 234
Job request
- Supervisor (7 departments)
7 * 1 hr * 52 weeks / 50% / 182
- Volunteer coordinator
1 hr * 52 weeks / 50% / 26
Job assignment
- Volunteer coordinator
10 hr * 52 weeks / 60% / 312
Total / 761.5

2.3ROI Analysis

Include summary or result of your Return-On-Investment (ROI) analysis and show evidence or assurance of project feasibility.

Table 5 shows the example of ROI analysis while Figure 1 indicates the result of ROI Analysis plotted in graphical information.

Table 5: Example ROI Analysis of Volunteer Tracking System

Year / Cost / Benefit
(Effort Saved) / Cumulative Cost / Cumulative Benefit / ROI
2007 / 425 / 0 / 425 / 0 / -1.00
2008 / 156 / 761.5 / 581 / 761.5 / 0.31
2009 / 171.6 / 761.5 / 752.6 / 1,523 / 1.02
2010 / 188.76 / 761.5 / 941.36 / 2,284.5 / 1.43

From the above table, the first row (year 2007) refers to the development period and is yet to have any benefits. 425 hours is invested into the system and is considered as a cost. In the next three rows: 2008, 2009, and 2010, they are considered to be in an maintenance period and require some personnel costs in terms of hours needed to maintain the system. The values 156, 171.6, and 188.76 hours are 10% increased annually while 761.5 hours of effort are saved from using this automated system.

Figure 1: Example ROI Graph of Volunteer Tracking System

The break-even point is where the ROI value is equal to the value of 1; that means the amount of cost is equal to the amount of returned value. From the figure, it can be concluded that you will get your return-on-investment within three years, or in 2009.

There are mistakes that are often done by students regarding the ROI calculation including:

-The unmatched cost/benefit values from benefit table and ROI calculation.

-Failing to include hardware/software related cost/benefit into the ROI calculation.

-Failing to use accumulated cost/benefit.

-Failing to point out the year that the system cost pays off. (Having ROI=1)

3.Architecture Feasibility

In this section you provide arguments to justify the feasibility of achieving the desire system architecture.

3.1Level of Service Feasibility

Provide arguments to justify the Level of Service (L.O.S.) requirements specified in the SSRD. You do this by demonstrating -- through analysis, detailed references to prototypes, models, simulations, etc. -- how the designs will satisfy the SSRD L.O.S. requirements (SSRD Section 5). Complete coverage of the L.O.S. requirements is essential. If applicable, provide links to files containing detailed evidence of the feasibility of required levels of performance, interoperability, and dependability.

Note that ambitious Level of Service requirements and their tradeoffs can be the most difficult requirements to satisfy, and the most difficult requirements for which to argue satisfiability in advance of actual implementation. Table 6 summarizes some useful product and process strategies for addressing Level of Service requirements.

Table 6: Level of Service Product and Process Strategies

Attributes / Product Strategies / Process Strategies
Dependability / Accuracy Optimization, Backup/ Recovery, Diagnostics, Error-reducing User Input/output, Fault-tolerance Functions, Input Acceptability Checking, Integrity Functions, Intrusion Detection & Handling, Layering, Modularity, Monitoring & Control, Redundancy / Failure Modes & Effects Analysis, Fault Tree Analysis, Formal Specification & Verification, Peer Reviews, Penetration, Regression Test, Requirements/Design V & V, Stress Testing, Test Plans & Tools
Interoperability / Generality, Integrity Functions, Interface Specification, Layering, Modularity, Self-containedness / Interface Change Control, Interface Definition Tools, Interoperator Involvement, Specification Verification
Usability / Error-reducing User Input/output, Help/ explanation, Modularity, Navigation, Parameterization, UI Consistency, UI Flexibility, Undo, User-programmability, User-tailoring / Prototyping, Usage Monitoring & Analysis, User Engineering, User Interface Tools, User Involvement
Performance / Descoping, Domain Architecture-driven, Optimization (Code/ Algorithm), Platform-feature Exploitation / Benchmarking, Modeling, Performance Analysis, Prototyping, Simulation, Tuning, User Involvement
Adaptability (Evolvability / Portability) / Generality, Input Assertion/type Checking, Layering, Modularity, Parameterization, Self-containedness, Understandability, User-programmability, User-tailorability, Verifiability / Benchmarking, Maintainers & User Involvement, Portability Vector Specification, Prototyping, Requirement Growth Vector Specification & Verification
Development Cost / Schedule / Descoping, Domain Architecture-driven, Modularity, Reuse / Design To Cost/schedule, Early Error Elimination Tools And Techniques, Personnel/Management, Process Automation, Reuse-oriented Processes, User & Customer Involvement
Reusability / Domain Architecture-driven, Portability Functions / Domain Architecting, Reuser Involvement, Reuse Vector Specification & Verification
Interoperability / System development with modular and layer design for future evolutions, integration testing / Interface change control, Interoperation Involvement, Specification Verification, Prototyping
All of Above / Descoping, Domain Architecture-driven, Reuse (For Attributes Possessed By Reusable Assets) / Analysis, Continuous Process Improvement, Incentivization, Peer Reviews, Personnel/Management Focus, Planning Focus, Requirement/ design V&V, Review Emphases, Tool Focus, Total Quality Management

3.2Capability Feasibility

Provide arguments to justify the capability requirements specified in the SSRD. You do this by demonstrating -- through analysis, detailed references to prototypes, models, simulations, etc. -- how the designs will satisfy the SSRD capability requirements (SSRD Section 3). Complete coverage of the capability requirements is essential.

3.3Evolutionary Feasibility

Provide arguments to justify the evolutionary requirements specified in the SSRD. You do this by demonstrating -- through analysis, detailed references to prototypes, models, simulations, etc. -- how the designs will satisfy the SSRD evolutionary requirements (SSRD Section 6). Complete coverage of the evolutionary requirements is essential.

4.Process Feasibility

In Table 7, process decision tableis shown. From this table, you should provide your choice of case that you find to be the most suitable with your project characteristics. Describe the rationales and supporting evidences for that decision.

Table 7: Common Risk-Driven Special Cases of the ICM

Special Case / Example / Size, Complexity / Change Rate % /Month / Criticality / NDI Support / Org, Personnel Capability / Key Stage I Activities : Incremental Definition / Key Stage II Activities: Incremental Development, Operations / Time per Build; per Increment
1. Use NDI / Small Accounting / Complete / Acquire NDI / Use NDI
2. Agile / E-services / Low / 1 – 30 / Low-Med / Good;
In place / Agile-ready, Med-high / Skip Valuation , Foundations phases / Scrum plus agile methods of choice / <= 1 day; 2-6 weeks
3. Architected Agile / Business data processing / Med / 1 – 10 / Med-High / Good; Most in place / Agile-ready, Med-high / Combine Valuation, Foundations phases. Complete NDI preparation / Architecture-based Scrum of Scrums / 2-4 weeks; 2-6 months
4. Formal Methods / Security kernel; Safety-critical LSI chip / Low / 0.3 / Extra High / None / Strong formal methods experience / Precise formal specification / Formally-based programming language; formal verification / 1-5 days; 1-4 weeks
5. HW component with embedded SW / Multi-sensor control device / Low / 0.3 – 1 / Med-Very High / Good;
In place / Experienced; med-high / Concurrent HW/SW engineering. CDR-level ICM DCR / IOC Development, LRIP, FRP. Concurrent Version N+1 engineering / SW: 1-5 days; Market-driven
6. Indivisible IOC / Complete vehicle platform / Med – High / 0.3 – 1 / High-Very High / Some in place / Experienced; med-high / Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve / Drop deferrable features to meet conservative cost. Strong award fee for features not dropped / SW: 2-6 weeks; Platform: 6-18 months
7. NDI- Intensive / Supply Chain Management / Med – High / 0.3 – 3 / Med- Very High / NDI-driven architecture / NDI-experienced; Med-high / Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/ architecture definition / Pro-active NDI evolution influencing, NDI upgrade synchronization / SW: 1-4 weeks; System: 6-18 months
8. Hybrid agile / plan-driven system / C4ISR / Med – Very High / Mixed parts: 1 – 10 / Mixed parts; Med-Very High / Mixed parts / Mixed parts / Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces) / Full ICM ,three-team incremental development, concurrent V&V, next-increment rebaselining / 1-2 months; 9-18 months
9. Multi-owner system of systems / Net-centric military operations / Very High / Mixed parts: 1 – 10 / Very High / Many NDIs; Some in place / Related experience, med-high / Full ICM; extensive multi-owner team building, negotiation / Full ICM; large ongoing system/software engineering effort / 2-4 months; 18-24 months
10. Family of systems / Medical Device Product Line / Med – Very High / 1 – 3 / Med – Very High / Some in place / Related experience, med – high / Full ICM; Full stakeholder participation in product line scoping. Strong business case / Full ICM. Extra resources for first system, version control, multi-stakeholder support / 1-2 months; 9-18 months

You also need to provide specific content of prioritized capabilities from SSRD and show the evidence that the project is feasible by using the estimated Effort (Person-months) and Schedule from Budgets (LCP Section 5) in the sense that staffing levels are sufficient and the project is achievable within the schedule. It is important to use a credible and repeatable estimation technique for the Effort and the Schedule; however, presenting a link to the actual calculation file/source (i.e. COCOMO tool) would be enough.