Independent Verification & Validation Program / Research Program / IVV 09-3
Revision: R
Effective Date:
February 09, 2009

DOWNLOADED AND/OR HARD COPY UNCONTROLLED

Verify that this is the correct version before use.

APPROVAL SIGNATURES / DATE
Gregory Blaney (original signature on file) / IMS Representative / 02/09/2009
REVISION HISTORY /
Revision / Description of Change / Author / Effective Date /
Initial / Initial Release / Cynthia Calhoun IT/309 / 08/22/1997
A / Added 5.2 Process Verifications.
Added NASA Policy Guideline NPG 1441.1 to References. Added section 8 (Metrics).
Format and content changes. / Cynthia Calhoun IT/309 / 04/09/1998
B / Format changes / Cynthia Calhoun IT/309 / 05/05/1998
C / Added “Financial Management” section.
Added references to 53.IT.0009-4 SLP in section 6.1.7.2.
Modified sections 1.0, 6.1.2, 6.1.4.3, 6.1.7.1, 6.1.7.2, 6.1.7.3, 6.1.8, 6.2 / Cynthia Calhoun IT/309 / 07/23/1998
D / Quality Record - format change / Cynthia Calhoun IT/309 / 08/26/1998
E / Revised the Resource Manager responsibility.
Added the OSMA Annual Review Report, OSMA Software Program Level 1 Technical Program Plan, and OSMA Software Program Operating Plan as Quality Records.
Updated Center Initiative Management Process to reflect the changes in the Level 1 Technical Program Plan. / Cynthia Calhoun IT/309 / 04/16/1999
F / References to Ames Quality Manual replaced with references to IV&V Facility Quality Manual / Cynthia Calhoun IT/309 / 09/10/1999
G / Reference to Process Verification Work Instruction (IVV 09-3-1) in section 6.2 / Cynthia Calhoun IT/309 / 03/10/2000
H / Format and Number changes; Delete Reference to Ames Research Center / Griggs / 12/06/2000
I / Rewrite of document to reflect refinement of procedure resulting from transfer to GSFC / Shandra Auvil / 07/02/2001
J / Changed item 6.5 / Wesley Deadrick / 05/15/2002
K / Rewrite to reflect NRA and DDF scope additions / Frank Gmeindl / 04/08/2003
L / Updated reference documents to include relevant standards. Evaluated and updated the metrics collected as a result of this process. / Wes Deadrick / 11/03/2004
M / Aligned with current practice / Frank Gmeindl / 06/08/2005
N / Aligned with other IV&V Facility documentation / Lisa Montgomery / 02/07/2006
O / Changes from 08/11/2006 audit / Frank Gmeindl / 08/16/2006
P / Remove references to cancelled 09-3-4, remove record, add research infusion initiatives, delete RCT / Frank Gmeindl / 09/07/2007
Q / Update process flow diagrams to align with Facility Management paradigm / Stephanie Ferguson / 08/05/2008
R / Changed “IV&V Facility” to “IV&V Program”; updated to align with current NASA IV&V roles and other documentation; updated Section 6.0, Records / Stephanie Ferguson / 02/09/2009
REFERENCE DOCUMENTS
Document / Title
IVV QM / IV&V Quality Manual
IVV 06-2 / Work Instruction for Processing Procurements for Grants and Cooperative Agreements
IVV 09-3-1 / Work Instruction for Evaluating Research Initiatives
IVV 09-3-2 / Work Instruction for Selecting OSMA SARP Research Initiatives
IVV 09-3-3 / Work Instruction for Selecting IV&V Facility Research Initiatives
IVV 09-3-5 / Work Instruction for Processing Research Products
IVV 09-3-6 / Work Instruction for Conducting Research Initiative Performance Reviews
IVV 09-3-7 / Work Instruction for the Publication/Presentation of Research Results
IVV 16 / Control of Records
NPR 1441.1 / NASA Records Retention Schedules

1.0  Purpose

The purpose of this system level procedure (SLP) is to establish guidance for defining and managing both the Office of Safety and Mission Assurance (OSMA) Software Assurance Research Program (SARP) and research undertaken for the NASA IV&V Program.

2.0  Scope

This SLP is applicable to:

1.  The day-to-day oversight and management of the SARP, sponsored by OSMA

2.  The NASA IV&V Program’s own internally funded Research Program

This SLP addresses research program planning; research proposal solicitation, evaluation and selection; and Research Initiative management, including the management and publication of results.

2.1  Research Funding

Funding for the OSMA SARP is managed by the NASA IV&V Program. Funds are used for Center Initiatives (CIs) and Research Infusion Initiatives (RIIs) conducted at various NASA locations. The funds’ delivery cycle generally begins with funding allocation recommendations generated by the SARP. The recommendations are approved by OSMA, which in turn distributes the funding to the various NASA locations, which can include NASA IV&V. The NASA locations and NASA IV&V then distribute the funding to the corresponding initiatives.

The NASA IV&V Program may choose to directly fund Facility Initiatives (FIs), which conduct research to improve IV&V methods and practices. These initiatives are often, though not exclusively, conducted at NASA IV&V.

2.2 Types of Initiatives

The NASA IV&V Program manages, or has oversight of, four types of research projects:

1.  Center Initiatives (CI) – OSMA SARP research conducted at NASA Centers or at NASA IV&V

2.  Facility Initiatives (FI) – IV&V research funded by the NASA IV&V Program

3.  Research Infusion Initiative (RII) – OSMA SARP projects conducted at NASA Centers or at NASA IV&V for the purpose of transferring technology that was developed under research into practice

2.3 Initiative Management Structure

The Chief of Research oversees all Research Initiatives. Each Research Initiative has a NASA Point of Contact (NPOC) and a Principal Investigator (PI).

The Chief of Research provides oversight to NPOCs and PIs. The NPOC ensures that a PI’s research products are accepted by NASA and are retained in the Center Initiative Management (CIM) Tool. The PI actually implements the research.

2.3.1 NASA IV&V Management of the OSMA SARP

All NPOCs are civil service employees or NASA Jet Propulsion Laboratory (JPL) employees. NPOCs are located at NASA Centers or at JPL.

Research Initiatives can be performed by NASA civil service employees, JPL employees, or contractors. If a Research Initiative is performed by a contractor, the NPOC is usually, although not exclusively, the Contracting Officer Technical Representative (COTR) for the contractual vehicle under which the work is conducted.

The NASA Centers and JPL have their own contracting offices. The NPOCs for FIs and the Contracting Officer (CO) for FIs reside at NASA IV&V.

PIs can be NASA civil service employees or contractors. PIs can be located at NASA Centers, JPL, NASA IV&V, or at contractor sites.

2.3.2 Commonalities between OSMA SARP and IV&V-Funded Initiatives

The following are common features of the NASA IV&V Program's management of the OSMA SARP and NASA IV&V Program-funded research.

·  The CIM Tool is the data repository for all Research Initiative proposals, contracts, deliverables, and progress reports.

·  All Research Initiatives have a designated NPOC and a PI.

2.3.3 Differences Between OSMA SARP and IV&V-Funded Initiatives

SARP Initiatives / DDF Initiatives
Selection Instrument / Level 1 Plan – Program Operating Plan (POP) call and/or NASA Research Announcement (NRA) / Notice of Intent (NOI), Request for Proposal (RFP)
Selection Process / Source Evaluation Board / Director and Chief of Research
Management Structure / OSMA delegates day-to-day management to the NASA IV&V Program / Director delegates day-to-day management to the Chief of Research
NPOC locations / NASA Centers
NASA IV&V Program / NASA IV&V Program only
Initiative types / CIs
RIIs / FIs


3.0 Definitions and Acronyms

Official NASA IV&V roles and terms are defined in the Quality Manual. Specialized definitions identified in this SLP are defined below.

3.0 

3.1  Acronyms

CI / Center Initiative
CIM / Center Initiative Management
FI / Facility Initiative
FRC / Federal Records Center
IMS / NASA IV&V Management System
JPL / Jet Propulsion Laboratory
NOI / Notice of Intent
NPOC / NASA Point of Contact
NPR / NASA Procedural Requirement
NRA / NASA Research Announcement
OSMA / Office of Safety and Mission Assurance
PI / Principal Investigator
POP / Program Operating Plan
QM / Quality Manual
RFP
RII / Request for Proposal
Research Infusion Initiative
RRWS / Research Results Web Site
SARP / Software Assurance Research Program
SAS / Software Assurance Symposium
SEB / Source Evaluation Board
SLP / System Level Procedure
SWG / Software Working Group

4.0  Process Flow Diagram

The following diagram depicts processes described in this document, and the responsibilities and actions that shall be performed by process participants. Any information supplemental to the depicted processes will appear after the diagram.

4.1  Determine Research Requirements

To determine research topics, the Chief of Research:

·  OSMA SARP topics

o  Queries or discusses with NASA project personnel, NASA Headquarters personnel, Software Working Group (SWG) personnel, and Software Assurance Working Group personnel

·  NASA IV&V topics

o  Queries stakeholders

The Chief of Research documents and publishes selected research topics on the NASA IV&V web site in preparation for proposal solicitation.

Please see IVV 09-3-1, Work Instruction for Evaluating Research Initiatives, for additional information.

4.2  Evaluate Ongoing Research

The Chief of Research conducts an annual evaluation of ongoing Research Initiatives to develop funding recommendations for the following year.

Please see IVV 09-3-1, Work Instruction for Evaluating Research Initiatives, and IVV 09-3-6, Work Instruction for Conducting Research Initiative Performance Reviews, for additional information.

4.3  Select OSMA SARP Initiatives

The Chief of Research is responsible for the formulation and management of the SARP Evaluation Board (SEB). The SEB recommends funding for Research Initiatives (continued and new) based on proposal evaluations and amount of available funds. OSMA selects proposals for award based on the SEB recommendations.

The Chief of Research drafts the Level 2 Plan, which defines the funding levels for ongoing Research Initiatives and serves as OSMA’s selection of proposals for award.

Please see IVV 09-3-2, Work Instruction for Selecting OSMA SARP Research Initiatives, for additional information.

4.4  Select Facility Initiatives

The Chief of Research solicits proposals when a need for research is identified by the NASA IV&V Program.

The Chief of Research evaluates the proposals and prepares a recommendation for the NASA IV&V Director. The NASA IV&V Director approves the plan, thus selecting for award the proposals that offer the most benefit directly to the NASA IV&V Program.

Please see IVV 09-3-3, Work Instruction for Selecting IV&V Facility Research Initiatives, for additional information.

4.5  Fund Ongoing Initiatives

NASA Centers fund existing grants or contracts according to the guidance contained in the approved Level 2 Plan. The assigned procurement officer funds existing contracts or grants and awards new grants or contracts for NASA IV&V.

Please see IVV 09-3-1, Work Instruction for Evaluating Research Initiatives, or IVV 06-2, Work Instruction for Processing Procurements for Grants and Cooperative Agreements, for additional information.

4.6  Manage Research Initiatives

The Chief of Research, PIs, and NPOCs use the CIM Tool to manage deliverables, status reports, and performance reviews.

Please see IVV 09-3-1, Work Instruction for Evaluating Research Initiatives; IVV 09-3-5, Work Instruction for Processing Research Products; and IVV 09-3-6, Work Instruction for Conducting Research Initiative Performance Reviews, for additional information.

4.7  Publish Results

Research results are published at the annual Software Assurance Symposium (SAS) and on the NASA IV&V Research Results Web Site (RRWS). The Chief of Research publishes deliverables that will provide significant value to the software assurance and engineering community.

Please see IVV 09-3-7, Work Instruction for the Publication/Presentation of Research Results, for additional information.

5.0  Metrics

Any metrics associated with this SLP are established and tracked within the NASA IV&V Metrics Program.

6.0  Records

The following records will be generated and filed in accordance with this SLP and IVV 16, Control of Records, and in reference to NASA Procedural Requirement (NPR) 1441.1, NASA Records Retention Schedules.

Record Name / Original / Vital / Responsible Person / Retention Requirement / Location /
OSMA SARP Level 1 Technical Program Plan / Y / Y / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / Filing System
Software Assurance NRA / N / N / Chief of Research / Destroy when superseded, obsolete, or when 3 years old, whichever is sooner.
(5/37C1) / Filing System
Research Portfolio Plan / Y / Y / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / Filing System
Proposals / N / N / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / CIM Tool
OSMA SARP Level 2 Technical Program Plan / Y / Y / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / Filing System
Grants and Contracts / N / N / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / Filing System
Research Initiative Record in the CIM Tool / Y / Y / Chief of Research / Delete/destroy 6 years, 3 months after completion of grant/contract/agreement. (5/37A) / CIM Tool

CHECK THE MASTER LIST at http://ims.ivv.nasa.gov/

VERIFY THAT THIS IS THE CORRECT REVISION BEFORE USE

1 of 12