Black-hat Performance
Criteria
For
Network Support Program
and
Space Operations & Maintenance
Support Program
April 1999
Lockheed Martin Confidential
Table of Contents
· Purpose…………………………………………………………….………………….3
· Scope…………………………………………………………….……………………3
· Types of Black-hats…………………………………………….……………………..3
· Approach..……………………………………………………..………………………4
· Inquiry Orientation…..………………………………………..…….…………………4
· Commonality of Investigation………………………………..……………………….5
· Issues………………………………………………………..…………………………5
· Recommendations………………………………….…….……………………………5
· Instructions…………………………………….…….….……………………………..5
· Program Management Data Sheet…………………….………………………………6
· RTS Operation Data Sheet…………………………….………………………………7
· Personnel Performance Data Sheet……………….…….……………………………..8
· Quality Program Data Sheet……………………….………………………………….9
· Security Program Data Sheet…..…………………………………………………….10
· Safety Program Data Sheet…………………………………………………………..11
· Operations Support Data Sheet………………………………………………………12
· Maintenance Support Data Sheet…………………………………………………….13
· Logistics Program Data Sheet………………………………………………………..14
· Software Support…….…………………………………………………..…………..15
· Local Area Networks Data Sheet…………………………………………….………16
· Training Program Data Sheet……………………………………………….………..17
· Civil Engineering Support Data Sheet…………………………………….…………18
· Staffing/Personnel Data Sheet…………………………………………….…………20
· CDRLS Data Sheet…………………………………………………………………..22
Black-hat Performance
Purpose
The purpose of the Black-hat is to make a critical review of the current NSP and SOM Programs in an effort to determine what changes, if any, are necessary to establish a more competitive position for the upcoming re-competition. The data gathered by the Black-hat Team will be retained by the proposal team for analysis and use in developing a competitive proposal for the Operational Space Services and Support (OSSS) acquisition. Currently, NSP and SOM are separate contracts, which will be combined under the OSSS acquisition.
Scope
The scope of the Black-hat review is limited to the areas currently within the NSP and SOM contracts. Other elements, to be added in the future, or currently under jurisdiction of other contractors or the Government are not within the scope of this task.
Types of Black-hats
There are two types of Black-hats – non-incumbent and incumbent. Both are designed to provide the same end result in regards to a proposal, but each has its methodology and approach. These are:
· Non-incumbent: in this position, everything is based on a competitive assessment of areas of the contract that should to be most affective in cost and performance related issues. As a result, the Black-hat becomes a review of program themes and discriminators, operational approaches, support concepts, and competitive analysis. In essence it is an analysis of what works, what doesn’t work, where the competition is vulnerable, and what the team has to offer.
· Incumbent: in this position, it is necessary to take the approach to first analyze the elements of the current operation to ascertain where, what, and how things are run and where improvements can be made. This initial thrust is independent of the solicitation with the focus on how we can make the operation more efficient, become more responsive, make the customer happier, and finally, lower the cost. Subsequent to this is the competitive analysis segment. In order to obtain a meaningful competitive position, the analysis must begin with an effective and efficient operation based on the existing contract elements.
Approach
In order to conduct an effective Black-hat, it is necessary that the review be performed in a logical sequence based on the elements of the current contracts. Therefore, a Black-hat should be based on a three-prong approach:
1. Review of current program operations to uncover problem areas, identify issues, define inefficiencies in operations, cite functional areas subject to award fee problems (history), analyze customer concerns, and finally review employee concerns and HR issues.
2. Make a composite analysis of how, what and where to correct/improve Black-hat findings to put the program on a more efficient and cost-effective baseline of operation.
3. Identify the areas noted by the Black-hat that need significant change or correction in order to make the operation become more competitive?
This approach, implemented in this order, will provide the greatest results from the Black-hat, and become the basis for a more realistic competitive analysis relative to the solicitation.
Inquiry Orientation
Initial Black-hat inquiries should be oriented toward those issues defined. Each question asked, or functional area reviewed, should be from a perspective of “what is the current functionality, what appears to be the issues, and can I suggest improvements”. The following needs to be considered, as required, when reviewing each assigned area:
· Specific charter and functionality of the organization.
· Interfaces with the Government and User organizations.
· Processes and procedures at each level.
· Areas impacted by Government and/or User actions
· Organization work flow.
· Communications.
· Effective utilization of staffing.
· Degree of automation.
Commonality of Investigation
There are a number of areas common to the NSP and SOM Programs as well as to each of the respective sites therein. Therefore, it will be necessary to apply the data sheets to each as applicable. Some examples are logistics, hardware and software maintenance, etc. Although there is a great degree of commonality in these areas, it is necessary that each is reviewed applicable to its respective operation and support task, and data sheets completed respectively.
Issues
Program issues, to the functional level, need to be identified and suggested resolutions made on the data sheets. These issues are applicable to operations, maintenance, training, personnel, and especially those from the customer. Do not make suggestions to program or site personnel!
Recommendations
Recommendations, based upon findings in each area, should to be annotated on the respective data sheet during the review. This is necessary in order that the competitive analysis and proposal team can derive the best results of the review. Also, it will greatly enhance the final analysis if a general summation of your review is included at the end of each data sheet package.
Instructions
The attached data sheets are designed to cover the areas deemed to provide the most information for an analysis. Complete a data sheet for each respective area, entering Not Applicable (NA) where necessary. Annotate each topic in sufficient detail to enable a subsequent summation of the critical points by the analysis and proposal team. There is no specific order of the areas to be reviewed; therefore it is at the discretion of the reviewer as to the review sequence.
Black-hat Data Sheet
Program Management
Applicability: All locations.
Site: Reviewer: Date:
1. Program Management:
2. Program issues:
3. Organization review:
4. Areas of concern:
5. Customer contacts:
6. Customer program level concerns:
7. Customer site issues:
8. Cost:
9. Program compliance:
10. Recommendations:
Black-hat Data Sheet
RTS Operation
Applicability: All RTS sites.
Site: Reviewer: Date:
1. Commonality of operations:
2. Standardized procedures:
3. Organization:
4. Maintenance tasks:
5. Operations reports:
6. Issues:
7. Information management utilization:
8. Operator certification requirements:
9. Task assignments/workload integration:
10. Recommendations:
Black-hat Data Sheet
Personnel Performance
Applicability: All areas.
Site: Reviewer: Date:
1. Appearance and attitude:
2. Motivation:
3. Knowledge of assigned tasks:
4. Turn-over rate/impact on operations:
5. Collateral duties:
6. Task assignment/related skills:
7. Task/workload allocations:
8. Personnel/HR issues:
9. Labor issues:
10. Recommendations:
Black-hat Data Sheet
Quality Program
Applicability: All areas.
Site: Reviewer: Date:
1. Implementation processes:
2. Operations and maintenance integration:
3. Collateral duty assignments:
4. Quality related issues:
5. Program effectiveness:
6. Recommendations:
Black-hat Data Sheet
Security Program
Applicability: All areas.
Site: Reviewer: Date:
1. Organization:
2. Staffing:
3. Response to program requirements:
4. Review of security violations:
5. Program impacts:
6. Degree of automation used:
7. Interfaces:
8. Headquarters support:
9. Recommendations:
Black-hat Data Sheet
Safety Program
Applicability: All areas.
Site: Reviewer: Date:
1. Organization:
2. Working conditions:
3. Safety environment:
4. Implementation process:
5. Adherence to requirements:
6. Safety history record:
7. Personnel violations; cause and affect:
8. Equipment violations; cause and affect:
9. Recommendations:
Black-hat Data Sheet
Operations Support
Applicability: Schriever AFB and all GSU’s
Site: Reviewer: Date:
1. Organization:
2. Interfaces:
3. Workload/schedule:
4. Skills assessment:
5. Operational environment:
6. Issues:
7. Engineering Support:
8. Meetings/reports:
9. Technical support/studies:
10. Recommendations:
Black-hat Data Sheet
Maintenance Support
Applicability: All AFSCN C-E & DSCS equipment.
Site: Reviewer: Date:
1. Systems availability history:
2. Organization:
3. Collateral duties:
4. Task assignment/workload integration:
5. Responsiveness:
6. Issues:
7. Vendor support:
8. Records/logs/data:
9. Supporting documentation:
10. Recommendations:
Black-hat Data Sheet
Logistics Program
Applicability: All AFSCN C-E & DSCS equipment.
Site: Reviewer: Date:
1. Organization:
2. Data collection and reporting process:
3. Purchasing procedures/actions:
4. Repair processes:
5. Equipment calibration:
6. Base support re-supply:
7. Vendor re-supply:
8. Logistics turn-around:
9. Provisioning levels (bench stock/central supply/depot):
10. Recommendations:
Black-hat Data Sheet
Software Support
Applicability: RTS, GA, MS, GPS, MCS, TSTS, & Engineering Analysis.
Site: Reviewer: Date:
1. Organization:
2. Task assignments/workload integration:
3. Responsiveness:
4. Vendor support:
5. Records/logs/data:
6. Testing & maintenance actions:
7. Interfaces:
8. Issues:
9. Recommendations:
Black-hat Data Sheet
Local Area Networks
Applicability: DGO, GTS, HTS, VTS & New Boston.
Site: Reviewer: Date:
1. Organization:
2. Supportability:
3. Task assignments/workload integration:
4. Responsiveness:
5. Records/logs/data:
6. SW/HW maintenance actions:
7. LAN management tasks:
8. LAN availability history:
9. Issues:
10. Recommendations:
Black-hat Data Sheet
Training Program
Applicability: All locations:
Site: Reviewer: Date:
1. Organization:
2. Training standards:
3. Course documentation:
4. Course materials:
5. Training aids/equipment:
6. Training effectiveness feedback:
7. Certifications:
8. Issues:
9. Recommendations:
Black-hat Data Sheet
Civil Engineering Support
Applicability: Real property & associated equipment – all sites.
Site: Reviewer: Date:
1. Organization:
2. Task assignments/workloads:
3. Responsiveness:
4. Records/logs/reports:
5. Maintenance actions:
6. Collateral duties:
7. Transportation:
8. Vendor support:
9. Health and safety:
10. Labor issues:
Black-hat Data Sheet
Civil Engineering
(continued)
11. Environmental issues:
12. Other issues:
13. Custodial services:
14. Work related violations:
15. Recommendations:
Black-hat Data Sheet
Staffing/Personnel
Applicability: All areas.
Site: Reviewer: Date:
1. Staffing mix:
2. Staffing levels:
3. Task assignments:
4. Workload distribution:
5. DOL compliant positions:
6. Turn-over rate:
7. Management issues:
8. Personnel issues:
9. Labor issues:
10. Training:
Black-hat Data Sheet
Staffing/Personnel
(continued)
11. Job performance:
12. Working environment:
13. Program Office/Headquarters support:
14. Problem areas:
15. Recommendations:
Black-hat Data Sheet
CDRLS
Applicability: All areas.
Site: Reviewer: Date:
1. Development:
2. Processes:
3. Data collection:
4. Delivery:
5. Issues:
6. Recommendations:
11