-1-

AIS-AIMSG/6 - SN/7

/ AIS-AIMSG/6 – SN/7
14/05/12

AERONAUTICAL INFORMATION SERVICES-AERONAUTICAL INFORMATION MANAGEMENT STUDY GROUP(AIS-AIMSG)

SIXTH MEETING

Buenos Aires, Argentina 21 to 25 May 2012

Agenda Item 4: AIM processes and requirements

4.1 AIM quality

VALIDATION AND VERIFICATION

(Presented by Gregory Pray)

SUMMARY
The group is invited to review and comment on the discussion herein concerning the definitions of validation and verification in Annex 15 and the proposed development of procedures for the correct application of validation and verification processes throughout AIM operations.

1.Background

1.1During the ICAO AIS-AIMSG/5 meeting it was identified that the current Annex 15 definitions of validation and verification are sourced from International Organization for Standardization (ISO) definitions with no additional guidance on the applicability of the definitions as they pertain to AIS/AIM:

Validation. Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled (ISO 9000*).

Verification. Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled (ISO 9000*).

Note 1.— The term “verified” is used to designate the corresponding status.

Note 2.— Confirmation can comprise activities such as:

— performing alternative calculations;

— comparing a new design specification with a similar proven design specification;

— undertaking tests and demonstrations; and

— reviewing documents prior to issue.

1.2Paragraph 3.5 of AIS-AIMSG/5 SN/2- Annex 15 Amendment 37 referenced several definitions of validation and verification currently in use and proposed new definitions. In addition to the definition proposals in SN/2 other proposals were discussed but no agreement could be reached.

1.3The group decided to defer the revision of the definitions to the Amendment 38 discussions. It was further noted that the discussion would benefit from material that could be madeavailable as a result of the work on Aeronautical Data Quality (ADQ) being carried on in Europe.

2.discussion/PROPOSAL

2.1The current definitions of validation and verification were included in Annex 15 to address AIS quality management system requirements. The current Annex 15 paragraph 3.2.1 states “Quality management systems shall be implemented and maintained encompassing all functions of an aeronautical information service” and thereafter the Annex focuses primarily on aeronautical data quality.

2.2In line with the focus on data quality paragraph 1.3.3 of the draft ICAO Aeronautical Information Management (AIM) Quality Manual refers to “the requirement in Annex 15 that validation and verification procedures be established which ensure that quality requirements (accuracy, resolution, integrity) and traceability of aeronautical data are met.” However, paragraph 3.8.2 of the manual also refers to the “verification and validation activities and the criteria for acceptability” in planning the processes to bring a product to fruition. This last reference is in line with AIM objectives to ensure the final product and/or service will be capable of meeting the needs of the end user.

2.3The current definitions are valid for quality management processes in general as described in the draft ICAO AIM Quality Manual. However, there is a need to provide clarity for application to aeronautical data quality processing that requires assurance of data integrity from input to and throughout the aeronautical data chain to output. The following are examples of definitions for validation and verification that are specific to data processing:

a)Validation:

EUROCONTROL Guidelines Supporting Implementation of ADQ: Data Validation - The process of ensuring that data meets the requirements for the specified application or intended use.

RTCA Document DO-200A and European Organization for Civil Aviation Equipment (EUROCAE) Document ED-76 — Standards for Processing Aeronautical Data: Validation - The activity whereby a data element is checked as having a value that is fully applicable to the identity given to the data element, or a set of data elements that is checked as being acceptable for their purpose

b)Verification:

EUROCONTROL Guidelines Supporting Implementation of ADQ: Data Verification: The evaluation of the output of an aeronautical data process to ensure correctness and consistency with respect to the inputs and applicable data standards, rules and conventions used in that process.

RTCA Document DO-200A and European Organization for Civil Aviation Equipment (EUROCAE) Document ED-76 — Standards for Processing Aeronautical Data: Verification - The activity whereby the current value of a data element is checked against the value originally supplied.

2.4In addition to considering the definitions of validation and verificationas described above for the development of data quality process guidance, there is need to consider other processes necessary to ensure data integrity as specified in the draft Amendment 37 to Annex 15, especially in reference to Note 1 in paragraph 3.3.3.2:

“3.3.3.2 The integrity of aeronautical data shall be maintained throughout the data process from survey/origin to distribution to the next intended user (the entity that receives the aeronautical information from the aeronautical information service provider) based on the following integrity classifications:

a)critical data: validation and verification procedures are followed to assure corruption does not occur at any stage of the entire process and include additional integrity assurance procedures to fully mitigate the effects of faults identified by thorough analysis of the overall system architecture as potential data integrity risks;

b)essential data: validation and verification procedures are followed to assure corruption does not occur at any stage of the entire process and may include additional processes as needed to address potential risks in the overall system architecture to further assure data integrity at this level; and

c)routine data: validation and verification procedures are followed throughout the data process.

Note 1.— The effects of faults in the entire process may be mitigated by additional data integrity assurance techniques as required such as application tests for critical data such as by flight check; the use of security, logic, semantic, comparison, and redundancy checks; digital error detection; and the qualification of human resources and process tools such as hardware and software.

Note 2.— Distribution to the next intended user will differ in the delivery method applied which may either be:

Physical distribution. The means by which aeronautical data and aeronautical information/data distribution is achieved through the delivery of a physical package, such as postal services;

or

Direct electronic distribution. The means by which aeronautical data and aeronautical information/data distribution is achieved automatically through the use of a direct electronic connection between the AIS and the next intended user.

Note 3.— Different delivery methods and data media may require different procedures to ensure the required data quality.”

2.5The data quality processes required to support the data integrity requirements as specified in the draft Amendment 37 to Annex 15 as outlined above need to be defined in the new Procedures for Air Navigation Systems (PANS) – AIM document for global application.

2.6To ensure that the appropriate guidance is provided as well as processes defined as necessary to achieve data quality and integrity, it is proposed:

a) that the current definitions of validation and verification in ICAO Annex 15 be retained to support the overall AIM quality management processes;

b)that further guidance for validation and verification to support data integrity is included in Chapter 8, The Data Quality Process, of the draft Aeronautical Information Management Quality Manual; and

c)that data quality processes including the data integrity requirements be defined in the new Procedures for Air Navigation Systems (PANS) – AIM document in support of the Annex 15 SARPS.

3.PROPOSED ACTION BY THE AIS-AIMSG:

3.1The study group is invited to:

a)discuss and comment on the content in this SN;

b)agree to retain the current definitions of validation and verification;

c)agree to include validation and verification guidance to support data integrity in the draft AIM Quality Manual; and

d)agree that data quality processes including the data integrity requirements as specified in the draft Amendment 37 to Annex 15 be defined in the PANS-AIM document.

— END —