VARSY Project / Code / L2b-ACM-CAP-SRS
Issue / 01
Date / 27/08/2012
Page / 11 of 16
EarthCARE Level 2 Documentation
ACM-CAP
Cloud and Precipitation Best Estimate
Software Requirements Specification
(SRS)
VARSY Project
Code: / L2-ACM-CAP-SRS
Issue: / 0.1
Date: / 27/08/2012
Reference:
Name / Function / Signature
Prepared by / Robin Hogan / Project Scientist
Reviewed by / Pavlos Kollias / Project Scientist
Approved by / Pavlos Kollias / Project Manager
Signatures and approvals on original
This page intentionally left blank
Document Information
Contract Number: / 4000104528/11/NL/CT
Contract Issuer: / ESA-ESTEC
Internal Distribution
Name / Unit / Copies
Pavlos Kollias / McGill University / 1
Aleksandra Tatarevic / McGill University / 1
Wanda Szyrmer / McGill University / 1
Internal Confidentiality Level
Unclassified / ¨ / Restricted / þ / Confidential / ¨
External Distribution
Name / Organisation / Copies
Tobias Wehr / ESA-ESTEC / 1
Michael Eisinger / ESA-ESTEC / 1
Dulce Lajas / ESA-ESTEC / 1
Robin Hoganogan / University of Reading / 1
Julien Delanoë / LATMOS / 1
Gerd-Jan van Zadelhof / KNMI / 1
David Donovan / KNMI / 1
Alessandro Battaglia / University of Leicester / 1
Archiving
Word Processor: / MS Word 2003
File Name: / VARSY-READING-SRS-001
Document Status Log
Issue / Change description / Date / Approved01-draft / Draft version. / 25/08/2012
Contents
1. Purpose and Scope 6
1.1. Applicable Documents 6
1.2. Reference Documents 7
1.3. List of Abbreviations 8
2. Compliance Matrix 9
1. Purpose and Scope
This document specifies the software requirements associated with the implementation of the ACM-CAP L2b algorithm.
1.1. Applicable Documents
Table 1: Applicable Documents
Reference / Code / Title / Issue /[SOW] / EC-SW-ESA-SY-0310 / Statement of Work: VARSY - 1-Dimensional VARiational Retrieval of SYnergistic EarthCARE Products / 1.0
[CC] / Appendix 2 to
AO/1-6823/11/NL/CT / Draft Contract (attachment to SOW) / 1.0
[AD 1] / EC-SW-ESA-SY-0152 / EarthCARE Level 2 Processor Development
General Requirements Baseline / 1.0
[AD 2] / EC.ICD.ASD.SY.00004 / EarthCARE Product Definitions. Vol. 0:
Introduction
[AD 3] / EC.ICD.ASD.SY.00005 / EarthCARE Product Definitions. Vol. 1:
Common Product Definitions / 1.0
[AD 4] / EC.ICD.ASD.ATL.00021 / EarthCARE Product Definitions. Vol. 2b:
ATLID level 1 / 1.0
[AD 5] / EC.ICD.ASD.BBR.00022 / EarthCARE Product Definitions. Vol. 3b:
BBR level 1 / 1.0
[AD 6] / EC.ICD.ASD.MSI.00023 / EarthCARE Product Definitions. Vol. 4b:
MSI level 1 / 1.0
[AD 7] / ECSIM-DMS-TEC-ICD01-R / ECSIM Simulator Interface Control Document
[AD 8] / PE-TN-ESA-GS-0001 / Ground Segment: File Format Standard / 1.0
[AD 9] / EC-TN-ESA-GS-0218 / Tailoring of the Earth Explorer File Format Standard for the EarthCARE Ground Segment / 2.0
1.2. Reference Documents
Table 2: Reference Documents
Reference / Code / Title / Issue /[RD1] / ECSIM-DMS-TEC-SUM-01-R / ECSIM System User Manual
[RD2] / ECSIM-KNMI-MAD01-R / ECSIM Model and Algorithms Document
[RD3] / EE-MA-DMS-GS-0001 / Earth Explorer Mission CFI Software:
General Software User Manual
[RD4] / EOP-SM/1567/TW / EarthCARE Mission Requirements Document
[ATLAS-FR] / EC-FR-KNMI-ATL-027 / ATLAS Final report / 1.0
[ATLAS-ACM-TC] / EC-TN-KNMI-ATL-ACM-TC-024 / L2b Classification ATBD / 1.2
13/03/08
[ATLAS-EBD] / EC-TN-KNMI-ATL-ATBD-A-EBD-021 / L2a ATLID Extinction, Backscatter and Depolarization algorithm ATBD / 1.1 27/04/09
[ATLAS-FM] / EC-TN-KNMI-ATL-ATBD-A-FM-010 / L2a ATLID Feature mask ATBD / 2.2
[RATEC-FR] / RATEC-FR-READING-1 / RATEC Final Report / 1.0, April 2011
1.3. List of Abbreviations
Table 3: List of abbreviations
Abbreviation / Name1D-VAR RS / 1-dimensional variational retrieval scheme
ATLID / Atmospheric Lidar (The EarthCARE lidar)
CASPER / Cloud and Aerosol Synergetic Products from EarthCARE retrievals
CPR / Cloud Precipitation Radar (The EarthCARE radar)
EarthCARE / The Earth Clouds, Aerosols and Radiation Explorer
ECSIM / EarthCARE Simulator
HSRL / High-Spectral Resolution Lidar
MSI / Multi-spectral Imager (The EarthCARE imager )
2. Compliance Matrix
The following table lists all the requirements in the General Requirements Baseline [AD1] related to the software and the degree of compliance with each requirement.
*C=compliant, P=partly non-compliant, N=not compliant, NY=not yet compliant but will be, NA=Not applicable
Reference / Requirement / Implementation / Compliance*Design Requirements
DES-1 / The level 2 processor shall be implemented as a set of independent executable entities, referred to below as level 2 modules, each one implementing a specific algorithm generating a specific level 2 data product. / The ACM-CAP is implemented as a stand-alone executable / C
DES-2 / No proprietary libraries or code shall be used / Libraries used:
GNU Scientific Library (GPL), LIDORT (permissive open source license), “Adept” automatic differentiation library (GPL), Multiscatter library (LGPL); note that Hogan is copyright holder for “Adept” and “Multiscatter”; Mishchenko’s T-matrix code (permissive BSD license); BHMIE does not specify a license so have emailed Bruce Draine. / C
DES-3 / Only portable libraries or code shall be used.
In this context, "portable" means that any libraries or code invoked within a level 2 module, including bespoke libraries and code, must not depend on any special features of, e.g., the operating system, the compiler, the specific programming language implementation on the development or target platform. / The ACM-CAP currently uses an extension of the GNU C++ compiler in permitting variable-length arrays (a C feature) in C++. / N
Functional Requirements
FUN-1 / Level 2 modules shall derive geophysical quantities (level 2 data products) from
EarthCARE level 1b data products. Synergistic (level 2b) modules may use EarthCARE level 2 data products on input as well (see also INT-10). / See associated ATBD / C
FUN-2 / Level 2 modules shall implement the algorithm specified in the corresponding ATBD. / C
FUN-3 / Level 2 modules shall compute and store into products: geophysical parameters, error descriptors and product confidence data (i.e., quality flags). / See associated PDD / C
FUN-4 / Level 2 modules shall check whether all required data are available before running. / C
FUN-5 / Level 2 modules shall respond in a controlled way to instrument degradation or failure or unexpected values in the input data as per INT-10. / Meaningful errors are reported and program gracefully exits if appropriate in response to missing or corrupt data / C
FUN-6 / Convergence criteria shall be established for each algorithm. These criteria shall be monitored during the execution of the algorithm. Abortion criteria shall be established and applied for deviation from accepted convergence profile. Respective error information shall be produced. / The convergence criterion is that the L2 norm of the gradient of the cost function falls to less than unity. If this is not reached in a specified number of iterations, the algorithm aborts. Error and convergence information is reported / C
FUN-7 / Level 2 modules shall provide a trace of their execution in a single Log File for each execution run. / C
FUN-8 / The configuration files of a level 2 module shall allow to configure the logging level of the level 2 module. / See associated ICD / C
FUN-9 / The following logging levels shall be supported for a level 2 module: Error, Warning, Info, Debug, Progress. / See associated ICD / C
FUN-10 / An Error shall be logged when an unexpected condition was detected and the
level 2 module cannot perform any error recovery and must exit. / See associated ICD / C
FUN-11 / A Warning shall be logged when an unexpected condition was detected but error recovery is possible. / See associated ICD / C
FUN-12 / An Info shall be logged when a significant condition occurred that directly or indirectly affects the level 2 module outputs. An info shall also be logged at module startup and module termination. / See associated ICD / C
FUN-13 / Debug should be used as needed for the development of the Level 2 module. / See associated ICD / C
FUN-14 / Progress shall be used to log the progress of the simulation from 0% to 100%, exploiting the processing steps of level 2 modules. / See associated ICD / C
Interface Requirements
INT-1 / The Unix operating system shall be used. The actual hardware/operating system
configuration shall be PC (multi-core/multi-processor system if needed)/Linux. Details
(including the Linux distribution and its version) shall be agreed with the agency. / Code developed under Linux / C
INT-2 / Coding shall be performed in a high-level programming language such as C/C++
or Fortran-90. Details (including compiler versions) shall be agreed with the Agency. / C (1990 standard), C++ (1998 standard) and Fortran (1977 and 1990 standards) are required / C
INT-3 / Compilation options and compiler versions shall be documented and justified. / GNU compiler collection is needed / C
INT-4 / Each level 2 module shall be able to run stand-alone. / C
INT-5 / Besides being able to run stand-alone, Level 2 modules shall be compatible with and run within the EarthCARE end-to-end simulator ECSIM. / Not yet / NY
INT-6 / Input to and output from level 2 modules shall be performed via files. / C
INT-7 / In addition, level 2 modules shall use Unix standard output for status messages. / Not yet, currently sends some messages to standard error / NY
INT-8 / Level 2 modules shall not use Unix standard input. / C
INT-9 / Level 2 modules shall not produce graphical output. Graphical output (e.g. for quicklook results) may be produced from additional tools. / C
INT-10 / Level 2 modules shall read the following files on input: 1. one EarthCARE level
1b data product (in case of synergistic processors: one or more EarthCARE level 1b data products from different instruments, and/or one or more level 2 data products) covering
up to 2 orbits worth of data, 2. a configuration file, 3. support files (e.g., meteorological data from ECMWF, look-up tables, topographic databases, etc.) as required. / C
INT-11 / Level 2 modules shall write the following files on output: 1. one level 2 data product, 2. a log file. / Currently no log file is written but this will be fixed / NY
INT-12 / All files (including configuration and ntermediate input/output files) shall be in XML or NetCDF format. The default format shall be XML while NetCDF shall be used for data products and large files only (i.e., where using XML would significantly degrade performance). Details (including the NetCDF version) shall be agreed with the agency. / Output data is in netcdf but configuration is via an ascii format that is much more easily edited. / NY
INT-13 / The functions of the Earth Explorer mission CFI software shall be used wherever possible. / What are these ? / N
INT-14 / The reference systems defined in the Earth Explorer mission CFI software shall
be used throughout, in particular for all outputs. / ??? / N
INT-15 / All processing parameters including the environment of level 2 modules (e.g.,
paths) shall be configurable in external configuration files without code updates or recompilation. / C
INT-16 / Level 2 modules shall read their configuration file(s) at run-time at start-up. / C
INT-17 / Level 2 products shall follow the general EarthCARE file format as specified in [AD1]. / Not sure the general baseline document defines a format? / NY
INT-18 / Level 2 modules shall generate one level 2 output data product per level 1 input data product. / C
INT-19 / Each level 2 module, complete with reference configuration and support files, shall be delivered as a single tar file, optionally compressed by gzip. / C
INT-20 / No unused or otherwise unneeded files shall be delivered with a level 2 module. / C
INT-21 / Each level 2 module shall be delivered with a script that plugs it into the ECSIM framework. / NY
INT-22 / Only relative path names shall be used in the installation scripts of Level 2 modules. / C
INT-23 / The tar file used to deliver a level 2 module shall conform to the naming convention described in Section 6.7 of the General Requirements Baseline Document [AD?] / NY
Testing Requirements
TST-1 / Test entry points shall be defined to facilitate testing and debugging. They shall allow running selected parts of the processing chain by providing breakpoints for reading intermediate input data and writing intermediate output data. / The ACM-CAP processor ins indivisible / N
TST-2 / ECSIM shall be used for end-to-end performance testing using appropriate ECSIM models (including scenes, forward models, instrument and L1b simulators). Furthermore, realistic scenes shall be used, derived from existing measurements. / NA
Configuration management requirements
CFG-1 / All elements and sub-elements composing the level 2 processor shall be put under configuration management. / What is configuration management? / N
CFG-2 / At least the following configuration items shall be managed under configuration
management:
· Software modules (source code and object code)
· Static configuration and support data
· Acceptance Test Data Sets
· Procedures and scripts
· Any static data, e.g., delivered tar files
· Software and hardware environment documentation / N
CFG-3 / Widely supported Software Configuration Management tools shall be used. / N
CFG-4 / It shall be possible to know, for any discrete point in time, the software
configuration of the level 2 processor. / ?
CFG-5 / It shall be possible to retrieve, for any discrete point in time, the level 2 processor
in this configuration. / ?
CFG-6 / To maintain software integrity throughout the life cycle, a proper and systematic
control of software changes shall be provided. / NY
CFG-7 / For all versions of any software module, the modifications with respect to the
previous version shall clearly be identified and documented in the software module itself. / NY
CFG-8 / All versions of all software modules shall be archived with a version identifier
and the time from which the software module was in use. / NY
CFG-9 / For each delivery of any Configuration Item, a Change Log detailing all changes
since the previous delivery shall be provided. / NY
CFG-10 / Each version of an Level 2 module shall be uniquely identified through an
Overall Version Number in the following format:
VERSION = Major Version.Minor Version, each over two digits / C
Coding Rules
COD-1 / Common code should be separately packaged and reused within a single level 2
module as well as across level 2 modules.
Code duplication should be eschewed as far as possible. / C
COD-2 / Code related to the interfaces (input/output) shall be separated (in different
subroutines and different source code files) from code implementing the algorithm. / C
COD-3 / File or path names shall not be used inside the code. / C
COD-4 / Redundant calculations in Level 2 modules should be avoided. / C
COD-5 / It shall be shown that code does not include any uninitialised variables, unused variables, memory leaks, and dead (i.e., unused) code. / All issues can be verified rigorously using various compiler options, except for memory leaks. The absence of any potential leaks would be difficult to rigorously demonstrate / P
COD-6 / Standard headers shall be used for subroutines. Details shall be agreed with the agency. / Define “standard header” / ?
COD-7 / Ample use of comments shall be made. / C
COD-8 / Comments shall be used to describe the structure of the code. / C
*C=compliant, P=partly non-compliant, N=not compliant, NY=not yet compliant but will be, NA=Not applicable