Validation Handbook

Validation Handbook

Version 1

SVDS-02

August 1999

Contents

1.Introduction

2.Sciamachy

2.1 The Instrument......

2.2 Scientific Objectives......

2.3 Sciamachy Data Products......

3.Validation Set-up

3.1Validation phases......

3.2Validation activities......

3.3Sciamachy data and information......

3.4Other Envisat data......

3.5Validation data......

3.6Constraints on validation activities......

3.7Rehearsal period......

3.8Reporting results

3.9Envisat validation......

3.10 Time schedule......

4.Core Validation Plan

4.1Spectral solar irradiance......

4.2Spectral earth radiance/Albedo......

4.3Polarisation......

4.4Ozone column......

4.5Ozone profile......

4.6NO column......

4.7NO profile......

4.8BrO column......

4.9BrO profile......

4.10HO column......

4.11HO profile......

4.12NO column......

4.13NO profile......

4.14CO column......

4.15CO profile......

4.16CO column......

4.17CO profile......

4.18CH column......

4.19CH profile......

4.20Temperature/Pressure profile......

4.21Cloud cover......

4.22Cloud top height......

4.23Aerosols......

4.24OClO column......

4.25SO column......

4.26HCHO column......

List of AO Validation Proposals

Details of AO validation proposals

List of Ground Based Validation Instrument Locations

Sciamachy Validation Campaign Contact List

D.1 SCIAVALIG members

D.2 Principle Investigators of Sciamachy AO Validation Proposals......

D.3 Others......

References

List of Acronyms

1.Introduction

The Sciamachy instrument will fly on board Envisat, which is currently planned for launch in November 2000. It is a joint project of Germany, Belgium and The Netherlands.

The primary scientific objective of Sciamachy is the global measurement of various trace gases in the troposphere and stratosphere, which are retrieved from the solar irradiance and Earth radiance spectra. The large wavelength range is also ideally suited for the determination of aerosols and clouds. Validation of Sciamachy is essential to ensure the quality of these derived products. The requirements for the validation of Sciamachy are described in the Sciamachy Validation Requirements Document (SVRD, 1998).

One of the tasks of the Sciamachy Science Advisory Group (SSAG) is to give advice and assistance concerning the validation of Sciamachy. These activities are co-ordinated by the Sciamachy Validation and Interpretation Group: SCIAVALIG.

This document describes the validation set-up and gives an overview of the core and AO validation activities. Additionally it provides information on: Sciamachy and validation data exchange, information exchange, workshops and timeschedule, Sciamachy data availability, validation constraints and availability of other Envisat data.

2.Sciamachy

The Sciamachy Project was initiated to provide global knowledge about the amounts and distributions of tropospheric, stratospheric and mesospheric trace constituents and parameters which play an important role in the physical and chemical processes, which determine the behaviour of the atmosphere (Burrows et al., 1988). Sciamachy was proposed for flight as part of the Polar Platform in response to the ESA Announcement of Opportunity. The development is managed by the German Aerospace Center (DLR) and the Netherlands Agency for Aerospace Programmes (NIVR) with a contribution from the Belgian Institute for Space Aeronomy (BIRA-IASB). This development is financially supported by the German Ministry of Science, Research and Technology within the ATMOS-programme, the Ministries of Economic Affairs, of Education, Culture & Science and of Housing, Spatial Planning & the Environment in the Netherlands and the Federal Office of Science, Technical and Cultural Affairs in Belgium.

2.1The Instrument

Sciamachy is a passive remote sensing spectrometer observing backscattered, reflected, transmitted, or emitted light from the Earth between 240 and 2380 nm, in the following viewing geometries:

• Same atmospheric volume first in limb and after a short time (8 min.) in nadir;
• Atmosphere in solar or lunar occultation;
• Direct solar or lunar observations for in-flight calibration.

A detailed description of the Sciamachy instrument is given in the Scientific Requirements Document (Bovensmann and Burrows, 1997) and Bovensmann et al. (1999).

2.2Scientific Objectives

The primary objective of Sciamachy is to determine vertical and horizontal distributions of important atmospheric constituents and parameters (trace gases, aerosols, radiance, irradiance, clouds, temperature and pressure) from measurements of radiances reflected by and upwelling from the atmosphere and Earth's surface (The Envisat Programme, ESA Bulletin 76; Scientific Requirements Document for Sciamachy Data and Algorithm Development, Chance et al. 1997). The measurement of the radiances will be done in different viewing geometries: nadir, limb and solar and lunar occultation (Bovensmann et al. 1999).

2.3Sciamachy Data Products

Table 2.1 lists the official Sciamachy data products. A distinction is made between Near-Real Time (NRT) products and Off-Line (OL) products. The Difference is the time of delivery: within 3 hours after sensing for NRT products, within 2-3 weeks for OL products. Another distinction is made between “nadir” products and “limb” products. Total column amounts of trace gases are retrieved from nadir measurements, and trace-gas profiles are retrieved from limb measurements.

Table 2.1 Sciamachy Level 1 and Level 2 Operational Products. NRT= Near-Real Time, products which are to be delivered in less than 3 hours after sensing; OL= Off-line, products which are to be delivered 2-3 weeks after sensing.

(*) Observed under special conditions or after averaging.

(**) Secondary product, necessary for retrieval of CO and NO.

(†) SSAG recommended products (not yet officially confirmed).

1

3. Validation set-up

3.Validation Set-up

3.1Validation phases

The validation of Sciamachy products can be divided in three phases: the commissioning phase, the main validation phase and the long-term validation phase.

Commissioning phase: The commissioning phase is from launch (L) to six months after launch (L+6). The first nominal measurements of the Earth shine spectrum are expected to occur at approximately L+3.

The preliminary data release is foreseen for L+9, so that within these six months a first notion of the quality of most of the products should be derived. The validation activities in this phase focus on a thorough inspection of the level 1 data products and quick-look comparisons and consistency checks for level 2 products, with emphasis on O3 and NO2 columns and profiles, on pressure and temperature, on cloud information and aerosol index.

Main validation phase: The main validation phase is from L+6 to L+18.

In this year the accuracies of all level 1 and level 2 products have to be derived. The results of this phase will be presented in a validation workshop around L+20. The official data release is foreseen for L+20.

Long-term validation phase: A regular, optimised repetition of the essential elements of the main validation is performed during the long-term validation phase. This phase extends over the entire lifetime of Sciamachy. Long-term validation results in a more detailed knowledge of the accuracy of the products and in a characterisation of slowly changing instrument parameters. It also enables the validation of products produced after the main validation phase.

3.2Validation activities

The validation activities will be divided into two complementary parts: a "core" validation and validation based on selected AO proposals.

The core validation programme provides a minimal but essential validation of all Sciamachy products. The core validation and possibly some AO projects begin in the commissioning phase. After the commissioning phase, the core validation and the AO validation will proceed in parallel contributing both to the main validation and the long-term validation.

The core validation activities are detailed in Chapter 4. The selected AO proposals are listed in Annex A and described in more detail in Annex B.

3.3Sciamachy data and information

Sciamachy data Policy

The Sciamachy data is distributed by ESA via D-PAC in Oberpfaffenhofen. Access to this data can be obtained by the Principal Investigators of the Envisat-AO approved proposals, after accepting the terms and conditions for using Envisat data ( These terms and conditions include among others: terms on intellectual property rights, conditions for publication, reporting conditions, terms on onward distribution of data, etc. In particular:

• Envisat data provided in the framework of the AO may be used exclusively for the purpose specified and approved in the project proposal and may not be copied or distributed to other users or projects rather than the approved Co-Investigators of the project.
• In all publications/presentations based on the Envisat data provided through the AO, proper reference to ESA and Envisat and the copyright of the data should be made.
• Regular progress reports should be generated and sent to ESA and the project results should be presented at ESA organised symposia/workshops.

Furthermore, ESA provides one copy of the level 0 and the ESA generated products at no charge to instrument providers DLR and NIVR jointly. NIVR shares its data rights with BIRA-IASB as co-funding agency. DLR and NIVR will distribute data to selected national research groups. These research groups will be listed in the Flight Operations and Data Plan (FODP). The terms and conditions for the use of this data will be based on the above mentioned terms and conditions.

After the commissioning phase Sciamachy data will be available for other research groups against reproduction costs.

Information

Three web-sites with up-to-date information will be opened, one with information on the instrument operations (maintained by the Sciamachy Operations Support Team), one with information on the Sciamachy data products and updates to the processor (maintained by ESA or DLR-DFD), and one with information on validation activities and results (maintained by KNMI). The latter can be used for the reporting of validation results for all groups working on the validation of Sciamachy (Section 3.7).

3.4Other Envisat data

MIPAS and GOMOS data are available for the Sciamachy validation groups, provided they have requested these data in their AO proposal. The same applies for other Envisat data (e.g. AATSR). The MIPAS and GOMOS data will be distributed via the User Services Facility at ESRIN.

3.5Validation data

A large amount of data will be collected during the commissioning phase and the main validation phase. This data will most likely be made available by the validation groups via the NILU data centre NADIR (NILU’s Atmospheric Database for Interactive Retrieval).

NADIR is put into operation for the EASOE campaign in 1991 and has been used for several campaigns and projects associated with atmospheric research (e.g. SESAME, THESEO, GOME).

In the next version of the Sciamachy Validation Handbook an annex will be included containing a description of the database and access protocols.

Furthermore there will be two special issues of Nadir News dedicated to Envisat: one for the rehearsal issued about Jan-Feb 2000, the second issue will be published at the time of launch with amendments, news, new products, etc.. During the whole validation period, NILU will maintain a 'dynamic' web page with the latest news.

3.6 Constraints on validation activities

Before setting up expensive validation campaigns, it is important to realise that several factors may place strong constraints on such campaigns, and on the analysis of the data. These are:

Time and geolocation constraints:

* Launch is currently scheduled for November 2000. This means that atmospheric measurements will probably be performed from February 2001, since the first three months after launch will be used for outgassing and functional testing of the instrument.

* Sciamachy observes one particular spot at the equator only once every six days, and at 60N or 60S only once every three days, when it is in alternating nadir-limb viewing mode. On days that Sciamachy does not pass over this spot, the maximum distance to the edge of the nearest observed ground pixel can be up to 1000 km at the equator, and up to 300 km at 60N or 60S.

* Sciamachy performs atmospheric measurements only at sun-lit regions of the Earth. So the highest latitudes (higher than ~67 degrees) can not be observed during part of the year.

* Once every month, three orbits will be used for calibration purposes. This means that the nadir/limb measurements in the first orbit are alternated with calibration lamp and solar measurements, and that there will be no nadir/limb measurements in the second and third orbit. The orbits used for this monthly calibration and the day of the month will be known in advance.

* Switch-off periods of the instrument may occur if necessary for mission operations. These can not be planned in advance. It is a risk factor to be added in the campaign planning.

* Whenever the moon can be observed by Sciamachy, moon-occultation and calibration measurements will be observed. The nominal nadir/limb measurements will be interrupted for about 2 minutes, corresponding to about 800 km.

Data availability constraints:

* Some data might not be processed directly after the measurement, due to data transport problems. A time delay of weeks to months can be expected in such case. Whether this is the case will hopefully be clear in the next version of this document.

* From previous missions it can be learned that it is always possible, although not foreseen, that some products might not yet be available from the start of the measurements. A few products might even take very long before becoming available, due to algorithm or implementation problems. This would probably not influence the measurement campaign itself but it would influence the time schedule for the data comparison.

1

3.7Rehearsal period

A validation rehearsal period will take place approximately one year before launch (end of 1999). This rehearsal should enable the validation groups to test at least the following aspects:

• accessibility of the NILU database
• file transfer to and from NILU
• availability of format conversion tools
• availability and usefulness of tools in the data base
• understanding of the data in the database
• understanding of the Sciamachy test data
• readiness of local data analysis tools
• capability of comparing different data sets
• feed-back to other research groups

Details of the rehearsal will be presented in November 1999. The evaluation will be in spring 2000.

3.8Reporting results

Validation results will be discussed in the Envisat and Sciamachy validation workshops (see section 3.9 and 3.10). A (protected) web site will be available for intermittent discussions on the validation, and posting of new results. It is important that all validation teams realise that these working discussions and sharing of information and data inevitably affect official validation publications. This should be visible in co-authorships, proper references and acknowledgements.

3.9Envisat validation

The Sciamachy validation will be closely linked to the validation of the other atmospheric chemistry instruments on board Envisat: MIPAS and GOMOS. Therefore the validation teams of these three instruments will have joint meetings regularly in the Atmospheric Chemistry Validation Team, consisting of the PIs of all relevant Envisat AO proposals.

Separate Sciamachy validation meetings will be organised occasionally, in between these ACVT meetings.

3.10 Time schedule

The current time schedule is shown in Figure 3.1. The launch date is planned for November 2000. Meetings will be organised at least once every year, possibly in conjunction with the ACVT meetings. In addition, meetings will be organised a few weeks before data releases and processing updates. Furthermore, there will be smaller working group meetings for separate (groups of) products, whenever they are necessary. These working group meetings are not indicated in the figure, but they will be announced at the information web page. All groups involved in the validation of the specific product will be invited.

Figure 3.1 Time schedule for Sciamachy validation

1

4. Core Validation Plan

4.Core Validation Plan

4.1Spectral solar irradiance

The following validation sources will be used to validate the spectral solar irradiance:

 SOLSTICE

 SOLSPEC

 GOME

 VIRGO

 Balloons

4.2Spectral earth radiance/Albedo