Validation Handbook
Version 1
SVDS-02
August 1999
Contents
1.Introduction
2.Sciamachy
2.1 The Instrument......
2.2 Scientific Objectives......
2.3 Sciamachy Data Products......
3.Validation Set-up
3.1Validation phases......
3.2Validation activities......
3.3Sciamachy data and information......
3.4Other Envisat data......
3.5Validation data......
3.6Constraints on validation activities......
3.7Rehearsal period......
3.8Reporting results
3.9Envisat validation......
3.10 Time schedule......
4.Core Validation Plan
4.1Spectral solar irradiance......
4.2Spectral earth radiance/Albedo......
4.3Polarisation......
4.4Ozone column......
4.5Ozone profile......
4.6NO column......
4.7NO profile......
4.8BrO column......
4.9BrO profile......
4.10HO column......
4.11HO profile......
4.12NO column......
4.13NO profile......
4.14CO column......
4.15CO profile......
4.16CO column......
4.17CO profile......
4.18CH column......
4.19CH profile......
4.20Temperature/Pressure profile......
4.21Cloud cover......
4.22Cloud top height......
4.23Aerosols......
4.24OClO column......
4.25SO column......
4.26HCHO column......
List of AO Validation Proposals
Details of AO validation proposals
List of Ground Based Validation Instrument Locations
Sciamachy Validation Campaign Contact List
D.1 SCIAVALIG members
D.2 Principle Investigators of Sciamachy AO Validation Proposals......
D.3 Others......
References
List of Acronyms
1.Introduction
The Sciamachy instrument will fly on board Envisat, which is currently planned for launch in November 2000. It is a joint project of Germany, Belgium and The Netherlands.
The primary scientific objective of Sciamachy is the global measurement of various trace gases in the troposphere and stratosphere, which are retrieved from the solar irradiance and Earth radiance spectra. The large wavelength range is also ideally suited for the determination of aerosols and clouds. Validation of Sciamachy is essential to ensure the quality of these derived products. The requirements for the validation of Sciamachy are described in the Sciamachy Validation Requirements Document (SVRD, 1998).
One of the tasks of the Sciamachy Science Advisory Group (SSAG) is to give advice and assistance concerning the validation of Sciamachy. These activities are co-ordinated by the Sciamachy Validation and Interpretation Group: SCIAVALIG.
This document describes the validation set-up and gives an overview of the core and AO validation activities. Additionally it provides information on: Sciamachy and validation data exchange, information exchange, workshops and timeschedule, Sciamachy data availability, validation constraints and availability of other Envisat data.
2.Sciamachy
The Sciamachy Project was initiated to provide global knowledge about the amounts and distributions of tropospheric, stratospheric and mesospheric trace constituents and parameters which play an important role in the physical and chemical processes, which determine the behaviour of the atmosphere (Burrows et al., 1988). Sciamachy was proposed for flight as part of the Polar Platform in response to the ESA Announcement of Opportunity. The development is managed by the German Aerospace Center (DLR) and the Netherlands Agency for Aerospace Programmes (NIVR) with a contribution from the Belgian Institute for Space Aeronomy (BIRA-IASB). This development is financially supported by the German Ministry of Science, Research and Technology within the ATMOS-programme, the Ministries of Economic Affairs, of Education, Culture & Science and of Housing, Spatial Planning & the Environment in the Netherlands and the Federal Office of Science, Technical and Cultural Affairs in Belgium.
2.1The Instrument
Sciamachy is a passive remote sensing spectrometer observing backscattered, reflected, transmitted, or emitted light from the Earth between 240 and 2380 nm, in the following viewing geometries:
- Same atmospheric volume first in limb and after a short time (8 min.) in nadir;
- Atmosphere in solar or lunar occultation;
- Direct solar or lunar observations for in-flight calibration.
A detailed description of the Sciamachy instrument is given in the Scientific Requirements Document (Bovensmann and Burrows, 1997) and Bovensmann et al. (1999).
2.2Scientific Objectives
The primary objective of Sciamachy is to determine vertical and horizontal distributions of important atmospheric constituents and parameters (trace gases, aerosols, radiance, irradiance, clouds, temperature and pressure) from measurements of radiances reflected by and upwelling from the atmosphere and Earth's surface (The Envisat Programme, ESA Bulletin 76; Scientific Requirements Document for Sciamachy Data and Algorithm Development, Chance et al. 1997). The measurement of the radiances will be done in different viewing geometries: nadir, limb and solar and lunar occultation (Bovensmann et al. 1999).
2.3Sciamachy Data Products
Table 2.1 lists the official Sciamachy data products. A distinction is made between Near-Real Time (NRT) products and Off-Line (OL) products. The Difference is the time of delivery: within 3 hours after sensing for NRT products, within 2-3 weeks for OL products. Another distinction is made between “nadir” products and “limb” products. Total column amounts of trace gases are retrieved from nadir measurements, and trace-gas profiles are retrieved from limb measurements.
Table 2.1 Sciamachy Level 1 and Level 2 Operational Products. NRT= Near-Real Time, products which are to be delivered in less than 3 hours after sensing; OL= Off-line, products which are to be delivered 2-3 weeks after sensing.
Level 1 / Earth radiance spectrum (240-2380 nm) (nadir, limb and occultation mode) Solar irradiance spectrum (240-2380 nm)
Earth fractional polarisation in 6 spectral bands (direction of polarisation in 1 band
PMD subpixel radiances in 7 bands
Level 2 / Nadir / Limb
UV/Vis / IR / UV-IR / UV/Vis / IR / UV-IR
NRT / O
NO
SO*
OClO*
HCO*
BrO* † / HO
NO
CO
CH** / Cloud cover frac.
Cloud top height
Aerosol abs. index
OL / O
NO
SO*
OClO*
HCO*
BrO
UV index † / HO
NO
CO
CH
CO
p, T / Cloud cover frac.
Cloud top height
Aerosol abs. index / O
NO
BrO † / HO
NO †
CO †
CH
CO
p, T / Aerosol abs. index
(*) Observed under special conditions or after averaging.
(**) Secondary product, necessary for retrieval of CO and NO.
(†) SSAG recommended products (not yet officially confirmed).
1
3. Validation set-up
3.Validation Set-up
3.1Validation phases
The validation of Sciamachy products can be divided in three phases: the commissioning phase, the main validation phase and the long-term validation phase.
Commissioning phase: The commissioning phase is from launch (L) to six months after launch (L+6). The first nominal measurements of the Earth shine spectrum are expected to occur at approximately L+3.
The preliminary data release is foreseen for L+9, so that within these six months a first notion of the quality of most of the products should be derived. The validation activities in this phase focus on a thorough inspection of the level 1 data products and quick-look comparisons and consistency checks for level 2 products, with emphasis on O3 and NO2 columns and profiles, on pressure and temperature, on cloud information and aerosol index.
Main validation phase: The main validation phase is from L+6 to L+18.
In this year the accuracies of all level 1 and level 2 products have to be derived. The results of this phase will be presented in a validation workshop around L+20. The official data release is foreseen for L+20.
Long-term validation phase: A regular, optimised repetition of the essential elements of the main validation is performed during the long-term validation phase. This phase extends over the entire lifetime of Sciamachy. Long-term validation results in a more detailed knowledge of the accuracy of the products and in a characterisation of slowly changing instrument parameters. It also enables the validation of products produced after the main validation phase.
3.2Validation activities
The validation activities will be divided into two complementary parts: a "core" validation and validation based on selected AO proposals.
The core validation programme provides a minimal but essential validation of all Sciamachy products. The core validation and possibly some AO projects begin in the commissioning phase. After the commissioning phase, the core validation and the AO validation will proceed in parallel contributing both to the main validation and the long-term validation.
The core validation activities are detailed in Chapter 4. The selected AO proposals are listed in Annex A and described in more detail in Annex B.
3.3Sciamachy data and information
Sciamachy data Policy
The Sciamachy data is distributed by ESA via D-PAC in Oberpfaffenhofen. Access to this data can be obtained by the Principal Investigators of the Envisat-AO approved proposals, after accepting the terms and conditions for using Envisat data ( These terms and conditions include among others: terms on intellectual property rights, conditions for publication, reporting conditions, terms on onward distribution of data, etc. In particular:
- Envisat data provided in the framework of the AO may be used exclusively for the purpose specified and approved in the project proposal and may not be copied or distributed to other users or projects rather than the approved Co-Investigators of the project.
- In all publications/presentations based on the Envisat data provided through the AO, proper reference to ESA and Envisat and the copyright of the data should be made.
- Regular progress reports should be generated and sent to ESA and the project results should be presented at ESA organised symposia/workshops.
Furthermore, ESA provides one copy of the level 0 and the ESA generated products at no charge to instrument providers DLR and NIVR jointly. NIVR shares its data rights with BIRA-IASB as co-funding agency. DLR and NIVR will distribute data to selected national research groups. These research groups will be listed in the Flight Operations and Data Plan (FODP). The terms and conditions for the use of this data will be based on the above mentioned terms and conditions.
After the commissioning phase Sciamachy data will be available for other research groups against reproduction costs.
Information
Three web-sites with up-to-date information will be opened, one with information on the instrument operations (maintained by the Sciamachy Operations Support Team), one with information on the Sciamachy data products and updates to the processor (maintained by ESA or DLR-DFD), and one with information on validation activities and results (maintained by KNMI). The latter can be used for the reporting of validation results for all groups working on the validation of Sciamachy (Section 3.7).
3.4Other Envisat data
MIPAS and GOMOS data are available for the Sciamachy validation groups, provided they have requested these data in their AO proposal. The same applies for other Envisat data (e.g. AATSR). The MIPAS and GOMOS data will be distributed via the User Services Facility at ESRIN.
3.5Validation data
A large amount of data will be collected during the commissioning phase and the main validation phase. This data will most likely be made available by the validation groups via the NILU data centre NADIR (NILU’s Atmospheric Database for Interactive Retrieval).
NADIR is put into operation for the EASOE campaign in 1991 and has been used for several campaigns and projects associated with atmospheric research (e.g. SESAME, THESEO, GOME).
In the next version of the Sciamachy Validation Handbook an annex will be included containing a description of the database and access protocols.
Furthermore there will be two special issues of Nadir News dedicated to Envisat: one for the rehearsal issued about Jan-Feb 2000, the second issue will be published at the time of launch with amendments, news, new products, etc.. During the whole validation period, NILU will maintain a 'dynamic' web page with the latest news.
3.6 Constraints on validation activities
Before setting up expensive validation campaigns, it is important to realise that several factors may place strong constraints on such campaigns, and on the analysis of the data. These are:
Time and geolocation constraints:
* Launch is currently scheduled for November 2000. This means that atmospheric measurements will probably be performed from February 2001, since the first three months after launch will be used for outgassing and functional testing of the instrument.
* Sciamachy observes one particular spot at the equator only once every six days, and at 60N or 60S only once every three days, when it is in alternating nadir-limb viewing mode. On days that Sciamachy does not pass over this spot, the maximum distance to the edge of the nearest observed ground pixel can be up to 1000 km at the equator, and up to 300 km at 60N or 60S.
* Sciamachy performs atmospheric measurements only at sun-lit regions of the Earth. So the highest latitudes (higher than ~67 degrees) can not be observed during part of the year.
* Once every month, three orbits will be used for calibration purposes. This means that the nadir/limb measurements in the first orbit are alternated with calibration lamp and solar measurements, and that there will be no nadir/limb measurements in the second and third orbit. The orbits used for this monthly calibration and the day of the month will be known in advance.
* Switch-off periods of the instrument may occur if necessary for mission operations. These can not be planned in advance. It is a risk factor to be added in the campaign planning.
* Whenever the moon can be observed by Sciamachy, moon-occultation and calibration measurements will be observed. The nominal nadir/limb measurements will be interrupted for about 2 minutes, corresponding to about 800 km.
Data availability constraints:
* Some data might not be processed directly after the measurement, due to data transport problems. A time delay of weeks to months can be expected in such case. Whether this is the case will hopefully be clear in the next version of this document.
* From previous missions it can be learned that it is always possible, although not foreseen, that some products might not yet be available from the start of the measurements. A few products might even take very long before becoming available, due to algorithm or implementation problems. This would probably not influence the measurement campaign itself but it would influence the time schedule for the data comparison.
1
3.7Rehearsal period
A validation rehearsal period will take place approximately one year before launch (end of 1999). This rehearsal should enable the validation groups to test at least the following aspects:
- accessibility of the NILU database
- file transfer to and from NILU
- availability of format conversion tools
- availability and usefulness of tools in the data base
- understanding of the data in the database
- understanding of the Sciamachy test data
- readiness of local data analysis tools
- capability of comparing different data sets
- feed-back to other research groups
Details of the rehearsal will be presented in November 1999. The evaluation will be in spring 2000.
3.8Reporting results
Validation results will be discussed in the Envisat and Sciamachy validation workshops (see section 3.9 and 3.10). A (protected) web site will be available for intermittent discussions on the validation, and posting of new results. It is important that all validation teams realise that these working discussions and sharing of information and data inevitably affect official validation publications. This should be visible in co-authorships, proper references and acknowledgements.
3.9Envisat validation
The Sciamachy validation will be closely linked to the validation of the other atmospheric chemistry instruments on board Envisat: MIPAS and GOMOS. Therefore the validation teams of these three instruments will have joint meetings regularly in the Atmospheric Chemistry Validation Team, consisting of the PIs of all relevant Envisat AO proposals.
Separate Sciamachy validation meetings will be organised occasionally, in between these ACVT meetings.
3.10 Time schedule
The current time schedule is shown in Figure 3.1. The launch date is planned for November 2000. Meetings will be organised at least once every year, possibly in conjunction with the ACVT meetings. In addition, meetings will be organised a few weeks before data releases and processing updates. Furthermore, there will be smaller working group meetings for separate (groups of) products, whenever they are necessary. These working group meetings are not indicated in the figure, but they will be announced at the information web page. All groups involved in the validation of the specific product will be invited.
Figure 3.1 Time schedule for Sciamachy validation
1
4. Core Validation Plan
4.Core Validation Plan
4.1Spectral solar irradiance
The following validation sources will be used to validate the spectral solar irradiance:
SOLSTICE
SOLSPEC
GOME
VIRGO
Balloons
Spectral solar irradiance with SOLSTICECo-ordination / BIRA-IASB
Method / - Direct comparison of daily acquired solar measurements from SOLSTICE/ UARS and Sciamachy in the UV (240-400 nm)
- When available, direct solar comparison in the visible and IR from SOLSTICE-II/EOS (2002).
Use of uncorrected MgII index to spot additional instrumental problem (etalon, wavelength shift, ...). Wavelength range 118-420 nm, res.: 0.2 nm for SOLSTICE/UARS; 120-2000 nm for SOLSTICE-II/EOS
Time schedule / SOLSTICE/UARS is fully operational. Validation can start as soon as the solar data are available. Accuracy/short term degradation can be estimated after the commissioning phase and will continue throughout the mission. SOLSTICE II/EOS (scheduled for 2002) is only available after its own validation phase (expected to be short, at least for the UV channels).
Location / n.a.
Time period / Daily solar spectrum available since October 1991
Altitude range / n.a.
Accuracy / 170-320nm: 5.7%; 280-420nm: 7.9%
Limitations / Data currently limited to the UV.
Requirements / Availability and accuracy of SOLSTICE-II yet unknown.
Spectral solar irradiance with SOLSPEC
Co-ordination / BIRA-IASB
Participation / IFE-Bremen
Method / Same method as with SOLSTICE except that a smaller number of spectra are available. The IR part of the spectrum is not yet released. Wavelength range: 180-3200 nm, res.: 1 nm in UV-visible and 20 nm in IR
Time schedule / Comparison with data from space shuttle flights of SOLSPEC can be performed during the commissioning phase: Spacelab 1 (1984), Atlas-I (1992), Atlas-II, Eureka (1993), Atlas-III (1994).
Data from SOLSPEC scheduled for ISS in 2004 (long duration mission, 18 months) will contribute to long-term studies.
Location / n.a.
Time period / n.a.
Altitude range / n.a.
Accuracy / 240nm: 5.1%; 300nm: 4%; 500nm: 2.1%; 650nm: 1.7%; 850nm: 2.5%
Limitations / None
Requirements / Atlas-II and -III not yet released. IR part (1000-1800 nm) of Eureka will be released soon. SOLSPEC/ISS will not be operational for the commissioning and most of the main validation phases but could provide useful additional information for long-term instrument characterisation when launched on ISS.
Spectral solar irradiance with GOME
Co-ordination / IFE-Bremen
Participation / BIRA-IASB
Method / Comparison of Sciamachy solar irradiance measurements with GOME solar irradiance measurements. Temporal cross-correlation between Sciamachy and GOME.
Time schedule / The comparisons will be performed on a case study basis throughout the mission with the highest frequency during the commissioning phase.
Location / n.a.
Time period / Continuous observations, 1 measurement per day
Altitude range / n.a.
Accuracy / To be determined (strongly dependent on wavelength range, altering with time due to instrument degradation)
Limitations / Only the GOME spectral range (240-800nm) is covered.
Requirements / GOME must be operational during Sciamachy mission.
NRT data required during commissioning phase.
Spectral solar irradiance with VIRGO
Co-ordination / BIRA-IASB
Participation / NASA
Method / VIRGO-DIARAD is an absolute radiometer measuring the solar constant (total integrated irradiance) with a very high precision. Direct comparison is not possible given the limited spectral range of Sciamachy. However, complemented with additional information (Extreme UV spectra, SOLSTICE, SPM/SOHO), DIARAD could be helpful to track anomalies in the long term
instrument monitoring. Still to be investigated.
Time schedule / VIRGO/SOHO is operational since early 1996 and will still be operational until complete propellant consumption (~1.5 year) or major instrument failure. Therefore, data are expected for only the early phase of the Envisat mission.
Location / n.a.
Time period / Total solar irradiance every 3 minutes since early 1996
Altitude range / n.a.
Accuracy / Accuracy of total irradiance limited by data acquisition system (< 1%)
Limitations / None
Requirements / The SOHO spacecraft has experienced several failures. Data availability is conditioned by the spacecraft stability and propellant consumption. This may seriously limit its usefulness for Envisat mission.
Spectral solar irradiance with Balloons
Co-ordination / University of Heidelberg/IFE-Bremen/MPI-Mainz
Participation / CNRS-LPMA
Method / The LPMA/DOAS balloon instruments will be provided with a radiometric calibration for each flight. This will be achieved by observation of calibrated sources (tungsten filament lamps) on the ground. The combined wavelength range of the instruments (320-2400nm) covers most of the Sciamachy spectral range at a comparable resolution. The solar irradiance is extrapolated from the calibrated solar occultation measurements using the Langley plot method.
Time schedule / The validation tasks are planned for a period of 36 months.
Pre-launch activities: Development of radiometric calibration facility (L-6 to L)
Commissioning phase: 2 balloon expeditions (L to L+6); Intercomparison and validation (L+3 to L+9); Progress report (L+9);
Main validation phase: 4 balloon expeditions (L+6 to L+18); Intercomparison and validation (L+6 to L+20); Main validation report (L+6)
Long-term validation: 2 balloon expeditions (L+18 to L+30)
Location / Arctic (Kiruna), mid-latitudes and tropics
Time period / The LPMA/DOAS payload measures either during balloon ascent during late
afternoon and solar occultation at dusk, or solar occultation during dawn and balloon descent in the early morning hours.
Altitude range / Up to 30-40 km (depending on balloon size and payload weight). The height resolution is approximately 200 metres.
Accuracy / 3%
Limitations / None
Requirements / None
4.2Spectral earth radiance/Albedo