Browse Guidelines Document

Browse Guidelines Document

Interoperable Catalogue System (ICS)

Browse Task Team (BTT)Interoperable Catalogue System (ICS)Interoperable Catalogue Systems (ICS)Interoperable Catalogue System (ICS)
System Design Document (SDD)Browse Guidelines Document (BGD)
CEOS
Working Group on Information Systems and Services
Browse Task Team / Doc. Ref.:CEOS/WGISS/PTT/SDDCEOS/WGISS/BTT/BGD
Date:March 1997August 1999
Issue:Version 1.2Version 1.0

Browse Task Team –Browse Guidelines Document Page 1

CEOS/WGISS/BTT/BGD Version 1.0 August 1999

Document Status Sheet

Issue / Date / Comments / Editor
0.1 / May 1999 / First draft issue for CEOS-Access Subgroup review. / O. Ochiai B. Burford
1.0 / August 1999 / First issue for public view. / O. Ochiai B. Burford

Acknowledgements

This document drew on material in the Guidelines for an International Interoperable Catalogue System written by Brian Thomas (Earth Observation Sciences Ltd.). Content and format guidance was provided by CEOS Data Subgroup member Ramachandra Suresh (NASA/Raytheon ITSS).

This document was published by:

Earth Observation Center

National Space Development Agency of Japan

Browse Task Team –Browse Guidelines Document Page 1

CEOS/WGISS/BTT/BGD Version 1.0 August 1999

Table of contents

1. INTRODUCTION1

1.1 Intended Audience1

1.2 Background Information and Scope of Document1

2. browse Products4

2.1 Browse as a Decision Process4

2.2 Role of Data Compression in Browse4

2.3 Browse Products5

3. attributes9

3.1 Attribute Definition9

3.2 Attribute Aspects11

3.3 Attribute Modification and Data Dictionary11

3.4 Extended Attributes12

3.5 Explanation of Browse Attributes12

4. conclusions15

appendix A16

Results of AVHRR Browse Survey

LIST OF FIGURES

1-1 Example of an AVHRR Browse Image2

LIST OF TABLES

1-1 Summary of Browse Functionality3

2-1 AVHRR Browse Products5

2-2 Survey of Browse Products6

3-1Example of Address Description9

3-2Example of Satellite Image Description10

3-3 Browse Attribute Definitions13

3-4 Example of Browse Attributes14

Browse Task Team –Browse Guidelines Document Page 1 of 21

CEOS/WGISS/BTT/BGD Version 1.0 August 1999

1. Introduction

This document has been developed by the Browse Task Team under the direction of the Access Subgroup of the Committee on Earth Observation Satellites (CEOS) [ Working Group on Information Systems and Services.

The purpose of this document is to provide information about browse data and recommendations for development of browse data systems. It is hoped that the information and recommendations provided will give the global change research community and other users of remote sensing information simpler and wider access to the wealth of data that is now available. This in turn will help to stimulate more data providers to make their information holdings available and encourage further development of global interoperable data systems.CEOS Working Groups and Subgroups are consensus organizations and the recommendations made within this document are non-binding.

1.1Intended Audience

This document is intended to assist data providers, including designers, developers and operators of Earth observation data systems to understand how a browse service might be made available to a wider audience.

1.2Background Information and Scope of Document

Data sets collected by Earth observation satellites are often quite large and it is impractical for data user's to examine complete data sets. In addition, providing Earth observation data is growing as a commercial field and an increasing number of data providers require payment for data sets. Earth observation data holdings are typically managed in a catalogue which can be queried by users to determine its contents and locate data sets. A catalogue system allows a user to identify individual data sets which fulfill specific requirements identified by the user. Data providers often offer a browse service which allows the user to review the selected items at a reduced resolution. This enables user's to establish whether the item is appropriate for the intended purpose. Figure 1-1 shows an example of a browse image for Advanced Very High Resolution Radiometer (AVHRR) instrument data archived by NOAA.

Figure 1-1 Example of an AVHRR Browse Image

Simple examples of the use of browse data might be:

  • Does the data cover the desired area?
  • Are the expected features present (e.g., an oil spill)?
  • Is the data of adequate quality (e.g., is the feature masked by cloud cover)?

Browse information is used for a variety of different purposes. These are summarized in Table 1-1 below:

Table 1-1 Summary of Browse Functionality

Browse function / Resolution / User type / Envisaged use
Viewing prior to ordering / partial / end user / check whether data is appropriate to the user's needs before placing an order
Identifying type and quality of data in a dataset (semi-guide) / full or partial / end user / for an inexperienced user (or user unfamiliar with the dataset) provides an overview of the characteristics of the dataset
Pre-processing check and selection of processing algorithms/parameters / full or partial / system operator / generates sample products from a subset of data prior to committing full resources to the processing task and enables evaluation of rival algorithms or algorithm parameters
Specification of processing parameters / full or partial / end user / interaction with a browse image allows the user to fine-tune the processing parameters to be applied to the associated data products
Reduced archiving and processing costs / full or partial / system operator / browse products can be used to support a policy of processing higher level products only on demand, thereby reducing archiving and routine processing overheads

In the context of a distributed system, however, some of these are likely to be mainly local requirements, such as the pre-processing check, which would normally be carried out only by system operators before generating a processed data item at the processing center. In addition, the use of browse to provide an illustrative example of the type of product that has been identified may be regarded as a capability of a guide service rather than a browse service. For this reason, this document focuses on the most widely accepted understanding of the use of browse as a means of establishing whether a specific data item is appropriate to the user's needs before placing an order for the retrieval of the full resolution data product from the archive.

2. Browse Products

Browsing in this context is part of a sequence of steps necessary for a researcher to evaluate individual data items from a data set (this should not be confused with “browser” software used on the World Wide Web). It is the interactive, visually oriented part of the resource searching and data quality assessment process which aids in the selection or prioritizing of information. The activity itself is dependent on the researcher's interests and criteria at the time that he or she is browsing, thus making it difficult to optimize browsing in a general way.

2.1 Browse as a Decision Process

The act of browsing is in essence a complex interactive decision process that is dependent on both explicit and implicit evaluation criteria applied by the user. It can be viewed as a hierarchical decision process where the number of decision levels possible is dependent on the information content of the browse data and the number of decision levels necessary is dependent on the user's evaluation criteria. While a particular browse object may not have enough content to allow a user to definitively determine its relevance based on his or her search criteria, it needs to be emphasized that being able to qualify browse data even at a first level of the decision hierarchy can significantly reduce the amount of information that is classified as relevant. And, as a result, significantly reducing the amount of information that the user must deal with.

2.2 Role of Data Compression in Browse

Perhaps the most important factor influencing accuracy in browse data is the data size reduction biasing introduced during browse data generation. Data size reduction is accomplished through techniques such as subsampling (and/or wavelet type processing) and compression of original data granules. The different methodologies bias the accuracy of the browse data in different ways and therefore knowledge of which methodology has been used provides relevant information about what is being represented. For example, there are a number of ways that an 8km resolution browse data object could be derived from a 1km resolution data granule. You could apply an 8x8 box filter which provided the mean of the 64 data values within the box, or use the highest data value within an 8x8 region or simply extract the value of every 8th pixel from every 8th line. Each of these methods provides a different representative data value and applies a different statistical bias to the browse data.

In the case of data compression a different kind of biasing can be introduced. Data compression can be utilized to generate a browse data object directly from the original data granule or applied to an existing browse data object as a means of decreasing its storage requirements. When a specific browse data object is requested, the compressed version is retrieved and is either; delivered to the end user's system in the compressed state (to increase network transfer efficiency), or decompressed at the production site and then transferred to the end user. Both lossless, (e.g.; Huffman, Lempel-Ziv) and lossy (e.g.; JPEG) compression algorithms have been used to advantage for satellite image data compression. With lossless compression, the decompressed object is an exact replicate of the original. Lossless compression algorithms optimize data storage by cataloguing repeated sequences and creating an index of their location in the original object. For AVHRR image data, Lempel-Ziv compression provides typically 4-to-1 or 5-to-1 compression. In comparison, lossy compression algorithms are capable of much greater compression ratios and may allow the user to control how the data is compressed. Higher compression ratios are enabled by reducing the accuracy of various operations within the processing algorithm (discrete cosine transform, real to integer assignments) which have the result of mapping multiple data values from the original to a single data value in the decompressed object (e.g.; values 22-32 from the original are mapped to 22 in the decompressed object). The tradeoff for higher compression ratios is that the decompressed object is not a replica of the original and it is difficult to infer how the biasing has modified the science data itself.

2.3 Browse Products

Within CEOS, different member systems utilize different methodologies for generating browse data objects. Information on what methodology is being used and a description of how it is being implemented will provide the user with some measure of the browse data accuracy. This information could be utilized by the user to adjust his or her evaluation criteria and take advantage of, or compensate for, the data reduction biasing in browse data. Table 2-1 is a sampling of AVHRR browse produced and held by CEOS member and other agencies.

Table 2-1 AVHRR Browse Products

Agency / Browse Characteristics / URL
CCRS / 500x500x8bits image, 1:4 subsampled, JPEG compression, from bands 1&2 /
DLR / 350x750x24bits image, 1:6 subsampled, JPEG compression, from bands 2&4 /
NASA / 640x640 image, 1:3 subsampled, 1 Mbyte / eos.nasa.gov/imswelcome
NOAA/
NCDC/
SAA / 205x191 image, 1:4 subsampled, JPEG compression, from band 1, 2 or 4 /
NSC / 158x304 image, GIF compression, from band 2 or 4 /
Univ. of Dundee / From 102x180 to 512x900 image, JPEG, GIF and UNIX compression, from bands 1 to 5 /
URI / 1024x1024 image, JPEG compression, from bands 4&5 / rs.gso.uri.edu/avhrr-archive/
archive.html
USGS/
EDC / Swathx408 image, reduced every 4th line and ever 5th sample, after JPEG compression (Q=75) size ~125 kB, from band 2 / edc
webglis
ESA/
ESRIN / 462xvariablex24bits image, subsampled at 6 Kms, JPEG compression / earthnet.esrin.esa.it

Within the nine examples of AVHRR browse data, every agency has chosen a different combination of subsampling ratio and image size. All agencies utilized compression, but different types of compression were used. All but one used JPEG compression. GIF and UNIX compression were also used. Different agencies utilized data from different bands to generate the browse images. For the same instrument (AVHRR), each agency chose a different way to generate their browse product.

Table 2-2 is a sampling of various types of browse produced and held by CEOS members and other agencies. The various agency datasets for a given type of browse are grouped together so that a comparison can be made of the way in which different agencies have chosen to implement browse for the same type of data.

Table 2-2 Survey of Browse Products

Agency / Satellite/Platform / Sensor/
Instrument / Browse Characteristics / URL
NASDA / JERS / SAR / 750x750x8bits image, subsampled 1:8 horizontal 1:8.53 vertical, JPEG compression / eus.eoc.nasda.go.jp
ESA/
ESRIN / ERS / SAR / 500xVariable image, 8-look, 200 pixel spacing, JPEG compression, Ground range / earthnet.esrin.esa.it
CCRS / Radarsat1 / SAR / 256x256 image, black and white, JPEG compression / ceocat.ccrs.
nrcan.gc.ca
NASDA / TRMM / PR / 3960x880/8bits image, subsampled 1:9, JPEG compression / eus.eoc.nasda.go.jp
NASA / TRMM / PR / 3960x880 image, HDF RLE compression / lake.nascom.nasa.gov/data/trmm
NASDA / Landsat / TM / 640x498x8bits image; descending composite 4,3,2 as RGB and subsampled 1:12.04 horizontal, 1:12.05 vertical; ascending band 6 and subsampled 1:3.01; JPEG compression / eus.eoc.nasda.go.jp
USGS/
EDC / Landsat 4-5 / TM / 350x350 image, composite 5,4,3 as RGB, each band reduced every 16th line and every 16th sample, JPEG compression (Q=75) size 75kB. / edc
ESA/
ESRIN / Landsat / TM / 960xvariable image, composite 7,5,2 RGB, Level0, subsampled 1:6, JPEG compression / earthnet.esrin.esa.it
CCRS / Landsat 4&5 / TM / 400x258 image, subsampled 1:16, ground sample spacing = 480m, usual band inclusion = 2,3,4, JPEG compression / ceocat.ccrs.
nrcan.gc.ca
NASDA / Landsat / MSS / 512x498 image, composite 7,5,4 as RGB, subsampled 1:6.33 horizontal, 1:4.37 vertical, JPEG compression / eus.eoc.nasda.go.jp
USGS/
EDC / Landsat 1-5 / MSS / 390x590 image, composite 7,5,4 as RGB, each band reduced every 6th line and every 6th sample, JPEG compression (Q=75) size 60kB. / edc
ESA/
ESRIN / Landsat / MSS / 1000xvariable image, composite 6,5,4 RGB, Level0 subsampled 1:3, JPEG compression / earthnet.esrin.esa.it
NASA / SeaStar / SeaWiFS / Level-1 browse is a subsampled (every other pixel, every other line) version of the band-8 raw radiance counts image.
Level-2 browse is a subsampled (every other pixel, every other line) version of the chlorophyll a image.
Level-3 browse is a subsampled (every 8th pixel, every 8th line) version of the SMI image array. / seawifs.gsfc.nasa.gov/cgibrs/seawifs_browse.pl
ESA/
ESRIN / SeaStar / SeaWiFS / 418xvariablex24bits image, subsampled at 4 Kms, JPEG compression / earthnet.esrin.esa.it

Table 2-2 Survey of Browse Products (Continued)

Agency / Satellite/Platform / Sensor/
Instrument / Browse Characteristics / URL
NASDA / SPOT / HX / 512x512x8bits image, subsampled 1:5.86, JPEG compression / eus.eoc.nasda.go.jp
NASDA / SPOT / HP / 512x512x8bits image, subsampled 1:11.72, JPEG compression / eus.eoc.nasda.go.jp
CCRS / SPOT 1-3 / Panchroma-
tic & Multi-
spectral / 250x250 image, subsampled ~1:12 or 1:24, ground sample spacing = 240m, usual band inclusion = 1,2,3 or pan, JPEG compression / Ceocat.ccrs.
nrcan.gc.ca
NASDA / JERS / OVN / 512x512x8bits image, subsampled 1:8 horizontal 1:6.05 vertical, JPEG compression / eus.eoc.nasda.go.jp
ESA/
ESRIN / JERS / VNIR / 1024xvariable image, composite 3,2,1 RGB, Level0 subsampled 1:4, 1024xvariable, JPEG compression / earthnet.esrin.esa.it
NASDA / ERS / AIM / 800x800x8bits image, subsampled 1:8 horizontal 1:8.5 vertical, JPEG compression / eus.eoc.nasda.go.jp
ESA/
ESRIN / ERS / ATSR / Quick Look size when framed according to ATSR product extension: 210 sample x 256 lines image.
Descending frames (most daytime): color composite ch 11micron inverted BT and 1.6 Refl.
Ascending frames (most nighttime) ch 11micron inverted BT. Lat/Long grids and coastlines. subsampled 1:2, JPEG compression / earthnet.esrin.esa.it
esapub.esrin.esa
.it/eoq/eoq52/
buon52.htm
NASDA / MOS / MES / 512x450x8bits image, composite 4,2,1 as RGB, subsampled 1:4, JPEG compression / eus.eoc.nasda.go.jp
NASDA / MOS / VTI / 512xvariablex8bits image, subsampled 1:1.95 horizontal 1:1.79 vertical, JPEG compression / eus.eoc.nasda.go.jp
NASDA / ADEOS / AVM / 512x512x8bits image, composite 4,3,2 as RGB, subsampled 1:9.77, JPEG compression / eus.eoc.nasda.go.jp
NASDA / ADEOS / AVP / 512x512x8bits image, subsampled 1:19.53, JPEG compression / eus.eoc.nasda.go.jp
NASDA / ADEOS / OCT / 2048x1024x8bits image, subsampled 1:2, JPEG compression / eus.eoc.nasda.go.jp
NASA / Meteor 3
Nimbus 7 etc. / TOMS / Level 3 browse is a 1 degree latitude by 1 ¼ degree longitude grid full resolution (not subsampled), GIF compression / toms.gsfc.nasa.
gov
ESA/
ESRIN / IRS-P3 / MOS / 384x384x24bits image, full resolution, JPEG compression / earthnet.esrin.esa.it
USGS/
EDC / Scanned Aircraft / Digital Orthophoto Quads / 950x750 image, B&W images subsampled by 2 followed by 2 wavelet passes, JPEG compression (Q=75) size 150kB. / edc

While table 2-2 is not a comprehensive survey of all browse products available, it does provide a sense of the wide array of browse data available and the various ways in which it can be produced. The final entry in Table 2-2 shows browse for photographic data taken from aircraft. This example is meant to point out that there are many types of data other than satellite data for which browse images are produced. Conversely, there are many types of data (e.g., Field Campaign data and Ground Truth data) which are not image data and for which browse may not be produced. Finally, there are browse products, such as histogram data, that are not image products.

These tables list examples of precomputed or static browse. There are also instances of interactive or dynamic browse. Some agencies'' client software allows the user to choose characteristics (e.g., image size, false color imaging) by which browse is produced “on the fly.” With advances in the World Wide Web, interactive browse will gain additional capabilities and will become more common.

3. Attributes

The result of a search is often in the form of a description of the object or objects that meet the search criteria. The description consists of values of attributes where the attributes have been chosen to convey necessary information about the object(s). Because different types of objects can be described by completely different sets of attributes, and because attributes can be changed at any time by the catalogue system, descriptions of attributes must be provided by the system's server. This means that the server has to provide not only the names of the possible attributes but also a full definition of every attribute, which includes, for example, the possible values and whether the attribute can be used in a query.

3.1 Attribute Definition

A set of attributes provides the means by which an object can be described in an information system. Different types of objects will be described by different attribute sets. In the context of attribute definitions for a catalogue system, different attribute sets will exist to describe collections of data, guides and products. Further, for different types of collections of data (e.g., a collection of image data or a collection of sea temperature profiles) and different types of products, different attribute sets may be defined.