Overview Presentation in Order to Explain the Issues on Implementation of Coveragesfrom

Jordi Escriu – Facilitator INSPIRE Thematic Cluster #3version 13.04.2016

o be submitted to MIG-T

Issues implementing INSPIRE Coverages

Executive summary

Implementers from the Member States have to make their data conformant to INSPIRE before 2020, for data sets under Annexes II and III of the Directive.This includes the Elevation and Orthoimagery themes, whereNational Mapping and Cadastral Agencies (NMCAs) play a central role as data producers.

The main spatial representation type of data belonging to these themes is the raster data format. In the scope of INSPIRE, such data shall be implemented and made available across the European data infrastructure as coverages according the concepts stemming from ISO 19123, taking into account the standards currently in force (ISO / OGC).

Whereas many NMCAs have already got significant experience about transformation and set up of download services for INSPIRE themes focused on vector data, transforming and serving coverage-based raster data for the purposes of the Directive is not yet widely spread and most of implementers are really willing to get more knowledge about these topics.

TheINSPIRE Thematic Clusters is an initiative, launched by the European Commission in the context of the Maintenance and Implementation Framework (MIF), aimed to sharing and consolidating the implementation experiences and make proposals for changes to the INSPIRE data specifications.

Activity in Thematic Cluster #3, focused on the Elevation, Orthoimagery, Coordinate Reference Systems and Geographical Grids themes, has been aligned from the beginning of this initiative with thecommunity of NMCAs through the EuroGeographics INSPIRE-KEN, jointly organizing the following recent activities in the scope:

Workshop about Transformation of Coverage-Based Data Themes and WCS – Barcelona(ICGC venues), 29-30 September 2015

The workshop aimed to share experiences about data transformation and implementation of coverage-based themes (with focus on Elevation and Orthoimagery), but also to other topics such as the use and application of Web Coverage Services (WCS). Many NMCAs from those involved in the activity were alsoparticipating in the ELF project, that involves the theme Elevation and probably Orthoimagery.

The main expected outcomes were:

Learn and improve knowledge about coverage data and WCS,in general.
Share transformation experiences on Elevation and Orthoimagery to get ideas about available paths (e.g. main difficulties, potential solutions, choice between options).
Check that implementers have the same interpretation of the related INSPIRE data specifications.
Identify potential common requests for change the mentioned INSPIRE data specifications, to be reported and discussed by MIG-T (Maintenance and Implementation Group).

The workshop was a good opportunity to meet and share experiences and issues about transforming and serving the INSPIRE coverage-based themes, with a total of 23attendees. One of the most interesting sections of the workshop was a Training Session on Coverages and WCS, given by Alex Dumitru (Jacobs University, Bremen).

The minutes of the workshop, the presentations and a summary of the implementation experiences from the participating Member States is available here:

Follow-up Webinar on Coverage Data and Services, 18 January 2016

After digesting the outcomes from the previous workshop, severalquestions and issues remained open.

Peter Baumann, coverage and Big Data expertfrom the Jacobs University in Bremen who organized the training session on coverages in theworkshop, kindly proposed to set up a follow-up webinar to clarify concepts and answer these questions.

The webinar was mainly dedicated to those who attended the workshop in Barcelona,but itwas open to everyone having particular interest in coverage data and services.

As a result of the mentioned webinar, a list of open issues about coverage implementation has been drafted by Thematic Cluster #3.

The main purpose of this document is to make this list available to all MIG-T members and open a discussion within the group in order to decide on the best way to proceed for solving these issues.

List ofopen issues on implementation of coverages

This section provides a summary of the outcomes and remaining issues from the Follow-up Webinar on Coverage Data and Services, co-organized by the EuroGeographics INSPIRE-KEN and INSPIRE Thematic Cluster #3, on 18th January 2016.

Documentation from this event is available at:

There will be 2 possible ways of providing coverage data for INSPIRE purposes:

Case 1: Download by predefined data sets (e.g. via ATOM Feeds).

This is the only way which is currently in force and operative. For this reason, at the moment implementers are focusing their efforts on obtaining XML snippets trying to reproduce the structure of theme-specific coverages in conformity with the corresponding INSPIRE (raster) data models.

Note that such models are based on the ISO 19123 – Schema for coverage geometry and functions plus minors INSPIRE extensions. In other words,data shall be implemented as a coverage – which should be in fact the natural response of a WCS to a GetCoverage request.

Case 2: Download by WCS.

This option is not available yet, since the rules governing the provision of coverage data through WCS are not finished – Drafting the rules and guidelines regarded to this option forms part of the tasks assigned to MIG-T MIWP-7b.

GML coverages file should be therefore derived - as much as possible - automatically from GetCoverage requests to a WCS server, preventing unnecessary manual intervention and related issues.

Many data providers,especially NMCAs, do not have previous experiencewith coverage data and services (WCS), neither at implementation nor at exploiting level. Due to this fact, they remain full of questions to be solved, and some issues may prevent a successful implementation in time.

The list of issues below is aimed at providing a first overview about the current situation, worth to be evaluated for drafting the structure and content of the new Technical Guidelines for providing INSPIRE coverage data using WCS- Task 2 of MIG-T MIWP-7b.

Issue 1 - Will the evolution in standardization (from GMLCOV to CIS v1.1) be taken into account by INSPIRE data specifications dealing with coverage data (e.g. EL and OI)? When it might occur?

CIS v1.1 will be adopted by OGC in spring 2016 – It will become the ISO 19123-2 standard, whereas OGC WCS will be treated in a new dedicated ISO standard.

Description of the issue -

Any possible changes in the INSPIRE Data Specifications derived from the adoption of CISv1.1 would not encourage implementers to transform their data quickly / according the roadmap.

However, CIS v1.1 model is probably not going to be adopted by INSPIRE in the short term - as other examples of standardization evolution (e.g. from GML to other more flexible and less-heavy formats). Emmanuel Devys (IGN-France), who participated in the examples included in CIS v1.1, would be a good candidate to develop INSPIRE examples.

Interesting link to be distributed:

Issue 2 - How to implement the concepts of mosaicking (OI) and tiling (EL & OI) in INSPIRE coverages?

Description of the issue -

The concept of mosaicking is introduced in the INSPIRE data specification on Orthoimagery (‘MosaicElement’ feature type and its subtypes): They are intended to identify the contributing area and the acquisition time of one or several input images used to generate a mosaicked orthoimage coverage. I.e. mosaic elements are used to precisely describe the temporal characteristics of mosaicked orthoimage coverages. The modelling of such elements corresponds to a thematic need. Hence, such concepts shall be encoded somehow within the INSPIRE coverage files.

The concept of tiling in the scope of the INSPIRE data specifications (at least for Elevation and Orthoimagery) is the tiling describing logical structures (e.g. mapsheets, administrative units like regions or districts, etc.). The link between different tiles to cover an area of the territory has been modelled with the concepts of “OrthoimageAggregation” and “ElevationGridCoverageAggregation” (in the Orthoimagery and Elevation themes, respectively).

Implementation of the previous concepts, mosaic elements and tile / coverage aggregation, constitute independent issues. However, it is not clear how to implement both of them in GMLCOV, e.g. might be different ways to implement them in practice. The following discussion thread was intended to bring discuss and bring light to this issue:

According Peter Baumann, a coverage can be modelled as (recursively nested) partitions in CIS v1.1 – This possibility is worth to be examined in the future, if CIS v1.1 is adopted by INSPIRE.

According Dominique Laurent, it is not clear why coverage aggregations are really needed, and implementation of this concept makes things complex that simpler.

The tiling approach selected in the implementation may have a direct impact on efficiency – See Issue 3.

Issue 3 - How to deal with huge volume of coverage data, in each case?

Case 1: Download by predefined data sets.
Case 2: Download by WCS.

Description of the issue -

According Jukka Rahkonen (leader of MIWP-7b WCS), there are some problems on the server side. Some WCS implementations build the final output of the GetCoverage request first totally in the RAM of the server before sending it out to the client. He reported the following information about such problems:

MapServer actually consumes memory about three times the size of the output - 3 GB of RAM is needed for making an output of 1 GB. The input data can easily be terabytes but the size of the subset or slice of the coverage is limited by the amount of RAM on the computer. Concurrent requests are all as expensive and for handling 2 concurrent GetCoverage requests of size 1 GB the peak memory consumption is about 6 GB.

The following thread describes the MapServer case pretty well:

GeoServer is using less memory and in a simple test I could serve 4 concurrent GetCoverages of 1.6 GB each with a test server that had 8 GB of RAM with an acceptable speed. After that adding more concurrent requests made all of them very slow. Based on these experiences, it can be concluded that with 8 GB RAM 1 GB subsets/slices should be safe if there are not many concurrent requests to handle, but bigger ones can make troubles. Of course it is possible to add memory to the server and put the limit higher. Other servers like Rasdaman may be much better with big outputs.

The following thread is dealing with an issue that is already fixed in GeoServer but it may be interesting that one of the core developers considers that synchronous WCS is not suitable at all for big downloads but suggests WPS instead

"For large extractions WCS is not a suitable protocol anyways, as it does not have an asynchronous mode, you might want to have a look at the WPS download module instead, it's a community module that we created with the sole purpose of replacing WFS/WCS for large extractions where the asynch call support is a must."

Case 1: Download by predefined data sets

Most data providers consider each tile as a coverage (e.g. IGN-France). As presented in the Workshop about Transformation of Coverage-Based Data Themes and WCS - Barcelona, 29-30 September 2015, this is a general trend among NMCAs.

In this way data providers are able to split the complete coverage over a huge territory in several pieces, for both: organizational and efficiency purposes. This might be implemented in practice by using the concept of tiling (described in Issue 2), linking them by coverage aggregations as foreseen in the INSPIRE data models for Elevation and Orthoimagery.

Are there any other and better solutions? E.g. using multiple rangeSets instead of tiling?

The discussion is open in INSPIRE Thematic Cluster #3:

Case 2: Download by WCS

During the webinar Peter Baumann commented that the implementation details for describing logical structures (tiling, as defined in the scope of the data specifications) is not treated by the WCS interface, and that a COVERAGE shall not be confused with an IMAGE – i.e. do not make tiling on the WCS server, keep the whole data set.

Therefore, it is not clear if the concept of tiling from the data specifications does make sense when serving coverage data through a WCS – To be further discussed and clarified.

Taking into account this assumption (tiling not to be implemented when serving coverage data through WCS), how should we prevent users to request too big data sets?

According Peter Baumann, the best solution to prevent suck kind of problem is the internal tiling (i.e. implemented in file formats e.g. Tiled TIFF).

Issue 4 - Partial conceptual redundancies between INSPIRE coverages attributes and GMLCOV components / INSPIRE coverage model extensions – E.g. How to deal with the redundancy about coverage extent?

Description of the issue -

The INSPIRE models dealing with coverages partially extends the standard way of modelling and implementing OGC coverages. Such INSPIRE extensions corresponds to thematic needs identified by the Thematic Working Groups (TWGs) in the process of drafting the Technical Guidelines and should be therefore considered in the implementation of INSPIRE data.

However, the need for such extensions should be revisited one-by-one. This is due to the fact that the work of the TWGs was mainly focused on the conceptual level and the mapping of some of these concepts with the implementation standards was not sufficiently explored. Hence, there is room for some duplicities when implementing INSPIRE coverage data according the INSPIRE models.

EXAMPLEA good example of such redundancies is the identification of the coverage extent – largely discussed in the following thread of Thematic Cluster #3:

There are 2 attributes dealing with this information:

-boundedBy (from GML): It is optional and it may be approximate.

It does not allow the identification of discontinuous spatial extents.

-domainExtent: Attribute included in the INSPIRE models for coverages (at least Elevation and Orthoimagery). It constitutes an extension to the standard way of implementing an OGC coverage (i.e. INSPIRE extension), hence it is not recognized at all by WCS.

The main benefit of this attribute is that supports the identification of discontinuous spatial extents for coverages (something needed e.g. when coverage aggregations are identified).

Is there a real need of domainExtent? Will it be used if download by predefined data sets?

There is a need of identifying the spatial extent of INSPIRE coverages, which may be discontinuous. But it must be clear how to implement it (i.e. by using boundedBy or by using domainExtent).

This issue is probably not regarded with the fact of delivering INSPIRE data with predefined data set downloading or any other INSPIRE conformant option. If the first option is used, the extents informed within the predefined data sets shall be consistent with such pieces / data units.

In summary, there is an agreement to minimize such INSPIRE extensions as possible. We need to examine each additional attribute proposed by INSPIRE case-by-case. Making a complete list of the INSPIRE deviations from OGC coverages would be highly desirable in order to proceed with this task. If CIS v1.1 is adopted by INSPIRE in the near future, it would be quite sensible to perform this task while updating the INSPIRE data models and the Technical Guidelines.

Peter Baumann mentioned during the webinar that there was not much danger if some information is duplicated in INSPIRE coverages, because this issue can be checked automatically with appropriate applications. The real danger here is that WCS v2.0 ignores INSPIRE extensions, i.e. the information included in such extensions is omitted / not read by software applications.

Issue 5 - Which GML component of the rangeSet shall be used to reference the coverage values when using multipart representation?

Description of the issue -

Current applicable standards (OGC 07-036 GML / OGC 09-146r2 GML Application Schema Coverages) do not clarify the question:

a)According to OGC 09-146r2 GMLCOV, it is valid to use the gml:rangeParameters component of gml:File.

b)According to OGC 09-146r2 GMLCOV, the rangeType is used to document the parameters of the rangeSet – NOTE: WCS2.0 clients do not use the rangeSet parameters, but use information provided under the rangeType.

c)According to OGC 07-036 GML, the coverage values shall be provided in an external file referenced through the gml:fileReference property, whereas the rangeParameters component is intended for defining further semantics on the structure of the underlying data.

This is probably due to the fact that there is not only one standard for coverages. Standardization of coverages was done progressively, drafting OGC 09-146r2 GMLCOV as a complement of OGC 07-036 GML.

A common approach and guidance should be provided for ensuring interoperability, while CIS v1.1 is not adopted.

Issue 6 - What is the purpose of having metadata redundancy for coverages? Which kind of metadata is expected at coverage level, in the metadata hook?

Coverages may have metadata at different levels:

-At data set level (xml metadata file of the data set).

-In the image file (header).

-At coverage level (optional metadata hook, as proposed in GMLCOV).

Description of the issue-

Guidelines about the purpose and content of the different levels of metadata are required.

Data set level metadata is supposed to play a role in discovering the data set through a Catalogue Service (CSW), whereas the image file header may contain several mandatory and/or optional parameters that are required by a specific file format (e.g. GeoTIFF) according to the format data specifications (implementation level). Such parameters play a role in reading the file by software applications.

The main purpose and content of the metadata hook was slightly explained by Peter Baumann during the webinar, but in general it was not clearly understood by the audience. Does it provide easier and quicker access to the metadata elements (without having to access any external resource) when using a WCS? - It would be worth to clarify this aspect again.

In principle, redundancy of metadata is not such an issue, since there is or may be automatic tools for checking consistency. It would be worth to have any guidelines on how to check this consistency and a list of applicable testing tools.

Issue 7 - What is the state-of-play of WCS servers and client applications?