Report on activities carried
under MIWP-16
Table of content
Mandate
Review of the monitoring indicators with the objective of automating their calculation
Proposed non mandatory additional elements to be reported for spatial data sets and/or services
Metadata UUIDs of spatial data sets
Metadata UUIDs of the discovery, view and download services related to a spatial data set
Metadata UUID of the discovery service servicing the metadata of a network service.
Direct accessibility of a service
New INSPIRE monitoring workflow available to Member States
Prototype tools to automate the inspire monitoring
Prototype dashboard for monitoring data officially reported by Member States
Proposed updates to the current monitoring requirements
Indicators proposed to be discarded
DSi1: Geographical coverage of spatial data sets
NSi3: Use of network services
Indicators proposed to be discarded in case of fully automated monitoring
MDi1: Existence of metadata
NSi1: Accessibility of metadata through discovery services
Monitoring variables proposed to be added (on a voluntary basis)
Indicator proposed to be added
Open Issues
Appendix A – Theoretical and Practical Mapping of Metadata Elements to Monitoring Variables
Remarks
MIWP-16 monitoring mapping - theoretical & practical
Indicator MDi1 (Existence of metadata)
Indicator MDi2 (Conformity of metadata)
Indicator DSi1 (Geographical coverage of spatial data sets)
Indicator DSi2 (Conformity of spatial data sets)
Indicator NSi1 (Accessibility of metadata through discovery services)
Indicator NSi2 (Accessibility of data sets through view and download services)
Indicator NSi3 (Use of network services)
Indicator NSi4 (Conformity of network services)
Raw data section
Appendix B - Questionnaire issued to Member States by the working group.
Questions related to the implementation of a dashboard
Review of the monitoring indicators
Questions related to the inclusion of reporting elements into the dashboard
Questions related to the current and foreseen use of monitoring information at European and Member State level
Appendix C - Analysis of the responses from member states to the questionnaire and answers to the questionnaire (Q1/2014).
Questions related to the implementation of the dashboard (questions D-B 1-13)
Questions related to the inclusion of reporting elements (Art. 11 to 16) into the dashboard
Questions related to the current and foreseen use of monitoring information at European and Member state level.
Review of the monitoring indicators
Answers to the questionnaire
Mandate
As a result of a meeting of Member States in Copenhagen on the 15/10/2013 a strong need was expressed for the improvement of the usefulness and the reliability of monitoring information. The MIWP-16 group was formed as a result of the meeting and the work programme of the group was defined in two phases. The objectives of the phases are shown below:
●Phase 1
○Review the indicators defined in Articles 3 to 10 of the Commission Decision of 5 June 2009, implementing Directive 2007/2/EC of the European Parliament and of the Council as regards monitoring and reporting, with the objective of automating their calculation.
○Analyse how to extract monitoring information from the metadata records available in the EU-Geoportal or from the metadata records available in national geoportals.
○ Analyse how to filter out the metadata records which are for INSPIRE datasets out of a catalogue containing metadata on more datasets than only INSPIRE datasets.
○Design a dashboard (including functional requirements) which would provide access to all monitoring information and related indicators for every Member State.
○Test the approach with pilot countries.
○Update the justification document for indicators as appropriate.
○Update the Technical Guidelines on monitoring as appropriate.
●Phase 2
○Possibly, propose evolutions of discovery metadata or monitoring data requiring modifications of some legally binding pieces of INSPIRE legislation.
Review of the monitoring indicators with the objective of automating their calculation
A questionnaire, available in Appendix B, was issued to all Member States asking questions about the monitoring and reporting set up and the perceived usefulness of the information provided by the member states. The group explored how the information in the Implementing Rules for Monitoring and reporting could be derived automatically from the Metadata that is provided by the member states relating to INSPIRE data.
It was clear at this stage of the work, that a small number of indicators could not be derived solely from the metadata without member states providing some additional information. A review of all indicator variables with their automatability status as well as a theoretical mapping to metadata elements is available in Appendix A. As a result of the questionnaire a very small number of indicators were considered not to be useful in the real world and thus we are proposing that these indicators be dropped. For those indicators that can’t be derived directly from the provided metadata records, the group has identified methods that would allow member states to supplement the metadata with additional information allowing for the calculation of the indicator from a combination of the metadata and the additional information.
Proposed non mandatory additional elements to be reported for spatial data sets and/or services
In order to improve the quality of the monitoring data, we propose to add to the current monitoring requirements a few elements which can be derived from the metadata for spatial data sets and services.
Metadata UUIDs of spatial data sets
A UUID is a unique and persistent identifier. Applied to metadata for datasets, it allows for the unambiguous identification of a metadata record describing a dataset.
Retrieving metadata uuids of spatial data sets can be done automatically and this has been demonstrated during the set-up of prototype tools to automate the Inspire monitoring.
Metadata UUIDs of the discovery, view and download services related to a spatial data set
Knowing the metadata UUID of the discovery, view and download services related to a spatial data set permits to establish the link between data sets and services. From this link, it is then easy to know which themes are related to any given service by analysing the information reported for the underlying datasets and there is no need anymore to provide the information required under column T of the XLS monitoring template. Furthermore this linkage improves significantly the quality of the reported information as, instead of simply knowing that a data set is available through a view and/or a download service, it would then be possible to know through which services the data set is actually available.
Retrieving metadata uuids of services related to a spatial data set can be done automatically in most cases and this has been demonstrated during the set-up of prototype tools to automate the Inspire monitoring.
Metadata UUID of the discovery service servicing the metadata of a network service.
This information provides more value than simply knowing that the metadata of a network service is available through an unknown discovery service.
Retrieving the metadata uuid of the discovery service servicing the metadata of a network service can be done automatically in most cases and this has been demonstrated during the set-up of prototype tools to automate the Inspire monitoring.
Direct accessibility of a service
In order to automatically validate the conformity of reported network services with the INSPIRE IRs, it is necessary to know if they are directly accessible from the internet (i.e. without login, key, etc.). Statistics on the number of data sets served by services directly accessible from the internet could also be computed, which would provide an estimate of the amount of data freely available.
To a certain extent, this information can be extracted from the conditions applying to access and use but is expressed differently by each Member State when there are guidelines on that topic in national metadata profiles. A common approach would be preferable. For example, the MIWP-8 is proposing a codelist for LimitationsOnPublicAccess in which noLimitations means that there is no limitation on public access to spatial data sets and services.
New INSPIRE monitoring workflow available to Member States
Until 2014 Member States could report their monitoring data either through an excel file formatted according to a template provided or through an XML file according to a schema which had never been updated since 2009. As a consequence more elements had to be provided via XLS than via XML. Only two Member States were delivering their monitoring data as XML files until 2014.
End 2014, the 2009 UML class diagram modelling the monitoring data has been updated in order to cover all the elements present in the XLS template as well as the additional elements proposed for inclusion in the INSPIRE monitoring by the MIWP-16. After approval of the diagram by the MIWP-16, the corresponding XSD schema has been updated as well.
For the 2015 monitoring exercise (on 2014 data), Member States have been encouraged to provide their monitoring data as XML according to the new schema. In order to facilitate the transition from XLS to XML, two set of tools have been developed: a webform (developed by the EEA) and the tools to generate the monitoring report out of the metadata contained in a discovery service (MIWP-16 activity).
The formerallows the generation of a monitoring XML file out of a monitoring XLS file and allows for the easy editing and printing of monitoring XML files while the latter allows for the creation of a monitoring XML file out of the metadata records contained in a discovery service (a detailed description is available in the next section).
Thanks to the work of the MIWP-16 and thanks to the set of tools which have been made available, the following workflow has been successfully used by some Member States in 2015:
- Generate a XML monitoring file automatically out of the content of their national catalogue
- Review the indicators automatically calculated and possibly proceed to updates in the catalogue (in which case, rerun 1)
- Optional: Load the XML file into the webform and edit if needed (e.g. to add datasets or services which are not in the national catalogue). Save the modified XML file.
- Upload the XML file to Reportnet
Prototype tools to automate the inspire monitoring
A set of tools has been developed in order to provide Member States with the possibility to:
- harvest one or several discovery service endpoints (and configure some harvesting parameters)
- compute the indicators based on the harvested data or a subset of them (e.g. to calculate indicators corresponding to the metadata provided by one responsible authority only)
- display the indicators and related variables in a dashboard similar to the dashboard for monitoring data officially reported by Member States. This provides an easy way to visualise (and analyse) the performance on the various indicators.
The specifications for the tools have been developed by the MIWP-16 and the development of a prototype has been done by Titellus through funding from the EEA. Released under an open-source licence, some Member States have deployed them in their infrastructure and are further developing them. The MIWP-16 has always insisted that such developments should be contributed back to the open-source community for Member States to benefit from them.
The rules applied to derive the monitoring indicators from the metadata are described in detail in Appendix A. Due to the lack of a commonly agreed validator (work on-going in MIWP-5) for metadata, it has been agreed within the group to rely on the validator available on the INSPIRE Geoportal and to consider a metadata record as valid if the level of conformity returned by the validator was greater or equal to 75%.
Figure 1: Harvesting configuration page (Austria and BE/Wallonie are displayed)
Figure 2: Form allowing to create and preview the monitoring report out of harvested metadata (here shown for NL)
These tools are available at access is protected in order not to confuse people with the dashboard for monitoring data officially reported by Member States. Once the functionalities requested and funded by the Netherlands will have been integrated into the instance hosted at EEA (e.g. possibility to validate conformity of services against the ELF validator to compare against the declared conformity in the metadata), the access will be granted to the INSPIRE reporters and any additional person nominated by a NCP.
In order to guarantee the sustainability of the tools developed to automate the INSPIRE monitoring, their maintenance process (project governance, management of change requests, funding, …) needs to be addressed by the MIG.
Prototype dashboard for monitoring data officially reported by Member States
In order to improve the usefulness of the monitoring indicators, the MIWP-16 has considered it be appropriate to provide a means to visualise and query the indicators, the corresponding variables as well as the underlying reported data. Specifications for a dashboard have therefore been drafted by the MIWP-16 and the EEA funded the development of a prototype dashboard together with the prototype tools to automate the INSPIRE monitoring. The source code for the dashboard is released under an open source licence and is available at Some Member States have installed it within their infrastructure.
All available monitoring data have been loaded to the dashboard which has been presented at the 2015 INSPIRE conference in Lisbon. The prototype dashboard is publicly available at
Figure 3: Landing page of the dashboard with a list of specialised dashboards on the right
Figure 4: Maps showing indicators trends over the past 3 or 5 years
Figure 5: Indicators and variables related to network services in 2014
Figure 6: Statistics computed out of the raw data reported with filtering capabilities (narrow your search section)
In order to guarantee the sustainability of the dashboard for monitoring data officially reported by Member States, its maintenance process (project governance, management of change requests, funding, …) needs to be addressed by the MIG.
Proposed updates to the current monitoring requirements
The review of the monitoring indicators carried out at the beginning and during the course of the project together with the tests performed during the setup and the operation of the dashboard prototypes lead us to propose some modifications to the current monitoring requirements:
●Abandon some indicators and stop the collection of their underlying data;
●Introduce the collection of some new elements in order to improve the quality of the monitoring information.
Whilst it is clear for most the indicators how the variables are calculated, for a small number of the indicators clarification of the calculation method for those variables needs to be more rigorously defined.
Indicators proposed to be discarded
Having completed a survey of member states on the usefulness of the indicators (see Appendix B and C), a meeting was convened in Arona (Italy) to discuss how the indicators could be derived from the metadata (that is published by the Member States and harvested to the INSPIRE GeoPortal) directly or if additional information was required to derive the values of the indicators.
One indicator series DSi1 (Geographical coverage of spatial data sets) although meaningful from a theoretical perspective, in practice does not provide the information that was intended. It was very difficult especially in a federated environment to provide the actual area and the relevant area. For any individual data set in the federated environment it is not always clear (certainly at this stage) which other datasets within the metadata for a Member State should be combined to give a total picture.
DSi1: Geographical coverage of spatial data sets
DSi1 corresponds to the ratio between the sum of the actual areas of all the spatial data sets of all Annexes and the relevant areas of all the spatial data sets of all Annexes. DSi1.1, DSi1.2 and DSi1.3 are sub-indicators for respectively Annexes I, II and III.
The relevant and actual areas of data sets are not part of the metadata elements required by INSPIRE to describe a data set and therefore, with the exception of Sweden that has included these pieces of information in their national metadata profile, it is not possible to derive DSi1 from the metadata served by INSPIRE Discovery services.
Furthermore most Member States representatives in the MIWP-16 explained that it is extremely difficult to get the relevant and actual area for each data set when they collect the material to prepare the annual INSPIRE monitoring. As a consequence, some Member States are either not providing any figures about the actual and relevant area of their spatial data sets or providing the exact same figures for every data set they report. In 2014 (reported in 2015) two Member States did not report at all on DSi1, 11 had a DSi1 greater than 0.99 out of which 5 had a DSi1 equal to 1 meaning that the actual and relevant areas were reported as identical.
It is proposed to stop the collection of the relevant and actual area of spatial data sets and to consequently remove DSi1 from the list of INSPIRE monitoring indicators. This may require the “monitoring and reporting” Implementing Rules to be reviewed, removing the reporting of this indicator. This indicator is identified in Chapter III of the Implementing Rules, Article 5. We recommend that this entire Article is dropped from the Implementing Rules.
NSi3: Use of network services
NSi3 corresponds to the ratio between the sum of the annual number of network service request for all services (discovery, view, download, transformation, invoke) and the number of network services. NSi3.1, NSi3.2, NSi3.3, NSi3.4 and NSi3.5 are sub-indicators to monitor the use of respectively discovery, view, download, transformation and invoke services. The sub-indicators themselves are the summation of all the service requests for all service requests for all Inspire services. A particular data provider may choose in any year to merge of split data layers between one or more services. For example a data providers for the historic environment may have 12 layers of information the correspond to INSPIRE. In year 1, the provider decides to have 1 service with all 12 layers. The next year they decide to have 1 layer in a service. So with no change in data provision the indicator would change significantly.