Of the Meeting of the Working Group

Luxembourg, 10 April 2017

Minutes

of the meeting of the Working Group

"Methodology"[1]

5 April 2017

Luxembourg, Eurostat - Bech building, room Ampere

1. Approval of the agenda

The chair of the meeting, Martina Hahn (head of the unit B1) welcomed all participants and proposed the agenda of the meeting. The agenda was approved.

1. Nature of the meeting

The meeting was not-public but open to delegates from the 28 Member States with observers from EFTA, ECB and UNCE. All documents are public and available at:

1. List of points discussed

Item 1: Validation

Item 1.1: Latest developments and foreseen activities

Presentation

Eurostat informed about the progress on the ESS Validation project. The Task Force on Validation created under the umbrella of the WGM progressed in line with the roadmap: (1) feedback was given on the methodological handbook, (2) the usability of VTL by statisticians was assessed and (3) the Business Architecture for validation in the ESS was finalised. In 2017, the work will mostly focus on the documentation and agreement on validation rules at WG level. In 2017 and Q1/2018, the “ESSnet Validat integration” will work on: (1) finalising the methodological handbook, (2) creating a standard validation report structure, (3) developing a cost/benefit model for the 3 scenarios of the Business Architecture and (4) organising 4 regional conferences to disseminate and discuss the outcomes of the work. The outcome of the ESSnet will also be useful for grants aimed at supporting the implementation of the scenarios of the Business Architecture planned at the end of 2017.

Eurostat presented the 3 scenarios of the Business Architecture that countries may optionally implement. The related IT building blocks, validation principles and data acceptance process were also introduced.

Discussion

After the ESSnet Validat integration was established, Eurostat suggested revisiting the Task Force composition, allowing 1-2 further member states to join. No new country expressed an interest.

The deployment of a standard framework for the clarification of validation rules and roles by statistical domain will be done progressively between this year and 2019. The clarification of the validation rules and the link with compliance monitoring should limit validation “ping-pongs” between Eurostat and countries, improve data quality and limit the risks of misunderstandings on the compliance of data transmitted to Eurostat.

Countries welcomed the Business and IT architecture for ESS Validation. It was explained that its implementation should contribute substantially to the transparency and burden reduction on validation activities.

Opinions and Actions

Eurostat will provide to the WGM a roadmap for the deployment of a standard framework for describing validation rules and roles in statistical domains.

The Business Architecture was approved by the WGM. It will then be submitted to the DIME/ITDG.

Item 1.2: BIRD (Bank's integrated Reporting Dictionary) project

Presentation

The ECB presented the BIRD (Bank's integrated Reporting Dictionary) project which uses VTL to describe validation and transformation rules. The presentation showed the role of the actors, the business process and the underlying data model. More information is available at

Discussion

The ECB explained that the system for the moment covers only the validation intra-data files (levels 0 and 1) but it evolves and in the future it will cover validation between datasets or for time series.

Opinions and Actions

The presentation of ECB was well appreciated. It showed that VTL starts to be used in wide communities.

Item 1.3: Break-out sessions (methodology, architecture, services, VTL)

Presentation

Four Break-out sessions took place in parallel:

• (Session 1) Methodology,
• (Session 2) Business architecture for ESS validation,
• (Session 3) Tools and services,
• (Session 4) VTL (Validation and Transformation Language).

Participants could contribute to 2 sessions (15 minutes each).

The main objective of the 4 break-out sessions was to exchange views on practical implications for NSI of the deployment of an ESS validation framework.

Discussion

(Session 1) Methodology:

Two topics were briefly discussed with WG participants including two representatives of the ESSnet that cover these topics in Work Packages:

(1) Standard validation reports for machines and statisticians:

• they should reflect the data, the validation rules and the result of the validation process,
• they should be used as an outcome of the validation of data prepared for Eurostat by all parties involved in data validation in the ESS,
• they could also be used as an outcome of validations performed in NSI for data in an earlier production stage as the data produced for Eurostat,
• the results could be used to produce metrics.

(2) Metrics aim at monitoring the quality of the validation ruleset and the results of the validation procedures. Over the time, the number of rules may grow. Metrics may help to keep control and limit redundancy between rules and limit complexity.

(Session 2) Business architecture for ESS validation:

• The concepts introduced in the Business Architecture (e.g. the classification of severity rules and the concept of establishing acceptance criteria) could be reused to modernise and harmonise validation practices at national level. However, NSIs do not have the same relationship with their data providers is NSIs as Eurostat has with NSIs.
• It would be beneficial to have more harmonisation of validation approaches within broad statistical domains (i.e. validation approaches and methods across business statistics should be consistent). This goes further than what is currently proposed by the business architecture, which only advocates for a common process across all domains.
• The scope of the validation architecture is quite narrow, as it concentrates on the validation of the data but not on the validation of methodologies. This could be an area of further work (and in some ways is already, taking into account the work of the 2 validation ESSnets on methodology).
• The idea of a Centre of Excellence to support MS in implementing the architecture was proposed.

(Session 3) Tools and services:

• There are big differences in the current IT infrastructure between Member States.
• Mixed approach between national tools and ESS services might be better.
• MS may not fully implement VTL.
• The ability to build the rules is very important.
• Implementation it a challenge.
• Some countries may have a concern that their data would leave NSI for validation before official submission.
• Data sharing between countries is to be seen in a different context where countries share data between them on the basis of bilateral agreements.

(Session 4) VTL (Validation and Transformation Language).

• In the majority of countries, there is no common approach, no common validation language and no common set of validation rules across statistical areas,
• In most cases validation rules are written in the language of the production system or database (SAS, R, …).
• It would be good to develop translators of VTL towards the languages the most commonly used in NSI (in particular SAS).
• It is very hard to motivate colleagues to start using VTL and to find relevant information or training material on the Web.
• Tools and services should be integrated in order that validation could be performed all along the data production process without any need to exit the process.

Opinions and Actions

• VTL Translators will be needed (towards SAS, R, …).
• The integration of validation all over the production process is important.
• Four regional workshops to discuss the implementation of the ESS validation policy will be organised by the ESSnet Validat Integration between the end of this year and March next year.
• Information on validation in the ESS (VTL, tools and services, …) should be centralised and more easily accessible.
• Domain specific working groups should progress in agreeing on validation rules on data to be sent to Eurostat.

Item 2 : Statistical confidentiality

Presentation

In this session 2 items were presented:

• 2.1 Report on statistical confidentiality

Eurostat reported on the projects and activities in the field of statistical confidentiality, in particular: on the results of the Centre of excellence on statistical disclosure control, on the development of public use files for EU-SILC and EU-LFS and on the progress in release of microdata for scientific purposes. A part of the presentation was devoted to the on-going work on the project "Harmonised protection of census data".

• 2.2 Waivers implementation in business statistics

Eurostat presented NSIs experiences and some recommendations for waivers' implementation. Waiver is an agreement of the statistical unit to publish the data reported.

Discussion

Item 2.1 Report: As far as the microdata access is concerned, the discussion was focused on the scopes and objectives of the Microdata Access Network Group to be established and transmission of scientific use files to researchers via EDAMIS. The other topics discussed were: application of European Business Statistics Manual in agriculture and disclaimer for ESS public use files.

Item 2.2: Waivers: The participants discussed the validity of the waivers given via tick-box on the questionnaire. In one country such tick-boxes are not considered valid by the legal advisers. The incentives for the statistical unit to give waivers were also discussed. These could be: un-blocking the cell value for the whole sector and consistency with other publications channels where the same data are available to the general public.

Opinions and Actions

Item 2.1 Report: The Group took note of the recent developments pertaining to access to ESS microdata and in particular the pilot on secure file transfer protocol transmission of LFS 2015 data as well as dissemination of public use files following the aligned methodology. Following some questions Eurostat will provide more details about the scope and objectives of the Group.

The group also noted the on-going work by the Centre of Excellence on Statistical Disclosure Control on the harmonised protection of census data and the on-going consultation on the European Business Statistics Manual chapter on confidentiality.

Members were invited to express by 27 April 2017 their NSIs’ interest to participate to the newly established micro-data access network group, which will have a first meeting on 20 June 2017 in Luxembourg (e-mail to ).

Item 2.2 Waivers: Following a brief exchange on this item, the WG undertook to comment by 14 May 2017 on the recommendations proposed for waivers in view of presenting a revised version of the document to the upcoming meeting of the BSDG in June 2017.

Item 3: Statistical methods

Presentation

In this session 3 items were presented:

• 3.1 Estimation Methods for Administrative data (by IT and NL)
• 3.2 Selectivity in Big Data Sources (by Eurostat). The presentation focused on the methods that could be used to reduce self-selection bias, namely reweighting, modelling and its combination.
• 3.3 Temporal Disaggregation (by Eurostat). The presentation focused on the draft ESS guidelines on temporal disaggregation, benchmarking and balancing. The aim is to help the ESS in the daily production of high frequency data from annual to quarterly to monthly figures

Opinions and Actions

For the item 3.1, Eurostat undertook to share the draft deliverable with the WG on Methodology. Comments were invited by 27 April 2017. The WG noted the newly established methodological helpdesk in the framework of the ESS Vision 2020 ADMIN project. Some Member States pointed out the risk regarding deviation from stat units (enterprise) when using administrative data.

For the item 3.2, in addition to providing the list of literature on CROS , Eurostat would publish a working paper with the main results on options for treating selectivity in Big Data.

For the item 3.3, the WG undertook to provide comments on the draft guidelines in writing by 27 April 2017 to . A virtual Task Force was established, to which AT, UK, NL, DK, IT volunteered to participate. Further participants may signal their interest by 16 April 2017. A first meeting would be organised on the 30 May 2017 and a second one in September with a view to finalise the work by the end of this year.

Item 4 : Research and Innovation

Presentation

In this session 2 items were presented:

• 4.1 Linked open data for the facilitation of third-party data analytics; DIGICOM Work Package 3.1 (by Eurostat). The presentation aimed at seeking feedback on the planned “linked open data” work package of DIGICOM.
• 4.2 NTTS 2017 Conference Debriefing

Opinions and Actions

For the item 4.1, the WG took note of the recent developments, including the call for proposal for an ESSnet in that area published at the day of the meeting.

1. Conclusions/recommendations/opinions

The WG welcomed the recent Eurostat work in the field of Methodology and Research and approved the future plans of Eurostat. All planned topics were addressed during the meeting.

1. Next steps

Eurostat will implement the decisions taken, notably in terms of data validation, microdata access and temporal disaggregation. For this latter item a devoted task force will be created.

1. Next meeting

The next Working Group meeting is tentatively scheduled to be held in April 2018.

1. List of participants (please see the annex)

Annex: List of participants of the Working Group on Methodology

EU national delegates

EFTA countries

Candidate and potential candidate countries

European Commission

1

[1] Published in the Register of Commission Expert Groups and Other Similar Entities, code number E02296