Title Exactly As Indicated in the Title Field
Modernisation and industrialisation of the online data collection
Keywords: business survey, questionnaires, metadata, online data collection
1. Introduction
To ensure that responding to surveys is as user-friendly as possible for companies, INSEE has rolled out the possibility of responding online via a single web: the Coltrane Project (acronym for COLlecte TRANsversale d’Enquêtes» in French, Cross-Disciplinary Survey Data Collection).
Up to now, INSEE provides several ways of responding to the enterprises which are surveyed: paper of course but also online response. Online data collection is varied: electronic response form, file drop service, mail sent for paper forms. Each of those services satisfies globally its users but this diversity and heterogeneousness involve some complexity for the NSI and for enterprises.
Indeed the NSI undergoes a quite high maintenance cost (because of several technical platforms et several questionnaires development teams, and because of some overtaken technologies). What for the enterprises, they have to manage several «accounts» (login/password) on different platforms to fill in our questionnaires.
Furthermore, the cover of the collection by Internet remained insufficient: only a part of the surveys can be collected by Internet, whereas in april 2011, the government requested that all statistical surveys should be put on the web. Existing tools couldn't allow it (specially for the SBS survey).
So a few years ago began the Coltrane Project whose main challenges were:
· to extend, modernise, industrialize and standardise online data collection for all surveys.
· to set up a metadata-driven approach. It reduces development costs, permits to have the documentation of the survey always in sync with the questionnaire in production. And beyond building collection instrument, this principle could be implemented on other sub-processes in the GSBPM.
· to provide a single access point for businesses to make responding easier.
· to enable businesses to better identify surveys that are indeed within the official statistics remit.
All with some technical and functional consistencies.
2. Methods
Coltrane is a platform offering a range of technical & business services, composed of independent work packages. Each of them is composed of several «services» which allows a real urbanization of our internet collect system and a more efficient creation of questionnaire, through a “generation of questionnaires”.
2.1. On one side, the project “Contact”
This subject is quite independent from the online collection itself.
It is based on the use of a LDAP directory to authenticate all the contacts: any «contact» in an enterprise will get a unique login and password. Why? Because on the side of the enterprises, there is no “difference” whether the survey is carried out by one service of the NSI or by an other (or by a ministerial statistical office) as they receive questionnaires from both).
After authentication, the contact can/has to change and personalize its password from its first connection. In case of loss of the password, the contact can ask on-line for its reset.
An internal application is also dedicated to this contact management. It allows Insee staff to create or «delete» a contact and its respective characteristics, to display the list of contacts of a given enterprise, or to display the list of the surveys of a given contact.
Within a few months we will be able to communicate with all the contacts concerned by one survey: we will have some technical services for the sending of mails or letters in a secure frame of communication.
2.2. On the other side, the project “Collect”
The aim of this project is to offer to the National Statistical Authorities (NSI and ministerial statistical Offices) both tools and an effective modus operandi to put surveys on the web. We would like to add more than 30 surveys within 2 years so there is a real need to be able to industrialize the process of integration of surveys on the Coltrane plateform.
To this end, several applications, called “project Collect”.
First of all, the so-called application “Questionnaires” which includes the automatic generation of questionnaires and the technical platform which accommodates them. It is maybe the main part of the project. The idea is that if several collection modes (paper and online for instance) exist, the questionnaires should be generated from the same metadata flow, which requires modelling.
Regarding online data collection, we use a quite innovative process : forms are generated from a formal description, i.e. from metadata describing the questionnaire. The questionnaire are modelized with a DDI model ; then the form is generated from its DDI description through the use of a questionnaire generator engine, called ENO, which generates a new format: web forms (called Xforms, an intermediate technical model).
The choice of DDI as a modelling format for questionnaire specification was motivated by:
· A strong need for a precise modelisation of the questionnaire in order to be able to render its content and logical flow in different modes (i.e a semantically rich model);
· DDI being an open standard;
· DDI being serialised in XML, and thus easily processable.
This type of questionnaire would also allow us to incorporate dynamic controls to check the format of the input fields, consistency between the different input fields and, where necessary, with earlier data.
A portal, called “My surveys”, has been developed and offers customized services to the enterprises, which means that, contrary to what was done yesterday, a unique Internet portal presents all the surveys, including the MSO surveys (as far as they have accepted to use our portal).
The portal is customized for each contact so that, once authenticated, he can see the surveys he is interested in, he can modify his own personal parameters (phone number, e – mail, address, etc.) and look at the remaining surveys he still has to answer as well as their deadlines.
The portal has also an on-line assistance: this assistance is contextual and brought in mainly two actors: some are dedicated to technical problems, whereas others are in charge of questions on the survey. An online form can be filled in to request assistance from the competent department.
In addition to this portal, an internal application, dedicated to the NSA, allows the survey managers to run the collection with management tools (opening/closing a campaign, organize reminders with mails or letters).
3. Results
The first version of COLTRANE was introduced in 2013 for the annual structural business survey and the European Innovation survey.
Since 2015, the online questionnaires accessible from COLTRANE are generated based on their formal description, i.e. from metadata describing the questionnaire.
After the landing of the last experimental survey in the beginning of 2017, almost all of the features of Coltrane, necessary for the integration of all the usiness surveys, will be available (The next development will be related to the generation of different types of letter announcing a survey, reminders and paper questionnaires).
4. Conclusions
Of course, the project Coltrane was quite long but it is far from the end in terms of benefits.
Benefits for respondents who can access a unique portal and see all the questionnaires that they must reply to, using a single login account. Furthermore all the questionnaires will be presented with the same ergonomy, which can be a good thing for the enterprise.
Benefits for the NSA: COLTRANE lightens the burden of work for the survey departments. It automatically generates all data collection instruments and ensures that they comply with standards. It is connected to the metadata database that allows :
· to be up to date with the metadata
· to reuse previous metadata for a new vesrion of a questionnaire.
All that remains now is to incorporate all the INSEE surveys, and then eventually all the surveys of the official statistical system that wish to use this procedure.
References
Thomas Dubois, Generating business survey questionnaires using active metadata, Presented at the Fifth International Conference on Establishment Surveys, June 2016
Thomas Dubois, Pogues & Eno generator: Specify and view a web questionnaire generated from its formal description Presented at the Fifth International Conference on Establishment Surveys, June 2016
Sigaud Éric, Romain Tailhurat, Franck Cotton and Éric van der Vlist. “XForms generation: a real world example.” Presented at International Symposium on Native XML User Interfaces, Montréal, Canada, August 5, 2013. In Proceedings of the International Symposium on Native XML User Interfaces. Balisage Series on Markup Technologies, vol. 11 (2013).
Beguin Jean-Marc, Redesigning French Internet Business data collection at Insee: the Coltrane project, Presented at the 4th International Workshop on Internet survey methods, Daejeon, September 2012
4