Translating and Testing a Victimisation Survey Module

Grant Contract No 11002.2008.003-2008.212

TRANSLATING AND TESTING A VICTIMISATION SURVEY MODULE

Final technical implementation report

Aldona Ablingienė

Project Manager

Social Protection and Health Statistics Division

Statistics Lithuania

29 Gedimino Ave,

LT-01500 Vilnius – LITHUANIA

Tel. (+370) 5 236 4914,

Fax (+370) 5 236 4666,

E-mail:

October 2009, Vilnius

Introduction

The activities performed during the implementation of the project “Translating and testing a victimisation survey module” arise from the Commission Communication to the European Parliament, the Council and European Economic and Social Committee on developing a comprehensive and coherent EU strategy to measure crime and criminal justice: An EU Action Plan 2006–2010. This plan intends to establish common instruments of survey (module) on victimisation and implement them in all the Member States.

A draft a victimisation survey module was developed by Eurostat and approved by the Eurostat Task Force in June 2007. The draft questionnaire, originally in English, was the document to be translated into national languages and used to conduct pilot tests.

Implementation of the project

Over the period of implementation of the project, all the works foreseen in the project were performed, excluding translation of the Module into Russian. All the documents foreseen in the project were prepared and submitted to Eurostat.

The implementation of the project was carried out by Statistics Lithuania in the Social Protection and Health Statistics Division. The project was implemented in cooperation with sampling specialists and programmer. Interviews were performed by the interviewers of two regional statistical offices, who had passed a special training course.Translation of the Module into Lithuanian language was performed by two native speakers (acting as subcontractors) who were selected according to the national procurement procedure.

The main activities performed during the implementation of the project “Translating and testing a victimisation survey module” in Lithuania were as follows:

Translation of the Module into Lithuanian language.
Field testing of the module.
Preparation of the Quality report

Project started on September 2008 and its duration was 12 months.

1. Translation of the Module into Lithuanian language

At the first stage of the work, two native speakers of the Law Institute with good knowledge of the English language and in-depth knowledge of the field of crime and criminal justice were selected according to the national procurement procedure.

In order to adopt a national version of the Module, some changes in the structure and contents of questions were made. It was done with a view to make the questionnaire clear and consistent; however, it allowed collecting all variables of the original Module.

In order to ensure the correct conception and qualitative translation of the survey instrument, a technical specification for translation with translation procedures protocol were prepared and provided to the expert translator together with the survey instrument.

Translation of the Module into Lithuanian language was performed by the first native speaker. The translation procedure protocol was followed and the translation process was described in a report.

Another step in the Module translation-related work was checking the translation of the Module into Lithuanian language and preparation of a motivated assessment report on the translation quality by filling in a special checking questionnaire. The expert checker, as the expert translator, had to follow the same translation/checking requirements and course.

The version of the EU Victimisation Survey Module in Lithuanian language was harmonized in the meeting, in which the expert translator, the expert checker and representatives of Statistics Lithuanian participated. The Module in Lithuanian language was discussed by analyzing each question and following the order according to which a conclusion concerning the quality of translation of the Module was arranged.

Taking into consideration the comments and proposals of the experts, the specialists and the linguist of Statistics Lithuania, a final version of the Module in Lithuanian language was prepared.

All problems occurred and solutions found during the translation process were described in the translation report and together with the full text of the Victimisation Survey Module in Lithuanian language were transmitted to Eurostat.

2. Field-testing of the Module

The second stage of the implementation of the project was field-testing of the Module. At this stage, cognitive testing of the questionnaire was conducted. For the participation in the testing, 6 specialists, working in different fields of statistics and not participating in the implementation of the current project, were invited. They were provided with paper questionnaires and asked to fill them in parallel with making written remarks on the contents, clearness, sequence of questions, problems in choosing answer categories, etc. All these remarks and problems raised during the cognitive testing were discussed by the project implementation team and some necessary amendments to the questionnaire were performed.

During the period of the preparation for the field-testing a two-day training session was organized at Statistics Lithuania on February 2009. The session was attended by 16 interviewers from Šiauliai and Panevėžys regional statistical offices, as well as two supervisors of interviewers groups. Questionnaire, Interviewers’ guidelines, Report for registration of non-response reason, Interviewer opinion questionnaire on the field-testing and materials of the training sessions were prepared in Lithuanian and distributed to participants. Letters for respondents were printed and delivered to regional statistical offices two weeks before the training session.

Victimisation survey module fieldwork was carried out in March 2009. The survey was as a stand-alone and based on voluntary participation of selected respondents. The testing of the Module was conducted face-to-face using the CAPI method for sections A–F of the questionnaire. The section on violence (G section) of the questionnaire was filled in through self-completion in a paper questionnaire.

The target population of the Victimisation Survey Module was persons aged 15 and over living in private households. The Population Register was used as a sampling frame. The sample size of the field-testing of the Victimisation Survey Modules covered 300 persons (150 from Šiauliai county and 150 from Panevėžys county). Stratified sample design was used.

Blaise software was used for data entry and logical testing, and the process of data cleaning and editing started simultaneously with fieldwork. The processing of the field-testing data of the Module was performed using a SAS program. A technical specification for creating a special program for entering data was prepared by specialist and computer programming of the questionnaire – by the programmer.

A testing report on the approach adopted for testing of the Victimisation Survey Module and Questionnaire NAT-01 (section A–F in Lithuanian), Annex to the Questionnaire NAT-01 (section G in Lithuanian) were transmitted to Eurostat.

3. Preparation of the Quality Report

After the field-testing of the module the questions related to the quality of the implementation of the project “Translating and testing a victimisation survey module”, recommendations and suggestions for further improvement and cost estimate of implementing such a module in Lithuania were reviewed in the quality report.

In the pilot testing organized in March 2009, 244 interviews were conducted; the total response rate made up 81.3%. 198 out of 244 (81.1%) interviews were complete (all questions were answered); however, notwithstanding all efforts taken by the interviewers, 46 (18.9%) interviews were stopped at different stages of testing on request of respondent. 5 (11%) interviews were stopped before the section G “Violence” , the rest – in between the questions on violence.

The average length of interviews (including interviews unfinished on request of the respondent) was 58 min.; the average length of complete interviews (all questions answered) was 1 hour 7 min.

The overall evaluation of the Module has been positive. However, taking into account the results of field-testing, reactions of the interviewers and respondents showed as follows:

In order to make the questionnaire clear and consistent some changes in the structure and contents of questions in a national version of the Module was done; it allows collecting all variables of the original Module.
The questionnaire is very long and complicated; therefore, in order to reduce the burden on respondents and to receive exact answers to questions, it is necessary to formulate questions and answers in the simplest and clearest way possible.
In order to avoid misunderstanding and/or inclusion of different meaning in the questions of the Module, concept cards should be prepared with comprehensive explanations about the context of questions, so that the national version of the Module isformulated in an absolutely clear and conceptually uniform way.
The section “Respondent and household characteristics” of the Module should be fully harmonized with the European module on Core Social Variables. The implementation of common variables is the best solution in order to avoid measurement and interpretation mistakes.
Questions of section C “Screening and crime accidents” should be constructed in a way that would allow the classification of accidents after the interview, using the information on the circumstances reported. In case the current questionnaire is planned to be used, very clear instructions (definitions of particular types of crime) should be obligatorily included in the questionnaire.
Taking into account the results of testing, reactions of the interviewers and respondents, it is recommended to shorten the questions and lists of answer categories in section G “Violence”. The scope and sequence of questions should also be revised. The differences in time periods covered in this section of the questionnaire make it even more difficult. The shorting of questions and lists of answer categories would make the questionnaire more suitable for respondents and encourage them to provide necessary information.
The CAPI interviewing method is the most appropriate one for the Victimisation Survey. Taking into account the fact that for a large segment of the population using a computer for the self-completion of sensitive topics is complicated, respondents should be provided with a paper questionnaire (section G).
A reference period is one of the very important aspects in terms of planning and organizing the survey, as well as interpretation of data. For data collection in the Victimisation Survey, a calendar year reference period could be recommended.

Quality report and results of the field-testing survey were submitted to Eurostat.

Project “Translating and testing a victimisation survey module” was implemented successfully. Taking into account the conclusions and recommendations mentioned above, we would advise against including this survey as a module of a broader European survey. In our opinion, it should be a stand-alone survey. The average cost of implementing such Victimisation Survey Module in Lithuania would be about EUR 22 per interview.