COMPETENCIES REQUIRED FOR ADAPTATION OF SURVEY INSTRUMENTS AS PREMISE FOR METHODOLOGICAL QUALITY
Different authors consider that competencies of survey instruments’ translation are an important premise for good measurement properties of translated questionnaire. Incorrectly translated questionnaires can result in the whole research project failure due to the lack of methodological quality. Nevertheless, only few attempts have been made to produce some empirical validations.
The author of this article formulated “a model of 4 competencies”, while looking for an answer to a question as to what determined good results of the questionnaires’ translation and cross-cultural adaptation. The suggested translation organization technique (based on “the model of 4 competences”) stipulates that competencies of a researcher-translator should converge towards the “ideal” profile of two cultural and two academic-professional competencies. That is – the linguistic competence, the culturalcompetence, the methodological competence in survey, the competence in a measured diagnostic construct. The duality of the competencies is essential – practical and theoretical knowledge of a subject.
The effectiveness of suggested translation organization technique was tested with empirical social research. It was supported with biographical survey and interview with the scholarly group who created, applied and tested the above mentioned technique. Linguistic adaptation of questionnaires and the quality of the cross-cultural transfer was tested with psychometric statistics and methodological re-analysis of five different surveys. The methodological quality of the adapted questionnaires was confirmed from analysis of 665 primary indicators and 76 scales (when N=5516). That allowed empirically recognizethat the suggested methods as effective.
Keywords: translation quality of survey instruments, adaptation of survey instruments, competencies of translator, four competencies model
Cross-cultural comparative researches presuppose the task of questionnaires’ linguistic adaptation (translation). The difficulty of this task was recognised in 1960-1970 by psychologists who commenced cross-cultural research (Berry, 1969; Brislin, 1986, 1970; Brislin et al, 1973; Elder, 1976; Triandis, 1980, etc.). In the past, the methodological reflection of questionnaires and tests had been developing within the boundaries of one discipline – cross-cultural psychology. At the present time, primarily because of the international global scale researches, the discourse on questionnaire translation becomes not only more “sociological”, but steps over the boundaries of the social science disciplines, thus becoming an object of interest for various groups of social researchers (sociology, political and educational sciences, management etc.).
Translation of questionnaires and tests and the reflection of this procedure within a particular theory and methodology of surveys has become an independent thematic direction of surveys. Different guidelines of researches have a half century history and during this period various linguistic and organisational techniques of questionnaire adaptation and translation have been created: simply direct translation,modified directtranslation, back translation, parallel blindtechnique, committeetranslation, etc.A paradoxical tendency was discovered after the analysis of practices of questionnaires’ theoretical and practical adaptation. The vast majority of published and recognised techniquesare not about translation techniques as a mental linguistic procedure, but rather about translation organisation techniques. Therefore a greater attention of researchers to the translation might launch a heuristic movement within a particular direction of the social research methodology. It is important to pay more attention to the translator and his/her competencies to translate a specific stuff – questionnaire.
The aim: in this article the focus is given to a set of competencies that are necessary for a translator/researcher in order to maintain a high quality of questionnaires cross-cultural transfer.
The nature of questionnaire and its translation
The survey instrument has the dual nature: it is aninstrument of measurement and at the same time it is a text determines what is said to or read by respondent. One of the goalsof questionnaire design is to optimise communication of intended stimulus and response (Harkness,Shoua-Glusber, 1998, p. 96). That is why questionnaires translation is so distinctive and differs from the translation of poetry, literature or other texts. Translation of literature and poetry gives more freedom to a translator than translating research instrument. This is because that in the first case an artistic metaphor is cross-culturally translated, whereas in the second case, a measuring instrument of psychical structures (opinions, stereotypes, preferences) is constructed.The situation becomes more complicated as the measuring instrument is not only created but it is reconstructed and reproduced, transferring it from one social cultural context to another. Namely the original status of the questionnaire (it is the measuring instrument of consciousness structures and statuses and it is based on the cultural attribute –language) makes its translation distinctive from the translation of literature or technical text.
The translation of questionnaire becomes even more complicated in such circumstances than questionnaire was not created for the cross-cultural research and has emic items.Due to linguistic and cultural differences a translated questionnaire may not automatically follow the same factor structure, and translated items may contain additional emic meaning(limited to a single culture), and be misinterpreted by respondents (Chapman, Carter, 1979; Gjersing et al, 2010; Mažeikienė, 2001; Merkys, 1994, 1997; 1999; Reichenheim, Moraes, 2007; etc.). Therefore the transfer of a survey instrument from one sociocultural environment to another is a very complex task.
Certainly, an adaptation of the survey instruments also faced with other risk of measurement errors that arise due to various reasons (named as Total Survey Error):problems with questionnaire design, sample, the quality of fieldwork, differences and incompatibility of culture, etc. (Groves et al, 2004; Groves, Lyberg, 2010; Lipps, Dorer, 2011; Oberski, 2011; Weisberg, 2005, etc.). Nevertheless the questionnaire translation phase is the one of the most important to have an appropriate (good) measurement instrumentand the role of translator is highly important.
The role of translator
In social research methodological literature from both theoretical and empirical level, often distinguished that one of the key factors to ensure the quality of the translation of the questionnaire - the translators (or team of them) expertise and competence (Behling and Law, 2000; Hambleton et al., 2005; Harkness et al., 2003, 2010; Weeks et al., 2007). The qualification of translators and their role in survey instruments translation has become a sensitive issue in intercultural survey research due to relevancies of quality of survey translation to data quality. It is thought that the translator‘s professionalism is a crucial factor in the quality of translation, and has a higher impact than the choice of translation strategy or methods (Temple, 1997; Weeks at al., 2007).
It should be noticed that the competence is an absolute good, which is never too much (as well as safety, health or technological progress) thus the importance and significance of translator competence should not be discussed at all. However it is necessary to define what should be the competence profile of questionnaire’s translator, what should be the structure of certain competencies and the coherence of them?
Systematic and comparative analysis of scientific literature revealed that most scientists focus on components of translation competencies, i.e. the components that a translator suitable for translation of survey instruments is supposed to have. Different models focusing on various components of translation competence are created (PACTE, 2003, 2011) and quite many recommendations about who and how should translate are formulated (Hambleton et al., 2005; Harkness et al., 2010; International Test Commission, 2000; Survey Research Center, 2010; U.S.Census Bureau, 2005).
The literature review (Geisinger, 1994; Hambleton et al., 2005; Hambleton, Patsula, 1999; Harkness, Shoua-Glusberg, 1998; Hilton, Skrutkowski, 2002: cited Weeks et al, 2007, etc.) revealed that the main competencies which translators should have in order to be able to translate survey instruments accurately are (also see table 1):
- being fluent in both languages (Harkness,Shoua-Glusberg (1998) advocates to the necessity for being bilingual);
- having cultural awareness;
- understanding empirical social science research and the nature of questionnaire design;
- being expert in both the characteristics and the content measurement on the instrument;
Requirements for translators of survey instruments
Authors / Requirements for translatorsBehling and Law (2000, p.54) / “must settle for a less-than-perfect mix of fluency, cultural awareness, and knowledge of the instrument and its uses.”
Geisinger (1994, p.306) / “must be fluent in both languages, extremely knowledgeable about both cultures, and expert in both the characteristics and the content measurement on the instrument and the uses to which the […] instrument will be put.”
Guthery and Lowe (1992) cited by Behling and Law (2000, p. 54) / suggest that a native speaker of the target language, other things being equal, is a better translator than someone who learned it as a second language.
Hambleton and Patsula, (1999, p.8) / “Seek out translators with language proficiency, knowledge of the relevant cultures, and some subject matter knowledge/knowledge of the construct of interest”.
Hambleton et al. (2005, p. 10). / “Should know the cultures very well, especially the target culture. […] subject matter knowledge in the adaptation of achievement test is highly desirable”.
Harkness (2003, p.44) / “should be ‘bilinguals’, ‘professional translators’, people who understand empirical social science research, or combination of these.”
Source: Pauliukaite et al, 2011
However analysis of many publications revealedfour points:
First of all, most scientists focus on competencies applied during the phase of questionnaire translation instead of analyzing the competencies applied during the entire process of questionnaire adaptation.Only the Team Approach emphasizes the competencies applied during the whole target questionnaire development process[1].
The second, commonly accepted requirements for translators of survey instruments do not exist and there are still debates who should translate the questionnaire: the professional translator or the researcher himself. Mostly is a view that the translation of questionnaires should be entrusted to professional translators, mainly for practical reasons. However another position says that the ideal circumstances would be if the researcher translates and adapts the questionnaire himself. Such a practise exists, particularly in ad hoc research. Few examples of such practice are analysed in this paper as well. It shows that it is not necessary to be a professional translator in order to translate and adapt the measurement instrument and maybe other characteristics are also important.
The third, most of all recommendations for questionnaire translators are simply lists of characteristics that define the translator, and usually do not suggest how these components are related to each other[2]. Only a few authors provided with some light on this issue(Behling and Law, 2000; Harkness et al., 2003, 2010; Harkness and Schoua-Glusberg, 1998).Most often is considered translating skills to be more important than the experience of survey instruments’ translation, taking in consideration that translators could be provided with guidelines and examples. Additionally, knowledge of the instrument and its use should be the least important criterion for selecting translators in most cases (Hambleton and Patsula, 1999; Hambleton et al., 2005).
The fourth, the need for empirical validation of two abovementioned issues are observable. That is, there is a lack of empirical research for an answer to a question what should be the competence profile of questionnaire’s translator, what should be the structure of certain competencies and the coherence of them in order to determine good results of the questionnaires’ translation and cross-cultural adaptation?
The empirical research
The author of this article did an empirically research while looking for the answers to these questions.Empirical base is comprised of five cases of cross-culturally adapted questionnaires based on 665 primary indicators, and secondary analysis of 76 scales. The general sample is 5516 respondents. The cases of cross-cultural adaptation of questionnaires analysed in this paper are as follows:“The questionnaire of Fair of school”, “The questionnaire of Intergenerational tension”, “The questionnaire of Quality of studies, QUISS II”, “The questionnaire of Organisational climate”, “Team work questionnaire Team Puls”(see Table 2).
Table 2Empirical basis of the research
Questionnaire / Creators of the question-naire / Researchers who adapted the questionnaire for the Lithuanian sample / The original language – target language of the question-naire / The scope of adaptation of the questionnaire in Lithuania
Num-ber of scales / Num-ber of indi-cators / N respondents
Fear of School Questionnaire / Rost and Schremer (1997) / Bagdonas (2007) / German – Lithuanian / 9 / 205 / 1363
The Intergenerational Tension Questionnaire / Teh
(2001, 2002) / Kalinauskaitė (2007) / English –
Lithuanian / 7 / 32 / 549
Quality in Studies and in Studies Social Conditions QUISS II Questionnaire / Bargel
(2000 - 2002) / Turčinskaitė (2009) / German, French, English –
Lithuanian / 34 / 299 / 964
Organisational Climate Questionnaire / Merkys, et al. (2005) / Pauliukaitė, Palkimaitė* (2007)
*In the Latvian sample / Lithuanian –
Russian / 20 / 63 / 52
Teamwork Team Puls Questionnaire / Wiedemann, Watzdorf , Richter (2000) / Dromantas (2008) / German – Lithuanian / 6 / 66 / 2588
Total: / 76 / 665 / 5516
These five questionnaires were chosen not accidentally. All of these had a high level of reliability and validityafter the adaptation to Lithuanian population (the results of reliability and validity of instruments were presented in the dissertations wrote by researchers who adapted these questionnaires for the Lithuanian sample).It encouraged looking for an answer to the question as to why the indicators of psychometric statistics had revealed the adapted questionnaires to be of high quality.As the literature review shows the importance of the role of translator, a primary hypothesis was raised that the quality of the adopted questionnaires was conditioned by the peculiarities of the translations: translations were entrusted to multiculturally and methodologically competent people (in some cases – to a single person).
To unfold the competencies used in the development process of these translated questionnaires,the elements of biographical method were used. It is important to notice that the biographical method was applied not in the exactly traditional way, that is, analysis did not encompass the informants’ entire biographies, but was purposefully targeted to the analysis of those life periods which provided a possibility to reconstruct if the biography let to arrive at and identify the presumptions which provided a possibility for the formation of the competences and experience profiles. Respondents’curriculum vitae (CV) analysis andqualitative content analysis of the data from semi-structured interview were applied. All researchers-translators and a leader of this scholarly groupwere interviewed on February and March 2011. Interviewees were asked to narrate the process of the questionnaire translation, to identify the main competencies necessary for accurate translation, to narrate (by telling biographical facts) how these competencies were acquired and to share the insights about the translation itself (i.e. what translation is to them,what a good survey translator should know, what is the most important during the translation process, what the main problems are, etc.). Interviews took two to three hours and have been fully transcribed and coded for an exhaustive qualitative data analysis.
Semi-structured interview analysis
While performing the qualitative content analysis of the responses, interviewees’ answers were grouped into four categories that reflect the main competencies used in translation, review and adjudication phases of translated questionnaire development process.
The first category “Language competence” highlights the importance of having language proficiency, concretizes the components of this competence and reflects in what way this competence was acquired by interviewees.
According to interviewees, language competence is constituted of such elements:
- excellent knowledge of the language (grammas, vocabulary, etc.);
- knowledge of live language (concurrent with the deep awareness of culture);
- intuition of the language, its feeling (which interviewed expert equates to “high verbal IQ”).
Analysis of interviews revealed that knowledge of live language is more important than having formal language competence[3] and that to feel the language is essential condition in order to translate questionnaires well[4].
Interview analysis gave details on how the degree of language competence, which is necessary for questionnaires translation, could be measured. According to the one interviewee it can be seen in such details as excellent knowledge of proverbs, saws, metaphors, etc.[5] According to the results of analysis, interviewees have high level of this competence in view of the facts that respondents started learning appropriate languages at school (e.g. one researcher-translator started his Germany studies from the first school grade with advanced German language level), and polished it up till now throughout their studies at universities or summer schools (e.g. translator took intensive academic French course in France, in Poitiers city) as well as throughout the livingabroad (one spent two years in Germany, other lived six years in France, etc.).
Live language is highly related to cognition of the culture. Therefore “Awareness of the culture” was the second category extracted. According to interviewees, deep sense of culture is essential condition for translating questionnaires and it is acquired by living in that culture at least six months.
Interviewees indicated that the quality of living abroad as well as the period spent there has a big impact on the acquisition of this competence. The importance to see culture in various facets was highlighted too. It was extracted that the main ways to saturate with the culture are those:
- being active and having informal social network[6];
- losing the contacts with home and living long time enough there[7].
Analysis of interview corroborates the importance of this competence to the translation of analysed questionnaires. It helped to “catch” all of subtleties and nuances for appropriate translation of the items[8].
The third competence identified by the qualitative content analysis of the responses was knowledge on measuring topic. This extracted category “Knowledge of the construct” splits into following subcategories. First of all, the interview analysis indicated the importance to know the construct from two sides: theoretical and practical[9]. According to the interviewees, theoretical and practical familiarity with research subject gives the ability for the researcher to formulate translated items in the most understandable way for the respondents[10]. In addition, according to interviewees, practical familiarity with the construct is more substantial than theoretical knowledge of it[11].
Interview analysis revealed that keeping in touch with the reality researched and taking consultation with experts help the accuracy of translations[12].