Chapter 7

Collecting quantitative data

Suggested solutions to questions and exercises

  1. Describe the role of the interviewer in the data collection process.

The role of the interviewer is twofold: he or she must contact people who match the recruitment criteria of the survey, encourage them to take part in an interview and administer the interview. This is a skilled job. It requires a high level of interpersonal skill, a sound understanding of the data collection and the research processes, including responsibilities under data protection legislation and professional codes of practice.

  1. Why are interviewer training and interviewer briefing important in the collection of good quality data?

Interviewers must be familiar and comfortable with the questionnaire and the interviewing process in order to collect good quality data. Interviewer training and interviewer briefing are two ways in which the quality of the process can be enhanced.

Much research has been done on the effect an interviewer has on response rates and on the quality of data collected. There is evidence to show that appearance, age, gender, social grade, ethnic background, religion and attitude or personality have an effect on the interviewing process and on the outcome of the interview. This is not confined to face-to-face interviews. Research shows that respondents in telephone interviews make judgements about an interviewer's characteristics on the basis of his or her voice. To minimise the effects of interviewer variance, interviewers are trained to carry out interviews according to the instructions provided and to do so in a professional, courteous and objective way.

Uniformity or consistency of approach is a key aim in structured and standardised quantitative research – data must be collected in the same way across the sample and any possible bias or error in asking questions or recording responses must kept to a minimum. It is important therefore that each respondent is asked the questions on the questionnaire in exactly the same way. Training and briefing are key to achieving this. Interviewers are trained to read out instructions and ask the questions exactly as they appear on the questionnaire, and in the way that they were briefed to do (a change of emphasis on a word can change the meaning). For some questions, such as those with an 'other' code in the list of pre-coded responses, or where the respondent says 'Don't know' or 'Not sure', the interviewer is trained and briefed to probe in the appropriate way. Where there are open-ended questions, questions that require the respondent to answer in his or her own words, the interviewer is trained to record the answers verbatim. If probing is needed at open questions, the interviewer is briefed to follow the specified probing/prompting procedure set out on the questionnaire or specified in his or her training and in the briefing for the particular study.

  1. How do you decide on the most appropriate method or methods of data collection for a particular research project?

In deciding on the most suitable method or methods of data collection, it is important to consider:

·  the study and its objectives

·  the topic or issues under investigation

·  reaching the right sample

·  achieving the right numbers

·  the time and budget available.

If you have a subject of a very sensitive nature, for example, telephone data collection may be the best option as it offers the respondent a degree of anonymity and distance that a face-to-face interview does not. If you have a sample that is hard to reach in person – a sample of business executives, for example, the telephone or a postal or email survey may be the only way of contacting them. If you need to show respondents stimulus material, for example, an advertisement, or get them to try a product, a face-to-face approach may be the only feasible one. If you need to achieve a particular sample size, you may not choose a postal survey or an email survey unless you are fairly sure that the return or completion rate (which can sometimes be hard to predict) will give you the numbers (and the sample) you need. If you are working to a tight budget, you might consider a postal survey – with no interviewer costs it can be cheaper than a telephone or face-to-face survey. If you are working to a tight deadline, a postal survey may not be appropriate – turnaround times are often relatively long – therefore a telephone or email survey might be considered.

  1. What are the advantages and disadvantages of face-to-face data collection? Give examples of the types of study in which face-to-face data collection is appropriate.

Face-to-face data collection has a number of advantages. The interviewer has the chance to build rapport with the respondent, which can help achieve and maintain co-operation and increase the quality of the data. Response rates can be relatively high in comparison to other methods. Face-to-face methods allow for a relatively high degree of flexibility in the interviewing process – the interview can last up to an hour; stimulus material can be used; complex questions explained and administered; and probing and prompting carried out. In central location or hall tests, the environment of the interview can be controlled.

There are some disadvantages to face-to-face data collection, particularly in relation to in-home interviews. It is relatively expensive and time-consuming. Finding respondents at home (or at work) at a suitable time (and willing to take part) can be difficult. To overcome this, it is often necessary (and almost essential for business-to-business interviews) to make an appointment with the respondent, either by telephone or in person to set a suitable time. Interruptions from other members of the household or workforce or the presence of someone else in the room during the interview can impact on the quality of the data collected. It is difficult to cover remote or rural locations. Cluster sampling methods, which serve to reduce travel time and costs, risk introducing sample bias. Representativeness of the sample can be affected in other ways: interviewers may be reluctant to interview in some (socially deprived) neighbourhoods; they may find it difficult to obtain interviews in higher income areas; potential respondents in any neighbourhood may be unwilling to open the door to a stranger. There is a greater tendency in face-to-face methods for the respondent to give socially desirable responses – showing him – or herself in the best possible light. With quality control procedures at more of a distance than in telephone interviewing (where interviewers’ work can be continuously monitored), there is greater scope for interviewer bias or cheating.

If you need a quota sample of shoppers, the interview is about product preferences and is likely to last no more than about 10 minutes, face-to-face interviews recruited and conducted in the street or in a central location such as a shopping centre may be appropriate. If you are conducting a random sample survey on household spending that lasts up to 35 minutes, a face-to-face, in-home interview may be appropriate. If you have stimulus material to show, for example, in an advertising pre-test, a face-to-face interview may be the only option.

  1. What are the advantages and disadvantages of data collection by telephone? For what types of research is the method particularly suitable?

Telephone interviewing, and especially CATI, has a number of advantages over face-to-face methods. A geographically dispersed sample (including those in remote and rural areas) can be obtained more easily. It may be the only way of reaching some populations – for example, the business community. It is possible to use a pure random sampling approach rather than the cluster sampling approach that is common in face-to-face methods, thus reducing the chance of sampling error, and all at a greatly reduced cost because interviewer travel time is not an issue. A telephone survey may also make it easier to reach a wider spectrum of respondents – for example, in socially deprived areas where interviewers may be reluctant to work and in higher income areas where access to homes may be difficult. It is possible to digitally record answers to open-ended questions in full. Greater quality control is possible (and so cheating is minimised), with interviewers being monitored ‘live’ rather than after the event, via back-checking (and clients and research executives can listen in, enabling them to get a feel for the findings). It is relatively easy to monitor interview length and the time taken for individual questions – this can facilitate a dynamic or rolling pilot study and questions can be altered if necessary. It is also relatively easy to determine the strike rate and refusal rate and so monitor the sample and control quotas. Call-backs are easily managed so that bias towards those more often at home is reduced. Centralised, face-to-face briefings are more common than face-to-face projects and supervisors and executives may be on-hand to answer queries during fieldwork. Multi-country studies can be conducted from a central telephone unit enabling greater control over administration and increased consistency. Telephone interviews offer respondents a greater degree of perceived anonymity than do face-to-face interviews. As a result, it is a useful method for collecting data on sensitive or intimate subjects, and it is useful in reducing social desirability bias. Telephone interviewing is faster than face-to-face – more questions can be asked in a shorter period of time and project turnaround times are faster. It can therefore be more cost-effective than face-to-face interviewing.

There are some disadvantages with this method of data collection. Although fixed line telephone ownership in the UK is almost universal this is not the case in other parts of the world – even among EU Member States. Those who do not have a fixed line telephone are different from those who do – they are more likely to be from lower income households and they tend to be older. This has implications for obtaining a truly random sample of these populations – not everyone has the same chance of being selected and not all of them are in the sampling frame or universe. This problem – of sampling error and bias – is exacerbated if the telephone directory is used as a sampling frame. Not all telephone numbers (and so individuals and households or businesses) are listed. In addition, the problem is further exacerbated by the fact that many people (particularly in younger age groups) have mobile rather than fixed line phones. The incidence of telephone answering machines, call screening, lines being used to access the Internet and more than one line per household add to the problems of access and sampling. The rise in telephone sales means that response rates can make people suspicious of bona fide telephone research.

In a telephone interview some of the benefits of social interaction and the chance to build rapport with a respondent are lost. It can be easier for the respondent to refuse an interview or end it early and harder for the interviewer to encourage the respondent to take part. It is difficult to include stimulus material, although this can be overcome by sending material out to respondents in advance of the interview. Long and complex questions are best avoided.

  1. Describe what is meant by the term ‘computer-aided interviewing’ and its advantages and disadvantages over the traditional pen and paper survey methods.

Computer-aided interviewing means that data are captured and recorded electronically on a PC or laptop computer or handheld tablet or personal digital assistant (PDA). There are several methods of computer-aided interviewing:

·  CAPI – Computer Aided Personal Interviewing

·  CATI – Computer Aided Telephone Interviewing

·  CASI – Computer Aided Self Interviewing

·  CAWI – Computer Aided Wireless Interviewing.

The questionnaire (and in some cases the stimulus material), data entry and data editing and verification procedures are programmed into the computer. To start a new interview the interviewer calls a new questionnaire up on screen and enters responses into the computer using the keyboard or touch screen. The program is designed so that it automatically brings up the next question relevant to that respondent as soon as the response to the previous question is entered. All completed questionnaires are stored in the computer’s memory and can be downloaded via modem or wireless connection to a central computer at the agency’s data processing centre.

There are many advantages in using computer-aided methods:

Fieldwork management

·  No printing of questionnaires required

·  Review of quotas and sample

·  Monitoring of interviewer performance and strike rates

·  Data on interview length

·  Electronic access to all questionnaires

Use of stimulus material

·  High quality images embedded in the questionnaire

·  Multi-media capabilities – can embed and play high quality video and audio material

Data processing

·  Checking and editing done automatically

·  No manual data entry – data downloaded via modem or wireless connection to central processing

·  Interim results easily obtainable

·  Fieldwork to tabulation time greatly reduced

Data quality

·  Smooth flow of interview

·  Digital recording of verbatim responses

·  Automatic routing reduces errors

·  No separate data entry eliminates keying errors

The main disadvantage is the capital investment required to buy and maintain the equipment and the software (although this is becoming less of an issue as devices become smaller and cheaper) and to train and support interviewers and other staff to use it.