9316
Discussion groups and researcher bias
Elaine Dawson, University of Sheffield
In January 1992, I began some research into the public perception of changes in higher education. In my previous job I had become familiar with the use of discussion groups, or focus groups, and felt that it would be a useful research method to explore these perceptions. In this way I could characterise and illustrate what certain groups felt, but I would not generalise.
An initial reason for adopting the discussion group method was that I could put my own perceptions aside and sit in amongst a group of people who would stimulate each other to explore their perceptions. I expected my role to be minimal and unobtrusive. I was there only to keep control of the group – be aware of group dynamics; stop break-away conversations; control digressions and dominant speakers; ensure the best possible conditions for tape recording.
By adopting the use of theoretical sampling whereby data is jointly collected, coded and analysed, and with the use of thematic and comparative analysis, I was able to control problems of interview and sampling bias. The previous group determined where to go for the next group by either explicitly stating or implicitly inferring. The groups themselves would raise issues which they felt were important and probe each other for further information. Groups were compared, transcripts analysed, themes were developed and the research moved forward smoothly. I kept a comprehensive research diary, documenting my thoughts and reflections on the method as the research progressed. However, I was soon to realise that these issues of interview and sampling bias were only the tip of the iceberg. The whole area of my own personal bias had been overlooked.
Two particular incidents stopped me in my tracks. One concerned a group of people with disabilities, the other a group of manual council workers. In the first incident I had decided to speak to a group of people with disabilities as disabled access had been mentioned previously. Before I began this research I had considered myself to be quite ‘sound’ and politically correct and it came as quite a shock to realise that this was not the case. An excerpt from my research diary illustrates the problem:
Today I had a discussion group with a disabled group. I went into the office and was met by a woman in a wheelchair, two men with cerebral palsy, a man on the telephone and another man reading a newspaper...I am not proud of myself – I thought I was not a prejudiced person to any group in society, but I obviously am...I automatically presumed the man on the telephone was in charge and the person to talk to because his disability wasn’t visible...I remember looking at the two men with cerebral palsy and wondered whether they had just popped in for a cup of tea. I looked at the other man with no visible disability and presumed him to be the minibus driver. What can I say? I’m not happy with these presumptions or with my thoughts when preparing for the meeting, eg. what language am I using for the questions? Will they be able to understand me? The implication obviously being that people with disabilities are less intelligent than able-bodied people.[1]
The other incident, at first, appeared not to be connected with this, but with careful reflection I realised that both situations had arisen from my own unconscious perceptions about those particular groups of people. A quotation from my research diary illustrates this second problem:
Yesterday I had a discussion group with manual workers from the council. It was a bit uncomfortable, there was definitely a ‘them and us’ situation which has not been so obviously present in other groups. I seemed to be viewed as a ‘them’ and I think some of the group regarded me with suspicion. There was a direct attack on me from one of the men who said he could tell I was a student or lecturer or something like that by what I was wearing, a flimsy dress and sandals. I thought I had dressed appropriately for the group in clothes I don’t usually wear – a Marks and Spencer dress and British Home Store sandals. It was a hot day.[2]
It was my preconceptions which prompted me to dress this way – I thought it was an appropriate outfit for the group. Similarly other preparations for the group were carried out under these preconceptions – how was I to conduct myself; what questions were appropriate? The role of the researcher in discussion groups is crucial and if she or he is not comfortable with that role the participants will pick it up and react accordingly, as happened with these manual workers. The groups which ran the most smoothly were with white, middle-class participants, whether female, male or mixed. Obviously other factors are important in their success, but with hindsight I believe it was especially due to my feeling comfortable with the group because of my preconceptions. The members of the group as a consequence appeared to be comfortable with me.
These realisations came only after careful reflection through writing in my research diary and by filling in contact summary sheets after each group[3]. Initially I attempted other explanations for groups not running as smoothly as I had hoped for, as Evans notes:
… people may well be unaware of the fact that their decisions are subject to bias and become adept at constructing rational sounding explanations after the event.[4]
As I gradually became more aware of these personal biases I was able carefully to consider my thoughts and actions before, during and after each discussion group. Unfortunately I found very little help in the Social Sciences literature. Cognitive and Applied Psychology publications covered the role of bias in quantifying judgements and human reasoning, but were too theoretical. American market research publications mentioned moderator bias in research groups and advocated psychotherapy or sitting in on at least two hundred discussion groups as a training programme[5].
Mclaren[6] stated ‘it is important...that the field researcher not use the field as a site for his or her psychotherapy’. This is true for a researcher once in the field, but personal biases can and do affect research and researchers must therefore be able to understand themselves and the influences in their lives which affect what they do, their motives and their thoughts. This point puts me in mind of a woman in a discussion group who said that we should stop looking at others and look at ourselves instead.
Due to time and financial constraints it is often impossible for a researcher to consider sitting in on two hundred focus groups as training for their role as moderator. Psychotherapy is not as popular here as in the States, and again time and financial constraints mean it is not an option open to many researchers. I found self-awareness increased through careful reflection and personal honesty. As happened with me, researchers might not like what they find, but revelations should be viewed in a positive light and seen as part of the personal learning process, important for self development. As Cottle noted when researching families living in inner city areas:
Encounters, implying as they do mutual acknowledgment of persons, must affect our sense of identity, for they cause us to become re-acquainted with ourselves.[7]
As the research progressed I became aware of another potential problem which could, again, be reflected in my behaviour before, during and after discussion groups. It concerned a need for consistency of results, as Kennedy notes:
Unconscious pressures to bias a group discussion are most powerful during the later stages of a study, particularly when the initial sessions have been highly consistent in their implications. When a group threatens to act in a way which is at variance with research-based expectations, and promises to present us with unexpected and unwelcome analytical and interpretive headaches, our unconscious impulse is to nip this insurrection in the bud – or as a last resort, to tune out, or reject such a group as ‘odd-ball’.[8]
I was aware that these impulses could manifest themselves in various ways, and again, awareness and recognition were the first steps to control. I watched out for probing favourable comments and ignoring those which did not follow existing patterns. Tapes were transcribed fully, including my own comments so that I could watch out for leading questions and treat responses with caution. I studied body-language and neuro linguistic programming to help me become more aware of my reactions and gestures, along with those of the participants. During analysis I tried to treat all transcripts and comments equally, attempting to be faithful to those who had taken part in the research and not misrepresent what they had said. I recognised that the work was highly subjective – what I attached importance to another researcher might not have. I attempted to be honest with the reader and describe the subjective nature of the work, pointing out areas which could have been affected by personal biases.
In conducting research on women and their working lives, Griffin was accused of carrying out biased work because it only concentrated on women, yet many famous studies have only used male participants and not been labelled biased. This led Griffin to note:
Those approaches that are labelled as unbiased usually are associated with the forces that maintain existing sets of power relations in society.[9]
I believe we all need to reflect on our research practices and address these issues of personal bias in a constructive way so that researchers like Griffin do not have to face criticism alone. I have related these issues to discussion groups because that is where my experience lies, but there are implications for all qualitative work. Literature on these issues is scarce. I therefore welcome the opportunity to reflect on personal research experiences with interested colleagues in a workshop. The following is a list of questions which colleagues may wish to consider. Please ignore the biased nature of the questions – the list has been developed from my own personal interests and observations.
1. Have other researchers recognised personal biases influencing their research? If so, to what extent? How did they recognise them? What steps did they take to control them? What advice can colleagues give on the recognition and control of personal bias?
2. To what extent do personal biases affect the research process? Are these issues purely concerns of qualitative researchers or are there implications for quantitative work? Are some research methods affected more severely than others by personal bias?
3. Do certain types of background and life experiences have more influence on the research process than others? Are these personal biases more prominent when meeting with groups with whom we have had little or no contact in our lives? If this is so, given the personal profiles and backgrounds of most academic researchers, what are the implications for future research?
[1] Personal Research Diary, 1992–1993, p12
[2] op.cit; p15
[3] Contact Summary Sheets are discussed in Matthew B Miles