Experiences of researching the new deal programme full-time education and training option in Sheffield
Paper presented at SCUTREA, 32nd Annual Conference, 2-4 July 2002, University of Stirling
Geoff Chivers, Loughborough University, England
Maryam Shafiei, University of Sheffield, England
Introduction
The New Deal Programme is the New Labour Government’s initiative to combat medium- to long-term unemployment of young people aged 18-24 years. We are researching the full-time education and training option within the New Deal Programme, and the early work involved has already been reported (Chivers and Shafiei, 2001).
In that paper we referred briefly to some of the practical problems we have faced in conducting the empirical research for this project. In this paper we wish to focus on these problems in the context of wider experiences of one of us (GC) in conducting research into government-funded vocational training programmes. The paper is also written against a backdrop of increasing concern about the pressures on academic researchers to produce research results that meet the expectations and demands of research funders.
A survey of the lecturers’ union NATFHE (National Association of Teachers in Further and Higher Education) has recently reported that 11.2 per cent of members have had research results interfered with by either their employer or their research sponsor. The survey of more than 800 union members showed that damaging external interference comes in several forms, including pressure to alter results, delay publication, not publish at all, or suppress particular findings. Of those who reported such pressure, 30 per cent said it was applied by the funding source.
Interference can take more subtle forms, and 23.6 per cent of respondents stated that the academic freedom to test received wisdom and put forward controversial ideas was ‘limited, poor or non-existent’ (Baty, 2002). It is true that some of the reasons for this lack of academic freedom may relate to lack of time for research, or pressures of the United Kingdom Research Assessment Exercise (RAE) to research only in areas decreed by higher institutional authorities. However, most of the loss of academic freedom is considered to be due to loss of freedom of speech and publication rights. The sister United Kingdom trade union, the Association of University Teachers, has recently published an article calling for stronger ethical guidance in this area, given that external research funding is becoming an increasingly important part of university research funding, indeed of all university funding (Copeland, 2002).
One outcome of this pressure from funders to come up with the ‘right results’ can be seen in the literature reporting scandals in the publication of research into the efficacy of new medicinal drugs (Boseley, 2002). In the worst cases senior academic medics have become willing to have their name attached as a co-author to papers written for them by ghost-writers paid by drug companies. Scientific research fraud is, of course, not a new phenomenon, and La Follette published a book on ‘Stealing into print; fraud, plagiarism and misconduct in scientific publishing’ in 1992. Nor is misconduct in research work restricted to the scientific community. Bailey, Haselbank and Karcher (2001) report that, in a survey of 250 American accounting researchers, 3.7 per cent admitted to having cheated in their research, and 21 per cent thought that other researchers cheated. This cheating may well result from the pressure to publish (or perish).
However, in this paper we wish to focus on the difficulties which can be faced by researchers who are not subject to pressure from research funders or pressure to get published for the United Kingdom RAE or achievement of tenure. In our case we believe the difficulties arose because a whole variety of stakeholders in the New Deal Programme were determined to ensure that no negative views of the work in which they were engaged would be derived from the empirical data that the researchers were attempting to collect.
At the outset some problems of access to people and data concerned with the New Deal Programme had been anticipated. This was partly because of the Data Protection Act requirement that personal information collected and held for one purpose cannot be revealed for another purpose without the express permission of the person involved. However, given that absolute confidentiality was offered to individuals and organisations involved, the bigger concern arose from the older researcher’s previous experience of conducting research into a United Kingdom government-funded vocational training initiative.
This experience concerned a large research contract held by another university concerned with researching barriers to the development of National Vocational Qualifications in Management (at NVQ levels 3, 4 and 5). The research conducted on a sub-contract by the co-author’s research team showed quite clearly that the heavy investment by the government into the development of higher level NVQs in Management was not achieving anything like the success that was being claimed in terms of the percentage of those attempting work-based NVQs actually gaining awards. Indeed, wherever the research team went the story was pitiful, with only 3-5 per cent achieving their NVQ award within any reasonable time period, and the vast majority reporting that they had dropped out because the work-based scheme was far too complex, time-consuming and unworkable.
However, when these results were reported to the sponsoring university they were regarded as quite unacceptable. When the sub-contracted research team continued their field research and continued to report equally negative findings, their research methodology was attacked (on the basis that it included focus groups as well as individual interviews). Finally, the work ceased in an acrimonious environment and the sub-contracting research team withdrew from the research programme. Their research findings were not included in the overall report to the government funders, their involvement was glossed over and they did not receive a copy of the final research report. The main contracting university staff member with overall responsibility for the project justified his actions on the basis that the government department needed a report that showed that NVQs in Management were a big success. He also stressed that his research team was very dependent on a succession of funded projects from this government department and could not afford to offend.
Difficulties faced in researching the New Deal programme option: full-time education and training
The field researcher, who is an Iranian university lecturer funded by her government to undertake a PhD study in the United Kingdom, needed to interview a variety of stakeholders in the New Deal programme and carry out surveys, by questionnaire, of young people involved with the Full-Time Education and Training Option.
The stakeholders she needed to interview included staff involved in managing the New Deal programme in Sheffield, private and public sector training providers, some of the actual staff involved in classroom training and some employers of New Deal trainees who had taken the relevant New Deal option.
The Employment Service refused from the outset to provide the names of any of these stakeholders. Given that the New Deal Programme is funded from public taxes and is now operating on such a large scale, it may seem surprising that the Employment Service should be so secretive about, for example, the New Deal training providers in Sheffield. The Employment Service in Sheffield also refused to distribute any questionnaires to New Deal trainees. Fortunately, the Sheffield Training and Enterprise Council was more forthcoming and agreed to help the researcher make contact with staff involved in setting up the relevant New Deal option in Sheffield, and training providers delivering full-time courses with New Deal funding.
Despite this breakthrough, serious problems continued. College staff involved with New Deal initially agreed to give interviews, help in setting up interviews with relevant colleagues and distribute questionnaires to New Deal trainees in their college. Some interviews did take place after long delays in setting these up. However, no help was given with distributing questionnaires. One of the staff informally allowed the researcher to go to the ‘job search room’ at the college, in the hope that a few New Deal trainees might be there. Indeed, on each day that the researcher visited there were a few New Deal trainees present, and very slowly and painstakingly a sample group built up. This method of data collection removed any possibility of generating a representative sample rather than just an opportunistic sample.
After a few months even this limited access to New Deal trainees was closed down when the researcher was told she was no longer allowed to visit the ‘job search room’. No explanation for this decision was given. No untoward event had taken place during these visits and no trainee seemed to be in any way disturbed by being asked to complete the questionnaire. Staffing changes at the college led to statements to the researcher that more help would be forthcoming, but this was not the case. Time and again, she was told ‘leave it with me’, but positive replies never arose from repeated phone calls.
While this lack of help produced particularly serious problems, there were many other examples of lack of support or extreme defensiveness, as follows:
a) The researcher made an appointment for an interview with a manager at one New Deal training site, and left an interview schedule for information. The following week, when she returned at the agreed time for the interview, she was told that the interview could not go ahead as the questions were too political and the interview could lead to New Deal staff losing their jobs!
b) The researcher made an appointment with an employer who insisted that the interview should take place at his home (outside Sheffield and difficult for the researcher to get to on public transport). He refused to let the interview be tape recorded and stated that he would prefer to generate written answers to the interview schedule questions and post these to the researcher. He stated that this would be a better way for him to answer the questions clearly. No apology was offered for the wasted journey. No reply was ever received.
c) Another training centre manager agreed to an interview. However, on two occasions the researcher went to his office by appointment and waited for over an hour, only to be told he was unfortunately not available. On the third occasion, after another long wait beyond the agreed time, he finally came out of his office and told the researcher, without any apology, that he could not answer the interview questions.
d) A manager at a New Deal training centre gave the researcher the home phone number of a New Deal trainer because she worked from home and had agreed to this. When the researcher phoned the trainer she stated that she would be happy to be interviewed. However, when the researcher arrived at the agreed time for the interview the trainer grilled her about who had released her telephone number. She was very unforthcoming in the interview and seemed to be disturbed by the appearance of the researcher.
e) Through a personal friend the researcher made contact with a staff member, involved with employees who had taken the New Deal option, at one of the Sheffield hospitals. The researcher phoned the staff member who stated that she would be willing to distribute the questionnaire to some of the young people she supervised who had taken the relevant New Deal option. However, when the researcher arrived with the questionnaires she denied that she had any such employees.
f) Another member of staff in a training centre asked the researcher to bring the questionnaire for employees so that she could distribute it. This was done and, after two weeks, when no completed questionnaires arrived with the researcher, a phone call revealed that she had forgotten to distribute the questionnaires. She promised to do so immediately. Three weeks later the researcher had occasion to return to her office and saw the undistributed questionnaires still on her desk, where the researcher had placed them five weeks before.
These are only some examples of the ongoing difficulties faced by the researcher. Numerous interviewees refused to let the interview be taped, despite the obvious problems that the researcher, working in her second language, would have in understanding and writing down the answers to her questions, given in rapid, colloquial English. Several of those interviewed refused to complete the simple ‘bio-data’ form asking for their age, qualifications and previous experience. This form did not ask for their name, contact details or current employer’s details.
A continual problem was being referred by one person to another person in the organisation, or another organisation, who in turn referred the researcher to a further person, or indeed the original person. In particular, the Employment Service referred the researcher to staff in the college system. When contacted, these staff simply referred the researcher back to Employment Service staff.
Some of the challenges faced by the researcher were cross-culture in nature. She was sometimes shocked by the dreadful premises in which ex-New Deal trainees were working. Interviewees were clearly disturbed by her appearance and the fact that she was not British. One was very disturbed that, due to unavoidable circumstances, the researcher had to bring her young daughter with her to the interview. However, we are of the view that the majority of the difficulties found were not due to the nationality of the researcher, but the fact that she was a loose cannon!