Faculty Reponses to the Postgraduate Research Experience Survey 2013
Humanities and Social Sciences
Introduction
This paper outlines the Doctoral College's response to Queen Mary’s results in the Postgraduate Research Experience Survey (PRES) for 2013 provided by the Higher Education Academy.
An initial analysis of the PRES results, comparing Queen Mary to the sector as a whole and to other Russell Group Universities and broken down by Faculty, was prepared by Planning and provided to the Doctoral College Management Group in September 2013. The results were discussed by the DC Management Group at its September meeting, with more detailed results - collated by School, and including free text comments, circulated by the Deputy Deans to the Directors of Graduate Studies in each School for consideration. Schools were asked to produce an Action Plan detailing their responses to any issues where satisfaction levels fell below the Russell Group average, for return to their Faculty Deputy Dean for Research by the 1st week of November for consideration and summary by the Doctoral College Management Group.
Whilst the Doctoral College values PRES, it recognises that on its own PRES does not always provide the fine grained analysis that is most useful to assessing student experience. As a result, from September 2013 the Doctoral College have circulated a more detailed Exit Survey to all Queen Mary students on completion of their PhD. Together with existing forms of feedback (individual student progress reports, reports from student representatives on School Graduate Studies Committees, and on the Doctoral College Management Group) and PRES, this survey will provide a much fuller picture of the Queen Mary PGR experience.
Humanities and Social Sciences
The inability to analyse PRES results at the level of individual programs makes it difficult for some Schools in HSS (for example, Law/CCLS, or English and Drama) to differentiate between the experience of different cohorts. Whilst response rates were significantly increased on 2011 (33% in 2013 compared to 17% in 2011) and in-line with the sector average, the small number of absolute responses in individual Schools (ranging from a high of 38 (47%) in History to a low of 8 (25%) in the School of Economics and Finance (SEF)) also sometimes makes it difficult to interpret results from PRES and to develop appropriate and proportionate responses to issues of concern that the results might seem to flag up. None-the-less, the Faculty welcomes PRES, which provides useful additional feedback by which the Faculty and individual Schools can assess the experience and satisfaction of their PGR students, and every School in HSS has responded with specific action plans to address any areas of concern raised in the survey.
Taken as a whole, HSS scored well in PRES, with overall satisfaction levels of 79% - in-line with Queen Mary's average. Though this was a little (2%) below the Russell Group average, the aggregate score was pulled down by responses from a small number of Schools (Economics and Finance 62%, Law 68%) with the majority of Schools in HSS recording scores of between 74% (History) and 84% (Geography), and some scoring very highly indeed: students in the School of English and Drama (SED) recorded 92% satisfaction, and Business and Management (SBM) students 91%. Students in HSS reported especially high levels of satisfaction with supervisory arrangements, with the provision of research skills training and with induction, and arrangements for and understanding of requirements for progress and examination. Levels of satisfaction with the provision of professional development training were more uneven across Schools, as too student's experience of their School's and wider College research culture. Students across HSS reported much lower levels of satisfaction with regards library provision and access to specialist resources for their research, and access to adequate work space and computing facilities emerged as a major concern in three Schools.
Below this general picture, levels of satisfaction vary between different aspects of the PhD experience, and between Schools, and it is at this level that Schools have drawn up actions to address any areas of concern raised by their students. Both these results, and the actions put in place by the Doctoral College, Faculty and individual Schools to address these concerns are outlined below.
Supervision
Schools in HSS scored very highly on crucial questions relating to the quality of supervision. When asked whether their supervisors have the skills and subject knowledge to support their research students in 7 of the 8 Schools in HSS students reported levels of satisfaction considerably above the Russell Group average (84%), ranging from 95% of students in Law, to 94% in SLLF, 92% in SBM, 90% in SPIR, 90% in History, to 89% in SED and Geography. Only SEF fell below the Russell Group average, with a score of 75%. HSS Schools, including SEF, also scored highly in relation to appropriate contact with supervisors: SBM 92%, SEF 100%, SED 89%, Geography 89%, and Law 87%, with only History (82%) and SLLF (81%) falling (marginally) below the RG average.
Actions: In response, SEF have reviewed their admissions procedures so as to ensure that there is as close a match as possible between projects and supervisors' interests for the 2014 in-take.
Research Training
Whilst HSS as a whole recorded high levels of satisfaction with research training, with some notable exceptions (SED 81%, and Geography 82%) HSS Schools scored less highly with regard to the extent to which supervisors help their students identify their training and development needs (SBM 66%, SEF 62%, History 68%, SLLF 75%, Law 67%). Student satisfaction in relation to their professional development more generally (as measured, for example, in responses to questions relating to understandings of research integrity) also differed between Schools, with some Schools exceeding the RG average (SPIR 100%,SBM 92%, Law 87%, SED 86%, Geography 84%) but with a number dipping some way below it (SEF 75%, History 65% and SLLF 56%).
Actions: All Schools in HSS provide opportunities for their students to engage in professional development. In addition all students in HSS have access to the professional development courses offered through Queen Mary's Centre for Academic and Professional Development (CAPD). Social Science students in SBM, SEF, SLLF, Geography, Law and SPIR may also access specialist research methodology training provided through the Queen Mary-Goldsmith's ESRC funded Doctoral Training Centre, with other Schools supplementing this provision by providing their students with access to more specialised training offered by external providers (for example, the Institute for Historical Research). In some cases (SED, History, SLLF, Law and SPIR) individual School programmes integrate specialist research methodology skills with subject specific professional development (in conference preparation, networking, publishing etc.). In others (SBM, SEF, Geography), whilst still providing opportunities for professional development (for example, with annual PhD symposia and conferences) these programmes focus more heavily on research methodology training, with students in these Schools working on their professional development mainly through the CAPD.
In some cases (for example, SPIR, Law, SED), it appears that the integrated subject specific programs developed at School level are very well received by students. In other cases (for example, History) these programmes appear to be less successful, whilst students in other Schools which rely more heavily on the CAPD's provision (SBM, Geography) record higher levels of satisfaction with professional training.
It is not entirely clear from these results which form of provision students most welcome, or why some School programmes appear to be better received than others. One issue may be that PRES asks only about some professional development opportunities (for example, research integrity) and not others, and that students are responding to the availability of these quite specific opportunities; another is that some students are reflecting on the training offered on these issues in their Schools and others via the CAPD (PRES does not distinguish). It is also possible that lingering concerns amongst some students over the apparent demands of recording their training via the College's points based training system has led them to record their dissatisfaction with this system rather than with such training itself.
These complexities make interpreting these results, and thus responding effectively, quite difficult. None-the-less a range of responses, at School level, and number of recent developments at Doctoral College and Faculty level should improve our students experience of professional training very significantly.
At the Doctoral College level, all CAPD courses were reviewed (against student feedback) in June 2013 to identify areas of strength and concern, and CAPD and School provision is now augmented with twice yearly cohort days organised by the Doctoral College focusing on key areas of professional development (for example, academic networking, impact). In addition, in September 2013 the Doctoral College introduced a new funding scheme (the Postgraduate Research Initiative Fund) to extend the financial support already available to Queen Mary PGR students through the Postgraduate Research Fund, for those wishing to develop innovative training opportunities of their own. In Spring 2014 the Doctoral College will carry out a review of the College's training points system (with a particular focus on the availability of training opportunities across different RCUK research training domains, and the weighting of training requirements across those domains and each year of study), and will introduce a new, more user friendly interface to the training database - now hosted (for easy review by students and supervisors alike) on the Doctoral College website.
At Faculty level a review of all School and central training provision in HSS conducted in October 2012 identified gaps in provision and areas of overlap between Schools and between Schools and the CAPD. As a result, the CAPD will be offering new training on research ethics from January 2014. In addition, each School in HSS has now opened up its specialist research methodology and professional development training to students from other Schools in HSS (made possible by the development of an on-line training calendar hosted on the Doctoral College website) so that students across the faculty can access the expertise offered by different Schools.
At a School level, Schools have responded to the PRES results in flexible ways appropriate to their more detailed understanding of their students concerns: History will now survey their students twice yearly to ascertain what additional training their students desire beyond that already offered through the School's fortnightly Graduate Training Forum, and the access the School provides to the Institute of Historical Research and University of London's School of Advanced Studies. SBM, SEF and Geography will extend their research skills training offering with subject specific professional development workshops to supplement that available via the CAPD - with new workshops on Getting Published, and Meet the Editors in SBM; Publishing, and Applying for Academic Jobs in Geography; and the appointment of a dedicated placement officer to offer guidance to students on job applications and to liaise with potential employers, and specific job market training (mock job market seminars, mock interviews) in SEF.
In response to concerns over the extent to which supervisors help their students identify their training and development needs, it is clear that in Schools where a review of training needs is built in to a student's annual progress reports (SED, Geography) these needs are being effectively identified. Elsewhere (SBM, SEF, History, SLLF, Law) though supervisors are regularly reminded by their Directors of Graduate Studies that a regular review of training needs is a key component of supervision, awareness of this still needs to improve amongst some supervisors, as too how supervisors can monitor their students training development using the College's skills training points database. This is already being tackled at a Faculty level through HSS's new supervisory refresher training, part of which focuses on exactly these responsibilities and on how to use the points database (with more than 100 HSS staff completing this training in 2012-13) and will also be a focus of the extended training to be provided by the Doctoral College (from December 2013) to all those new to supervision at Queen Mary. In addition, all Schools in HSS will be required to ensure that their annual progress reports - completed by students and supervisors, and reviewed by the Director of Graduate Studies, include a report on training completed and still needed.
Academic Process, Progression and Standards
PRES also monitors students understanding of, and satisfaction with, the ways in which their progress is monitored, the standards required for the award of a PhD, knowledge of examination procedures, and the processes put in place to induct students in to their program. To enable an assessment of levels of satisfaction around these issues across a number of Schools, Schools in HSS were benchmarked against a combined average score across the Russell Group for these four questions. All but one School in HSS (SEF, 52%) either met or exceeded the RG average of 77%: Geography 87%, SED 87%, SPIR 84%, SBM 79%, History 77%, SLLF 76%, Law 76%. Student's understanding of the support system at Queen Mary (captured in a question asking whether they knew who, other than their supervisor, they should turn to if concerned about aspects of their degree programme) was more diverse. Whilst students in a number of Schools had a clear understanding of this
(SEF 84%, SED 83%, Geography 87%) in other School these processes are not yet so well understood (SBM 62%, History 56%, Law 74%, SPIR 72%).
Actions: the generally high level of satisfaction in response to these questions reflects the high quality of information that Schools in HSS make available to their students via their School handbooks and at School inductions, though there remain a few areas of concern. Students in some Schools reported lower levels of understanding of the examination process (SBM 58%, History 72%, SLLF 69%, Law 73%, SPIR 72%), of the standards required for the award of a PhD (SLLF 75%, Law 71%), and who they can turn to, other than their supervisors, if they have concerns about their program (SBM 62%, History 56%, Law 74%, SPIR 72%) and these issues will be addressed at the next revision of the relevant School's handbooks. Understanding of these issues should also improve following the launch of the Doctoral College website in October 2013, which includes information for students and supervisors on the examination process, including bringing all the relevant paperwork for examinations in to one easily accessible place, and a HELP button detailing the range of support on offer to students within and beyond their Schools. SEF scored poorly across all but one of these questions, and the Deputy Dean will review the information provided to students in SEF via the School handbook and induction to ensure it matches the quality of information provided in other Schools.