Joint

Funding Bodies Review of Research Assessment

Reponse from the SE London National Health Service Research Health Authority

I write on behalf of the SE London National Health Service research community in response to your invitation for interested parties to debate the issues and contribute to the scope of the review of research assessment.

First, we welcome the review and the opportunity to make a contribution to the shape of future arrangements for assessing research in universities and would like your reassurance that the views of the NHS will be taken into account.

The NHS and universities work in partnerships for research and teaching in the health sector. The NHS provides substantial financial support to universities through ‘Culyer’, or R&D Support Funding which is paid to NHS trusts but in effect also supports research in universities.

1. Quality of research

Questions of quality in research should include value added to professional practice, applicability, and impact beyond the research community. We would like to add to this a criterion for relevance.

We fully support the necessity for research to meet rigorous scientific standards in design and conduct. We think it is equally important that health research responds to issues in health and health care, recognising that while not all research can or should be immediately applicable it should have obvious relevance, even if it is at the abstract, theoretical stage of development.

Measurement of these additional aspects of quality will require new methods to be employed, and suggestions are made under ‘Approaches to assessment’.

We would also like to emphasise the value of translational research which is often rated poorly because it is not thought to be original. However, many Randomised Controlled Clinical Trials have such strict entry criteria that the recruited set of patients represents only a small subset of those encountered in practice. Thus trials of the broader applicability of the findings of these "exemplary" and "highly rated" studies are often needed before judgements can be made as to the general applicability of the findings in practice.

2. Health services research

Using “internationally significant” as a main criteria in health services research (HSR) inevitably devalues the research which is conducted in applied health services research particularly in those areas which provide “value added to professional practice, and applicability and impact beyond the research community”, such as primary care research. These areas of research are fundamental to the NHS but perceived as less likely to have the same international significance to health organisations in other countries, even though there are international journals reporting HSR and these do have an international readership.

HSR was treated badly in the last RAE due to difficulties in assessing its international significance. This has caused a tendency for universities which had an interest and focus in HSR to redirect their efforts into subjects that were assessed more generously. The result is that the NHS finds it increasingly difficult to commission academic units to respond to problems in health services that would benefit from research.

This gap in our ability to provide evidence for solving health service problems has obvious detrimental consequences for the health service. In the longer term, a lack of commitment to HSR work in the universities will result in ever decreasing capacity to undertake such work as fewer researchers receive training and experience in HSR methods.

3. Approaches to assessment

You have listed approaches to assessment as follows:

  • expert review (including peer review)
  • algorithm based entirely upon quantitative metrics
  • self-assessment
  • historical ratings

We suggest that the approach taken to expert review could be adapted to obtaining measures of relevance. At present experts are asked to comment on scientific work throughout the research commissioning and publication process to assess quality. Users of research (both immediate and ultimate users – see below) could be consulted in a similar way to comment on relevance, value for professional practice, applicability and impact on population health and health care. Perhaps some version of the ‘citizens jury’ could be considered.

The overall assessment approach will need to take account of the problem that bibliometric indices count heavily against qualitative work and the community-based sciences, and balance this with appropriate alternatives.

We would like to support the use of a range of tools to assess research, and emphasise the danger of repeating the exercise with a strong focus on bibliometric measures.

4. Research and teaching

The contribution made by universities to research training should be recognised in the RAE so that academic units with a heavy teaching load are not penalised. Research assessment should acknowledge that research training is an essential role of a research active institution.

5. Steering group representation

The steering group is made up mostly of academic representatives. It also has representatives from industry and one charity, but we are disappointed to see that it does not include an NHS, or even a public sector representative outside universities. We believe the review will miss an important perspective in the debate if it does not provide a seat at the table for the NHS, a significant user and funder of research.

Professor Sir Leszek Boriszeiwicz is an eminent clinical scientist but he will be seen to represent the scientific rather than applied branches of health research and not the NHS.

6. User perspective

In health research, users can be interpreted as those who implement change (providers, practitioners and service managers) and the ultimate beneficiaries of research – service users. The Department of Health has recognised the importance of involving service users and now requires service users to be included in designing research programmes.

We consider the user perspective to be essential to ensure that research is appropriate from the outset and ask how your review will ensure that users’ views are heard in this debate.

Finally, I would like to express our support once again for this review, and indicate our further willingness to be involved in your efforts to consider the health services perspective.