Response to the Joint Funding Bodies’ review of research assessment
from the United Kingdom Arts & Design Institutions Association (UKADIA)
Individual UKADIA specialist institutions will each be making initial contributions to the Joint Funding Bodies’ open invitation. The following response is submitted on behalf of UKADIA, and focuses on common issues and themes which have emerged to date in the consultation process, as well as those which are of particular relevance to specialist arts institutions.
Expert Review
UKADIA members regard the use of expert review as one which retains credibility, provided that the community of experts is sufficiently broad to have the confidence and support of the specialist areas from which they are drawn. Such experts should demonstrate the skills of judgement regarding both subject knowledge and ‘submission expertise’ and a clearly defined ‘job description’ would be required. The use of expert review facilitates interpretation, which, provided it is supported by sound subject knowledge and is transparent in its use of evaluative criteria, is necessary in judgements about research.
UKADIA supports the combination of both prospective and retrospective assessment.
Level of Assessment
The question posed in the consultation document concerning whether assessment should be made at the level of individual, groups, department, research institutes or higher education institutes, is of particular relevance to specialist institutions, as is the question about the organising of assessment around subjects or thematic areas.
Clearly a specialist institution has the opportunity to align both subject and research strategy closely to its mission, whereas large Universities accommodate a broad range of subjects, each with specific emphasis on and varying relevance to mission.
A creative arts institution may welcome the opportunity to submit evidence across, say, Art & Design, Media, Communication, Performing Arts and related ‘Histories’ (currently in different units) in a coherent single submission which is mission-aligned. This would avoid divisiveness and possible exclusion of relevant research due to lack of critical mass within in one ‘unit’. There is also a need for some flexibility in the use of a unit of assessment so that research centres and/or thematic entries can be accommodated.
Algorithm
UKADIA would not support an assessment system based entirely upon metrics, which are mechanistic, can be unreliable quality indicators, can be open to manipulation and distort behaviour and disadvantage innovative or unusual research. UKADIA members suggest that, if metrics were to be used as part of the process the following would be relevant to Art & Design:
- research income from AHRB and other agencies
- research student data
- staff data
- research outputs, particularly exhibitions, performances, design commissions
Regarding 1, levels of research income are significantly lower in Art & Design, as the dual support system has not and does not operate equally across the total subject range. The inequity of the dual support system is particularly relevant in any consideration of a ‘rolling’ programme of assessment based on research metrics.
Regarding 2, doctoral education is relatively new in Art & Design and Performing Arts and has been historically under funded. Therefore, at this stage, numbers are relatively low. However, comparisons between similar providers would be fair and appropriate.
Measures of visibility and measures of impact upon the economy are relevant to our subjects, although they would require substantial further development.
Self Assessment
UKADIA is supportive of the increasing role that self-assessment has taken in previous research assessment exercises. The qualitative data which can be provided in a statement of self-assessment is essential in providing a context for statistical data.
Self-assessment should include information concerning mission, priorities, strategy and SWOT analysis. For specialist Institutions, reference to mission and demonstration of mission alignment would be particularly relevant. Self-assessment should involve a prospective and retrospective approach. Self-assessment could also accommodate joint approaches, between Institutions or subjects, which would be useful to specialist institutions. Self-assessment could also be used to demonstrate the research contribution to, for example, cultural economy and business and the community. However some UKADIA members are wary of the potential for self-assessment to become burdensome, arguing that visits by trained reviewers would be required to ensure credibility.
But, self-assessment is seen as a more sensitive tool with which to judge the different types of Institution submitting for assessment. Qualitative information could be provided on collaborative activities, research infrastructure, contributions to regional cultural development, etc.
Historical Ratings
UKADIA is not supportive of an approach which could consolidate research in current large centres, creating a second tier of non-research institutions. The 2001 Research Assessment Exercise demonstrated the capacity for rapid development, particularly in the area of Art & Design. It would be counter- intuitive to design a system which does not accommodate development or encourage potential in new emerging disciplines.
However, an equitable approach which took a long term view of a research community’s performance is necessary. Mapping of performance over a series of research assessment exercises should be feasible. Assessment of value for money by comparing the level of output achieved with the level of research income received is also important.
Cross-Cutting Themes
In relation to defining excellence in research, some UKADIA members support a pluralistic approach whereby creativity and applicability can be equally or alternatively valued. One means of ensuring that we capture creativity and applicability would involve assessing on the basis of, for example, contribution to cultural economy, regional economy, business and industry.
In considering the determination of ‘subject pots’, UKADIA would value the support of research which underpins the rapidly developing creative industries. A strategic judgement on the importance of our areas to the UK should take account of this.
UKADIA welcomes the consultative paper’s sensitivity to institutional size and diversity of mission. Specialist institutions, of varying sizes, are likely to be clearly focused and each has a strong institutional sense of purpose and mission. A larger university will invariably cover more breadth. It seems more sensible that research assessment should compare like with like.
UKADIA would also welcome an approach which provided a ladder of improvement which enabled researchers and institutions to demonstrate and achieve potential. The opportunity for Joint Submission from smaller institutions should be considered, where collaboration has demonstrated a capacity to share and achieve research goals.
1