HRC Bibliometric Study 2014

Measuring up

To better understand the impact of the health research we fund, we studied all health research articles with New Zealand funders between 2005 and 2009, analysing how often they were quoted in the literature and how they ranked internationally. The results show that HRC systems work well – we are backing the best.

Bibliometrics is the study of the impact of publications. There are many other, important, research impactsbut they are hard to objectively measure.Data on publications for New Zealand and the world can be purchased,allowing us to benchmark our performance both nationally and internationally. This provides information that we cannot gain by any other means.

This table provides a brief summary of what we did, following an identical methodology used by the Australian National Medical Research Council to undertake their triannual bibliometric review.

Data purchased from Thomson Reuters ISI, NZ health research publications 2005-2009, Journal Citation Index 2010
HRC-funded publications: / 1560
Articles with no HRC funding: / 11,209
Four sectors defined by author addresses & HRC contract data: / HRC, University, Hospital and Other
Duplication of articles across all sectors but HRC: / Articles attributed to multiple sectors according to authorship, except when funded by HRC
ANZ Fields of Research Groups used / 13 of 22 categories with enough articles to analyse

We adjusted the number of times our articles were quoted in the international literature by the world average in each field to get a ratio called the Relative Citation Impact (RCI). To get an idea of the relative quality of the journals those articles were published in, we also adjusted our citation rate by average citation rate of the journals those articles were published in. This gave us the Relative Journal Impact (RCI). Our graphs show the RCI and the RJI in relation to the world average (the blue line) which, because this is a ratio, is at one for both measures. This allows us to compare both together on the same graph for reassurance that one sector is not being quoted highly in low impact journals.

Clear international strengths

HRC articles were cited at or above the world average in every field that we could analyse. Compared to the other funding sectors, the HRC had the highest RCI in Cardiovascular Medicine & Haematology; Clinical Sciences; General Biological Sciences, Genetics, Immunology;

Neurosciences; Nutrition & Dietetics; Paediatrics & Reproductive Medicine; Pharmacology & Pharmaceutical Sciences and Public Health and Health Services Research. The only areas in which the HRC did not have a higher RCI was in Biochemistry and Cell Biology and Oncology and Carcinogenesis.

HRC was the only sector to achieve 20% of articles ranked in the top 20% for impact worldwide

Looking at the percentile ranking of articles removes the potential problem of highly cited articles influencing the results. The percentile data reinforces the results from RCI and shows that HRC performance is particularly strong in the areas of Paediatrics and Reproductive Medicine; Immunology; Genetics and Clinical Sciences.