GUIDELINES ON THE COMPLETION OF A RATING REVIEWER REPORT

Particulars of Reviewer

Surname / Title / Initials
Institution
Department
Position
Email address
Area(s) of specialisation

Before you commencewith the review it is important that you take cognisance of the following:

CONFIDENTIALITY AGREEMENT AND CONFLICT OF INTEREST

This will certify that in the review of all applications for evaluation and rating submitted to the NRF:

i.I will treat all information contained in all applications for evaluation and rating and/or reviewer reports in the strictest of confidence and will not reveal that information to any third party without the prior written consent of the NRF.Please note that the review is anonymous and no correspondence with the applicant should be entered into.

ii.I will not do or allow anything to be done which might compromise the interest of the NRF or any of the proposers in respect of any intellectual property rights flowing from the confidential information.

iii.I will not use the information contained in those applications for evaluation and rating for any reason other than for the purpose of providing a peer reviewed report or assessment for the NRF.

iv.I will not participate in a review process where a conflict of interest exists. Should there be doubt about an apparent conflict of interest, I will advise the NRF, who will then indicate whether participation in the review process is permissible or not.

Please
1. / I have read the confidentiality agreement and conflict of interest and agree to abide by it.
2. / This is an assessment of the applicant’s research outputand not a request to review a funding proposal or an application for a job promotion.
3. / The usability of your report will be assessed by the members of the Assessment Panel. A useful report will address all of the issues raised in the preamble to each section. Superficial, unsubstantiated and/or biased reports will unfortunately not be used. (See attached document.)
4. / Reviewers should be familiar with the research field(s) of the applicant, e.g. have previously encountered and/or read the applicant's work, which could include reading the five best outputs listed and attached on the application.
5. / The review should focus on the outputs of the previous eight years (i.e. 2010 – 2017) and not the applicant’s lifetime achievements.
6. / The review should focus on the quality and impact of the research outputs (referred to in item 4. above) and not on the character of the applicant.
7. / Only standing derived from the quality and impact of the research outputs should be commented on in item 4. above.
8. / The review should focus on the contribution of the individual and not on the team/research group.
  1. Background knowledge of the applicant
  1. Please indicate whether you:

Yes / No
  • Have knowledge of the field(s)/research of the applicant to enable an assessment of the research portfolio (during the last eight years).

Yes / No
  • Have read the five best research outputs*embedded in the application after being requested to perform this review.

*Copies available as part of the reviewer documentation

Any additional information:

Please provide information to explain your above responses:
  1. Please indicate whether you have:

Yes / No
  • Collaborated/co-published with the applicant in the past.

  • Cited any of the applicant's work.
/ Yes / No
OR
  • Any other relevant form of association or collaboration with the applicant (e.g. doctoral supervisor)
/ Yes / n/a
Any additional information:
Add any additional comments with regard to the three points above that you may wish to include here:
  1. Appraisal of research outputs in the last eight years
  • Please focus your critique only on the quality of all research outputslisted in the application over the past eight years (2010 – 2017). Kindly note that the best copies of the self-selected research outputs (as indicated in the application) are attached for your convenience. Discuss the impact, if any, you feel that the applicant’s work has had on its specific research field and whether it has impacted on other fields.
  • We particularly value it if you are able to provide judgements which are related to specific aspects of the applicant's work.
  • It is not necessary to provide a quantitative summary of outputs (for example, a count of publications in refereed journals, published conference proceedings, etc.) or a detailed exposition of the content of the research.
  • Your opinion on the standing and appropriateness of the journals, books, conference proceedings and other forms of research outputs which the applicant may have listed would be very useful.In addition, should you be of the opinion that any of the outputs have appeared in “predatory journals”, kindly comment accordingly. For more information on how to identify predatory journals and deceptive publishers you can consult the following guidelines:

-

-

  • If the research outputs are those of a group, kindly assess the applicant's contribution to the group.
  • What is the significance of the research in the broader context of the discipline/field?
  • If you are able to, please describe any impact the research outputs may have had on industry or society.

Please add comments here:

3.Estimation of applicant’s current standing as a researcher

Please indicate how you would rate the applicant relative to his/her peers in the field and justify/substantiate your reasons with reference to the quality of the research outputs. If possible, comment on both national (usually South African, though some applicants may have conducted their recent research in another country) and international standing. As the applicant's current standing is being evaluated, please base your judgement primarily on the research outputs of the last eight years (2010 – 2017). The members of the Assessment Panel would appreciate your comments on the size of the applicant's research field (as indicated, for example, by the number of researchers working in it), and on the current importance of the field of the applicant's research within the discipline.

Please add comments here:

4.Reviewer’s recommendations for future development of research

You may wish to formulate your own feedback relating to the applicant’s future planning of his/her research that you feel potentially helpful.

Please add comments here:

Assessment of Reviewers’ Reports

Members of Assessment Panels must play an interpretative role when they assess reviewers’ reports. For example, if a reviewer states that the applicant is publishing in top journals, yet the journals are in the bottom tier of journals in the field, then the validity of the reviewer’s report must be called into question. Similarly they should recognise the weakness of reviewers’ reports which overly praise the importance of the applicant’s work and where the work concerned is clearly not of the calibre suggested in the report.

Reviewers have to indicate that they have actually read the best five research outputs and/or are well acquainted with the applicant’s research outputs, especially those produced in the last eight years.

Reports by reviewers are assessed and scored as follows:

1Excellent report: Reviewer gives a thorough analysis and critique of recent research outputs and substantiates comments/judgements on the quality and impact of the research outputs of the last eight years as well as the international/national standing of the applicant. [Reports classified in this category but where the reviewer went the “extra mile” should be noted at the meeting for recognition by the NRF]

1/2Good report: Reviewer gives a good analysis of recent research outputs and comments on the quality and impact of the research outputs of the last eight years as well as the international/national standing of the applicant.

2Satisfactory report: Reviewer gives an analysis of the quality and impact of recent research outputs and comments on the international/national standing of the applicant.

2/3Partially usable report:* Some aspects of the report are usable while other aspects are unsatisfactory. (e.g. no analysis of the outputs or standing is provided). These reports state, for example:

“It appears that the applicant’s work is…” /”I have heard from one of my colleagues…” / “It seems that his/her recent research outputs are…”

*Note: A 2/3 (“partially usable”) report has a relatively low weighting and cannot be used to determine an outcome in borderline cases. It can only be used as one of the usable reports if it confirms the opinion expressed by the majority of the other reports that are of acceptable quality (scored 1; 1/2 or 2)

3Unsatisfactory report:** Typical reasons:

  1. Reviewer has not read any of the research outputs of the last eight years.
  2. Report by reviewer:
  3. Is a testimonial / “over the top”
  4. Is superficial
  5. Contains sweeping and/or unsubstantiated statements not supported by the application
  6. Over-generalises and/or provides no assessment, or
  7. Fails to focus on the last eight years.
  8. Reviewer is inappropriate when:
  9. a reviewer is not a peer
  10. a reviewer is no longer active in the research field
  11. a reviewer’s report is biased or hostile

**Note: Unsatisfactory reports are not to be taken into account when deciding on a rating. For audit purposes a reason should be provided why the report was discarded.