PRELIMINARY ASSESSMENT FORM
The task of the review committees is to give an independent assessment of every group/programme from an international perspective. There are four aspects:
a) the quality of the academic activities,
b) the quantity of the academic output,
c) the academic and the societal relevance,
d) the academic perspective
I Aspects of Assessment
The primary focus for the review committee is the quality of the research group. An assessment of each of the following aspects is required for each group/programme:
(a) Academic quality of the output of the research group: what is the quality of the outcome?
What position has been achieved in the national and international context?
(b) Academic productivity of the research group, taking into account the input of human resources.
(c) The relevance of the research programme: what significance does the research have to the development of the academic field? In academic fields with a strong strategic or applied element this also includes the societal and/or technological impact: what significance has the research had for the development of societal and/or technological applications?
(d) Long-term viability of the research programme: what is the viability of the selected problem areas and approach and what are its prospects bearing in mind national and international competition?
In addition, the review committee is asked to assess the faculty research profile and give a critical review of the situation in the discipline or academic area as a whole and in each faculty or institute, from an international perspective. In this part of the assessments the specific context of the work carried out in a group or in a faculty or institute should be taken into account.
Aspects of the Assessment Further Defined
Academic quality:
Academic quality is based on the quality of the output of the research group; dissertations, academic publications, and, where relevant, professional publications, patents and other academic products (tests, prototypes, software). Aspects of the assessments include the academic level of the publications, with respect to the publication media, the originality and coherence of the research, and the contribution to the development of the discipline or area.
Due regard is given to the international standing of (the members of) a research group in assessing the quality of its achievements. Note is taken of participation in international cooperative projects, membership of editorial boards of international journals, academic awards, invitations to international conferences, visiting professorships, research funding acquired from ANR, EEC etc.
Academic productivity:
The committee assesses academic productivity by relating the output (the number of publications in total and in each category) to the input of human resources. In order to do justice to those groups with missions supplementary to strictly "curiosity-driven" research, due attention should also be given to other forms of academic output. In the comment accompanying the assessment, the review committee therefore compares the academic quality and productivity with the objectives or "mission" of the research programme as submitted by the research group. The research group's policy regarding publications should be taken into account. For this reason it is not always feasible to assess productivity in the form of a rating.
Relevance:
Assessment of the output cannot be considered independently of its relevance: what significance does the research have for the development of the academic field? Are the issues and the approaches chosen with insight, given the international situation of scholarship in the discipline concerned?
In academic fields with a strong strategic/applied research background the issue of academic relevance is inseparable from that of societal/technological impact. In the pure sciences the social and technological impact may or may not be important. It can thus be taken into account in the assessment. As in the case of productivity, here too the context should be taken into account and the relevance should be specified in the description of the group's mission. The minimum requirements for these missions will be a basic academic framework (is the work of a kind that can be expected from an academic group?). Furthermore, missions could include: a contribution to the front lines of science; support for other academic areas; support for applied sciences.
There is a growing interest in assessing the societal impact of research too. Here again, the stated mission of the group should be taken into account. In the comments accompanying the assessment, the committee's views of the relevance of the research are set against the research group's own viewpoint as expressed in the research group's mission. Because of the many different aspects of this criterion, it will not always be feasible to express the assessment in a rating. The verbal comment will be of more importance here.
Academic viability:
The assessment must take some account of the direction in which the research programme is developing. As far as possible, the committee should comment on the viability of the issues chosen and the research approach in the international academic arena. In some areas competitive strength will soon depend largely on factors of scale and the academic infrastructure available. The cohesion of the programme elements should also be a part of the assessment of academic viability.
In the comments accompanying the assessment, the committee considers the future position of the group involved as expressed in the faculty outline research profile.
II Aspects of the Assessment
The Committee assessed the research of the group/programme on the aspects:
Ø scientific quality (Q)
Ø scientific productivity (P)
Ø scientific relevance (R)
Ø longterm viability (V)
àIn the assessment of quality, the Committee considered in particular the quality of the research output and the international prominence of the research group.
àIn the assessment for productivity, the Committee considered the total scientific output, that is the contribution of the group to science and technology. Important indicators for this are
- the number of dissertations
- the number and nature of publications in scientific journals and books
- the number of patents
- the number of invited lectures.
àIn assessing the scientific relevance, the Committee weighed the issues and approaches chosen, the importance of the research for the advancement of knowledge in chemistry and in other sciences, and the possible impact and application in future technologies.
àLong-term viability was assessed on the basis of submitted plans and ideas for future research, and the availability of personnel and facilities.
We remark that the aspects quality and productivity refer to the past performance of the group, for the period 2004-2007. Thus the quality and productivity assessments take into account the scientific output (publications, etc.) of the academic staff, associated with the group during that period, including those who have left the group or are retired.
III Ratings used for the assessments
The Committee used five levels for the rating of their assessment. The five level are,
5 excellent
4 good
3 satisfactory
2 unsatisfactory
l poor
A more precise description of the levels for the ratings is as follows
The criteria for quality are:
Q = 5 excellent: The research developed in the group is equal to that of the world's best in the field.
Q = 4 good: The quality is above the world's average for that field and may contain elements that are excellent in quality.
Q = 3 satisfactory: The quality of the research developed in the group is comparable to the world's average in that field, and may contain aspects that are important nationally.
Q = 2 unsatisfactory: The quality of the research developed in the group is below the world's average in that field.
Q = l poor: The quality is far below the world's average in that field.
The criteria for productivity are:
P = 5 excellent: The productivity is comparable to the productivity of the world's best groups in that field.
P = 4 good: The productivity is above the world's average.
P = 3 satisfactory: The productivity is comparable to the world's average.
P = 2 unsatisfactory: The productivity is below the world's average.
P = l poor: The productivity is far below the world's average.
The criteria for relevance are:
R = 5 excellent: Notable and influential contributions have been made to prominent fields. The research group plays an important role in relevant scientific communities.
R = 4 good: Either less influential contributions have been made to prominent fields or else influential contributions have been made to less prominent fields.
R = 3 satisfactory: The group has performed moderately well on not very prominent but useful subfields of chemistry.
R = 2 unsatisfactory: The research is not of great relevance to the actual or potential advancement of knowledge in chemistry, and has little impact on future technologies.
R=l poor: The research is of no relevance.
The criteria for long-term viability are:
V = 5 excellent: The group is judged to have clear and coherent plans in line with international trends in the field. It has reached, and is likely to keep, a distinguished position in international networks. Continuity of funding is guaranteed. Highly qualified staff will continue to be available.
V = 4 good: The group is seen as competent to do the proposed research. The scientific issues being researched seem to be fruitful. The research group's position in the field is guaranteed. There are no doubts about future funding or the availability of the competent staff needed to maintain the programme of research.
V = 3 satisfactory: There are some reservations about one or more of the aspects mentioned above, but there is a reasonable chance that the group will survive and will continue to contribute adequately to its scientific field.
V = 2 unsatisfactory: Based on the plans presented, the Committee has serious doubts about the continued viability of the research group, and considers that without additional measures for strengthening the group, it will not be able to function adequately.
V = l poor: For one or more reasons, the research programme is judged to be non-viable, and should not be continued.
Generally, a score for viability which is higher than the score for quality indicates that the Committee has reasons to expect that the scientific quality of the group will be higher in the future; this can be the case for young, promising groups.
If the faculty has decided not to continue the programme, the Committee has given no score for viability (V=-). In cases where a programme director recently retired or left the group and a successor has been appointed recently or will be appointed in the near future, a formal score of V=3 is given for the viability, in expectation of future scientific output.
PRELIMINARY ASSESSMENT FORM
REVIEW COMMITTEE
Laboratory : Reviewer:
group/programme:
Manager :
Your preliminary assessment is only for use in the committee meetings and will not be published in the committee report.
A. QUALITYHow do you evaluate the quality of the programme with respect to the: / 1 / 2 / 3 / 4 / 5
1. originality of the approach and ideas
2. coherence and cumulative character of the research
3. prominence of the director and the other members of the programme
4. distribution of published output over the team members
5. quality of the scientific publications
Overall assessment of quality: / 0 / 0 / 0 / 0 / 0
B. PRODUCTIVITY
How do you evaluate the scientific productivity in view of the input of human and material resources with respect to the: / 1 / 2 / 3 / 4 / 5
1. number of PhD theses
2. number & nature of scientific publications
3. number of patents
4 number & nature of invited lectures
Overall assessment of productivity: / 0 / 0 / 0 / 0 / 0
C. RELEVANCE
How do you evaluate the relevance of the research with respect to: / 1 / 2 / 3 / 4 / 5
1. the advancement of knowledge or expertise
2. impact and application in future technologies
3. position in relevant networks (national and international)
4. the balance between fundamental and applied research
Overall assessment of relevance: / 0 / 0 / 0 / 0 / 0
D. LONG TERM-VIABILITY
Considering the available personnel and facilities, how do you evaluate the long-term viability of the programme (taking into account the long term needs of industry and society and of other users of scientific and technological knowledge): / 1 / 2 / 3 / 4 / 5
1. in view of what has been achieved so far
2. in view of the plans and ideas for the future
Overall assessment of long-term viability: / 0 / 0 / 0 / 0 / 0
Points of attention, remarks
Questions (to the programme director, faculty board or research committee)