Guidelines for assessment of manuscripts and presentations by PhD-students.
Supporting material for PhD student and assessors.
Four parameters are evaluated: the first two refer to the written manuscript, the third to the presentation quality and the fourth to the ability to discuss the presentation/poster with the referees.
1) MQ, 30 points. Material organization, accuracy, language, discussion of results, references
Manuscript Quality / Score range: 0-20MQ / Excellent (6) / Good (5) / Average (4) / Fair (3) / Poor (2)
Material organization
Accuracy
Language
Discussion of results
References
Material organization: The referee must assess whether the text of the oral, poster or mini-poster communication published in the Book of Proceedings was drafted in accordance to the guidelines for authors; included all the required sections; the objectives were clearly stated; all figures and tables were referred to in the text and provided with clear legends/headings, the presentation of results was clear and concise; the reference list was drafted according to the instructions.
Accuracy: The referee must assess whether the material and methods were clearly illustrated, the results section was complete, Tables and Figures showed the results with the necessary accuracy.
Language: The referee must assess whether the text is correct in terms of spelling, grammar, English usage.
Discussion of results: The referee must assess whether the results were discussed in view of the objectives and of current literature and not merely described; the authors presented hypotheses supported by the data; the discussion was not speculative.
References: The referee must assess whether the references are sufficient to illustrate the state of art (as related to the subject of the article) and correctly used to discuss any correspondence or disagreement with the author's findings.
2) NC, 20 points, research and methodology, novelty, statistical analysis, data-supported conclusions
Research and Methodology Novelty and Data-Supported Conclusions (NC): / Score range: 0-20NC / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Research methodology
Novelty
Statistical analysis
Data supported conclusions
Research methodology: The referee must assess whether the research methodology described in the text published in the Book of Proceedings was appropriate for the topic at hand (given the resources available);
Novelty: The referee must assess whether the work was novel in terms of approach, methodology, results, conclusions and there was no evidence of plagiarism (including self-plagiarism).
Statistical analysis: The referee must assess whether the statistical methods were appropriate for the topic at hand and the results were clearly presented, without redundancy and unnecessary details. It is worthy of pointing out that there is no excuse for lack of statistical treatment of the data.
Data supported conclusions The referee must assess whether the conclusions were fully supported by the data and any discrepancy with the objective, hypotheses, literature was addressed and/or explained.
3) PQ, 20 points. quality of the presentation, clarity, language property, mastery of the topic
Presentation Quality (PQ): / Score range: 0-20PQ / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Quality of the Presentation
Clarity
Mastery of the topic
Italian / Foreign Language
Language usage / 0
Quality of the presentation: The referee must assess whether the oral, poster or miniposter communication was clearly organized, did not exceed the allotted time (15 min for the oral communication or 5 min for the poster and miniposter one), was clearly readable/visible.
Clarity: The referee must assess whether the introductory material, the methods and the results were clearly presented, the results were discussed and compared with the literature and with the objective of the work
Mastery of the topic: The referee must assess whether the author clearly appeared to be current with the pertinent scientific literature, mastered the methods and understood the results and their implications.
Language usage: The referee must assess whether the spoken language was correct in terms of grammar, English usage, and pronunciation. If the presentation is in Italian the score is automatically nil.
4) AR, 30 points, Accurate reply to referee's questions
Accurate reply to referee's questions (AR) / Score range: 0-30Excellent (30) / Good (25) / Average (20) / Fair (15) / Poor (10)
AR
Textbooks on scientific writing are available in Italian and in English: a brief list follows.
A brief example of Primary trait analysis can be found at http://www.assessment.gatech.edu/wp-content/uploads/slide_shows/Using_the_Grading_Process_for_Assessment.pdf but more material is available on the Internet.
A NIH ORI publication “Avoiding Plagiarism, Self-Plagiarism, and Other Questionable Writing Practices: a Guide to Ethical Writing” (http://ori.dhhs.gov/education/products/plagiarism/) clearly addresses issues in plagiarism an unethical writing.
Booth V. 1993. Communicating in science. 77 pagine. Cambridge University Press.
Bruni F., Alfieri G., Fornasiero S., Tamiozzo Goldman S. 1997. Manuale di scrittura e comunicazione. Zanichelli. 448 pagine. ca. € 20,00.
Davis M. 1985. Scientific papers and presentations. 345 pagine. Academic Press.
Emilio Matricciani. 2007. La scrittura tecnico-scientifica. Casa Editrice Ambrosiana. 382 pagine, ca € 34,00
Roberto Lesina. 1994. Il nuovo manuale di stile. Edizione 2.0. Zanichelli. 384 pagine, ca. € 27,00
ALCUNI ESEMPI PRATICI
FORM FOR THE EVALUATION OF ORAL/POSTER COMMUNICATIONS
NAME X Y….. ………………………
DISCUSSION DATE: 17.09.2010
TITLE ABC
REFEREE: Giovanni Bianchi
Evaluation ParameterManuscript Quality (MQ):
Score range: 0-30 / Excellent (6) / Good (5) / Average (4) / Fair (3) / Poor (2)
Material organization / X
Accuracy / X
Language / X
Discussion of results / X
References / X
Brief Account of MQ weaknesses / Fig. 1 unclear, misspelled words English; Fig. 2 undiscussed; no reference for
solute determination
Research and Methodology Novelty and Data-Supported Conclusions (NC): Score range: 0-20 / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Research methodology / X
Novelty / X
Statistical analysis / X
Data supported conclusions / X
Brief Account of NC weaknesses / Conventional approach; no statistical treatment of data; discrepancy among these
results and those generally reported in the literature
Presentation Quality (PQ): Score range: 0-20 / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Quality of the Presentation / X
Clarity / X
Mastery of the topic / X
Italian / Foreign Language
Language usage / 0 / X
Brief Account of PQ weaknesses / Presentation exceeding the time allotted; Unreadable texts; Introduction section
too long; Language used ITALIAN; limited discussion of results
Accurate Reply to referees’ questions (R): Score range: 0-30 / Excellent (30) / Good (25) / Average (20) / Fair (15) / Poor (10)
Reply / X
Brief Account of R weaknesses / Some uncertainty in replying to …….
Total: score range 0-100 / 74
The overall evaluation of the PhD student is obtained by averaging the scores attributed by any referee and will be expressed as
A (very good) level: average score range: ³86
B (good) level: average score range: 71-85
C (sufficient) level: average score range: 60-70
D (insufficient) level: average score range: £59
FORM FOR THE EVALUATION OF ORAL/POSTER COMMUNICATIONS
NAME X Y….. ………………………
DISCUSSION DATE: 17.09.2010
TITLE ABC
REFEREE: Giovanni Bianchi
Evaluation ParameterManuscript Quality (MQ):
Score range: 0-30 / Excellent (6) / Good (5) / Average (4) / Fair (3) / Poor (2)
Material organization / X
Accuracy / X
Language / X
Discussion of results / X
References / X
Brief Account of MQ weaknesses
Research and Methodology Novelty and Data-Supported Conclusions (NC): Score range: 0-20 / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Research methodology / X
Novelty / X
Statistical analysis / X
Data supported conclusions / X
Brief Account of NC weaknesses
Presentation Quality (PQ): Score range: 0-20 / Excellent (5) / Good (4) / Average (3) / Fair (2) / Poor (1)
Quality of the Presentation / X
Clarity / X
Mastery of the topic / X
Italian / Foreign Language
Language usage / 0
Brief Account of PQ weaknesses
Accurate Reply to referees’ questions (R): Score range: 0-30 / Excellent (30) / Good (25) / Average (20) / Fair (15) / Poor (10)
Reply / X
Brief Account of R weaknesses
Total: score range 0-100 / 95
The overall evaluation of the PhD student is obtained by averaging the scores attributed by any referee and will be expressed as
A (very good) level: average score range: ³86
B (good) level: average score range: 71-85
C (sufficient) level: average score range: 60-70
D (insufficient) level: average score range: £59