An Instructional Guide for Peer Reviewers of Biomedical Manuscripts

An Instructional Guide for Peer Reviewers of Biomedical Manuscripts

Recommended References

Reporting in clinical trials - descriptive and inferential statistics (including problems with p-values)

1. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. JAMA 1997;277:927-934. Available at <a href=http://jama.ama-assn.org/info/auinst_req.html target=_blank>http://jama.ama-assn.org/info/auinst_req.html</a>.

2. Bailar JC, Mosteller F. Guidelines for statistical reporting in articles for medical journals. Amplifications and explanations. Ann Intern Med 1988;108:266-73.

3. Altman, D. Statistics in medical journals: some recent trends. Statist Med 2000;19:3275-3289.

4. Greenland S. Randomization, statistics, and causal inference. Epidemiology 1990;1:421-429.

5. Maclure M, Schneeweis S. Causation of bias: the episcope. Epidemiology 2001;12:114-122.

6. Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994;121:200-206.

7. Braitman LE. Confidence intervals assess both clinical significance and statistical significance. Ann Intern Med 1991;114:515-517.

8. Gallagher, EJ. Threshold for decerebrate genuflection. Acad Emerg Med1999;6:1084-1087.

9. Goodman SN. Toward evidence-based medical statistics. 1: the p-value fallacy. Ann Intern Med 1999;130:995-1004.

10. Sterne JAC, Smith GD. Sifting trough the evidence – what’s wrong with significance tests? BMJ 2001;322:226-231.

11. Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. NEJM 1987;317:426-432.

12. Hart A. Mann-Whitney test is not just a test of medians: differences in spread can be important. BMJ 2001;323 391-393.

13. Browner WS, Newman TB. Are all significant p values created equal? The analogy between diagnostic tests and clinical research. JAMA 1987;257:24592463.

14. Leamer, EE Specification searches: ad hoc inference with non-experimental data New York: Wiley, 1978.

15. Modern Epidemiology 2nd ed. Kenneth J. Rothman, Sander Greenland. Philadelphia : Lippincott-Raven, 1998. Bayesian Analysis

16. Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Ann Intern Med 1999;130:1005-1013.

17. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. An introduction to bayesian methods in health technology assessment. BMJ 1999;319:508-512.

18. Brophy JM, Joseph L. Placing trials in context using Bayesian analysis. GUSTO revisited by Reverend Bayes. JAMA 1995;273:871-875.

19. Ashby D, Smith AFM. Evidence-based medicine as Bayesian decision-making. Statist Med 2000;19:3291-3305.

20. Lehmann HP, Goodman SN. Bayesian communication: a clinically significant paradigm for electronic publication. J Am Med Inform Assoc 2000;7:254-266. Number Needed to Treat (NNT)

21. Cordell WH. Number needed to treat (NNT). Ann Emerg Med 1999;33:433-436.

22. Chatellier G, Zapletal E, Lemaitre D, Menard J, Degoulet P. The number needed to treat: a clinically useful nomogram in its proper context. BMJ 1996;312:426429.

23. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. NEJM 1988;26:1728-1733.

24. Smeeth L, Haines A, Ebrahim S. Numbers needed to treat derived from metaanalyses – sometimes informative, usually misleading. BMJ 1999;318:15481551.

25. Altman DG, Andersen PK. Calculating the number needed to treat for trials where the outcome is time to an event. BMJ 1999;319:1492-1495.

Baseline Comparisons, Multiple Comparisons and Subgroup Analysis

26. Altman DG, Dore CJ. Randomisation and baseline comparisons in clinical trials. Lancet 1990;335:149-153.

27. Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA 1991;266:93-98.

28. Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses. Ann Intern Med 1992;116:78-84.

29. Smith DG, Clemens J, Crede W, Harvey M, Gracely EJ. Impact of multiple comparisons in randomized clinical trials. Am J Med 1987;83:545-550.

30. Greenland S, Rothman K. Multiple Comparisons in, Fundamentals of Epidemiologic Data Analysis, in: Modern Epidemiology. 2 nd ed. Philadelphia, PA: Lippincott, Williams & Wilkins, 1998; 225-229.

31. Mills JL. Data torturing. NEJM 1993;329;1196-1199.

Multivariate models

32. Braitman LE, Davidoff F. Predicting clinical states in individual patients. Ann Intern Med 1996;125:406-412.

33. Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariate models. Ann Intern Med 1993;118:201-210.

Chart Review Methods

34. Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: where are the methods. Ann Emerg Med 1996;27:305-308.

Reporting Agreement and Inter-rater Reliability

35. Bland JM, and Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet 1986;8476:307-310.

36. Guggenmoos-Holzmann I. The meaning of kappa: probabilistic concepts of reliability and validity revisited. J Clin Epid 1996;49:775-782.

37. Bloch DA, Kraemer HC. 2 x 2 kappa coefficients: measures of agreement or association. Biometrics 1989;45:269-287.

Unit of Analysis

38. Whiting-O’Keefe QE, Henke C, Simborg DW. Choosing the correct unit of analysis in Medical Careexperiments. Medical Care1984;22:1101-1114.

Graphing

39. Briscoe MH. Preparing Scientific Illustrations: A Guide to Better Posters, Presentations, and Publications. 2 nd edition. New York, NY: Springer-Verlag; 1996.

40. Iverson C, Flanagin, A, Fontanarosa PB, Glass RM, Glitman P, Lantz JC, Meyer HS, et al. eds. Manuscript Preparation, in: American Medical Association manual of style: a guide for authors and editors. 9 th ed. Maryland: Williams & Wilkins, 1998; 7-85.

41. Tufte ER: Visual Display of Quantitative Information. Cheshire, CT: Graphic Press, 1983.

42. Cleveland WS: Visualizing Data. Summit, NJ: Hobart press, 1993.

43. McNeil D. On graphing paired data. The American Statistician 1992;46:307-311.

44. Schriger DL, Cooper RJ. Achieving graphical excellence: suggestions and methods for creating high-quality visual displays of experimental data. Ann EM 2001; 37:75-87.

Randomized Controlled Trials

45. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-639.

46. Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-1194.

Randomized Controlled Trials vs Observational Studies

47. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Interpreting the evidence: choosing between randomized and non-randomized studies. BMJ 1999;319:312-315.

48. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. NEJM 2000;342:1878-1886.

49. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. NEJM 2000;342:1887-1892.

Meta-analysis and Systematic Reviews – (reporting, how to use, and issues of quality scoring)

50. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM group. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Lancet 1999;354:18961900.

51. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, et al for the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) Group. Meta-analysis of observational studies in Epidemiology. A proposal for reporting. JAMA 2000;283:2008-2012.

52. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126:376-380.

53. Lau J, Ioannidis JPA, Schmid CH. Quantitative Synthesis in Systematic Reviews. Ann Intern Med 1997;127:820-826.

54. Oxman AD, Cook DJ, Guyatt GH, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature VI. How to use an overview. JAMA 1994;272:1367-1371.

55. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282:1054-1060.

56. Greenland S. Quality scores are useless and potentially misleading. Am J Epidemiology 1994;140:300-301.

Cost-effectiveness and Economic Analyses

57. Drummond MF, Jefferson TO on behalf of the BMJ Economic Evaluation Working Party. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313:275-283.

58. Detsky AS, Naglie IG. A clinician's guide to cost-effectiveness analysis. Ann Intern Med 1990;113:147-154.

59. Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC, for the Panel on Cost-Effectiveness in Health and Medicine. The role of cost-effectiveness analysis in health and medicine. JAMA 1996;276:1172-1177.

60. Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB for the Panel on Cost-Effectiveness in Health and Medicine. Recommendations of the panel on cost-effectiveness in health and medicine. JAMA 1996;276:1253-1258.

61. Siegel JE, Weinstein MC, Russell LB, Gold MR for the Panel on CostEffectiveness in Health and Medicine. Recommendations for reporting costeffectiveness analyses. JAMA 1996;276:1339-1341.

62. Drummond MF, Richardson S, O’Brien BJ, Levine M, Heyland D, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? JAMA 1997;277:1552-1557.

63. O’Brien BJ, Heyland D, Richardson S, Levine M, Drummond MF, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? JAMA 1997;277:1802-1806.

64. Nixon J, Khan KS, Kleijnen J. Summarising economic evaluations in systematic reviews: a new approach. BMJ 2001;322:1596-1598.

Diagnostic Tests

65. Jaeschke R, Guyatt G, Sackett DL for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA 1994;271:389-391.

66. Jaeschke R, Guyatt GH, Sackett DL for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? JAMA 1994;271:703-707.

67. McMaster Clinical Epidemiology Rounds. How to read clinical journals: II. To learn about a diagnostic test. CMAJ 1981;124:703-720.

68. Arkin CF, Wachtel MS. How many patients are necessary to assess test performance? JAMA 1990;263:275-278.

69. Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research. Getting better but still not good. JAMA 1995;274:645651.

70. Mower WR. Evaluating bias and variability in diagnostic test results. Ann EM 1999;33:85-91.

71. Lijmer JG, Mol BW, Heisterkamp A, Bonsel GJ, Prins MH, van der Meulen JHP, Bossuyt PMM. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999;282:1061-1066.

72. Buderer NFM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med1996;3:895-900.

73. Radack KL, Rouan G, Hedges J. The likelihood ratio. An improved measure for reporting and evaluating diagnostic test results. Arch Pathol Lab Med 1986;110:689-693.

74. Hayden SR, Brown MD. Likelihood ratio: a powerful tool for incorporating the results of a diagnostic test into clinical decisionmaking. Ann Emerg Med 1999;33:575-580.

Qualitative Research

75. Green J, Britten N. Qualitative research and evidence based medicine. BMJ 1998;316:1230-1232.

76. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ 2000;320:50-52.

77. Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ 2000;320:114-116.

78. Giacomini MK, Cook DJ for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature XXIII. Qualitative research in health care. A. Are the results of the study valid? JAMA 2000;284:357-362.

79. Giacomini MK, Cook DJ for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature XXIII. Qualitative research in health care. B. What are the results and how do they help me care for my patients? JAMA 2000;284:478-482.

Clinical Decision or Prediction Rules

80. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Annals EM 1999;33:437-447.

Clinical Guidelines

81. Hayward RSA, Wilson MC, Tunis SR, Bass EB, Guyatt G, for the EvidenceBased Medicine Working Group. Users’ guides to the medical literature VIII. How to use clinical practice guidelines. A. Are the recommendations valid? JAMA 1995;274:570-574.

82. Wilson MC, Hayward RSA, Tunis SR, Bass EB, Guyatt G, for the EvidenceBased Medicine Working Group. Users’ guides to the medical literature VIII. How to use clinical practice guidelines. B. What are the recommendations and will they help you in caring for your patients? JAMA 1995;274:1630-1632.

83. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature. IX. A method for grading health care recommendations. JAMA 1995;274:1800-1804.

84. Harbour R, Miller J for the Scottish Intercollegiate Guidelines Network Grading Review Group. A new system for grading recommendations in evidence based guidelines. BMJ 2001;323:334-336.

85. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peerreviewed medical literature. JAMA 1999;281:1900-1905.

86. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines. Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:527-530.

87. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines. Developing guidelines. BMJ 1999;318:593-596.

88. Feder G, Eccles M, Grol R, Griffiths C, Grimshaw J. Clinical guidelines. Using clinical guidelines. BMJ 1999;318:728-730.

Quality of life research

89. Muldoon MF, Barger SD, Flory JD, Manuck SB. What are quality of life measurements measuring? BMJ 1998;316:542-545.

90. Guyatt GH, Naylor D, Juniper E, Heyland DK, Jaeschke R, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature XII. How to use an article about health-related quality of life. JAMA 1997;277:1232-1237.

91. Staquet MJ, Hays RD, Fayers PM. Quality of Life Assessment in Clinical Trials. Methods and Practice. New York: Oxford University Press;1998.

92. Fischer D, Stewart AL, Bloch DA, Lorig K, Laurent D, Holman H. Capturing the patient’s view of change as a clinical outcome measure. JAMA 1999;282:11571162.

93. Coons SJ, Rao S, Keininger DL, Hays RD. A comparative review of generic quality-of-life instruments. Pharmacoeconomics 2000;17:13-35.

An Instructional Guide for Peer Reviewers of Biomedical Manuscripts

<i>An Instructional Guide for Peer Reviewers of Biomedical Manuscripts</i>

<b>Recommended References</b>