How do I choose the quality assessment method for my systematic Review?

You have three Available Choices

Essentially you have three main choices:

(i) to use the best available appraisal tool for each study design

(ii) to use a stable or "family" of tools from the same developers

(iii) to use a generic tool that has been developed for use with multiple designs.

Your choice is therefore determined not only by the study types but also what your plans for synthesis are. If you are planning to INTEGRATE evidence from different study types within a single review (e.g. a single chapter of findings) then options (ii) and (iii) are the best as they offer a degree of comparability. If you are going to consider each set of study types separately and summarise each "set" of evidence at a time within your report (e.g. a chapter on RCTs then a chapter on non-RCTs etcetera then you do not need to have any commonality of approaches.

(I)USE the best available appraisal tool for each design

For option (I) the Cochrane Risk of Bias tool is far and ahead the best tool for RCTs. A Risk of Bias Tool for Non-Randomised Controlled Trials (ACROBAT) also been developed allowing a large proportion of the important study types to be covered by these two similar tools - beyond this then the choice of additional checklists becomes more diverse and less consensual.

(II)USE a “STABLE” OR “FAMILY” of TOOLS FROM THE SAME DEVELOPERS

For option (ii) CASP, University of Glasgow General Practice, BestBETS andthe Joanna Briggs Institute are among those that have different tools for different studies offering an overall consistent approach or philosophy while acknowledging differences between study designs.

The CASP Checklists ( are as follows:

  • CASP Systematic Review Checklist
  • CASP Qualitative Checklist
  • CASP Randomised Controlled Trial Checklist
  • CASP Case Control Checklist
  • CASP Cohort Study Checklist
  • CASP Clinical Prediction Rule Checklist
  • CASP Diagnostic Checklist
  • CASP Economic Evaluation Checklist

The University of Glasgow Checklists ( are:

  • Checklist for an article on treatment or prevention
  • Checklist for an article on qualitative research
  • Checklist for an article on decision analysis
  • Checklist for an article on an educational intervention
  • Checklist for an article on prognosis
  • Checklist for an article on harm or causation
  • Checklist for an article on guidelines
  • Checklist for an article on diagnosis or screening
  • Checklist for a systematic review
  • Checklist for economic evaluations

The BestBETS Checklists ( are:

  • Case-control checklist (including harm)
  • Cohort
  • Decision rule
  • Diagnosis
  • Economic
  • Educational intervention
  • Guideline
  • Prognosis
  • Qualitative
  • Randomised control trial
  • Review or meta-analysis
  • Screening
  • Survey (including pre-test probabilities)

The Joanna Briggs Checklists are included as Appendices to the following document:

Appendix2: JBICriticalAppraisalforExperimentalStudies...... 3

Appendix 3:JBICriticalAppraisalChecklist forDescriptive/CaseSeries...... 4

Appendix4:JBICriticalAppraisalChecklist forComparable Cohort/Case Control..5

Appendix5:JBIDataExtractionFormforExperimental/ObservationalStudies....6

Appendix6:JBICriticalAppraisalChecklist forNarrative,ExpertOpinion Text..8

Appendix7:JBIQARIChecklistforInterpretiveCriticalResearch…..…..…..…..…...9

Appendix8:JBICriticalAppraisalChecklist forSystematicReviews...... 10

(III)USE a generic tool developed for use with multiple designs

For option (iii) the main tool available is the Alberta Heritage tool:

Kmet L M, Lee R C, Cook L S. Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Edmonton: Alberta Heritage Foundation for Medical Research (AHFMR). HTA Initiative #13. 2004. Available at:

(IV)FURTHER INFORMATION

A summary of a discussion on this topic can be found at:

Dr Andrew Booth School of Health and Related Research (ScHARR), University of Sheffield (19/05/2016)