Instructions
This tool is a templatized version of the Delfini Short Critical Appraisal Checklist for use in doing projects or exercises. While these questions are meant to guide you through a fairly complete review, do add comments, questions and observations as you believe relevant for assessing validity and clinical usefulness. Risk of bias ratings (or use your own): high, medium, borderline, low. Study grade (or use your own): A, B, BU, U (uncertain). Supplementary questions at bottom are highly condensed.
Project
Tips: When we do critical appraisal, we usually only grade until we are confident about a study grade. Adding the study abstract in the entry column near the beginning may be useful. For evidence-synthesis work, entry columns from other appraisals can be pasted next to each other for collective viewing.
Exercise
For this exercise, identify as many threats to internal validity as you can. Ignore the greyed out areas unless otherwise instructed.
# / Item/Consideration / Entry1. / Project
2. / Appraiser/recorder (i.e., your name)
3. / Date
4. / Study identification (i.e., author/year)
5. / Note sponsorship, funding and affiliations, recognizing that any entity or person involved in research may have a bias.
6. / Evaluate concerns or potential threats to validity in the absence or lack of quality of the following ê
7. / Study Design Assessment (for cause and effect, potential exception, all-or-none results)
o Is the design appropriate to the research question? Is the research question useful?
o For efficacy, use of experimental study design (meaning there was no choice made to determine intervention)
o Clinically significant area for study (morbidity, mortality, symptom relief, functioning and health-related quality of life) and reasonable definitions for clinical outcome such as response, treatment success or failure
o If composite endpoints used, reasonable combination
o Ensure prespecified and appropriate 1) research questions, 2) populations to analyze, and 3) outcomes / Bias Assessment
Notes
Risk of Bias Rating =
8. / Selection Bias
o Groups are appropriate for study, of appropriate size, concurrent and similar in prognostic variables
o Methods for generating the group assignment sequence are truly random, sequencing avoids potential for anyone affecting assignment to a study arm and randomization remains intact (allocation by minimization may be acceptable)
o Concealment of allocation strategies are employed to prevent anyone affecting assignment to a study arm / Bias Assessment
Notes
Risk of Bias Rating =
9. / Performance Bias
o Double-blinding methods employed (i.e., subject and all working with the subject or subject’s data) and achieved
o Reasonable intervention and reasonable comparator used (e.g., placebo)
o No bias or difference, except for what is under study, between groups during course of study (e.g., intervention design and execution, care experiences, co-interventions, concomitant medication use, adherence, inappropriate exposure or migration, cross-over threats, protocol deviations, study duration, changes due to time etc.) / Bias Assessment
Notes
Risk of Bias Rating =
10. / Data/Attrition Bias
o Evaluate bias in measurement activities
o Might attrition, including missing data, discontinuations or loss to follow-up, have resulted in distorted outcomes? / Bias Assessment
Notes
Risk of Bias Rating =
11. / Assessment Bias & Chance Assessment
o Assessors are blinded
o Low likelihood of findings due to chance, false positive and false negative outcomes
o Non-significant findings are reported, but the confidence intervals include clinically meaningful differences
o If variables are dichotomous, Intention-to-Treat Analysis (ITT) performed for efficacy (not safety) (all people are analyzed as randomized + reasonable method for imputing missing values). (May not be an issue if missing values are very few.)
o If time-to-event analysis performed, appropriate, transparent and unbiased. Evaluate censoring rules.
o Analysis methods are appropriate and use of modeling only with use of reasonable assumptions
o No problems of selective reporting or selective exclusion of outcomes / Bias Assessment
Notes
Risk of Bias Rating =
12. / Meaningful Clinical Benefit
o Clinically significant area + sufficient benefit size = meaningful clinical benefit (consider efficacy vs effectiveness)
o Safety (caution re: new interventions, caution re: non-significant findings) / Assessment
1. / Efficacy / Assessment
2. / Safety / Assessment
3. / Overall Grade and Summary (see supplements below if applicable) / Assessment
4. / Non-Inferiority & Equivalence Supplement: Absence of the following problems: lack of sufficient evidence confirming efficacy of referent treatment; study not sufficiently similar to referent study; inappropriate Deltas; or significant biases or analysis methods which would tend to diminish an effect size (e.g., conservative application of ITT analysis, insufficient power, etc.) / Assessment
5. / Diagnostic Test Supplement: New test requires better outcomes or value. Test is compared to gold standard or reasonable comparator and finds same abnormality and within time period that does not result in a change in diagnosis. Test is applied to all or random sample of subjects with and without disease. Assessors are blinded. There is minimal bias from indeterminate results. Measures of test function are useful. / Assessment
6. / Screening Supplement: Early diagnosis and treatments determined to be effective will improve outcomes more than later diagnosis and treatment. Beneficial outcomes are not explained by bias (e.g., lead time, length, overdiagnosis or volunteer bias). / Assessment
7. / External Validity
How likely are research results to be realized in the real world considering population and circumstances for care?
Review n, inclusions, exclusions, baseline characteristics and intervention methods ― this is a judgment call. / Assessment
8. / Patient Perspective
Consider benefits, harms, risks, costs, uncertainties, alternatives and satisfaction / Assessment
9. / Provider Perspective
Satisfaction, acceptability (includes adherence issues, potential for abuse, dependency issues), likely appropriate application and actionability (e.g., FDA approval, affordability, external relevance, circumstances of care, able to apply, tools available) / Assessment
10. / Overall / Assessment
www.delfini.org © 2002-2015 Delfini Group, LLC. All Rights Reserved Worldwide. Page 1 of 4