Oryan Henig, Dafna Yahav, Leonard Leibovici, Mical Paul

Guidelines for the treatment of pneumonia and UTI: evaluation of methodological quality using the AGREE-II instrument

Supplementary data

Table S1. AGREE II score items1 and interpretation specific to bacterial infections

AGREE II items / Criteria included in the item / Interpretation specific to bacterial infections
Domain 1. Scope and purpose
1. The overall objective(s) of the guideline is (are) specifically described. / • health intent
• expected benefit or outcome (of the guidelines)
• target(s) / The purpose/ objectives of the document are described:
· Scope of guidelines defined (prevention, antibiotic treatment, admission criteria, supportive treatment etc.)
· Expected benefits or change in patient outcomes expected when guidelines will be implemented
2. The health question(s) covered by the guideline is (are) specifically described / • intervention(s)
• comparisons
• outcome(s) (of the disease under consideration)
• health care setting or context / In the background or chapter describing the condition:
· Description and gold standard diagnosis of the disease (e.g. definition of pneumonia)
· The morbidity and mortality associated with the disease (patient relevant outcomes, outcome tree)
· Review of the epidemiology of the disease in the locale targeted by the guidelines, including bacterial distribution and antibiotic susceptibilities
3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described. / • target population, gender and age
• clinical condition
• severity of disease
• comorbidities
• excluded populations / Definition in background of
· Patients considered by the guidelines
· Patients populations excluded
· Patient subgroups for which treatment effects are expected to differ (can appear under the description of the condition)
Minimum characteristics that should be addressed include: age, place of infection acquisition, immune-compromise and infection severity. In addition:
For CAP: COPD/ chronic lung diseases,
For HAP/VAP: time in hospital
For UTI: sex, catheter, diabetes
Domain 2. Stakeholder involvement
4. The guideline development group includes individuals from all relevant professional groups and preferences have been sought. / For each member of the guideline development group, the following information is included:
• name
• discipline/content expertise
• institution
• geographical location
• a description of the member’s role in the guideline development group / The guidelines should address in text the authorship choice and expertise. The guideline development group should include experts from: infectious diseases, infection control, microbiology, pharmacology, community GPs, geriatrics, intensive care, acute care, and the relevant society (thoracic society for pneumonia and urological/ gynecological societies for UTI).
5. The views and preferences of the target population (patients, public, etc.) Have been sought. / • statement of type of strategy used to capture patients’/public’s’ views and preferences
• methods by which preferences and views were sought
• outcomes/information gathered on patient/public information
• description of how the information gathered was used to inform the guideline development process and/or formation of the recommendations / None
6. The target users of the guideline are clearly defined. / • clear description of intended guideline audience (e.g. specialists, family physicians, patients, clinical or institutional leaders/administrators)
• description of how the guideline may be used by its target audience (e.g., to inform clinical decisions, to inform policy, to inform standards of care) / None
Domain 3. Rigour of development
7. Systematic methods were used to search for evidence / • named electronic database(s) or evidence source(s) where the search was performed
• time periods searched
• search terms used
• full search strategy included / In the methods section:
· Sources and electronic databases defined and are acceptable for a systematic search strategy (PubMed search is not sufficient)
· Time periods defined
· The search strategy is given (without addressing its content)
8. The criteria for selecting the evidence are clearly described / Description of the inclusion criteria, including
• target population characteristics
• study design
• comparisons (if relevant)
• outcomes
• language
• context
Description of the exclusion criteria / Address the content of the search strategy and whether the items specified by AGREE are defined in the search strategy. The search strategy should address bacterial epidemiology and resistance
9. The strengths and limitations of the body of evidence are clearly described / Descriptions of how the body of evidence was evaluated for bias and how it was interpreted by members of the guideline development group:
• study design(s) included in body of evidence
• study methodology limitations (sampling, blinding, allocation concealment, analytical methods)
• appropriateness/relevance of primary and secondary outcomes considered
• consistency of results across studies
• direction of results across studies
• magnitude of benefit versus magnitude of harm
• applicability to practice context / · Definitions of evidence grading and level of recommendation in the methods section
· Methodological assessment of individual studies, within evidence grades (e.g. outcome selection in an RCT), in the context of the recommendations for antibiotic treatments
· Emphasis on “appropriateness/relevance of primary and secondary outcomes considered”. The guidelines should mention the outcome/s of relevance for the disease as specified in item #2 in the section on antibiotic treatment selection and describe how one treatment or the other is expected the affect the outcomes
10. The methods for formulating the recommendations are clearly described / • description of the recommendation development process (e.g., voting procedures)
• outcomes of the recommendation development process (e.g., extent to which consensus was reached)
• description of how the process influenced the recommendations (e.g., alignment with recommendations and the final vote) / Details on the technical procedures. Statements such as correspondence by emails, number of meetings etc. are not sufficient. Higher score for details on how a consensus was attained
11. The health benefits, side effects, and risks have been considered in formulating the recommendations / • supporting data and report of benefits
• supporting data and report of harms/side effects/risks
• reporting of the balance/trade-off between benefits and harms/side effects/risks
• recommendations reflect considerations of both benefits and harms/side effects/risks / Scored for antibiotic treatment recommendations
Description of the reasoning leading to the antibiotic choice must include:
· The effect of the antibiotic on outcomes as defined in item 2
· Adverse events of the antibiotic
· The ecological impact of the antibiotic compared to other antibiotic choices
12. There is an explicit link between the recommendations and the supporting evidence / • the guideline describes how the guideline development group linked and used the evidence to inform recommendations
• each recommendation is linked to a key evidence description/paragraph and/or reference list
• recommendations linked to evidence summaries, evidence tables in the results section of the guideline / · Reference to primary studies in the section presenting the antibiotic selections
· Antibiotic treatments are presented with the level of evidence and grade of recommendations.
13. The guideline has been externally reviewed by experts prior to its publication / • purpose and intent of the external review
• methods taken to undertake the external review
• description of the external reviewers
• outcomes/information gathered from the external review
• description of how the information gathered was used to inform the guideline development process and/or formation of the recommendations / IDEM
14. A procedure for updating the guideline is provided. / • a statement that the guideline will be updated
• explicit time interval or explicit criteria to guide decisions about when an update will occur
• methodology for the updating procedure is reported / IDEM
Domain 4. Clarity of presentation
15. The recommendations are specific and unambiguous / • statement of the recommended action
• identification of the intent or purpose of the recommended action (e.g., to improve quality of life, to decrease side effects)
• identification of the relevant population (e.g., patients, public)
• caveats or qualifying statements, if relevant (e.g., patients or conditions for whom the recommendations would not apply) / 1. Antibiotic treatment recommendations are stratified at least by (or refer to a certain subgroups of):
· Place of infection acquisition
· Age groups
· Infection severity (or alternatively location of treatment)
2. Antibiotic recommendations address duration of treatment
3. Recommendations address the epidemiological settings for which the recommendations are relevant for
16. The different options for management of the condition or health issue are clearly presented. / • description of options
• description of population or clinical situation most appropriate to each option / · Different antibiotic treatment options given with explanation on how to choose between the options, or with a ranking that allows for prioritizing treatment.
· Options address different epidemiological settings
· Alternatives that have not been tested in RCTs, but that are similar in coverage and activity are addressed
17. Key recommendations are easily identifiable / • description of recommendations in a summarized box, typed in bold, underlined, or presented as flow charts or algorithms
• specific recommendations are grouped together in one section / IDEM
Domain 5. Applicability
18. The guideline describes facilitators and barriers to its application / • identification of the types of facilitators and barriers that were considered
• methods by which information regarding the facilitators and barriers to implementing recommendations were sought
• information/description of the types of facilitators and barriers that emerged from the inquiry
• description of how the information influenced the guideline development process and/or formation of the recommendations / Discussion of expected difficulties in using and implementing the guidelines or circumstances that will allow for easy use/ application. Might include a survey of barriers and facilitators to guidelines implementation.
19. The guideline provides advice and/or tools on how the recommendations can be put into practice. / An implementation section in the guideline
Tools and resources to facilitate application:
• guideline summary documents
• links to check lists, algorithms
• links to how-to manuals
• solutions linked to barrier analysis
• tools to capitalize on guideline facilitators
• outcome of pilot test and lessons learned
• directions on how users can access tools and resources / Practical recommendation on guidelines application (when, how, where, electronic or manual etc.). The guidelines should address antibiotic stewardship programs and how to implement recommendations within antibiotic stewardship programs.
20. The potential resource implications of applying the recommendations have been considered / • identification of the types of cost information that were considered
• methods by which the cost information was sought
• information/description of the cost information that emerged from the inquiry
• description of how the information gathered was used to inform the guideline development process and/or formation of the recommendations / The direct costs of the recommended antibiotics (drug, administration and monitoring costs) mentioned and considered when selecting between treatment options
21. The guideline presents monitoring and/or auditing criteria / • identification of criteria to assess guideline implementation or adherence to recommendations
• criteria for assessing impact of implementing the recommendations
• advice on the frequency and interval of measurement
• descriptions or operational definitions of how the criteria should be measured / The guidelines provide recommendations on how to monitor adherence with recommendations.Antibiotic audits addressed. The guidelines address performance measures for the indicated disease (e.g. monitor time to antibiotic treatment)
Domain 6. Editorial independence
22. The views of the funding body have not influenced the content of the guideline / • the name of the funding body or source of funding (or explicit statement of no funding)
• a statement that the funding body did not influence the content of the guideline / The funding statement must include the funder and its role in the preparation and publication of the guidelines (including potential influence or a statement on lack of influence). The statement must include explanations on who funded the group meetings and travel.
23. Competing interests of guideline development group members have been recorded and addressed. / • description of the types of competing interests considered
• methods by which potential competing interests were sought
• description of the competing interests
• description of how the competing interests influenced the guideline process and development of recommendations / Competing interests mentioned and if there are competing interests, their possible effect (or lack of) on recommendations addressed

1 Taken from the AGREE-II user’s manual available at: http://www.agreetrust.org

Table S2. Scores of the 23 AGREE-II items for guidelines on pneumonia

(See item definitions in supplementary Table 1)

AGREE-II item / 1 / 2 / 3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12 / 13 / 14 / 15 / 16 / 17 / 18 / 10 / 20 / 21 / 22 / 23
Swedish society of Infectious Diseases
2012, CAP [1] / 2.3 / 6.0 / 4.3 / 3.3 / 1.0 / 2.3 / 5.0 / 4.3 / 3.3 / 1.7 / 4.0 / 3.7 / 1.0 / 2.7 / 5.7 / 5.0 / 6.0 / 1.0 / 1.3 / 1.0 / 1.0 / 6.3 / 7.0
SWAB /NVALT1
2012, CAP[2] / 5.3 / 5.3 / 6.7 / 6.7 / 1.0 / 3.3 / 7.0 / 5.0 / 5.3 / 5.0 / 5.7 / 7.0 / 1.3 / 3.0 / 6.7 / 6.3 / 6.7 / 1.0 / 1.0 / 1.0 / 7.0 / 1.3 / 3.7
ESCMID/ERS2
2011, LRTI[3] / 4.5 / 5.5 / 4.0 / 5.3 / 1.0 / 6.0 / 3.0 / 5.3 / 5.5 / 3.0 / 4.0 / 4.3 / 2.0 / 2.0 / 4.7 / 5.5 / 5.3 / 1.0 / 1.0 / 1.0 / 1.0 / 3.7 / 1.3
SEPAR3
2010, CAP[4] / 4.3 / 3.7 / 6.7 / 1.0 / 1.0 / 1.0 / 1.0 / 1.0 / 5.0 / 1.3 / 3.3 / 3.3 / 1.0 / 2.0 / 4.7 / 4.0 / 4.3 / 1.0 / 1.3 / 1.0 / 1.0 / 1.0 / 1.0
BTS4
2009, CAP[5] / 5.3 / 6.0 / 6.7 / 7.0 / 1.0 / 7.0 / 6.7 / 3.7 / 5.3 / 7.0 / 7.0 / 6.3 / 3.7 / 7.0 / 7.0 / 6.3 / 7.0 / 1.0 / 6.0 / 2.0 / 6.7 / 1.0 / 6.3
BTA5
2009, CAP[6] / 4.0 / 4.0 / 3.3 / 2.3 / 1.0 / 1.0 / 1.3 / 1.0 / 3.0 / 4.7 / 3.0 / 4.7 / 1.0 / 1.7 / 3.0 / 3.7 / 3.3 / 1.7 / 1.0 / 1.0 / 1.0 / 6.3 / 1.0
ABA6
2009, VAP[7] / 5.3 / 4.7 / 6.0 / 1.7 / 1.7 / 6.7 / 5.3 / 4.3 / 3.7 / 1.7 / 4.0 / 6.0 / 1.0 / 1.0 / 3.0 / 1.7 / 4.3 / 1.0 / 1.7 / 1.0 / 1.0 / 1.0 / 1.0
BSAC7
2008, HAP[8] / 4.7 / 5.3 / 6.3 / 6.3 / 5.7 / 1.0 / 5.7 / 2.7 / 6.7 / 5.3 / 4.3 / 6.0 / 6.7 / 2.7 / 5.7 / 5.0 / 4.3 / 2.0 / 2.3 / 1.3 / 1.7 / 3.0 / 3.3
South African Thoracic Society
2007, CAP[9] / 6.7 / 5.0 / 6.3 / 4.7 / 1.0 / 2.0 / 1.3 / 1.7 / 1.7 / 5.0 / 5.0 / 1.7 / 1.0 / 2.0 / 5.0 / 5.0 / 3.3 / 1.3 / 1.0 / 1.7 / 1.0 / 1.0 / 1.0
GCC8
2007, CAP[10] / 6.7 / 4.7 / 3.7 / 7.0 / 1.7 / 7.0 / 2.0 / 1.7 / 3.3 / 4.3 / 3.3 / 2.7 / 4.0 / 2.0 / 5.0 / 5.0 / 6.3 / 1.3 / 4.0 / 3.3 / 4.3 / 1.0 / 1.0
IDSA/ATS9
2007, CAP[11] / 6.0 / 6.0 / 6.7 / 5.3 / 3.0 / 7.0 / 1.7 / 2.0 / 5.7 / 6.7 / 6.0 / 5.3 / 5.3 / 1.7 / 7.0 / 5.7 / 7.0 / 2.3 / 3.7 / 1.3 / 7.0 / 5.7 / 3.3
IDSA/ATS9
2005, HAP/ VAP/ HCAP[12] / 5.7 / 6.3 / 6.7 / 5.3 / 1.0 / 3.0 / 2.3 / 1.7 / 4.7 / 6.0 / 4.7 / 5.3 / 1.0 / 2.3 / 5.3 / 5.3 / 7.0 / 1.0 / 5.3 / 1.0 / 5.3 / 6.7 / 3.7
JRS10
2004, HAP[13-23] / 5.7 / 3.7 / 2.3 / 1.3 / 1.0 / 5.7 / 1.0 / 1.0 / 1.0 / 1.7 / 2.3 / 1.0 / 5.0 / 4.0 / 3.0 / 3.7 / 4.3 / 1.0 / 1.0 / 1.0 / 1.7 / 1.0 / 1.0

1 The Dutch Working Party on Antibiotic Policy (SWAB)/Dutch Association of Chest Physicians (NVALT)