Accuracy of quick- Sequential Organ Failure Assessment (q-SOFA) score and systemic inflammatory response syndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: A meta-analysis of observational studies
Souvik Maitra, Anirban Som, Sulagna Bhattacharjee
e supplement 1: Search strategy
PubMed search strategy
Quick[All Fields] AND Sequential[All Fields] AND ("Organ"[Journal] OR "organ"[All Fields]) AND Failure[All Fields] AND ("Assessment"[Journal] OR "assessment"[All Fields])
quick[All Fields] AND sofa[All Fields]
q-SOFA[All Fields]
3 Organ Dysfunction Scores/ or Emergency Service, Hospital/ or Sepsis/ or Quick Sequential Organ Failure Assessment.mp. or Adult/
EMBASE search strategy
'qsofa sepsis':ti,ab,kw OR 'q-sofa':ti,ab,kw OR 'qsofa':ti,ab,kw OR 'quick sofa':ti,ab,kw OR 'quick sequential organ failure assessment score':ti,ab,kw
e supplement 2: Statistical methods
Following statistical methods were used: sensitivity=tp/(tp+fn); specificity=tn/(tn+fp); 95% confidence intervals (CIs); overall accuracy=(tp+tn)/(tp+fn+fp+tn); positive predictive value (PPV)=tp/(tp+fp); negative predictive value (NPV)=tn/(tn+fn); positive likelihood ratio (PLR)=(tp/(tp+fn))/(fp/(fp+tn))=sensitivity/(1-specificity); negative likelihood ratio (NLR)=(fn/(tp+fn))/(tn/(tn+fn))=(1-sensitivity)/specificity; diagnostic odds ratio (DOR)=(tp*tn)/(fp*fn); Initially a bivariate random effects regression model for diagnostic meta-analysis was used to obtain weighted summary estimates of the sensitivity and specificity of q-SOFA and SIRS criteria. The bivariate approach assumes a bivariate distribution for the logit-transformed sensitivity and specificity and respects binomial data structure. We calculated performance statistics, which includes pooled sensitivity, specificity, positive and negative likelihood ratios for each study and also at the pooled level with the midas command in STATA (STATA SE 12.0, STATA Corp, College Station, TX, USA). Likelihood ratio denotes the magnitude by which probability of death in the hospital or within 30-days in a given patient is modified by the result of the scoring systems. It incorporates both sensitivity and specificity, and less affected by the prevalence of sepsis. Pooled diagnostic odds ratio (OR) was also calculated by random effects model by metandi command. Diagnostic OR is regarded as a performance indicator that summarizes the diagnostic accuracy of the index test as a single number that describes how many times greater the chance is of getting a positive result in a person with the disease than in someone without the disease. It incorporates both sensitivity & specificity and higher value of DOR indicates greater overall accuracy.
Quantification of heterogeneity11 was done by I2 value that was obtained by midas command in STATA.
A hierarchical summary ROC (HSROC) 12 curve was generated by metandiplot command in STATA (Stata SE 12.0, STATA Corp, College Station, TX, USA). Cook’s distance is a measure of the influence of a study on the model parameters and can be used to check for particularly influential studies. A typical cut-point for declaring a value of Cook’s D to be “large” is four times the number of parameters divided by the number of studies. Cook’s distance was calculated by predict command after metandi (predict cooksd, cooksd).
A summary ROC curve with AUC and 95% CI was also calculated with midas command. Clinical utility of a diagnostic test was evaluated using the likelihood ratios to calculate post-test probability (PTP) based on Bayes’ theorem as follows:
Pretest Probability=Prevalence of target condition; PTP= LR × pretest probability/ [(1- pretest probability) × (1-LR)]. A Fagan nomogram was also generated to graphically plot pretest and posttest probability. We did not calculate Q point, as it is not recommended in Cochrane methodology.
Possible reasons for heterogeneity among different studies were searched for using a regression model (midas tp fp fn tn, reg) and following covariates were used: sample size, clinical setting, prevalence of disease, age of the patients, cut of value of the index test. Publication bias was tested by visual inspection of funnel plot and Deeks’ regression model13 with the midas pubbias command.
e figure 1: Summary of methodological quality in the included studies
e table 1: Characteristics of the included studies
Author / Year / Country / Center / Setting / Design / Sample size / Patients / Primary outcomeAsai / 2017 / Japan / Single / Hospital / Retrospective cohort (2016) / 78 / Adult patients with health care associated with pneumonia / In-hospital/ 30 day mortality
Askim / 2017 / Norway / Single / ED / Retrospective cohort (January 2012- December 2012) / 1535 / Patients (age>16 yrs) presenting with suspected infection / Usefulness of qSOFA as a risk stratification tool
Amland / 2017 / US / Multi / Hospital / Retrospective cohort (January- March 2016) / 5992 / Adult patients meeting the definition of
Sepsis-3 suspected infection / compare the utility of qSOFA to the St. John Sepsis Surveillance Agent among patients with suspected infection
Burnham / 2017 / US / Single / Hospital / Retrospective cohort (2009-13) / 510 / Enterobacteriaceae sepsis / Prediction of mortality as per qSOFA and Sepsis classification
Chen / 2016 / China / Single / ED / Retrospective cohort (2012-14) / 1641 / Adult patients with pneumonia / 28-day mortality, hospitalisation and ICU admission with regard to qSOFA, CRB and CRB-65 scores
Churpek / 2016 / US / Single / ED & hospital ward / Retrospective cohort (2008-16) / 30677 / Patients with suspected infection / qSOFA, SIRS, MEWS, and NEWS compared for predicting death and ICU transfer
Costa / 2018 / Brazil / Single / ICU / Retrospective cohort (2014-15) / 450 / Patients with sepsis admitted in ICU / In-hospital mortality
del Castillo / 2017 / Spain / Multi / ED / Retrospective cohort (2015- 16) / 1071 / Older (>75 yrs) with suspected infection / Accuracy of the SIRS criteria and the qSOFA and GYM scores to predict all-cause 30-day mortality
Donnelly / 2017 / US / Not applicable / Community database / Retrospective analysis of longitudinal cohort (2003- 07) / 2593 / Patients who had hospital admissions attributed to serious infection. / Accuracy of SIRS, qSOFA and SOFA for prediction of mortality
Estella / 2018 / Spain / Multi / ED / Prospective / 1662 / Adult (age>75y) patients with acute infection / Prognostic performance of qSOFA ≥ 2 for the prediction of 30-day-mortality according to the site of infection.
Finkelsztein / 2017 / US / Single / ED/ Hospital ward / Retrospective cohort / 152 / Adult patients with suspected infection / Accuracy of qSOFA & SIRS for predicting mortality & ICU free days
Forward / 2017 / Australia / Single / Hospital ward / Retrospective cohort (May- August 2015) / 161 / Adult patients with suspected infection / Predictive validity of qSOFA and SIRS for in-hospital mortality/ ICU admission/ positive blood culture
Freund / 2017 / Europe / Multi / ED / Prospective cohort (May- June 2016) / 879 / Adult patients with suspected infection / Predictive validity of qSOFA, SOFA and SIRS for in-hospital mortality
Fukushima / 2017 / Japan / Single / ED/ hospital ward / Retrospective cohort (2012- 17) / 141 / Adult patients with acute pyelonephritis / In-hospital mortality
Giamarellos-Bourboulis / 2016 / Greece / Multi / ICU/ hospital ward / Retrospective cohort (2006- 16) / 3436 (non-ICU)
1058 (ICU) / Adult patients with infections and at least two signs of
SIRS / Sensitivity of qSOFA and sepsis definition
to predict mortality
Goulden / 2018 / UK / Single / ED / Retrospective cohort (2016- 17) / 1818 / Adult patients with suspected sepsis / In-hospital mortality
Groot / 2017 / The Netherlands / Multi / ED / Retrospective cohort (2011-15) / 2280 / Adult patients with suspected infection / Association between qSOFA, PIRO, MEWS and NEWS with in-hospital mortality
Guirgis / 2017 / USA / Single / ED / Retrospective cohort (2013- 16) / 3297 / Adult patients
with a diagnosis of sepsis, and with a primary discharge
diagnosis of sepsis / In-patient mortality
Hayder / 2017 / US / Single / ED / Retrospective cohort (2014- 15) / 199 (A random sample was taken from a database) / Adult patients with suspected infection / Sensitivity of the qSOFA score in diagnosing
sepsis
Henning / 2017 / US / Multi / ED / Secondary analysis of 3 prospective cohort (2003- 06) / 7637 / Adult patients with suspected infection / sensitivity and specificity of qSOFA score
for in-hospital mortality
Ho / 2017 / Australia / Single / ICU / Analysis of a prospectively collected data (2008- 13) / 2322 / Adult patients who are not intubated & sedated / Prognostic significance of qSOFA for in-hospital mortality
Huson / 2016 / Gabon / Single / Hospital ward / Analysis of a prospective cohort (2012- 13) / 329 / Adult patients with two SIRS criteria / Prognostic accuracy of qSOFA for in-hospital mortality
Huson / 2017 / Malawi / Single / Hospital ward / Prospective cohort (November- December 2016) / 518 / Adult patients with suspected infection / Prognostic accuracy of qSOFA for in-hospital mortality
Hwang / 2017 / Korea / Single / ED / Retrospective cohort (2008-14) / 1395 / Adult patients who met the criteria for severe sepsis or septic shock / Performance of positive qSOFA score for predicting 28-day mortality
Jouffroy / 2017 / France / Single / Registry of calls / Retrospective cohort (April- May 2011) / 141 / Adult patients with suspected infection / Admission to an
ICU, either directly after intervention
Khwannimit / 2018 / Thailand / Single / Sepsis registries of patients / Retrospective cohort / 2350 / Adult patients with sepsis / All cause hospital mortality
Kim / 2017 / Korea / Single / ED / Retrospective cohort / 125 / Adults (> 18 years of age) with a discharge diagnosis of CAP / Predictive accuracy of SOFA, qSOFA, APACHE II for 28-day mortality
LeGuen / 2018 / Australia / Single / ED / Prospective cohort (2016) / 97 / All patients with suspected infection receiving rapid response call / In-hospital mortality
Minsoo / 2017 / Korea / Single / ED / Retrospective analysis of prospective registry (January- December 2015) / 615 / Adult patients with febrile neutropenia / Predictive
performance of the qSOFA score as a screening tool for
sepsis, mortality, and ICU admission
Moore / 2017 / Africa / Multi / Hospital ward / Pooled analysis of hospital based cohort (2009-15) / 5573 / Adult patients / Comparison of the performance
of the UVA score with that of MEWS and qSOFA,
Moskowith / 2017 / US / Single / ED / Retrospective cohort (2010-14) / 24164 / Adult patients with suspected infection / Requirement of received critical care intervention within 48 hours
Muller / 2017 / Switzerland / Single / ED / Retrospective cohort (2011- 13) / 527 / Adult patients with pneumonia / Prognostic performance of qSOFA for in-hospital mortality
Park / 2017 / Korea / Single / ED / Retrospective cohort (2007-16) / 1009 / Adult patients with a suspected infection / Predictive accuracy of qSOFA for new organ dysfunction
Peake / 2017 / Australia / Multi / ED / Retrospective analysis of a RCT (2008-14) / 1591 / Adult patients with early septic shock / Predictive accuracy of qSOFA and Sepsis-3 definition for mortality
Piano / 2017 / Italy / Multi / Hospital ward / Retrospective cohort (2011- 16) / 259 / Adult cirrhosis patients with suspected infection / prognostic accuracy of Sepsis-3 criteria and qSOFA
Quinten / 2017 / The Netherlands / Single / ED / Retrospective cohort (2012-14) / 193 / Adult patients with suspected infection with two SIRS criteria / Predictive accuracy of PIRO and qSOFA for ICU admission
Raith / 2017 / Australia/ Nz / Multi / ICU / Retrospective cohort (2000-15) / 184875 / Adult patients with infection related primary diagnosis / Prognostic accuracy of SOFA, qSOFA and SIRS for in-hospital mortality
Rannikko / 2017 / Finland / Single / ED / Retrospective cohort (2012-14) / 497 / Adult patients with blood culture positive sepsis / Hospital mortality
Ranzani / 2017 / Spain / Multi / ED / Retrospective cohort (1996-2015) / 6874 / Adult patients with clinical diagnosis of CAP / All cause mortality
Seymour / 2017 / International / Multi / ED/ICU / Retrospective cohort (2010- 12) / 74454 (validation cohort) / Adult patients with suspected infection / In-hospital mortality
Singer / 2016 / US / Single / ED / Retrospective review (2014- 15) / 22530 / Adult patients with suspected infection / In-hospital mortality
Szakmany / 2017 / UK / Multi / ED / Prospective / 380 / Adult patients with suspected sepsis / 30- day mortality
Umemura / 2017 / Japan / Multi / ICU / Retrospective review (2010-11) / 624 / Adult patients with severe sepsis / Predictive validity of qSOFA for in-hospital mortality
Wang / 2016 / China / Single / ED / Retrospective cohort (July- December 2015) / 477 / Adult patients with clinically suspected infection / Performance of the qSOFA in predicting mortality and ICU admission
Williams / 2016 / Australia / Single / ED / Retrospective analysis of prospective cohort (2007-11) / 8871 / Adult patients with suspected or potential infection / 30- day mortality
e table 2: Reported prognostic accuracy of qSOFA for in-hospital mortality/ 30-day mortality in the included studies
Author / Year / Estimated mean age of the population / Baseline risk of mortality / AUCROC (95% CI)Asai / 2017 / 80 / 0.077 / NR
Askim / 2017 / 62 / 0.044 / 0.55(NR)
Amland / 2017 / 65 / 0.037 / 0.67(0.65- 0.68)
Burnham / 2017 / 60 / 0.0131 / 0.72(0.66- 0.77)
Chen / 2016 / 73 / 0.333 / 0.65(0.63- 0.68)
Churpek / 2016 / 58 / 0.054 / 0.69(0.67- 0.70)
Costa / 2018 / 59.6 / 0.590 / 0.69(0.64- 0.74)
del Castillo / 2017 / 84 / 0.067 / 0.69(0.61- 0.76)
Donnelly / 2017 / 65 / 0.063 / 0.76(NR)
Estella / 2018 / 84.7 / 0.101 / 0.73(NR)
Finkelsztein / 2017 / 64 / 0.284 / 0.74(0.66- 0.81)
Forward / 2017 / NR / 0.155 / NR
Freund / 2017 / 67 / 0.084 / 0.80(0.74- 0.86)
Fukushima / 2017 / 73 / 0.078 / 0.78(0.64- 0.92)
Giamarellos-Bourboulis / 2016 / 0.252 / NR
Goulden / 2018 / 68 / 0.146 / 0.62(0.59- 0.66)
Groot / 2017 / 61 / 0.062 / 0.68(0.63- 0.72)
Guirgis / 2017 / 59 / 0.101 / 0.68 (NR)
Hayder / 2017 / 71 / 0.110 / 0.68(0.58- 0.78)
Henning / 2017 / 58 / 0.044 / NR
Ho / 2017 / 57 / 0.120 / 0.67(0.64- 0.71)
Huson / 2016 / 34 / 0.046 / 0.80(0.74- 0.93)
Huson / 2017 / 35 / 0.231 / 0.73(0.68- 0.78)
Hwang / 2017 / 65 / 0.163 / 0.60(0.57- 0.63)
Jouffroy / 2017 / 64 / 0.085 / NR
Khwannimit / 2018 / 62 / 0.445 / 0.81(0.80- 0.83)
Kim / 2017 / 68 / 0.104 / 0.81(0.73- 0.87)
Minsoo / 2017 / 54 / 0.325 / 0.65(0.51- 0.79)
Moore / 2017 / 36 / 0.173 / 0.69(0.67- 0.72)
Moskowith / 2017 / 64 / 0.049 / 0.71(0.69- 0.72)
Muller / 2017 / 66 / 0.133 / 0.58(0.52- 0.66)
Park / 2017 / 67 / 0.158 / 0.71(0.64- 0.83)
Peake / 2017 / 63 / 0.151 / NR
Piano / 2017 / 61 / 0.187 / 0.78(0.72- 0.84)
Quinten / 2017 / 60 / 0.041 / 0.82(0.71- 0.94)
Raith / 2017 / 63 / 0.187 / 0.61(0.60- 0.611)
Rannikko / 2017 / 63 / 0.197 / NR
Ranzani / 2017 / 66 / 0.062 / 0.70(0.67- 0.72)
Seymour / 2017 / 61 / 0.043 / 0.66(0.64- 0.68) [ICU]
0.81(0.80- 0.82) [NON- ICU]
Singer / 2016 / 54 / 0.012 / 0.76(0.73- 0.78)
Szakmany / 2017 / 74 / 0.205 / 0.57(0.49- 0.64)
Umemura / 2017 / 73 / 0.242 / 0.62(0.56- 0.67)
Wang / 2016 / 0.275 / 0.67(0.61- 0.72)
Williams / 2016 / 49 / 0.037 / 0.78(0.76- 0.81)
e figure 2: Forest plot showing sensitivity and specificity of qSOFA2 for predicting in-hospital or 30-day mortality in non-ICU patients at individual study level and pooled analysis level
e figure 3: Univariate meta-regression analysis for baseline risk of mortality, sample size and estimated mean age of the patients (Data from the non-ICU patients have been included here)
e figure 4: Fagan’s nomogram for pre-test and post-test probability of in-hospital mortality in non-ICU patients when qSOFA2
e figure 5: Deek’s funnel plot asymmetry test for publication bias for in pooled analysis of diagnostic odds ratio for qSOFA2 for predicting in-hospital mortality (Data from non-ICU patients included here)
e figure 6: A summary ROC curve (bivariate model) for pooled analysis of sensitivity and specificity of qSOFA2 for predicting in-hospital mortality [Data from both ICU and non-ICU patients included here]
1