Systematic review and evaluation of physiological track and trigger warning systemsfor identifying at risk patients on the ward

Electronic Supplementary Material (ESM)

Search strategy for electronic databases

Ovid MEDLINE(R) <1966 to November Week 3 2004>

  1. *health status indicators/ or exp *"severity of illness index"/ or *sickness impact profile/ (8850 records)
  2. *Risk Assessment/ (5692)
  3. severity of illness ind$.tw. (90)
  4. health status ind$.tw. (514)
  5. risk assess$.tw. (11356)
  6. sickness impact profile$.tw. (753)
  7. early warning score$.tw. (9)
  8. ews.tw. (648)
  9. (track and trigger).tw. (21)
  10. modified early warning score$.tw. (7)
  11. mews.tw. (9)
  12. *Point-of-Care Systems/ (1295)
  13. point-of-care system$.tw. (38)
  14. point of care system$.tw. (38)
  15. ((trigger or calling) adj5 criteria).tw. (39)
  16. serious$ ill$.tw. (2790)
  17. or/1-16 (30375)
  18. exp *Critical Care/ (15634)
  19. critical care.tw. (7683)
  20. intensive care.tw. (41157)
  21. exp *Intensive Care Units/ (14487)
  22. (intensive care adj2 (unit$ or ward$ or clinic$ or department$ or section$)).tw. (29240)
  23. exp *Emergency Service, Hospital/ (14197)
  24. hospital emergency service$.tw. (155)
  25. medical emergency team$.tw. (36)
  26. met.tw. (44331)
  27. hospital emergency team$.tw. (2)
  28. patient emergency team$.tw. (0)
  29. exp *Patient Care Team/ (13621)
  30. patient care team$.tw. (135)
  31. patient at risk team$.tw. (2)
  32. patient-at-risk team$.tw. (2)
  33. par.tw. (3491)
  34. patient-at-risk.tw. (1703)
  35. (outreach adj (service$ or team$)).tw. (256)
  36. shock team$.tw. (2)
  37. or/18-36 (134622)
  38. 17 and 37 (1707)
  39. limit 38 to (human and "all adult (19 plus years)" and yr=1990-2005) (691)
  40. from 39 keep 1-691 (691)

Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations <January 14, 2005>

  1. health status ind$.tw. (17)
  2. severity of illness ind$.tw. (3)
  3. risk assess$.tw. (668)
  4. sickness impact profile$.tw. (13)
  5. early warning score$.tw. (0)
  6. ews.tw. (75)
  7. (track and trigger).tw. (2)
  8. modified early warning score$.tw. (0)
  9. mews.tw. (1)
  10. point of care system$.tw. (5)
  11. point-of-care system$.tw. (5)
  12. serious$ ill$.tw. (63)
  13. ((trigger or calling) adj5 criteria).tw. (1)
  14. or/1-13 (848)
  15. critical care.tw. (286)
  16. intensive care.tw. (1587)
  17. (intensive care adj2 (unit$ or ward$ or clinic$ or department$ or section$)).tw. (1272)
  18. hospital emergency service$.tw. (4)
  19. medical emergency team$.tw. (5)
  20. met.tw. (2114)
  21. hospital emergency team$.tw. (1)
  22. patient emergency team$.tw. (0)
  23. patient care team$.tw. (5)
  24. patient at risk team$.tw. (0)
  25. patient-at-risk team$.tw. (0)
  26. par.tw. (342)
  27. (outreach adj (service$ or team$)).tw. (15)
  28. shock team$.tw. (0)
  29. patient at risk$.tw. (69)
  30. patient-at-risk$.tw. (69)
  31. or/15-30 (4312)
  32. 14 and 31 (20)
  33. from 32 keep 1-20 (20)

Embase (NHS Dialog)

  1. Disease Severity.MJ
  2. Injury Scale.MJ
  3. Heath Survey.MJ
  4. Health Status.DE
  5. or/1-4
  6. ind$
  7. 5 and 6
  8. sickness impact profile$
  9. severity of illness (indices or index$)
  10. health status (indices or index$)
  11. early warning score$
  12. modified early warning score$
  13. track and trigger
  14. point of care system$
  15. point-of-care system$
  16. (trigger OR calling) criteria
  17. or/8-16
  18. 7 or 17
  19. Intensive Care.DE
  20. Intensive Care Unit.DE
  21. Emergency Health Service.DE
  22. critical care OR intensive care
  23. hospital emergency service$
  24. medical emergency service$
  25. hospital emergency team$
  26. patient emergency team$
  27. patient at risk team$
  28. patient-at-risk team$
  29. outreach (service$ OR team$)
  30. shock team$
  31. or/19-30
  32. 18 and 31

CINAHL - Cumulative Index to Nursing & Allied Health Literature <1982 to December Week 2 2004>

  1. *Health Status Indicators/ (558)
  2. *"Severity of Illness Indices"/ (218)
  3. *Sickness Impact Profile/ (16)
  4. *Risk Assessment/ (1616)
  5. severity of illness ind$.tw. (51)
  6. health status ind$.tw. (111)
  7. risk assess$.tw. (1424)
  8. sickness impact profile$.tw. (593)
  9. early warning score$.tw. (1)
  10. ews.tw. (10)
  11. (track and trigger).tw. (6)
  12. modified early warning score$.tw. (1)
  13. mews.tw. (6)
  14. point of care system$.tw. (15)
  15. point-of-care system$.tw. (15)
  16. serious$ ill$.tw. (492)
  17. ((trigger or calling) adj5 criteria).tw. (7)
  18. or/1-17 (4566)
  19. *Critical Care/ (2184)
  20. critical care.tw. (5564)
  21. intensive care.tw. (6769)
  22. *Intensive Care Units/ (1528)
  23. (intensive care adj2 (unit$ or ward$ or clinic$ or department$ or section$)).tw. (5075)
  24. exp *Emergency Service/ (4138)
  25. hospital emergency service$.tw. (12)
  26. medical emergency team$.tw. (9)
  27. met.tw. (5084)
  28. hospital emergency team$.tw. (0)
  29. patient emergency team$.tw. (0)
  30. *Multidisciplinary Care Team/ (2428)
  31. patient care team$.tw. (44)
  32. patient at risk team$.tw. (1)
  33. patient-at-risk team$.tw. (1)
  34. par.tw. (290)
  35. patient-at-risk.tw. (271)
  36. patient at risk.tw. (271)
  37. (outreach adj (service$ or team$)).tw. (149)
  38. shock team$.tw. (0)
  39. or/19-38 (25007)
  40. 18 and 39 (244)
  41. limit 40 to ((adult <19 to 44 years> or middle age <45 to 64 years> or aged <65 to 79 years> or "aged <80 and over>") and yr=1990 - 2005) (98)
  42. from 41 keep 1-98 (98)

PsychInfo

  1. health status ind* (116)
  2. severity of illness ind* (24)
  3. sickness impact profile* (196)
  4. risk assess* (1849)
  5. early warning score* (0)
  6. ews (17)
  7. track and trigger (2)
  8. modified early warning score* (0)
  9. mews (11)
  10. point of care system* (4)
  11. point-of-care system* (0)
  12. (trigger or calling) near criteria (52)
  13. critical care (411)
  14. intensive care (1110)
  15. intensive near (unit* or ward* or clinic* or department* or section*) (1808)
  16. hospital emergency service* (37)
  17. medical emergency team* (3)
  18. hospital emergency team* (0)
  19. patient emergency team* (0)
  20. patient care team* (11)
  21. patient at risk team* (0)
  22. patient-at-risk team* (0)
  23. patient at risk (41)
  24. patient-at-risk (0)
  25. outreach near (service* or team*) (699)
  26. shock team* (1)
  27. #1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9 or #11 or #12 (2266)
  28. #13 or #14 or #15 or #16 or #17 or #18 or #19 or #20 or #21 or #22 or #23 or #24 or #25 or #26 (3179)
  29. #27 and #28 (27)

(Note: “#1 or #2 or #3” indicates that all the results from searches 1 to 3 should be returned and is equivalent to the MEDLINE syntax “or/1-3”)

Cochrane Library (Wiley InterScience website)

  1. Health Status Indicators (MeSH)
  2. APACHE (MeSH)
  3. Severity of Illness Index (MeSH)
  4. Sickness Impact Profile (MeSH)
  5. Karnofsky Performance Status (MeSH)
  6. or/1-5
  7. Emergency Service, Hospital (exp MeSH)
  8. Point-of-Care Systems (exp MeSH)
  9. Critical Care (exp MeSH)
  10. medical emergency team*
  11. hospital emergency team*
  12. patient at risk team*
  13. patient-at-risk team*
  14. outreach team*
  15. outreach service*
  16. or/7-15
  17. 6 and 16

Web of Science

  1. severity of illness ind*
  2. health status ind*
  3. risk assess*
  4. sickness impact profile*
  5. early warning score*
  6. ews
  7. modified early warning score*
  8. mews
  9. track and trigger
  10. point of care system*
  11. point-of-care system*
  12. (trigger or calling) SAME criteria
  13. or/1-13
  14. critical care
  15. intensive care
  16. hospital emergency service*
  17. medical emergency team*
  18. hospital emergency team*
  19. patient emergency team*
  20. patient care team*
  21. patient at risk team*
  22. patient-at-risk team*
  23. outreach SAME (service* or team*)
  24. or/14-23
  25. 13 and 24

List of professional bodies and experts contacted toreview the completeness ofthe list of identified papers

  • Intensive Care Society
  • RCN Critical Care Forum
  • RoyalCollege of Anaesthetists
  • British Association of Critical Care Nurses
  • National Outreach Forum
  • RoyalCollege of Physicians
  • RoyalCollege of Physicians and Surgeons of Glasgow
  • RoyalCollege of Surgeons of England
  • Steering Group members/research team (n = 21)
  • Other clinical experts (n = 9)

Table S1: Modified DoCDat criteria for assessing the coverage (A–D) and accuracy (E–F) of physiological track and trigger warning system databases

Level 1 / Level 2 / Level 3 / Level 4
A. Was the spectrum of patients representative of the patients who will be monitored with the TT in practice? / No evidence or unlikely to be representative.
(e.g. only patients referred to CCOS) / Some evidence that eligible population is representative.
(e.g. all patients seen by CCOS with CC follow-up can be identified) / Good evidence that eligible population is representative.
(e.g. all patients on selected wards) / Total population in your current setting.
(e.g. all patients on all wards that could be attended by CCOS)
B. Were selection criteria clearly defined? / No / Yes
C. Completeness of TT variables / Only summary TT variables or scores. / Summary TT variables or scores;
at least admission to CC and death were recorded as minimum outcomes / Raw physiological data;
at least admission to CC and death were recorded. / Raw physiological data;
all outcomes;
important confounders
D. Completeness of data (% variables at least 95% complete) / Few (<50%) / Some (50-79%) / Most (80-97%) / All or almost all (>97%)
E. Use of explicit definitions and rules for variables / None / Some (<50%) / Most (50 -97%) / All or almost all (>97%)
F. Extent to which data are validated / No validation / Range or consistency checks / Range and consistency checks / Range and consistency checks plus external validation using alternative source

CC: critical care; CCOS: critical care outreach service; TT: physiological track and trigger warning system.

Table S2: Sensitivity and specificity of physiological track and trigger warning systems by hospital and patient subgroup

Hospital / Sub-groups / Patients
n (%) / Sensitivity
(95% CI) / Specificity
(95% CI) / PPV
(95% CI) / NPV
(95% CI) / Prevalence
(95% CI)
A / CC follow-up / 701 (74.1) / 32.8
(21.3,46.0) / 85.5
(82.5,88.1) / 17.7
(11.2,26.0) / 93.0
(90.7,95.0) / 8.7
(6.7,11.0)
Referral / 245 (25.9) / 48.2
(41.8, 54.6)
B / CC follow-up / 209 (44.4) / 42.9
(17.7,71.1) / 95.4
(91.2,98.0) / 42.9
(17.7,71.1) / 95.4
(91.2,98.0) / 7.4
(4.1,12.1)
Referral / 262 (55.6) / 31.8
(26.4,37.6)
C / Referral / 405 (100) / 26.4
(22.2, 31.0)
D / CC follow-up / 1098 (46.3) / 3.2
(0.1,16.7) / 99.2
(98.4,99.6) / 10.0
(0.3,44.5) / 97.2
(96.1,98.1) / 2.8
(1.9,4.0)
Referral / 1273 (54.7) / 35.1
(32.5, 37.8)
E / CC follow-up / 1119 (34.3) / 43.3
(32.9,54.2) / 90.4
(89.0,91.7) / 17.4
(12.7,23.0) / 97.2
(96.3,97.9) / 4.5
(3.6,5.4)
Referral / 2149 (65.7) / 43.2
(40.0,46.3)
F / CC follow-up / 289 (87.6) / 60.9
(48.4,72.4) / 64.2
(57.5,70.6) / 35
(26.5,44.2) / 83.8
(77.4,89.1) / 24
(19.2,29.4)
Referral / 41 (12.4) / 29.3
(16.1,45.5)
G / MAU patients / 750 (100) / 65.1
(49.1,79.0) / 65.8
(62.1,69.3) / 10.4
(7.0, 4.6) / 96.9
(94.9,98.2) / 5.7
(4.2,7.6)
H / All CCOS / 1051 (100) / 36.7
(33.6,39.8)
I / Referral* / 2463 (100) / 64.7
(62.8,66.6)
J / CC follow-up / 1512 (78.4) / 15.8
(9.8,23.6) / 99.1
(98.5,99.6) / 61.3
(42.2,78.2) / 93.2
(91.8,94.4) / 7.9
(6.6,9.4)
Referral / 417 (21.6) / 54.7
(49.8,59.5)
K / CC follow-up / 323 (85.0) / 69.2
(48.2,85.7) / 89.5
(85.4,92.7) / 36.7
(23.4,51.7) / 97.0
(94.3,98.7) / 8.1
(5.4,11.7)
Referral / 57 (15.0) / 40.4
(27.6,54.2)
L / CC follow-up / 240 (70.8) / 100
(89.7,100) / 14.3
(6.7,25.4) / 38.6
(28.4,49.6) / 100
(66.4,100) / 35.1
(25.6,45.4)
Referral / 99 (29.2) / 52.5
(46.0,59.1)
M / MAU patients
(no CCOS) / 1672 (72.0) / 25.4
(15.8,37.1) / 92.9
(90.5,94.9) / 30.5
(19.2, 43.9) / 91.0
(88.4,93.2) / 10.9
(8.6,13.6)
MAU patients
(with CCOS) / 649 (28.0) / 19.1
(14.0,25.0) / 95.4
(94.2,96.4) / 38.0
(28.8,47.8) / 88.9
(87.2,90.4) / 12.9
(11.3,14.6)
N / CC follow-up / 520 (20.4) / 84.3
(71.4,93.0) / 47.3
(42.7,52.0) / 14.9
(11.0,19.5) / 96.5
(93.2,98.5) / 9.8
(7.4,12.7)
Referral / 2028 (79.6) / 35.5
(33.4,37.6)
O / CC follow-up / 412 (69.6) / 78.0
(65.3,87.7) / 50.4
(41.1,59.7) / 43.8
(34.1,53.8) / 82.2
(71.5,90.2) / 33.1
(26.3,40.6)
Referral / 180 (30.4) / 61.1
(56.2,65.9)

*Includes 1167 (47.4%) CC admissionsnot seen by the CCOS; CC: critical care; CCOS: critical care outreach service; MAU: Medical Admissions Unit; NPV: negative predictive value; PPV: positive predictive value.

Table S3: Results of meta-regression on log diagnostic odds ratio (lnDOR)

Physiological parameters* / Coefficient / 95% confidence interval
Temperature / -1.2 / (-4.4, 1.9)
Urine output / -0.1 / (-1.9, 1.8)
Oxygen saturation / 0.2 / (-1.7, 2.1)
Outcome variables† / Coefficient / 95% confidence interval
Cardiopulmonary resuscitation / 1.2 / (-1.4, 3.9)
Do not attempt resuscitation / 0.7 / (-1.9, 3.4)
Patient group / Coefficient / 95% confidence interval
Critical care follow-up patients‡ / 1.0 / (-1.6, 3.7)

*The following physiological parameters were included in all systems: heart rate, blood pressure, respiratory rate, conscious level; The following physiological parameters were included in a single system only: respiratory support, acid base disturbance;†The following outcomes were recorded in all datasets: admission to critical care, death; ‡Comparison group: all Medical Admissions Unit patients.

1

Figure S1: Receiver operator characteristic (ROC) curves for composite outcome in critical care follow-up patients

1

Figure S2: Forest plot of logdiagnostic odds ratio (lnDOR)

The size of each square is inversely proportional to the variance of lnDOR.The horizontal lines are 95% confidence intervals for lnDOR.

1