Electronic Supplementary Material (ESM).
PROGNOSTIC VALUE OF SOMATOSENSORY EVOKED POTENTIALS IN COMATOSE CHILDREN: A SYSTEMATIC LITERATURE REVIEW
1,2Riccardo Carrai, MD; 1,3Antonello Grippo, MD; 1,4Lori Silvia, MD; 1Francesco Pinto, MD; 1Aldo Amantini, MD.
1SOD Neurofisiopatologia – DAI Scienze Neurologiche, – Azienda Ospedaliera Universitaria Careggi – Firenze, Italia;
2UO Riabilitazione Respiratoria, Fond. Don C. Gnocchi ONLUS – IRCCS Pozzolatico, Firenze, Italia;
3UO Riabilitazione Neurologica Fond. Don C. Gnocchi ONLUS – IRCCS Pozzolatico, Firenze, Italia.
4Neurologia Pediatrica, Azienda Ospedaliera Universitaria “A. Meyer”, Firenze, Italia;
SOD Neurofisiopatologia – DAI Scienze Neurologiche, Azienda Ospedaliera Universitaria Careggi; Viale Morgagni 85, 50134, Firenze, Italia
Tel: +390557949410
FAX: +39055794409
Figure ESM 1. Flow charts used to identify studies for analysis
Reasons for exclusion of some papers that met the inclusion criteria from analysis (bibliography numbers referred to main article)
Fourteen articles met inclusion criteria (Fig ESM 1) but for some papers data were not complete as indicated below:
1) In Goodwin’s article [25], 34 out of 41 cases were available for analysis (four had no SEP data and three died of uncertain causes).
2) In White’s [43] article, only 59 out of 62 subjects were considered (three non-neurological deaths).
3) We did not calculate the predictive value for Parain’s [32], Chéliout-Heraut’s [19] and Wohlrab’s articles [18] because 2×2 tables showed two cells with zero value.
4) With regard to Carter’s studies [14,20], only the first one was included in our analysis because the patients’ characteristics were similar in two papers and in the second [14] we were not able to find data to generate 2×2 tables.
At last eleven studies were analysed.
Quality of the studies
Figure ESM 2 Quality study. Methodological quality graph: Review authors’ judgments about each methodological quality item presented as percentages across all included studies.
Figure ESM 3. Quality study. Methodological quality summary: Review authors’ judgments about each methodological quality item for each included study.
Meaning of Sensitivity, Specificity, Positive Predictive Value and Negative Predictive Value
According to Sackett et al. [1] and Dinnes et al [2], we defined the predictive values as indicated below:
STUDY CONDICTIONTEST / Present / Absent / Total
Positive / TP / FP / TP+FP
Negative / FN / TN / FN+TN
Total / TP+FN / FP+TN / TP+FP+FN+TN
TP, true positive; FP, false positive, FN, false negative; FP, false positive; TN, true negative
Sensitivity = TP/ (TP+FN) => Proportion of subjects with study condition who have positive test respect to all subjects with study condition
Specificity = TN/ (TN+FP) => Proportion of subjects without study condition who have negative test respect to all subjects without study condition
Positive Predictive Value = TP/(TP+FP) => Proportion of subjects with study condition who have positive test respect to all subjects with positive test
Negative Predictive Value = TN/(TN+FN) => Proportion of subjects without study condition who have negative test respect to all subjects with negative test
Positive likelihood ratio (sensitivity/(1-specificity) => It is the ratio of the probability of the specific test result in people who do have the study condition to the probability in people who do not
Diagnostic odds ratio = (TP x TN)/(FN x TP) => it describes the ratio of the odds of a positive test result in a patient with disease compared with a patient without disease).
Pre-test probability = (TP+FN)/(TP+FP+TN+TN).
Post-test probability = odds post-test/(odds post-test+1).
The predictive values of SEP were estimated by calculating the relationship between the presence (or the absence) of cortical SEP and the outcome of patients (PPV, NPV). The PPV for a favourable outcome estimates the percentage of patients who will recover when a cortical SEP is evoked. In contrast, the NPV for an unfavourable outcome estimates the percentage of patients who will not recover when no cortical SEP is evoked. The relationship between SEP and outcome can also be assessed from the outcome (SENS, SPE). The SEN for an unfavourable outcome (or the SPE for a favourable outcome) estimates the percentage of patients who will not have a cortical SEP when the outcome is bad. In contrast, the SEN for a favourable outcome (or the SPE for an unfavourable outcome) estimates the percentage of patients who will have a cortical SEP when the outcome is good.
For clarity the meaning of various predictive values in each analysis are indicated in the following tables (Table ESM 1 and ESM 2)
1
Table ESM 1 Significance of True Positive, False Positive, True Negative, False Negative [1]
PRIMARY STUDIES ANALYSIS (figure 1 of main paper) #TRUE POSITIVE / FALSE NEGATIVE / TRUE NEGATIVE / FALSE POSITIVE
SEPs present to predict GOS 3-5 / patients with PRE SEPs and AW / patients with ABS SEPs and AW / patients with ABS SEPs and N-AW / patients with PRE SEPs and N-AW
SUB-GROUP ANALYSIS
SEP grading analysis (table 5 of main paper)
TRUE POSITIVE / FALSE NEGATIVE / TRUE NEGATIVE / FALSE POSITIVE
NOR vs PAT+ABS
SEPs normal to predict GOS 3-5 / patients with NOR-SEPs and AW / patients with PAT+ABS SEPs and AW / patients with PAT+ABS SEPs and N-AW / patients with NOR SEPs and N-AW
PAT vs NOR+ABS
SEPs patological to predict GOS 3-5 / patients with PAT-SEPs and AW / patients with NOR+ABS SEPs AW / patients with NOR+ABS SEPs and N-AW / patients with PAT SEPs and N-AW
ABS vs NOR+PAT
SEPs absent to predict GOS 1-2 / patients with ABS SEPs and N-AW / patients with PRE SEPs and N-AW / patients with PRE SEPs and AW / patients with ABS SEPs and AW
SEP grading analysis (table 6 of main paper)
TRUE POSITIVE / FALSE NEGATIVE / TRUE NEGATIVE / FALSE POSITIVE
NOR vs PAT+ABS
SEPs normal to predict GOS 4-5 / patients with NOR-SEPs and GOOD-R / patients with PAT+ABS SEPs and GOOD-R / patients with PAT+ABS SEPs and BAD-R / patients with NOR SEPs and BAD-R
PAT vs NOR+ABS
SEPs patological to predict GOS 4-5 / patients with PAT-SEPs and GOOD-R / patients with NOR+ABS SEPs GOOD-R / patients with NOR+ABS SEPs and BAD-R / patients with PAT SEPs and BAD-R
ABS vs NOR+PAT
SEPs absent to predict GOS 1-3 / patients with ABS SEPs and BAD-R / patients with PRE SEPs and BAD-R / patients with PRE SEPs and GOOD-R / patients with ABS SEPs and GOOD-R
#, for only papers in which is possible to define these category outcome; AW, awakening; N-AW, non-awakening; SEP Grading: PRE, present; NOR, Normal; PAT, pathological; ABS, absent; GOS 3-5, awakening; GOS 1-2, non-awakening; GOS 4-5, good recovery; GOS 1-3 bad recovery; BAD-R, bad recovery; GOOD-R, good recovery
Table ESM 2 Significance of Sensitivity, Specificity, Positive predictive Value, Negative Predictive Value [1]
PRIMARY STUDIES ANALYSIS (table 3, figure 1 of main paper) #SENSITIVITY / SPECIFICITY / POSITIVE PREDICTIVE VALUE / NEGATIVE PREDICTIVE VALUE
SEPs present to predict GOS 3-5 / patients with PRE SEPs and AW respect to all patients with AW / patients with ABS SEPs and N-AW respect to all patients with N-AW / patients with PRE SEPs and AW respect to all patients with PRE SEPs / patients with ABS SEPs and N-AW respect to all patients with ABS SEPs
SUB-GROUP ANALYSIS
SEP grading analysis (table 5 of main paper)
SENSITIVITY / SPECIFICITY / POSITIVE PREDICTIVE VALUE / NEGATIVE PREDICTIVE VALUE
NOR vs PAT+ABS
SEPs normal to predict GOS 3-5 / patients with NOR-SEPs and AW respect to all patients with AW / patients with PAT+ABS SEPs and N-AW respect to all patients with N-AW / patients with NOR-SEPs and AW respect to all patients with NOR-SEPs / patients with PAT+ABS SEPs and N-AW respect to all patients with PAT+ABS SEPs
PAT vs NOR+ABS
SEPs patological to predict GOS 3-5 / patients with PAT-SEPs and AW respect to all patients with AW / patients with NOR+ABS SEPs and N-AW respect to all patients with N-AW / patients with PAT-SEPs and AW respect to all patients with PAT-SEPs / patients with NOR+ABS SEPs and N-AW respect to all patients with NOR+ABS SEPs
ABS vs NOR+PAT
SEPs absent to predict GOS 1-2 / patients with ABS SEPs and N-AW respect to all patients with N-AW / patients with PRE SEPs and AW respect to all patients with AW / patients with ABS SEPs and N-AW respect to all patients with ABS SEPs / patients with PRE SEPs and AW respect to all patients with PRE SEPs
SEP grading analysis (table 6 of main paper)
SENSITIVITY / SPECIFICITY / POSITIVE PREDICTIVE VALUE / NEGATIVE PREDICTIVE VALUE
NOR vs PAT+ABS
SEPs normal to predict GOS 4-5 / patients with NOR-SEPs and GOOD-R respect to all patients with GOOD-R / patients with PAT+ABS SEPs and BAD-R respect to all patients with BAD-R / patients with NOR-SEPs and GOOD-R respect to all patients with NOR-SEPs / patients with PAT+ABS SEPs and BAD-R respect to all patients with PAT+ABS SEPs
PAT vs NOR+ABS
SEPs patological to predict GOS 4-5 / patients with PAT-SEPs and GOOD-R respect to all patients with GOOD-R / patients with NOR+ABS SEPs and BAD-R respect to all patients with BAD-R / patients with PAT-SEPs and GOOD-R respect to all patients with PAT-SEP / patients with NOR+ABS SEPs and BAD-R respect to all patients with NOR+ABS SEP
ABS vs NOR+PAT
SEPs absent to predict GOS 1-3 / patients with ABS SEPs and BAD-R respect to all patients with BAD-R / patients with PRE SEPs and GOOD-R respect to all patients with GOOD-R / patients with ABS SEPs and BAD-R respect to all patients with ABS SEPs / patients with PRE SEPs and GOOD-R respect to all patients with PRE SEPs
#, for only papers in which is possible to define these category outcome; GOS 1-2, non-awakening; GOS 3-5, awakening; GOS 1-3 bad recovery; GOS 4-5, good recovery; N-AW, non-awakening; AW, awakening; BAD-R, bad recovery; GOOD-R, good recovery; SEP Grading: PRE, present; NOR, Normal; PAT, pathological; ABS, absent
1
Testing for heterogeneity [3]
The heterogeneity was evaluated by:
Cochran’s Q test statistic that is computed by summing the squared deviations of each study’s estimate from the overall meta-analytic estimate, weighting each study’s contribution in the same manner as in the meta-analysis. P values are obtained by comparing the statistic with a 2 distribution with k−1 degrees of freedom (where k is the number of studies).
I2 test which describes the percentage of total variation across studies that is due to heterogeneity rather than chance. It can be readily calculated from basic results obtained from a typical meta-analysis as I2 = 100%×(Q − df)/Q, where Q is Cochran’s heterogeneity statistic and df the degrees of freedom. Negative values of I 2 are put equal to zero so that I2 lies between 0% and 100%. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity.
The results of heterogeneity analysis (Table ESM 3) were categorised according to Q (and p) analysis [4] and I2 categorisation of Higgins et al., [3,5] as indicated below:
· 0% to 40%: might not be important;
· 30% to 60%: may represent moderate heterogeneity
· 50% to 90%: may represent substantial heterogeneity
· 75% to 100%: considerable heterogeneity
Table ESM 3 Results of heterogeneity analysis of single studies and different sub-groups analysis
SEPs present to predict FAV outcome (as defined in primary studies). / 27.522 / 10 / 0.002 / 63.612
SEPs absent to predict UNFAV outcome (as defined in primary studies). / 24.218 / 10 / 0.007 / 58.708
Subgroup analysis / Q / df / p / I2
SEPs grading
NOR vs PAT+ABS
SEPs normal to predict GOS 3-5 / 4.356 / 4 / 0.360 / 8.172
PAT vs NOR+ABS
SEPs patological to predict GOS 3-5 / 7,377 / 4 / 0,117 / 45,774
ABS vs NOR+PAT
SEPs absent to predict GOS 1-2 / 0.595 / 4 / 0.964 / 0.000
NOR vs PAT+ABS
SEPs normal to predict GOS 4-5 / 8.709 / 5 / 0.121 / 42.589
PAT vs NOR+ABS
SEPs patological to predict GOS 4-5 / 7.992 / 5 / 0.157 / 37.441
ABS vs NOR+PAT
SEPs absent to predict GOS 1-3 / 1.775 / 5 / 0.879 / 0.000
Single studies analysis: SEPs, somatosensory evoked potentials; FAV, favourable; UNFAV, unfavourable; SEP Grading: NOR, Normal; PAT, pathological; ABS, absent; GOS 1-2, non-awakening; GOS 3-5, awakening; GOS 1-3 bad recovery; GOS 4-5, good recovery
Summary Receiver Operating Curve
Summary receiver operating characteristic curve (sROC) plots the sensitivity (true positive rate) on the vertical axis against the false positive rate (FPR= 1–specificity) on the horizontal axis using the un-weighted method as described by Moses and colleagues [6,7,8]. In SROC curves each data point on the curve represents a separate study, and the curve shows the trade-off between sensitivity and specificity for different thresholds corresponding to different studies. The SROC curve allows to assess whether the variability in the threshold used by different studies could explain the variability in study results in terms of diagnostic performance. SROC curves represent a powerful tool to synthesise the available evidence on diagnostic tests and provide the reader with the information needed for an effective evidence-based diagnostic decision-making. The sROC curve was generated to predict both favourable and unfavourable outcomes as defined in primary studies using Meta-Test software (ver.0.6, Boston, MA).