Supplementary Appendix – Explanation of Difference-in-Differences Method
The difference-in-differences method attempts to estimate the impact of large-scale policies and programs by using a control group to isolate the underlying secular or temporal change from change observed in the intervention group. Table S1 illustrates the conceptual layout of this analysis. In this analysis, patient outcomes (in-hospital mortality and hospital length of stay) were compared pre- vs. post-implementation for patients in collaborative hospitals (A1 – A2 = Difference 1) and in non-collaborative hospitals (B1 – B2 = Difference 2). The independent effect of participation in the collaborative is represented by the DID estimator, which is the difference-in-differences between collaborative and non-collaborative hospitals (Difference 1 – Difference 2 = DID estimator).Table S1. Conceptual layout of Difference-in-Differences (DID) analysis comparing outcome measures pre- vs. post-implementation for patients in collaborative and non-collaborative hospitals, and estimating the independent effect of the MHA Keystone Sepsis Collaborative (i.e. the DID estimator).
Patient Cohort / Outcome Measures / Pre- vs. Post-Implementation
Comparison a / Independent
Effect of MHA
Keystone Sepsis b
Pre-Implementation / Post-Implementation
Collaborative / A1 / A2 / A2 – A1 = Difference 1 / Difference 1 – Difference 2
Non-Collaborative / B1 / B2 / B2 – B1 = Difference 2
a Statistical significance tests for Diff 1 and Diff 2 represent significant changes in outcomes following implementation
b Statistical significance tests of DID estimator indicates significant effect of the MHA Keystone Sepsis collaborative
In a statistical model, the DID estimator corresponds to the interaction term between intervention status (i.e. collaborative vs. non-collaborative) and pre- vs. post-intervention. This is illustrated in the following generalized linear model, where the terms for collaborative and post represent binary dummy variables for intervention status and pre- vs. post-intervention, respectively, and represents a set of patient and/or hospital characteristics used in risk adjustment.
The Table below specifies the interpretation of the beta coefficients in the DID model. The pre vs. post-implementation changes are specified by the sum of for collaborative hospitals, and simply for the non-collaborative hospitals. The relative difference in pre- vs. post-implementation for collaborative and non-collaborative hospitals is () - = . Hence, the DID estimator isolates the effect of the collaborative, and is estimated by the model interaction coefficient (). If the model is a logistic regression, the interaction term is represented by an odds ratio. The odds ratio is interpreted as the relative odds of the outcome (i.e. mortality) in patients treated in collaborative hospitals compared to non-collaborative hospitals. Conversely, if the model is a linear model, the interaction term is represented as the absolute change in the outcome (i.e. length of stay) in patients treated in collaborative hospitals compare to non-collaborative hospitals. A statistically significant interaction term would indicate that the intervention had an additional impact (either positive or negative) on patient outcomes, beyond changes observed in the control group.Table. Parameter interpretation for Difference-in-Differences model comparing collaborative and non-collaborative hospitals pre and post collaborative implementation
Pre- Implementation / Post-Implementation / Pre/Post Difference / Difference-in-Differences
Collaborative / / / /
Non-Collaborative / / / / -