Supplementary information 1: Main issues raised by EAC and MTAC; submissions 1 - 6

Parafricta Bootees and Undergarments / The geko device / The MAGEC system / The E-vita open plus / Ambu aScope2 / Watch BP Home A
Searching for economic studies
EAC Critique / The sponsors search was somewhat limited. A more comprehensive search was tested, but no additional relevant results found. / NR / The sponsor's search strategy was considered unsystematic. The EAC repeated the sponsor’s search identifying more results. One economic study was included and it was not clear how this study was selected. / The purpose of the search reported by the sponsor was unclear, outdated and did not use the required databases. The EAC conducted their own search, finding no additional studies. / The EAC made some improvements to the search strategy to increase its sensitivity. However, no additional studies were identified. A study identified by the sponsor, but excluded from the review on the basis that it was a simple preliminary cost analysis, was included by the EAC. / The EAC criticised the sponsor’s literature search as crude and poorly reported. The sponsor used studies from the clinical evidence submission and no economic studies were reported in the submission.
Committees considerations / NR / NR / NR / NR / NR / NR
Target population and subgroups
EAC Critique / NR / NR / NR / The scope issued by NICE specified two subgroups to be considered, but clinical data were not available for these subgroups. Therefore, the EAC agreed with the sponsor's decision to omit these subgroup analyses. / NR / The patient population used in the economic analysis was different from that specified in the scope. The proportion of patients assumed to be asymptomatic was judged to be unrealistic.
Committees considerations / NR / NR / The MAGEC system might not be cost saving in older children with less than 35 months growth potential. Use of this technology in older children needs careful patient selection. / NR / NR / The cost impact of using Watch BP Home A in a younger population is unknown due to lack of evidence on the prevalence of AF and associated stroke in this group.
Setting and location
EAC Critique / NR / NR / NR / NR / The scope suggested that a relevant model of care might include the complementary use of reusable scopes and the AmbuaScope in different clinical scenarios. This analysis was not conducted. / The external validity of using this device in settings specified by the scope was not well supported by the submitted evidence.
Committees considerations / NR / NR / NR / NR / Additional modelling to model more reasonable scenarios for the use of AmbuaScope, including using the devicewhere multiple-use endoscopes are not available; and using the devicewhere multiple-use endoscopes are normally available but are inaccessible. The further modelling allowed the committee to come to a recommendation. / Although the findings of the economic modelling indicated that it could potentially prevent a number of strokes, the committee considered that use in home settings would not be cost saving.
Perspective
EAC Critique / NR / NR / NR / NR / NR / The perspective of the economic analysis was not reported; however, the EAC assumed it was performed from the NHS perspective based on the model inputs and outputs.
Committees considerations / NR / NR / NR / NR / NR / NR
Comparator
EAC Critique / NR / NR / NR / NR / NR / The comparator used in the economic analysis was nurse or GP-led pulse palpation. The EAC considered this as appropriate comparator; but judged it was not the same comparator described in the clinical section of the sponsor submission.
Committees considerations / NR / NR / NR / NR / NR / NR
Choice of health outcomes
EAC Critique / Evidence was not provided for all outcomes defined in the scope. / NR / NR / NR / NR / There was no attempt to assess effectiveness and, therefore, the incremental costs and benefits were not explicitly determined.
Committees considerations / NR / NR / NR / NR / NR / NR
Estimating resources and costs
EAC Critique / The costs of pharmaceuticals used to treat people with pressure ulcers and costs of complications were not included in the model. Some costs (including bed-days) were amended by the EAC. / Whilst resources and costs were generally estimated well, the EAC criticised one cost calculation where the incorrect cost of a nurse had been used. This had a minor impact on the results of the model. / The micro-costing was based on only 14 patients from a single centre and may not be generalisable to other NHS Trusts. Using national sources would have been more appropriate. No discounting was applied within the model. / NR / Resource use data were obtained from a questionnaire of a sample of NHS centres. The EAC questioned the generalisability of data from the sample (for some values this only included 6 centres). / The cost of stroke was estimated using a societal perspective, which overestimates NHS and PSS costs. An inflation rate of 5% per year was applied for all costs. The EAC considered this too high and again would overestimate costs.
Committees considerations / NR / NR / The impact of the potential reduction in spinal cord monitoring during rod lengthening procedures was considered and advised the cost savings generated are likely to be conservative. / NR / NR / The capital cost of the device was consideredin two scenarios:
1) immediate wholesale replacement of existing monitors at GP Practices;
2) replacements at the end of the device life spans.
Both scenarios would be cost saving over a 10 year time horizon as modelled.
Choice of model
EAC Critique / Some assumptions in the sponsor’s cost model were deemed to beunlikely. The EAC modified these to produce a revised model. / NR / No adverse events other that complete device failure were included. There was no separation in the model of single and dual rod patients and the costing used was that of dual rods. No validation of the model was undertaken. / NR / The model was based on an unsubstantiated assumption - that a policy of using single-use devices will reduce the chance that the necessary scope will not be available in urgent scenarios. / The simple model structure did not capture longer term health consequences. This would have been better dealt with by using a state transition model.
Committees considerations / The hospital model structure was appropriate. However, the committee noted that limited information on the resource implications of having pressure ulcer were included and the pressure ulcer grade not considered. The committee considered that the EAC’s analysis based on these revisions was more appropriate. / NR / The resource use evidence used to inform the model was based on a single unpublished study. However, it considered the additional work carried out by the EAC addressed many of the uncertainties and provided sufficiently robust findings to make a decision. / NR / NR / NR
Time horizon
EAC Critique / The sponsor’s submission stated that the time horizon for the model was one year. However, the EAC noted that the time horizon was based on the inpatient stay (up to 4 months). / NR / A time horizon of 6 years was used (lifespan of device). However, experts advised that was not appropriate due to device replacement following growth. The EAC found that the system would become cost saving at around 5.5 to 6 years after insertion. / The EAC criticised the 1 year time horizon of the model, given that it was not long enough to capture all cost differences relating to complications resulting from E-vita and its comparators. / NR / NR
Committees considerations / NR / NR / NR / In the short term, E-vita may be cost incurring compared with current practice. However, taking a longer term view (using the additional modelling by the EAC over a longer time horizon) showed E-vita to be cost saving from year 2 onwards. / NR / The longer term cost impact of using the technology in primary care and new estimates of the cost of stroke over time based on additional modelling work carried out by the EAC was accepted by the committee.
Measurement of effectiveness
EAC Critique / NR / The effectiveness of the intervention was modelled using data from an alternative device due to a paucity of data. The EAC judged this to be a weak assumption. This input parameter was a key driver of the economic analysis and the EAC therefore stated that this assumption undermined the results of the model. / NR / NR / Some of the model's inputs were from older cohort studies (1992-1994). The EAC were also concerned that the relative risk of the intervention in reducing clinical events was an arbitrary assumption which had not been validated. / NR
Committees considerations / NR / The threshold analysis that was undertaken by the sponsor around the risk reduction of deep vein thrombosis was considered and judged unlikely that the device would become cost incurring, based on the values presented. / NR / The committee considered the people treated with E-vita who might need a second procedure. The EAC did not include re-intervention within its model due to a lack of data. The EAC advised the committee that re-intervention would not substantially change the findings of its model. / NR / NR
Parameters
EAC Critique / The EAC updatedparameters within the sponsor’s model by reanalysing the source data. / NR / The frequency of rod lengthening was critiqued by the EAC with additional analysis around this parameter. / The EAC reported that a number of unit costs and resource usage estimates utilised within the model were either incorrect or were not referenced in enough detail to be validated. / NR / The EAC considered the approach used by the sponsor to calculate the population eligible to use the device to be incorrect and overestimated.
Committees considerations / The committee accepted the mean LoS calculated by the EAC were appropriate, but noted the values were inconsistent and uncertain due to limited data and the complex relationship between ulcer severity and LoS. They concluded that further research would be necessary to inform a more robust analysis. / NR / The frequency of lengthening using the MAGEC system may increase in future based on expert advice. However, calculations based on an additional work carried out by the EAC which concluded that even with more frequent lengthening (6 weeks), the MAGEC system would remain cost saving of £7,000+ per child over 6-year time horizon. / NR / NR / NR
Characterising uncertainty
EAC Critique / The PSA conducted by the sponsor was based on arbitrary distributions. This was updated and re-run by the EAC to reflect the true uncertainty. / NR / The sponsor only carried out limited scenario analysis. The EAC carried out additional deterministic sensitivity analysis. / NR / NR / Univariate and multivariate DSA were performed for some parameters. The EAC considered this to be limited and performed additional multivariate sensitivity analyses.
Committees considerations / The committee considered the expected cost savings in the community were uncertain. Further, research around the effectiveness of the device in reducing the incidence and severity of pressure ulcers was recommended. / NR / NR / NR / There were too many uncertainties in the sponsor's economic model to make recommendations. The committee requested additional modelling from the EAC. The further modelling conducted by the EAC reduced these uncertainties and allowed the committee to come to a decision. / The committee acknowledged the uncertainty associated with the cost of stroke and the 1 year time horizon of the model.
Presentation and interpretation of results
EAC Critique / NR / NR / NR / Multiple comparators were considered within the model and the results reported as a weighted average across all comparators. The EAC judged that the sponsor should have reported head-to-head comparisons between E-vita and each comparator. / EAC judged that the uncertainty in the assumptions on which the model was built were so great that the results are not reliable. This was not captured in the sponsor's interpretation of the results. / NR
Committees considerations / NR / NR / NR / NR / NR / NR
DSA = deterministic sensitivity analysis
EAC = External assessment centre
GP = General practice
NHS and PSS = National Health Service and personal social services
NR = Not reported
PSA = probabilistic sensitivity analysis

Supplementary information 1: Main issues raised by EAC and MTAC; submissions 7 - 12

The VeriQ system / Inditherm Patient Warming Mattress / Ambulight PDT / The MIST Therapy system / The BRAHMS copeptin assay / SeQuent Please balloon catheter
Searching for economic studies
EAC Critique / The EAC considered the sponsor's search strategy to be inadequately reported and not extensive. However, the EAC's search did not identify any further studies. / The sponsor's literature search did not include search terms specific to cost or cost effectiveness and there was no search of NHSEED. / The EAC noted that there were a large number of duplicates in the sponsor's search as they searched the same database twice (Medline). / NR / The use of an additional 'AND' operator reduced the number of publications identified and a more sensitive search strategy should have been adopted. However, an improved search was undertaken by the EAC, and no further studies were identified. / The sponsor’s literature searches were not sensitive and were conducted on a limited range of databases. The search strategies were poorly reported in places.
Committees considerations / NR / NR / NR / NR / NR / NR
Target population and subgroups
EAC Critique / NR / The subgroups outlined within the scope were not considered within the economic model. / NR / NR / NR / The cost model did not include the full patient group specified in the scope. The subgroups outlined within the scope were also not considered.
Committees considerations / NR / NR / NR / NR / NR / NR
Setting and location
EAC Critique / NR / NR / NR / NR / NR / The study used to inform the model was based in a German healthcare setting and may not be generalisable to the UK NHS.
Committees considerations / NR / NR / The cost of setting up facilities to deliver the technology may outweigh the potential benefits from using the technology as the impact of the technology on costs is determined by whether there is adequate service provision of standard care in the community. / NR / NR / NR
Perspective
EAC Critique / NR / NR / NR / NR / NR / NR
Committees considerations / NR / NR / NR / NR / NR / NR
Comparator
EAC Critique / NR / NR / NR / NR / The EAC recommended that the sponsor provide a clear description of the model comparator. / The EAC noted that only one comparator was included in the model. This comparator is the most widely used, hence the significance of this may be limited.
Committees considerations / NR / NR / NR / NR / NR / Only one comparator (paclitaxel-eluting stent) was used in the model. The committee considered this as the most relevant comparator and therefore appropriate for the model.
Choice of health outcomes
EAC Critique / The lack of adverse events in the model was noted as a limitation by the EAC. However, no adverse events were included in the EAC's additional work. / NR / NR / The EAC raised the issue that the outcome measure used in the model are primary outcomes from clinical trial and have to be linked with other evidence to generate quality of life implications. / NR / NR
Committees considerations / NR / NR / NR / NR / NR / NR
Estimating resources and costs
EAC Critique / NR / Some of the costs used within the model were implausible and inconsistent. Other costs were erroneously excluded. Discounting was not incorporated. / Some cost elements were taken from an expert and, therefore, may not be representative of clinical practice across the UK. Other costs were double counted. Costs were converted into pounds sterling with no exchange rate provided.
The average selling price for the device rather than that based on large volume orders should have been used. / A societal perspective rather than NHS and PSS perspective was used.
Large uncertainty existed around many downstream costs. / The unit cost of a key driver of the model was judged to be invalid. The sponsor provided an updated cost post submission. / The EAC questioned the durations for drug use presented in the model as these did not correspond with the durations presented in the clinical trial. The EAC also raised the issue that the cost of SeQuent Please is higher than the selected HRG code.
Committees considerations / The sponsor excluded savings from reductions in ICU stay and reduced readmission rates, potentially underestimating savings. The sponsorunderestimated usage of their device and, therefore, the equipment cost per procedure would be lower than estimated.. / The model included purchase of components of the Inditherm PWS in addition to the mattress, and so the cost savings estimated by the model might be an underestimate. The mattress might be used in combination with FAW, with the aim of additional patient benefit. However, the costs of that combined approach were not available. / NR / There were concerns around the different approaches used to calculate the costs of treatment with the MIST Therapy system and with standard wound care. The committee also discussed that the exclusion of procedures with substantial long-term cost implications (e.g. amputation) may underestimate the potential cost savings of adopting the system. / The committee was advised there would be additional costs if a hospital laboratory does not have access to a compatible analyser for the technology, and there may also be additional costs for the technician time.