Evidence Sub Group &
Economic Evidence Sub Group
Health Improvement Review
Assessment of Stop Smoking Wales
The following report includes the assessments of the evidence base produced by the Evidence subgroup (section 1- Evidence of Effectiveness) and the Economic Evidence sub-group (section 2 – Evidence of Cost effectiveness).
Section 1:Evidence of Effectiveness
1.Introduction
A core component of the Health Improvement Review (HIR) has been the assessment of the evidence-base for initiatives included in the HIR. This report describes the methodology for, and findings of, the assessment of the Stop Smoking Wales programme.
2.Methodology
Assessment of the initiative employed a dual approach:
- Assessment of the potential effectiveness of the initiative by review of research on the effectiveness of similar initiatives or of component interventions (some initiatives involve more than one intervention).
- Assessment of the actual effectiveness or impact of the initiative by review of any available evaluation reports for the initiative in Wales.
2.1Review of potential effectiveness
The methodology adopted for this review followed systematic review principles of transparency, a priori setting of the research question, search strategy, inclusion/exclusion criteria, critical appraisal and standardised data extraction.
2.1.1A ‘question’ was developed for each initiative following the PICO format[1]. For Stop Smoking Wales, the question was: Is smoking cessation an effective intervention, with specific reference to group intervention. Is there evidence for the effectiveness of brief intervention training for health professionals in promoting quit attempts and referral to services?
2.1.2Due to the time constraints of the Health Improvement Review, a ‘best available evidence’ approach was taken for the reviews of research on potential effectiveness of initiatives. Key words and search terms were derived from the question and a pragmatic search strategy designed using specified health databases and search-engines. For initiatives where recent high quality secondary analyses of the primary literature were found, searches were narrower and terminated at an earlier stage. Searches for questions that yielded little high quality data initially were broadened by date or by search terms in an attempt to capture related work. The time-constraints did not enable hand-searching or contacts with experts in the field (external to Public Health Wales) to search for missed or unpublished data, however, an iterative process of related article searches were run on key papers for some questions to try and capture information that the initial search strategy had not identified. All reasonable efforts were made to locate the most relevant and highest quality evidence in the short-time frame allocated.
The search terms used for Stop Smoking Wales Search strategy
Smoking cessation; smoking cessation services
Databases searched
TRIP Database
NHS Evidence
Campbell Collaboration
EPPI Centre
PubMEd
Google Scholar
Health Evidence Canada
WHO
2.1.3Retrieved articles were recorded in the ‘Evidence Mapping Table’ for the initiative (Table 1) and were screened for inclusion by two reviewers independently (disagreements resolved by discussion), on the basis of direct relevance to the initiative or component interventions and type of article, thus single studies were not included if higher level evidence was available:
Primary group of sources: NICE guidance, Single systematic reviews from Cochrane, Campbell Libraries, the EPPI-Centre
Secondary group (include if no primary group evidence items available): RCT or evaluation of robust design looking at appropriate outcomes
Other study designs to be included if no primary or secondary sources are available, the
quality of these to be judged separately/recorded on a case-by-case basis.
2.1.4Information was extracted from each included article into a standardised template – the Evidence Mapping Table, for each initiative.
2.1.5Each included article was assessed in terms of ‘reliability’, strength and direction, using the following ‘Evidence Grading Scheme’:
++ Directly relevant evidence that the intervention evaluated is beneficial/ useful/ effective - the evidence comes from a reliable source[2] and is guidance based on RCTs, SRs or robust evaluations of appropriate outcomes or is a well conducted systematic review.
+ Directly relevant evidence that the intervention evaluated is beneficial/ useful/effective -the evidence comes from a reliable source and is a robust/ large RCT or robust evaluation of appropriate outcomes.
+/- Conflicting evidence (from reliable sources) about the usefulness/efficacy of the intervention being evaluated.
0 Directly relevant evidence on effectiveness of an intervention the same as, or similar to, the initiative, of acceptable reliability, is lacking.
-- Directly relevant evidence that the intervention being evaluated is not
beneficial/useful or is ineffective - the evidence comes from a reliable source and is guidance based on RCTs, SRs or robust evaluations of appropriate outcomes or is a well conducted systematic review.
- Directly relevant evidence that the intervention evaluated is not beneficial/useful or is ineffective -the evidence comes from a reliable source and is a robust/large RCT or robust evaluation of appropriate outcomes.
QlWell conducted studies using robust qualitative research methods which cast light on how/why intervention might be effective/ineffective or have important implications for interpretation of findings or other included studies.
The ‘evidence grades’ for each included article were recorded in the Evidence Mapping Table for the initiative (Table 1).
2.1.6A subjective judgment of the overall balance of evidence grades given to included articles was then made by one reviewer, to give a ‘Summary Evidence Grade’ for the Initiative:
++There is consistent, strong relevant evidence from reliable sources that the intervention/approach employed in the initiative has the potential to be effective.
+There is some relevant evidence from reliable sources that the intervention/approach employed in the initiative has the potential to be effective.
+/-Relevant evidence (from reliable sources) about the likely effectiveness of the intervention/approach employed in the initiative is conflicting.
0Directly relevant evidence (from reliable sources) about the likely effectiveness of the intervention/approach employed in the initiative is lacking.
- -There is consistent, strong relevant evidence from reliable sources that the intervention/approach employed in the initiative has the potential to be ineffective.
-There is some relevant evidence from reliable sources that the intervention/approach employed in the initiative has the potential to be ineffective.
Checks were made for consistency of application of these ‘Summary Evidence Grades’ through comparison and discussion amongst the reviewer team.
2.2) Assessment of initiative
An ‘Initiative Assessment Log’ was then completed (Table 2). Information from any evaluation or other reports about the initiative in Wales was considered for relevance at this stage and pertinent information summarised into the log along with the Summary Evidence Grade and other information. A final ‘Initiative Grade’ was then applied by one reviewer using set criteria (see Annex 1). This therefore takes into account both the evidence of potential effectiveness and evidence of actual effectiveness in Wales, where available. This Initiative Grade’ will feed directly into the Programme Budgeting and Marginal Analysis which forms the decision-making framework for the Health Improvement Review. Initiative grades were checked for consistency by comparison and discussion amongst the review team.
3 Findings of the assessment Table 1: Evidence Mapping
Initiative Evidence Assessment Report SSW FinalPage 1
Public Health Wales Observatory / Health Improvement ReviewEvidence Sub Group &
Economic Evidence Sub Group
Study Details / Study Design / Outcome measures / Main findings / Results / Evidence Grading / Include/ Exclude
NICE, 2008
Smoking cessation services
Smoking Cessation
Whole population / Systematic Review / Not specified – smoking cessation / There are an extensive number of recommendation in this guideline, this is a summary of the recommendations (refer to full guidance for detail)
NHS Stop Smoking Services should be
- Based on an understanding of local population of smokers including those from minority ethnic and socially disadvantaged groups
- Have adequate staffing and a full time co-ordinator
- Have realistic performance targets relating to uptake and cessation rates or at least 5% of smokers per year and 35% quit rates at 4 weeks validated by CO monitoring
- Undertaking regular audits which are made available and undertake audits of cessation rates < 35% or >70% at 4 weeks
- Establish links with services reaching women of childbearing age
- Offer evidence based treatments
- Tailor support to clients needs, particularly disadvantaged and minority ethnic groups and aim to reach these groups at least in proportion to the smoking population
- Telephone quitlines should provide timely and authoritative responses; consider language of choice and ensure staff are appropriately trained
- Healthcare professionals should be able to advice on use of NRT and other pharmacological aids; prescribe where appropriate in line with guidance
- Nicotine Assisted Reduction to Stop (NARS) should only be offered as part of a clinical trial
- Hospital patients, particularly those undergoing elective surgery
- Smokers with cardiovascular disease, particularly unstable and respiratory disease
- Pregnant women, those planning a pregnancy and their partners and family
- Women with infants and young children, breastfeeding women and their partners and family
- Institutions including prisons, care settings and mental health inpatient facilities
- Workplaces
NICE, 2010
Smoking in Pregnancy
Smoking Cessation
Adults of working age / Not specified – smoking cessation / Summary of the recommendations (refer to full guidance for detail)
- Midwives should asks all pregnant women about smoking status; take CO reading; refer all smokers and those quit <2 wks to specialist services; provide information on benefits of quiting, record and review action
- Other professionals in contact with pregnant women should ask about smoking status; refer to specialist services; record and review
- Telephone all women referred and if no contact made consider other ways to make contact e.g. clinic visits
- Review and reduce barriers to access
- Provide information, inc to those who opt out
- Provide evidence based interventions e.g. CBT, MI and structured self help support
- Use NRT and other pharmacological support
- Take steps to reduce barriers to access by disadvantaged pregnant smokers
- Offer multi-component interventions to partners
This guidance is based on limited evidence of effective interventions
Stead, et al 2012
Combined behavioural and pharma -cotherapy interventions
Smoking Cessation
Working Age Adults / Systematic Review / Smoking Cessation minimum 6 months FUP / Interventions that combine pharmacotherapy and behavioural support increase smoking cessation success compared to a minimal intervention or usual care. Further trials would be unlikely to change this conclusion. We did not find strong evidence from indirect comparisons that offering more intensive behavioural support was associated with larger treatment effects but this could be because intensive interventions are less likely to be delivered in full /
- 41 studies more than 20,000 participants.
- Most studies provided NRT. Behavioural support was typically provided by specialists in cessation counselling, 4-8 sessions. The planned maximum duration of contact was typically more than 30 minutes but less than 300 minutes.
- One large study contributed heterogeneity due to a substantially larger treatment effect than seen in other studies (RR 3.88, 95% CI 3.35 to 4.50). Since this study used a particularly intensive intervention and hence it was not pooled in other analyses.
- Remaining 40 studies (15,021 participants) good evidence for a benefit of combination pharmacotherapy and behavioural treatment compared to usual care or brief advice or less intensive behavioural support (RR 1.82, 95% CI 1.66 to 2.00) with moderate statistical heterogeneity (I² = 40%).
- The pooled estimate for 31 trials that recruited participants in healthcare settings (RR 2.06, 95% CI 1.81 to 2.34) was higher than for eight trials with community-based recruitment (RR 1.53, 95% CI 1.33 to 1.76). Pooled estimates were lower in a subgroup of trials where the behavioural intervention was provided by specialist counsellors versus trials where counselling was linked to usual care (specialist: RR 1.73, 95% CI 1.55 to 1.93, 28 trials; usual provider: RR 2.41, 95% CI 1.91 to 3.02, 8 trials) but this was largely attributable to the small effect size in two trials using specialist counsellors where the take-up of the planned intervention was low, and one usual provider trial with a large effect. Little indirect evidence that the relative effect of an intervention differed according to whether participants in a trial were required to be motivated to make a quit attempt or not. There was only weak evidence that studies offering more sessions had larger effects and there was not clear evidence that increasing the duration of contact increased the effect, but there was more evidence of a dose-response relationship when analyses were limited to trials where the take-up of treatment was high
Stead LF and Lancaster T, 2005
Group behaviour therapy programmes
Smoking Cessation
Adults / Systematic Review / Abstinence from cigarettes at follow up at least six months after the start of treatment. / Group therapy is better for helping people stop smoking than self help, and other less intensive interventions. There is not enough evidence to evaluate whether groups are more effective, or cost-effective, than intensive individual counselling. There is not enough evidence to support the use of particular psychological components in a programme beyond the support and skills training normally included. / 53 trials were included for one or more of the comparisons in the review.
- group programme compared to a self-help programme (N = 4375, RR 1.98, 95% CI 1.60 to 2.46; 13 trials).
Bryant,J. et al, 2011.
Behavioural smoking cessation interventions in selected disadvantaged groups.
Smoking Cessation
Adults / Systematic Review / Abstinence rates at short-term (< 3 months) and long-term (6 months or the longest) follow-up. / There is some evidence of effective interventions in the short term with low income female smokers and in the longer term with individuals with mental illness. /
- Thirty-two relevant studies were included. The majority (n=20) were of low quality.
- Results of the meta-analysis showed a significant increase in cessation for behavioural support interventions targeted at low-income female smokers at short-term follow-up [RR 1.68, CI 1.21-2.33]
- behavioural support interventions targeted at individuals with a mental illness at long-term follow-up (RR 1.35, CI 1.01-1.81).
- Results of the narrative review showed several promising interventions that increased cessation rates at 6-month or longer follow-up.
Grimshaw G, et al, 2010
Tobacco cessation interventions
Smoking Cessation
School Age Children / Systematic Review / Primary: change in smoking behaviour / There is insufficient evidence to recommend an approach. Complex approaches show promise, especially those incorporating elements sensitive to stage of change and using motivational enhancement and CBT. More data is needed on sustained quitting that allows for the episodic nature of much adolescent smoking. There were few trials with evidence about pharmacological interventions (nicotine replacement
and bupropion), and none demonstrated effectiveness for adolescent smokers. /
- 24 trials involving over 5000 young people met inclusion criteria (11 cluster-RCTs, 11 RCTs and 2 CTs).
- 2 trials based mainly on TTM interventions achieved moderate long-term success, with a pooled odds ratio (OR) at one year of 1.70 (95% (CI) 1.25 - 2.33).
- The 11 trials that included some form of motivational enhancement gave an estimated OR of 1.70 (95% CI 1.31 to 2.20).
- The six trials which included cognitive behavioural therapy did not individually achieve statistically significant results, but there was a marginally significant effect of pooling four studies using the Not on Tobacco programme (OR of 1.77, 95% CI 1.00 to 3.11) although 3 of the trials used abstinence for as little as 24 hours at six months as the cessation outcome.
- Neither of the small trials using NRT products detected statistically significant effects. A trial of bupropion alone, showed no significant effect for either a standard or lower dose.
Brose LS et al, 2011
What makes for an effective stop smoking service?
Smoking Cessation / Secondary data analysis from specialist smoking cessation services / Abstinence from smoking 4 weeks after the target quit
date, CO verified / Routine clinic data support findings from randomised controlled trials that smokers receiving stop-smoking support from specialist clinics, treatment in groups and varenicline or combination NRT are more likely to succeed than those receiving treatment in primary care, one-to-one and single NRT. All smokers should have access to, and be encouraged to use, the
most effective intervention options. The most effective forms of intervention are uncommon, many services do not offer the optimal options. Smokers using the more successful intervention options tend to be more dependent, so higher success rates are unlikely to be due to an ‘easier’ clientele. / 126 890 treatment episodes in 24 SSSs in 2009/10 were used to assess the association between intervention characteristics and success rates, adjusting for key smoker characteristics. There was substantial variation in success rates across intervention characteristics after adjusting for smoker characteristics. Single NRT was associated with higher success rates than no medication (OR 1.75, 95% CI 1.39 to 2.22); combination NRT and varenicline were
more successful than single NRT (OR 1.42, 95% CI 1.06 to 1.91 and OR 1.78, 95% CI 1.57 to 2.02, respectively); group support was linked to higher success rates than one-to-one support (OR 1.43, 95% CI 1.16 to 1.76); primary care settings were less successful than specialist clinics (OR 0.80, 95% CI 0.66 to 0.99). / + / Include
Coleman T et al ,2010
Relapse prevention in UK Stop Smoking Services: