Abstract
Objective:
To review the quality of current smoking cessation guidelines that include recommendations for people with severe mental illness.
Methods:
A systematic search of scientific databases, central government health authority websites, psychiatry peak bodies, guideline clearing houses and Google was undertaken for relevant smoking cessation guidelines. Three reviewers independently assessed guideline quality using the AGREE II (
Results:
Thirteen guidelines met the inclusion criteria. Seven guidelines scored ⩾60% in at least four domains. Median scores for ‘Editorial independence’, ‘Rigour of development’, ‘Stakeholder Involvement’ and ‘Applicability’ were less than 60%. The highest median scores were for ‘Scope and purpose’ (87%, 69–96%) and ‘Clarity of presentation’ (87%, 56–98%). ‘Editorial independence’ (33.3%, 0–86%) and ‘Rigour of development’ (54%, 11–92%) had the lowest median domain scores. The guidelines varied greatly in their recommendations but the majority recommended nicotine replacement therapy, bupropion or varenicline as first-line pharmacotherapy, along with behavioural support.
Conclusion:
Many guidelines did not adequately report their methods or the competing interests of the authors. Future guidelines development may benefit from more specifically addressing AGREE II criteria and the needs of smokers with severe mental illness.
Introduction
Smoking prevalence among people with severe mental illnesses (SMI), such as schizophrenia, bipolar disorder and major depression, is 3–5 times higher than for the remainder of the population (Lawrence et al., 2009). Smoking-related morbidities such as cardiovascular and respiratory diseases contribute substantially to the 10–15 years lower life expectancy of people with SMI compared to the general population (Galletly et al., 2016; Lawrence et al., 2013). Reducing smoking is critical to improving the physical health of people with SMI (Galletly et al., 2016). However, the combination of heavy dependence on nicotine, lingering pro-smoking culture in many mental health institutions, socio-economic disadvantage experienced by many people with SMI and insufficient smoking cessation support provided by mental health practitioners pose substantial barriers to people with SMI quitting smoking (Sharma et al., 2016a).
The evidence-base to inform smoking cessation interventions in this population is limited (Peckham et al., 2017). However, the available research shows that most smoking cessation interventions that are effective for smokers without SMI are also effective for smokers with SMI (Peckham et al., 2017). Behavioural interventions (e.g. brief advice and motivational interviewing) and pharmacotherapy (e.g. nicotine replacement therapy [NRT], bupropion and varenicline) are currently the mainstays of smoking cessation treatment for people with and without SMI (Peckham et al., 2017). However, smokers with SMI have lower quit success rates than general population smokers, even when assisted with cessation aids (Ferron et al., 2011). Non-abstinence or harm-reduction strategies such as switching to NRT or e-cigarettes for longer term use (i.e. >6 months) have also been recommended for those who have tried and failed to quit smoking using traditional smoking cessation methods (Gartner and Hall, 2015).
Practice guidelines for smoking cessation help translate research into clinical practice. Hence, it is vital that guidelines reflect best practice, are kept up to date and cover areas of need. We assessed the quality of smoking cessation clinical practice guidelines which include recommendations for smoking cessation among people with SMI.
Methods
Search strategy
We used the National Library of Medicine’s (NLM) Medical Subjects Heading (MeSH) definition of practice guidelines to identify relevant documents for inclusion in this study. This definition states that practice guidelines are a set of directions or principles to assist the healthcare practitioner with patient care decisions about appropriate diagnostic, therapeutic, or other clinical procedures for specific clinical circumstances [that are] developed by government agencies at any level, institutions, organizations such as professional societies or governing boards, or by the convening of expert panels.
We searched medical databases (PubMed, CINAHL, EMBASE and PsycINFO), central government health authority websites, psychiatry peak bodies and guideline clearing houses of Australia, New Zealand, Canada, the United States and the United Kingdom for guidelines on smoking cessation or tobacco harm reduction that were specific or explicitly relevant to smokers with SMI (Supplemental Table 1). This was supplemented by a general Internet search using the Google search engine. The search terms used were ‘guidelines’, ‘guidance’, ‘mental illness’, ‘smoking’, ‘tobacco cessation’ and ‘tobacco harm reduction’. Where multiple versions of a guideline were found, the most recent guideline was reviewed. Guidelines were included if they
Explicitly stated that they were designed for smoking cessation among people with mental illness or included a dedicated section for addressing smoking cessation/harm reduction among people with mental illness;
Were endorsed or funded by central government bodies, national health departments or medical professional bodies. State, regional or provincial guidelines were excluded;
Used English as the reporting language.
The search strategy is given in Figure 1 .

Search strategy.
Evaluation
The guidelines were appraised for their methodological rigour and quality of reporting using the AGREE II (
The AGREE II instrument evaluates the process of practice guideline development and reporting quality, using 23 items which represent 6 quality domains (see Table 1). Each item is given a score on a scale, ranging from 1 (strongly disagree) to 7 (strongly agree).
AGREE II instrument.
Apart from the domain score, AGREE II methodology provides two subjective measures of overall assessment, not calculated from the domain scores:
Overall assessment 1. The overall quality of each guideline is rated on a seven-point scale (lowest possible quality to highest possible quality) and expressed as a percentage, similar to the domain scores.
Overall assessment 2. Each assessor provides a recommendation on whether to use the guideline in practice or not (recommendation for use: ‘yes’, ‘yes with modifications’ or ‘no’).
Three researchers (R.S., D.P. and K.A.) independently appraised the guidelines using the online AGREE II platform. Prior to starting the evaluation, all assessors read the AGREE II instruction manual, watched the tutorial videos and completed a practice guideline evaluation available from the AGREE II website.
Additionally, R.S. and D.P. extracted content on the specific interventions discussed in each guideline. Data were extracted according to a pre-determined template of interventions, designed in consultation with C.G. and C.M., who are experienced researchers in the field of tobacco cessation and mental health. Content pertaining to the endorsement or disendorsement of interventions and qualifying details were extracted from guidelines.
Analysis
The analysis followed the AGREE II methodology (Brouwers et al., 2010). For each guideline, the points given by all evaluators for the items relating to each quality domain were added together to give the domain total for that guideline. The domain scores of each guideline were then calculated as follows: (domain total − minimum possible score)/(maximum possible score − minimum possible score). The maximum possible score was calculated as: (number of questions) × (number of reviewers) × 7. The minimum possible score was calculated as: (number of questions) × (number of reviewers) × 1. The domain scores were expressed as a percentage, where 0% is the worst possible score and 100% is the best possible score. A similar method was used to calculate the overall assessment 1. Frequencies were calculated for each item of overall assessment 2. Medians and ranges were calculated for the domain scores for all guidelines. We also calculated the medians for the six domain scores of each guideline.
The AGREE II manual does not make any suggestions about a minimum domain score which can be considered acceptable for a guideline. Hence, we followed the examples of other published studies which deem a score of 60% or greater in the quality domains as acceptable (Armstrong et al., 2016). Intra-class correlation coefficients (ICCs) were calculated for each domain to assess the inter-rater reliability.
The extracted recommendations were tabulated (see Table 2), appraised and summarised for their relevance to smoking cessation and harm reduction for smokers with SMI by three of the authors (R.S., C.G. and C.M.).
Extracted guideline recommendations.
Information available at http://www.health.govt.nz/our-work/preventative-health-wellness/tobacco-control/e-cigarettes
Results
In total, 13 guidelines met the study’s inclusion criteria (see Table 3).
Characteristics of included guidelines.
AGREE evaluation
The ICC (inter-rater reliability) ranged from 0.7 to 0.9 (good to excellent). The domain scores for all the guidelines are presented in Table 4. The highest median domain score was achieved by the AHRQ guidelines (89%, range = 98.1–66.7%), while the Guidance for frontline staff (31.5%, range = 0–83.3%) scored the lowest. Seven guidelines scored ⩾60% in at least four quality domains.
Domain scores of each guideline.
Scope and purpose
All 13 guidelines scored more than 60% in this domain with the AHRQ guidelines (96.3%) and NICE-PH45 guidelines (96.3%) scoring the highest. The median score for ‘Scope and purpose’ was 87% (range = 68.5–96.3%).
Stakeholder involvement
Five out of 13 guidelines obtained a score of 60% or more on this criterion. The median score for this domain was 59.3% (range = 24.1–88.9%). The maximum scores were achieved by the NICE-PH48 guidelines (88.9%) and AHRQ guidelines (88.9%). Guidance for frontline staff scored the lowest (24.1%).
Rigour of development
Six out of 13 guidelines scored more than 60% with NICE-PH48 guidelines (91.7%) scoring the maximum and Guidance for frontline staff (11.1%) scoring the lowest. The median score for rigour of development was 54.2% (range = 11.1–91.7%).
Clarity of presentation
Most guidelines’ recommendations were explicit and easily identifiable with 11 guidelines achieving scores greater than 60% in this domain. The median score was 87% (range = 55.6–98.1%). AHRQ guidelines (98.1%) scored the maximum, while Tobacco-Free Toolkit (55.6%) scored the lowest in this domain.
Applicability
Five out of 13 guidelines scored more than 60% for applicability with an overall median of 58.3% (range = 22.2–84.7%). Maximum domain scores were achieved by the NICE-PH45 guidelines (84.7%) while RCGP guidelines (22.2%) scored the lowest.
Editorial independence
Five out of 13 guidelines scored more than 60% in this domain. Most guidelines performed poorly with a median domain score of 33.3% (range = 0–86.1%). The maximum domain score was obtained by CAN-ADAPTT guidelines (86.1%) and the minimum by the Guidance for frontline staff (0%) and NCSCT guidelines (0%).
Overall guideline assessment
Six out of 13 guidelines were scored as ‘recommended’ (CAN-ADAPTT, 2011; Fiore et al., 2008; Kleber et al., 2006; NICE, 2013a, 2013b; RACGP, 2011) and ‘recommended with modifications’ (Campion et al., 2014; Friend et al., 2010; Ministry of Health, 2014; Morris et al., 2013; Ruther et al., 2014; NCSCT, 2014), respectively, by at least two of the three reviewers. Of these, two guidelines were scored as recommended by all reviewers (APA guidelines and AHRQ guidelines) (Kleber et al., 2006; Tobacco Use and Dependence Guideline Panel, 2008). The AHRQ guidelines received the highest median domain score (89.2% [66.7–98.1%]), while the highest overall assessment 1 score was obtained by AHRQ guidelines (94.4%) and APA guidelines (94.4%).
Guideline recommendations
Key guideline recommendations have been summarised in Table 5.
Summary of key smoking cessation guideline recommendations.
Discussion
AGREE evaluation
This systematic guideline review is the first to assess the methodological rigour and quality of smoking cessation guidelines aimed at smokers with SMI. The guidelines included in our assessment varied considerably in their quality, and only two guidelines were recommended by all the reviewers. Most of the guidelines were from the United Kingdom, where a dedicated health authority (NICE: National Institute for Health and Care Excellence) provides national guidelines on health and social care, in addition to those produced by professional bodies and national health departments. Unlike some other guideline evaluations in other fields, where more recent guidelines were of better quality than older ones, we found an even spread of scores over the years (Polus et al., 2012).
In keeping with guidelines for other topics, the ‘Scope and purpose’ domain consistently received high scores (⩾60%), followed by ‘Clarity of presentation’ where all but two guidelines scored high (Polus et al., 2012; Sabharwal et al., 2014; Xie et al., 2016). Many of the assessed guidelines presented their recommendations using flow charts, boxes, tables and bold fonts as required by the AGREE II scoring criteria.
By contrast, ‘Rigour of development’ was not adequately reported in the assessed guidelines. This could have resulted from the fact that not all guidelines conducted their own independent and extensive literature reviews to formulate the recommendations. For example, some UK guidelines borrowed recommendations from NICE guidelines (Friend et al., 2010; NCSCT, 2014), while those from New Zealand were based on a systematic review done for a previous version of the guideline. Thus, guidelines such as these were unable to fully report on methods of the literature review or criteria for inclusion of evidence with consequent low scores.
Only a minority of guidelines (CAN-ADAPTT, 2011; Kleber et al., 2006; RACGP, 2011; Ruther et al., 2014; Tobacco Use and Dependence Guideline Panel, 2008) satisfactorily fulfilled the criteria for ‘Editorial independence’. According to AGREE II criteria, this involves listing and addressing the potential conflicts of interest of the guideline developers, and explicitly confirming to the reader that funding bodies did not influence the recommendations of the guideline. Many published practice guidelines did not report or under-reported conflict of interests of guidelines’ panel members (Neuman et al., 2011; Papanikolaou et al., 2001). Establishing editorial independence is relevant to smoking cessation guidelines because funding from the pharmaceutical industry could present a conflict of interest in terms of recommendations that encourage the use of cessation pharmacotherapy (Etter et al., 2007). Declaring conflicts of interest is also an implementation issue because poor reporting may detract from the trustworthiness of the guidelines, reducing their uptake by health professionals.
Smoking cessation by people with SMI involves a concerted effort by health practitioners, smokers, caregivers and policymakers. For instance, consumer participation can improve the quality of services and the production of new or improved information sources (Crawford et al., 2002). However, the guidelines we reviewed scored low in stakeholder involvement, meaning that they did not provide sufficient evidence of stakeholder relevant input into their guideline’s development. Resulting guidelines may therefore be less successful (Cluzeau et al., 2012), given that smokers with SMI may have personal preferences for different smoking cessation approaches (Meurk et al., 2016).
As in other fields, most of the guidelines did not satisfactorily fulfil the criteria for ‘Applicability’ (Armstrong et al., 2016). ‘Applicability’ refers to the inclusion of practice tools (such as patient leaflets, algorithms and referral forms), monitoring criteria and the appropriate financial and human resources for implementation (Brouwers et al., 2010). The main reason for low scores in this domain was that very few guidelines addressed the human and financial implications of implementation that are specific to smokers with SMI, despite mentioning other resources such as links to telephone quit lines or support groups. The highest scoring guidelines were those formulated by NICE (2013a, 2013b), and included both detailed cost effectiveness analysis and economic modelling.
Smoking cessation guidelines for people with SMI also need to address practitioner barriers such as time and financial constraints and competing priorities in delivering smoking cessation (Sheals et al., 2016). Cost-effectiveness and affordability are particularly relevant for both practitioners and their patients, given that many smokers with SMI have limited incomes (Francke et al., 2008). Information about government subsidies for smoking cessation assistance, and about how to access this support, needs to be communicated to health practitioners via guidelines (RACGP, 2011).
Relevance of guidelines’ recommendations for smokers with SMI
Smoking cessation interventions effective in the general population are also effective for smokers with mental illness (Peckham et al., 2017). However, smokers with SMI may require more intensive counselling, closer monitoring for changes in psychiatric symptoms during a quit attempt, prolonged pharmacotherapy and titration of their psychotropic medication (Ruther et al., 2014). Recent guidelines for the management of schizophrenia and mood disorders published by the Royal Australian and New Zealand College of Psychiatrists recommend pharmacotherapy, peer-supported group therapy and adjustment of antipsychotic medication for those smokers who reduce or stop smoking (Galletly et al., 2016; Malhi et al., 2015; Mendelsohn et al., 2015). Tobacco harm-reduction techniques such as long-term maintenance on NRT and/or e-cigarettes may need to be considered for smokers with SMI who may be more dependent on nicotine and find it difficult to quit or do not want to quit nicotine (Gartner and Hall, 2015). Guidelines not tailored to smokers with SMI may be poorly implemented as mental health practitioners may consider them unsuitable for their patients, or because smokers with SMI may be unable to adhere to treatment due to logistic, including financial, constraints (Davis and Taylor-Vaisey, 1997).
Four out of the 13 guidelines (EPA guidelines, Guidance for frontline staff, RCGP guidelines and APA guidelines) were exclusively targeted at smokers with SMI, while the remaining nine were general smoking cessation guidelines with some consideration of smokers with SMI (Campion et al., 2014; Kleber et al., 2006; Robson and Potts, 2014; Ruther et al., 2014). These provided varying levels of detail on the required adaptations in practice to adequately meet the needs of smokers with SMI.
The majority of the guidelines recommended NRT, bupropion and varenicline as a first-line pharmacotherapy for nicotine dependence, with most warning about neuropsychiatric symptoms and recommending close monitoring, especially when prescribing varenicline to smokers with SMI. There is evidence that varenicline is more effective than either NRT or bupropion, and recent data suggest it is not more likely to cause adverse events in people with SMI compared to the general population or when compared to placebo or NRT (Anthenelli et al., 2016). In the light of this evidence, the US Food and Drug Administration (FDA) removed the requirement for boxed warning about neuropsychiatric side-effects for varenicline in 2016. Despite superior effectiveness, mental health practitioners can be reluctant to prescribe varenicline possibly due to fears about side-effects (Chen et al., 2016). This indicates the need to update smoking cessation guidelines regularly, as clinical guidelines can be a key vehicle for translating research into practice by informing the decisions of clinicians.
All guidelines mentioned that the dosage of certain psychotropic drugs may require monitoring and re-evaluation following smoking cessation although many did not provide a comprehensive list of relevant agents. Such a list might help non-specialists to decide whether and when it would be appropriate to refer the patient for medication review (Mendelsohn et al., 2015).
Four out of 13 guidelines did not recommend or expressed a sceptical view of using NRT for an extended period to prevent relapse, and none recommended the extended use of varenicline or bupropion. This is despite the fact that smokers with SMI are more likely to relapse and, thus, may require extended pharmacotherapy and counselling (Evins et al., 2017; Shahab et al., 2016).
No guideline recommended or endorsed e-cigarettes as a smoking cessation method. Three UK guidelines mentioned that e-cigarettes may be less harmful than cigarettes, in line with official support for tobacco harm reduction in the United Kingdom (Campion et al., 2014; NICE, 2013a, 2013b). E-cigarettes are battery-operated devices which heat a solution of nicotine and other additives to produce inhalable nicotine vapour. The use of these devices remains controversial because of uncertainty surrounding their role in smoking cessation, adverse effects, potential renormalisation of smoking and concerns about uptake by young non-smokers (Fairchild et al., 2014). However, there is early evidence to suggest that e-cigarettes may have similar efficacy to NRT in assisting smokers with mental illness to stop or reduce smoking (O’Brien et al., 2015). Surveys of health practitioners in the United States and Europe have shown that a significant proportion recommend e-cigarettes to their patients (Kandra et al., 2014; Moysidou et al., 2016). Research also shows that smokers with mental illness and their carers are interested in using e-cigarettes for smoking cessation and reduction, and may approach healthcare providers for information on using e-cigarettes as a smoking cessation method (Meurk et al., 2016; Sharma et al., 2016b). Given this, guidelines should include information about e-cigarettes to enable mental health clinicians to deliver accurate advice about e-cigarettes, including which products and nicotine concentration are the most appropriate and how to use and maintain these devices. As with conventional NRT, specific guidance is needed for health practitioners on the need to monitor or adjust the dosage of psychiatric medications when a patient switches from cigarettes to e-cigarettes (Sharma et al., 2016b).
Limitations
This study has several limitations. First, our sample was restricted to guidelines endorsed by central government or professional peak bodies of predominantly high-income countries. We chose this strategy to include guidelines from similar health systems. As a result, we are not able to comment on the quality of guidelines originating in low- and middle-income countries. Second, despite using several ways to supplement our search strategy, there is a possibility that we may not have captured all relevant guidelines, mainly due to there being no consolidated national or international database for smoking cessation guidelines. Third, our study was possibly limited by use of three assessors, instead of the preferred four. For adequate reliability, AGREE II instrument requires that a minimum of two and preferably four assessors appraise the guidelines; however, our high inter-rater agreement between the three assessors suggests a fourth assessor was not needed. Fourth, AGREE II scores may have underestimated the rigour of development of some guidelines that referenced other documents, such as research articles or older versions of the guideline, rather than providing explicit details of the methodology underlying guideline development within the guideline document itself. Finally, the AGREE II instrument evaluates the method of development of the guidelines but not the clinical appropriateness of their content resulting in possible dissonance between the scores and the clinical relevance of some guidelines. For example, RACGP and New Zealand guidelines received modest scores on AGREE II due to a lack of detailed reporting. However, recommendations made in these guidelines were judged as being up to date and clinically relevant for smokers with SMI. To overcome this limitation of the AGREE II instrument, another instrument called AGREE-REX (
Conclusion
Our evaluation indicates that there is substantial room for improvement of smoking cessation guidelines for smokers with mental illness. Guidelines that are evidence-based, detailed, comprehensible, address practical factors that affect implementation and account for the views of different stakeholders are highly desirable. Several good-quality smoking cessation guidelines do exist; however, they could be improved by careful and detailed reporting in accordance with the AGREE II guidelines. To optimally support the smoking cessation efforts of people with SMI, regular updates of existing guidelines are necessary to incorporate the latest evidence into smoking cessation practice. Guidelines also need to be tailored to jurisdictional regulatory, policy and financing frameworks.
Footnotes
Author contribution
All authors contributed to the idea and design of the study. R.S., K.A. and D.P. performed the AGREE II evaluation. R.S. and D.P. extracted the recommendations of the guidelines. R.S. did the data analysis and wrote the initial draft of the paper. All authors contributed to revising and finalising the manuscript.
Declaration of Conflicting Interests
R.S. and K.A. hold an Australian Government Research Training Scholarship. C.G. holds a National Health and Medical Research Council Career Development Fellowship (GNT#1061978). C.M. is funded by the National Health and Medical Research Council (NHMRC) Centre for Research Excellence in Mental Health Systems Improvement (GNT1041131).
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
