A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Abstract

Objective

This study sought to develop a WeChat mini-program for smoking cessation behavioral interventions (named the WQ mini-program) and evaluate its usability, acceptability, and preliminary efficacy among Chinese smokers.

Methods

The WQ mini-program was designed based on behavioral change theories and clinical practice guidelines, and clinical smoking cessation experts participated in the development process. Fifty Chinese smokers and five software experts were involved in a single-arm trial. Smokers were asked to use the WQ mini-program at least once a day for 4 weeks and to complete a weekly online follow-up questionnaire. Software experts were asked to complete an online follow-up questionnaire after using all functions of the WQ mini-program. Primary outcomes were usability and acceptability of and satisfaction with the mini-program tested by the System Usability Scale (SUS) and the Mobile App Rating Scale (MARS). Self-reported 7-day point prevalence abstinence (PPA) was used to evaluate its preliminary efficacy for smoking cessation. Optimization suggestions for the mini-program were collected from all participants through an open-ended question at the last follow-up and were analyzed by thematic analysis.

Results

The mean SUS and MARS total scores for the WQ mini-program as evaluated by smokers were 82.1 ± 13.8 and 84.5 ± 3.3 and by software experts were 4.21 ± 0.32 and 4.27 ± 0.15, respectively. Most smokers reported being willing to recommend this mini-program to other smokers (85.4%) and would continue to use it (95.8%). The mean satisfaction score for the mini-program was 4.23 ± 0.69 (out of 5 points) among smokers. Self-reported 7-day PPA among smokers at the 4-week follow-up was 50% (25/50).

Conclusions

This study demonstrated that the WQ mini-program would be a feasible and potentially effective method to encourage Chinese smokers to quit smoking. However, future research is needed to confirm its efficacy through a randomized controlled trial.

Keywords

Smoking cessation mHealth WeChat mini-program behavioral interventions usability

1. Introduction

Quitting smoking is the only effective measure to prevent and reduce smoking-related health hazards.¹ Due to the high addictive nature of nicotine, only 3–5% of smokers can successfully quit smoking for 6–12 months after a given quit attempt.² Therefore, it is necessary to provide professional cessation assistance to smokers.³

Given that more than 70% of smokers visit a physician each year, clinical and other healthcare settings are the most effective channels for delivering cessation interventions.⁴ However, due to time constraints and a lack of knowledge, training, and confidence, most physicians could not consistently deliver all steps of the 5A's approach (Ask, Advise, Assess, Assist, Arrange).⁴ An alternative widely used approach is the Ask–Advise–Refer (AAR) approach,⁵ which involves a physician asking about tobacco use, advising smoking patients to quit, and referring interested patients to other professional cessation services, such as smoking cessation clinics or quitlines. However, smoking cessation services are very limited in low- and middle-income countries.⁶ As a result, many smokers do not have access to cessation services, and their successful rate in quitting is low.⁶

This situation also exists in China, where there are more than 300 million current smokers.⁷ Among these smokers, the smoking cessation rate was only 20.1%, and more than 90% of them had never received any professional help.⁷ This may be related to the limited resources of smoking cessation services in China. A nationwide survey showed that there were only 366 smoking cessation clinics in about 1 million hospitals in mainland China, and most of them were located in tertiary hospitals in large cities, making it inconvenient to visit.⁸ Although there are three national quitlines and many local hospital-based quitlines available in mainland China,^9,10 all of them face the challenge of low utilization.^11–13 One possible reason may be that more people prefer to communicate using chat tools on their smartphones instead of telephones.¹⁴ Therefore, it is necessary to develop effective approaches to promote smokers easy access to cessation services, to meet their needs, and to suit their preference.

With the rapid development of mobile health (mHealth) technology, the above concerns can be addressed. These approaches have characteristics that can remove or reduce time and transportation issues that may hinder face-to-face service delivery. In addition, during the COVID-19 pandemic, people preferred to use mHealth approaches for health services.^15,16 There are a variety of mHealth approaches to delivering behavioral interventions for smoking cessation, such as short message services, web-based interventions, mobile applications (apps), and social media, which have been shown to be effective in helping smokers quit.¹⁷ The WeChat app is the most popular social media platform in China and has become a major tool for communication, entertainment, and payment among Chinese smartphone users.¹⁸ Many studies have explored the use of the WeChat app to send cessation messages.^19–21 However, it is important to note that most of these interventions were untailored and had limited interaction intensity. In addition, some studies have used the WeChat app to provide social media-based interventions^22,23 in a similar way to Twitter and Facebook,^24–26 which were more tailored and interactive. But their effectiveness or efficacy is still uncertain.²⁷ WeChat mini-programs embedded in the WeChat app work in a similar way to mobile apps, which could combine text messaging and web elements to create tailored, more interactive, and intensive interventions. Unlike regular apps, mini-programs do not require installation and can run inside the WeChat app,²⁸ which makes them more accessible and convenient and therefore more likely to be widely used. There are only two WeChat mini-programs focused on smoking cessation that have been reported in China (SCAMPI,²⁹ QuitAction³⁰). However, it is important to note that no clinical experts were involved in the development of the SCAMPI mini-program, and the QuitAction was developed solely on the basis of clinical smoking cessation guidelines without any behavioral change theories. These aspects may have affected their quality and reliability.

This study aimed to develop a WeChat mini-program for smoking cessation behavioral interventions (named the “Way to Quit” mini-program or WQ mini-program) based on behavioral change theories and clinical practice guidelines led by clinical experts on smoking cessation treatment. We also sought to evaluate its usability, acceptability, and preliminary efficacy for smoking cessation among Chinese smokers.

2. Methods

2.1 Development process

The WQ mini-program was funded as part of a World Health Organization project and developed by a multidisciplinary team that included clinical experts from a smoking cessation clinic in a general hospital in Beijing. This hospital has outstanding clinical and research capabilities in smoking cessation services, including setting up the first smoking cessation clinic and a national tobacco quitline in mainland China.¹² Other team members were software engineers from a professional medical technology service company. The development process consisted of three sequential steps.

Step 1. Building the WQ mini-program framework

We explored evidence-based behavioral interventions recommended in clinical practice guidelines for smoking cessation,^31,32 along with two classical theoretical models, the “COM-B model” (C, capability; O, opportunity; M, motivation; and B, behavior)³³ and the transtheoretical model (TTM).³⁴ The COM-B model proposes that three potential factors (capability, opportunity, and motivation) influence behavior and have been widely used to improve the design of behavior change interventions.³³ The TTM model posits that stage-matched interventions could increase effectiveness.³⁵ As our target population was smokers who planned to quit smoking within the next month, the WQ mini-program primarily offered interventions for people in the preparation, action, and maintenance stages of smoking cessation. Based on the behavior change wheel (BCW),³³ which is a useful tool for designing behavior change interventions, we selected intervention functions that were suitable for provision through the WeChat mini-program, along with evidence-based behavioral interventions for smoking cessation, such as cognitive therapy,³⁶ motivational intervention,³⁷ contingency management, and incentive-based interventions.³⁸ Next, we linked these interventions to behavioral change techniques (BCTs)³⁹ and found that 36 BCTs were deliverable (Table 1). To provide stage-matched interventions, all interventions were classified on the basis of the TTM. Finally, these interventions were transformed into corresponding functions, which formed the WQ mini-program demo.

Step 2. User-centric design process

Table 1.
Functions of the Way to Quit mini-program and corresponding COM^a components, BCW^b intervention functions, BCT^c codes, and stage of smoking cessation.

Functions Sub-functions COM components BCW intervention functions BCT codes ²¹ Stages

Registration Assess current smoking behavior NA Provide information for incentivization, training, and restriction RI1 Preparation

Assess history of quit attempts RI3

Assess the level of nicotine dependence RI1

Assess the current readiness BM9, RI2

Facilitating action plan Set a quit date Opportunity Restriction BS3, BS4, BM6 Action

Set a plan for reducing smoking Opportunity Restriction BS3, BS4 Preparation

List reasons for quitting Motivation Persuasion BM9, BS5 Preparation Action

Reset a quit plan Motivation Persuasion BM6 Maintenance

Recording quitting Record the duration of quitting Opportunity Enablement BM3, BS6 Action

Provide rewards for quitting Motivation Incentivization BM4 Action

Share quitting achievements Motivation Persuasion
Incentivization BM2, BM3, BM8 Action

Benefits of quitting Hazards reduced by quitting Motivation Education
Persuasion BM1, BM2, RD1 Action

Cost saving from quitting Motivation Incentivization BM1, BM2, RD1 Action

Health benefits of quitting Motivation Education
Persuasion
Incentivization BM1, BM2, RD1 Action

Coping with cravings Record smoking craving Capability Training
Enablement BS1, BS6 Action

Motivational messages Motivation Persuasion BM2, BS1 Action

Coping toolkit Capability Training
Enablement BS1, BS7, BS10, RD2 Action

Reducing smoking Calculator for daily smoking cigarettes Opportunity Enablement BM3, BS6 Preparation

Quitting diary Record smoking cues Opportunity Enablement BS1, BS6 Preparation

Advise on changing routine Opportunity Environmental restructuring BS7 Preparation

Quitting tasks Set graded tasks Capability Training BS9 Preparation
Action
Maintenance

Assess smoking cessation progress Capability Enablement BM3 Preparation
Action
Maintenance

Facilitate barrier identification and problem-solving Capability Training BS1, BS7, BS8, BS10, BS11, A1, A2, RI4, RC6, RC10 Preparation
Action
Maintenance

Strengthen ex-smoker identity Motivation Modeling BM8 Maintenance

Relapse prevention and coping BS2 Action
Maintenance

Medication reminder Assess medication(s) using experiences Opportunity Enablement A4 Preparation
Action
Maintenance

Set a medication use plan BS3

Reminder medication use A1

Additional supports Quitline Opportunity Enablement A5 Preparation
Action
Maintenance

Smoking cessation clinic Opportunity Enablement A3

Online consultation Capability Opportunity
Motivation Enablement A5, RC2

Online self-help materials Motivation
Capability Education BM1, BM5, RC5, RC6

Users Information Quitting information Capability Enablement BM3, RC9 Preparation
Action
Maintenance

Motivation to quit Motivation Persuasion BM9, RC9

Achievements Motivation Incentivization BM4, BM7, RC9

Functions	Sub-functions	COM components	BCW intervention functions	BCT codes ²¹	Stages
Registration	Assess current smoking behavior	NA	Provide information for incentivization, training, and restriction	RI1	Preparation
	Assess history of quit attempts	RI3
	Assess the level of nicotine dependence	RI1
	Assess the current readiness	BM9, RI2
Facilitating action plan	Set a quit date	Opportunity	Restriction	BS3, BS4, BM6	Action
Set a plan for reducing smoking	Opportunity	Restriction	BS3, BS4	Preparation
List reasons for quitting	Motivation	Persuasion	BM9, BS5	Preparation Action
Reset a quit plan	Motivation	Persuasion	BM6	Maintenance
Recording quitting	Record the duration of quitting	Opportunity	Enablement	BM3, BS6	Action
Provide rewards for quitting	Motivation	Incentivization	BM4	Action
Share quitting achievements	Motivation	Persuasion Incentivization	BM2, BM3, BM8	Action
Benefits of quitting	Hazards reduced by quitting	Motivation	Education Persuasion	BM1, BM2, RD1	Action
Cost saving from quitting	Motivation	Incentivization	BM1, BM2, RD1	Action
Health benefits of quitting	Motivation	Education Persuasion Incentivization	BM1, BM2, RD1	Action
Coping with cravings	Record smoking craving	Capability	Training Enablement	BS1, BS6	Action
Motivational messages	Motivation	Persuasion	BM2, BS1	Action
Coping toolkit	Capability	Training Enablement	BS1, BS7, BS10, RD2	Action
Reducing smoking	Calculator for daily smoking cigarettes	Opportunity	Enablement	BM3, BS6	Preparation
Quitting diary	Record smoking cues	Opportunity	Enablement	BS1, BS6	Preparation
Advise on changing routine	Opportunity	Environmental restructuring	BS7	Preparation
Quitting tasks	Set graded tasks	Capability	Training	BS9	Preparation Action Maintenance
Assess smoking cessation progress	Capability	Enablement	BM3	Preparation Action Maintenance
Facilitate barrier identification and problem-solving	Capability	Training	BS1, BS7, BS8, BS10, BS11, A1, A2, RI4, RC6, RC10	Preparation Action Maintenance
Strengthen ex-smoker identity	Motivation	Modeling	BM8	Maintenance
Relapse prevention and coping			BS2	Action Maintenance
Medication reminder	Assess medication(s) using experiences	Opportunity	Enablement	A4	Preparation Action Maintenance
Set a medication use plan		BS3
Reminder medication use		A1
Additional supports	Quitline	Opportunity	Enablement	A5	Preparation Action Maintenance
Smoking cessation clinic	Opportunity	Enablement	A3
Online consultation	Capability Opportunity Motivation	Enablement	A5, RC2
Online self-help materials	Motivation Capability	Education	BM1, BM5, RC5, RC6
Users Information	Quitting information	Capability	Enablement	BM3, RC9	Preparation Action Maintenance
Motivation to quit	Motivation	Persuasion	BM9, RC9
Achievements	Motivation	Incentivization	BM4, BM7, RC9

COM, capability, opportunity, motivation; ^bBCW, behavior change wheel; ^cBCTs, behavior change techniques

²¹

Michie S, Hyder N, Walia A, et al. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav 2011; 36(4):315-9.

The demo was assessed by potential users in accordance with the principles of user-centric design.⁴⁰ At this stage, 20 Chinese smokers (aged 18–50) who planned to quit within 1 month were recruited from the smoking cessation clinic in the study hospital. A scale of 0–5 points was used to rate the demand for each function (0 = not at all; 5 = very high demand). Appendix 1 shows that the proportions of high demand (4 points) and very high demand (5 points) for each function were all greater than 50% (range 56.3%–84.4%). The layout of functions was optimized according to demand ranking.

Step 3. Development of the WQ mini-program version 1.0

Finally, the WQ mini-program version 1.0 was developed based on WeChat mini-program design guidelines⁴¹ within 3 months. It included 11 functions: (1) registration, (2) facilitating action plan, (3) recording quitting, (4) benefits of quitting, (5) coping with cravings, (6) reducing smoking, (7) quitting diary, (8) quitting tasks, (9) medication reminder, (10) additional supports, and (11) user information. The registration function assessed baseline smoking behavior, readiness to quit, and nicotine dependence level. Functions (2) to (9) provided behavioral interventions, and function (10) (additional supports) provided links to other professional cessation services, such as smoking cessation clinics, quitline, and online counseling. Table 1 provides a detailed description of each function and corresponding COM components, BCW components, BCT codes, and smoking cessation stages. Figure 1 shows selected screenshots of the WQ mini-program. The WQ mini-program was accessed by scanning a quick response (QR) code (Appendix 2) or searching for Chinese “Jie Yan You Dao” in the WeChat app.

Figure 1.

Selected screenshots of the “Way to Quit” mini-program.

2.2 Preliminary evaluation procedures

2.2.1 Study design

A single-arm trial was conducted from December 2021 to February 2022 in China. The study protocol was approved by the Institutional Review Board of Beijing Chao-Yang Hospital, Capital Medical University (No. 2021-KE-260). All eligible participants were invited to click the “agree” button on the electronic consent form.

2.2.2 Participants and recruitment

Participants were recruited through online advertisements via WeChat official account. Inclusion criteria for smokers were (1) 18–50 years of age, (2) having smoked 100 or more cigarettes in their lifetime and currently smoking five or more cigarettes per day, (3) planning to quit within the next 30 days, (4) speaking Chinese, (5) having a smartphone and WeChat account, and (6) willing to complete follow-up questionnaires on time. Exclusion criteria were (1) using or planning to use other smoking cessation interventions (e.g., medications, quitline, e-cigarettes, other quitting apps and websites) and (2) psychological and mental illness or other substance dependence. Software experts were required to meet the following criteria: (1) having a minimum of 2 years of professional experience in developing mobile apps or WeChat mini-programs and (2) a master's degree or above. Exclusion criteria for software experts included participation in the design, development, and testing of the WQ mini-program.

2.2.3 Interventions

Participating smokers were required to use the WQ mini-program at least once a day for 4 weeks and could use any function as needed. Software experts were required to use all functions within 1 week.

To improve compliance, we invited all participating smokers to join a WeChat group chat, an online group chat feature embedded in the WeChat app. Two staff members (SC and HJ) guided and encouraged smokers to use the mini-program on a one-to-many basis. However, they would not provide smoking cessation counseling unless smokers experienced serious withdrawal symptoms (defined as intolerable withdrawal symptoms self-reported by smokers and determined by smoking cessation specialists that they needed intensive intervention). No participants left the group before the end of the study.

2.2.4 Data collection

For participating smokers, baseline data were collected through an online questionnaire sent via the WeChat app, including demographic information, smoking characteristics, comorbidities, and previous experience with smoking cessation apps. Nicotine dependence was measured by the Fagerstrӧm test for nicotine dependence (FTND).⁴² Participants completed the online follow-up questionnaire sent via the WeChat app once a week for 4 weeks. Follow-up information collected included smoking status, quitting attempts, and the WQ mini-program used in the past 7 days. Smoking status was measured by the question “Have you smoked a cigarette, even a puff, in the past 7 days?” Quit attempts were measured by the question “Have you tried to quit smoking for at least 24 h in the past week?” In addition to the functions that participants used, the data collected included the frequency and duration of their daily use. At the 4-week follow-up, they were required to complete an additional usability assessment questionnaire (Appendix 3) based on the System Usability Scale (SUS)⁴³ and the Mobile App Rating Scale (MARS).⁴⁴ The SUS is a 10-item scale with 100 points, which provides a global usability assessment of the system and focuses on users’ perceptions and satisfaction with the apps they use.⁴³ The MARS is a multidimensional tool that assesses the objective quality and content of mobile health apps across four subscales (Engagement, Functionality, Aesthetics, and Information), with responses to each item on a 5-point scale (1 = Inadequate, 2 = Poor, 3 = Acceptable, 4 = Good, 5 = Excellent).⁴⁴ Moreover, participants were encouraged to provide their views and optimization suggestions for the WQ mini-program with an open-ended question “Do you have any further comments on the mini-program? Please specify.” We have obtained permission from copyright holders to use all applicable tools/questionnaires and images.

Software experts were also asked to complete an online questionnaire delivered via the WeChat app that covered demographic information, smoking characteristics, and their previous experiences in developing or testing mobile apps or WeChat mini-programs. They were required to complete the same usability assessment questionnaire as participating smokers (Appendix 3) after using all functions.

To improve follow-up compliance, participants received 100 RMB (about $ 14.95) in the form of the WeChat red packet (Hongbao, which is similar to an electronic cash reward)⁴⁵ upon completing the follow-up questionnaires. The compensation had no impact on the use of the WQ mini-program, retention in the WeChat group, and cessation results.

2.2.5 Outcomes

Usability measures included the SUS score, the MARS total score, and sub-scores for participating smokers and software experts, as well as smokers’ self-reported use data during the study period. Acceptability of the WQ mini-program was assessed by measuring participating smokers’ willingness to recommend the mini-program to other smokers, their satisfaction scores, and their perceptions and suggestions. The primary outcome of efficacy for smoking cessation was self-reported 7-day point prevalence of abstinence (PPA) at the 4-week follow-up, defined as the proportion of smokers who reported not smoking in the past 7 days.⁴⁶ Secondary outcomes were self-reported 7-day PPA at 1-week, 2-week, and 3-week follow-up and quit attempt rate at each follow-up. The quit attempt rate was defined as the percentage of smokers who made at least one attempt to quit (over 24 h) while continuing to smoke^.⁴⁷

2.2.6 Statistical analysis

Analyses were performed using SPSS 22.0 (IBM Corp LP, USA). Data were presented as mean and standard deviation for continuous variables with normal distribution, median (interquartile range, IQR) for continuous variables without normal distribution, and proportions for categorical variables. Kruskal–Wallis tests were used to examine differences in mean MARS scores by different software experts, described as the intra-class correlation coefficient (ICC). Those who lost follow-up were classified as continuing smokers without attempting to quit and were included in efficacy analysis but excluded from usability and acceptability analysis. Statistical significance in this study was defined as a significance level of 0.05 (two-tailed).

Thematic analysis followed Braun and Clarke's six-phase framework.⁴⁸ Responses to the open-ended question were independently coded by two members of the research team (SC and LF). The themes were readily apparent due to the descriptive nature of the data, and the researchers reached consensus after a single discussion without any differences.

3. Results

3.1 Participants’ characteristics

A total of 50 smokers and five software experts participated in the baseline survey. Forty-eight smokers (96.0%) completed four follow-ups, and all software experts completed the usability assessment questionnaire.

Among smokers, the mean age was 34.8 ± 8.3 years old. The majority of smokers were male (48/50, 96.0%) and had a college degree or higher (42/50, 84.0%). They had smoked for a median duration of 14.5 (IQR 9.4–20.0) years, consumed a median of 15.0 (IQR 10.0–20.0) cigarettes per day, and had a moderate or severe nicotine dependence (FTND score ≥4) (39/50, 78%). Most smokers (37/50, 74%) had a history of quitting within the past year. However, few (5/50, 13.5%) used mobile apps to help quit smoking (Table 2).

Table 2.
Demographic, smoking characteristics, and comorbidities of the participating smokers.

Characteristics n = 50

Demographic characteristics

Sex, n (%)

Male 48 (96.0)

Female 2 (4.0)

Age (years), mean ± SD 34.8 ± 8.3

18–30 16 (32.0)

31–40 25 (50.0)

41–50 9 (18.0)

Education, n (%)

Middle school and below 2 (4.0)

High school 6 (12.0)

College and above 42 (74.0)

Occupations, n (%)

Business/service staff 21 (42.0)

Administrative staff 11 (22.0)

Technical staff 9 (18.0)

Others^a 9 (18.0)

Smoking characteristics

Cigarettes per day, median (IQR) 15.0 (10.0, 20.0)

Smoking duration (years), median (IQR) 14.5 (9.4, 20.0)

FTND^b score, n (%)

0–3 11 (22.0)

4–6 37 (74.0)

7–10 2 (4.0)

Prior quit attempt, n (%)

Yes 37 (74.0)

No 13 (26.0)

Smoking cessation interventions used, n (%)

Cold-turkey 26 (70.3)

E-cigarette 8 (21.6)

Medications^c 8 (21.6)

Counseling 6 (16.2)

Quitline 6 (16.2)

Smartphone apps 5 (13.5)

Comorbidities, n (%)

With at least one comorbidity 10 (20.0)

Hypertension 7 (14.0)

Hyperlipemia 3 (6.0)

COPD^d 1 (2.0)

Others^e 2 (4.0)

Characteristics	n = 50
Demographic characteristics
Sex, n (%)
Male	48 (96.0)
Female	2 (4.0)
Age (years), mean ± SD	34.8 ± 8.3
18–30	16 (32.0)
31–40	25 (50.0)
41–50	9 (18.0)
Education, n (%)
Middle school and below	2 (4.0)
High school	6 (12.0)
College and above	42 (74.0)
Occupations, n (%)
Business/service staff	21 (42.0)
Administrative staff	11 (22.0)
Technical staff	9 (18.0)
Others^a	9 (18.0)
Smoking characteristics
Cigarettes per day, median (IQR)	15.0 (10.0, 20.0)
Smoking duration (years), median (IQR)	14.5 (9.4, 20.0)
FTND^b score, n (%)
0–3	11 (22.0)
4–6	37 (74.0)
7–10	2 (4.0)
Prior quit attempt, n (%)
Yes	37 (74.0)
No	13 (26.0)
Smoking cessation interventions used, n (%)
Cold-turkey	26 (70.3)
E-cigarette	8 (21.6)
Medications^c	8 (21.6)
Counseling	6 (16.2)
Quitline	6 (16.2)
Smartphone apps	5 (13.5)
Comorbidities, n (%)
With at least one comorbidity	10 (20.0)
Hypertension	7 (14.0)
Hyperlipemia	3 (6.0)
COPD^d	1 (2.0)
Others^e	2 (4.0)

Others include workers, farmers, students, retirees, etc.

FTND, Fagerström test of nicotine dependence.

Medications include nicotine replacement therapy, bupropion, and varenicline.

Chronic obstructive pulmonary disease.

Including gastric ulcer and arthritis.

All software experts were male and under 30 years. Four experts were software developers, and one was a software product designer. One expert was a current smoker, and the others had never smoked.

3.2 Usability

The mean SUS and MARS total scores for the WQ mini-program as evaluated by smokers were 82.1 ± 13.8 and 84.5 ± 3.3 and by software experts were 4.21 ± 0.32 and 4.27 ± 0.15, respectively. Among smokers, the mean MARS sub-scores (Engagement, Functionality, Aesthetics, and Information) ranged from 4.04 to 4.41, while among software experts, these scores ranged from 4.13 to 4.40 (Figure 2). The usability rating of each function ranged from 4.02 to 4.58 among smokers and from 3.20 to 5.00 among software experts (Figure 3). Overall, smokers (44/48, 91.7%) rated the mini-program four stars or higher. In terms of the mean MARS total score, the ICC of the five software experts was 0.83 (95% confidence interval 0.75–0.89), indicating a high degree of reliability.

Figure 2.

MARS sub-scores of the “Way to Quit” mini-program among smokers and software experts. Note: MARS is the Mobile App Rating Scale.

Figure 3.

Usability score of each functional module on four dimensions of the “Way to Quit” mini-program among smokers and software experts.

During the study period, most smokers reported using the mini-program one to five times (range 68.8%–75.0%) for 1–5 min (range 43.8%–68.8%) daily. The functions with a high proportion of self-reported use were “Recording quitting” (83.0%–89.6%), “Coping with cravings” (54.2%–63.8%), and “Benefits from quitting” (36.2%–52.1%) (Table 3).

Table 3.

Self-reported usage of the Way to Quit WeChat mini-program (n = 48).

Usage data	1-week	2-week	3-week	4-week
Usage times per day (n, %)
1–5	37 (87.1)	35 (72.9)	37 (77.1)	39 (81.2)
>5	11 (22.9)	13 (27.1)	11 (22.9)	9 (18.8)
Usage duration per day (n, %)
<1 min	5 (10.4)	8 (16.7)	11 (22.9)	10 (20.8)
1–5 min	33 (68.8)	28 (58.3)	21 (43.8)	25 (52.1)
>5 min	10 (20.8)	12 (25.0)	16 (33.4)	13 (27.1)
Function used
Recording quitting	40 (83.3)	39 (83.0)	42 (89.4)	43 (89.6)
Coping with cravings	26 (54.2)	30 (63.8)	30 (63.8)	26 (54.2)
Benefits of quitting	25 (52.1)	21 (44.7)	17 (36.2)	22 (45.8)
Quitting diary	14 (29.2)	8 (17.0)	11 (23.4)	10 (20.8)
Quitting tasks	12 (25.0)	11 (23.4)	8 (17.0)	12 (25.0)
Additional supports	6 (12.5)	3 (6.4)	6 (12.5)	9 (18.8)

3.3 Acceptability

Most smokers thought the WQ mini-program was attractive (40/48, 83.3%) and would recommend it to other smokers (41/48, 85.4%). In addition, most (46/48, 95.8%) reported that they would continue to use the mini-program in the future. The mean satisfaction score for the WQ mini-program was 4.23 ± 0.69 (out of 5 points) among smokers.

Smokers and software experts offered many suggestions for optimization. Over 16 unique codes were identified, and four themes emerged: (1) Functionality, (2) Operation, (3) Aesthetics, and (4) Information. The majority of smokers’ suggestions focused on adding new features, improving the design, and gathering more information (Appendix 4). The software experts’ suggestions focused on improving the functional design, adding more operations to existing functions, and optimizing the function interface design (Appendix 5).

3.4 Preliminary efficacy

The self-reported 7-day PPA at 4-week follow-up was 50.0% (25/50) and at 1-week, 2-week, and 3-week follow-up were 40.0% (20/50), 42.0% (21/50), and 44.0% (22/50), respectively. In addition, another 44.0% (22/50), 46.0% (23/50), 46.0% (23/50), and 38% (19/50) of smokers reported that they did not quit smoking but made a quit attempt at the 1-week, 2-week, 3-week, and 4-week follow-up, respectively (Figure 4).

Figure 4.

Self-reported 7-day PPA and quit attempt rate among smokers. Note: 7-day PPA is 7-day point prevalence abstinence.

4. Discussion

This single-arm trial described the development and preliminary evaluation of a WeChat mini-program for smoking cessation interventions (WQ mini-program) in China. It was found that smokers rated the WQ mini-program usability and satisfaction very high. Half of smokers reported quitting after using the mini-program for 4 weeks, and another 40% reported making attempts to quit. These findings demonstrated the feasibility, acceptability, and preliminary efficacy of the WQ mini-program for smoking cessation in Chinese smokers.

This study found that most smokers highly rated the usability of the WQ mini-program. A mean SUS score of over 80 indicated that the overall usability of the WQ mini-program was excellent.⁴³ The mean MARS total score and sub-scores were over 4, demonstrating the high quality of the WQ mini-program on aesthetics, functionality, engagement, and information.⁴⁴ Overall satisfaction with the mini-program was relatively high. In addition, a majority of smokers were willing to recommend this mini-program to other smokers and would continue to use it. Good usability and acceptability may be due to the user-centric design process,⁴⁰ which has made the mini-program more likely to meet the needs of potential users. In addition, the system-level usability assessment was objectively conducted by five software experts. The usability evaluation results among software experts were essentially similar to those of smokers. However, software experts had relatively low ratings for the usability of “quitting diary” and “quitting tasks” functions. This indicated that the design of these functions still needed to be optimized. Furthermore, the thematic analysis for optimization suggestions showed that the WQ mini-program still had room for improvement. For example, some existing functions need to be optimized (e.g., the button for coping with cravings cannot be clicked continuously), and many new functions have been suggested to be added (e.g., the function to share quit achievements should be to share pictures directly without saving them). These useful suggestions could help us further optimize the mini-program.

This study demonstrated a promising high self-reported quit rate. Self-reported 7-day PPA at the 4-week follow-up was 50%. Compared with studies focused on cessation apps, the abstinence rate at 4 weeks in this study was higher than that in some studies (15.9%–31.6%),^49–53 comparable with others (43.8%–44.5%),^54,55 and lower than a study from Japan (71.6%).⁵⁶ Variations in abstinence rates may be due to differences in intervention and study design. Most studies with lower abstinence rates may be due to the lack of involvement of clinical smoking cessation experts in app development. Prior studies have found that input from healthcare professionals during the development of mHealth apps can ensure that apps are evidence-driven, contain reliable content, and are of high quality.⁵⁷ In the study conducted in Japan, significantly higher abstinence rates can be attributed to the inclusion of cessation medication and intensive face-to-face counselling.⁵⁶ This suggested that combining smoking cessation apps with pharmacotherapy and intensive smoking cessation counseling could potentially further increase quit rates. Compared to WeChat app-based studies, the abstinence rate of the WQ mini-program was higher than text message-based interventions delivered by the WeChat app (11.9%,²¹ 20.6%²⁰). Currently, only two WeChat mini-programs for smoking cessation (SCAMPI, QuitAction) have been reported. The abstinence rate in our study was higher than QuitAction (26.1%) which was developed solely on the basis of clinical smoking cessation guidelines³⁰ and comparable to that for SCAMPI (63.0%) which was also developed on the basis of the COM-B model, BCW, and clinical smoking cessation guidelines.²⁹ However, it should be noted that this study and the study of QuitAction were single-arm clinical trials,³⁰ while the SCAMPI evaluation was a pilot randomized controlled trial (RCT).²⁹ Therefore, the efficacy of the WeChat mini-program for smoking cessation needs to be further confirmed in RCTs with large sample sizes.

The higher smoking cessation rate in this study can be attributed to the following factors. First, it was developed using behavioral change theories (COM-B³³ and TTM³⁴) and clinical practice guidelines.^21,22 In addition, two useful tools for designing behavioral change interventions (BCW³³ and BCTs³⁹) were used to build the WQ mini-program framework. This meant the WQ mini-program provided evidence-based, stage-matched, and tailored smoking cessation interventions that fit the individual's needs. Numerous studies have demonstrated the effectiveness or efficacy of cessation apps.^49–56 However, despite the significant increase in the number of cessation apps, some studies have found that most of the apps available in app stores were of poor quality and did not comply with treatment guidelines or BCTs.^58,59 Therefore, it is necessary to develop cessation apps based on clinical smoking cessation guidelines and BCTs. Second, experts in clinical smoking cessation interventions contributed to the development of the intervention content and provided extensive experience in smoking cessation treatment, ensuring that behavioral interventions were effective and reliable. In addition, the WQ mini-program also provided convenient referrals for smokers in need of more intensive interventions, such as online appointments at smoking cessation clinics, a “One Click to Call” quitline, and online cessation counseling. In this way, traditional smoking cessation services can be integrated with a mHealth-based approach, which is expected to be more effective than just using the WeChat mini-program.

This study had several limitations. Firstly, we evaluated participants’ smoking abstinence based on self-report rather than biological validation. Inevitably, this will lead to measurement bias as smokers may have misrepresented their quitting status,⁶⁰ resulting in an overestimation of abstinence rates. Second, this was a single-arm trial rather than an RCT, so we could only assess the preliminary efficacy of the WeChat mini-program and not provide conclusive evidence. However, this study provides valuable insights for future RCTs to verify its efficacy. Third, we only followed participants for 4 weeks, so we could not evaluate the long-term efficacy of the WQ mini-program. Fourth, all participating smokers were young to middle-aged and highly educated, and it is unclear whether the WQ mini-program would be feasible and effective for older smokers and those with low literacy. Fifth, the Chinese translation versions of the SUS and MARS scales used in this study were not validated. Finally, the current version of the WQ mini-program is unable to record users’ interest in features, frequency of use, or time spent on each access. As a result, we were unable to determine which features of mini-program use contributed to successful quitting.

Despite these limitations, with the WeChat app having more than 1 billion users,¹⁸ 450 million of whom are daily active WeChat mini-program users,⁶¹ and the WQ mini-program has the potential to be used in a wider area, which would address the lack of smoking cessation services in China.

5. Conclusion

This study demonstrated that the WQ mini-program would be a feasible and potentially effective method to encourage Chinese smokers. Its efficacy on smoking cessation needs to be further verified in a well-conducted RCT after it has been optimized according to the results of this study.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231208553 - Supplemental material for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Supplemental material, sj-docx-1-dhj-10.1177_20552076231208553 for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial by Shuilian Chu, Lin Feng, Hang Jing, Di Zhang, Zhaohui Tong and Lirong Liang in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076231208553 - Supplemental material for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Supplemental material, sj-docx-2-dhj-10.1177_20552076231208553 for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial by Shuilian Chu, Lin Feng, Hang Jing, Di Zhang, Zhaohui Tong and Lirong Liang in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076231208553 - Supplemental material for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Supplemental material, sj-docx-3-dhj-10.1177_20552076231208553 for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial by Shuilian Chu, Lin Feng, Hang Jing, Di Zhang, Zhaohui Tong and Lirong Liang in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076231208553 - Supplemental material for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Supplemental material, sj-docx-4-dhj-10.1177_20552076231208553 for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial by Shuilian Chu, Lin Feng, Hang Jing, Di Zhang, Zhaohui Tong and Lirong Liang in DIGITAL HEALTH

Supplemental Material

sj-docx-5-dhj-10.1177_20552076231208553 - Supplemental material for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial

Supplemental material, sj-docx-5-dhj-10.1177_20552076231208553 for A WeChat mini-program-based approach to smoking cessation behavioral interventions: Development and preliminary evaluation in a single-arm trial by Shuilian Chu, Lin Feng, Hang Jing, Di Zhang, Zhaohui Tong and Lirong Liang in DIGITAL HEALTH

Footnotes

Acknowledgments

We would like to thank the entire “Way to Quit” team, including the research team (besides SC, LF, HJ, DZ, ZT, and LL) and the app developer team from Yi Lai Yi Wang Co. Ltd.

Contributorship

LL, SC, and ZT researched the literature and conceived the study. LF, HJ, and DZ were involved in protocol development, ethical approval, patient recruitment, and data analysis. SC wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The ethics committee of Beijing Chao-Yang Hospital, Capital Medical University, approved this study (REC number: 2021-KE-260).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the World Health Organization, the National Health Commission of the People's Republic of China, Beijing Key Specialists in Major Epidemic Prevention and Control, and the Reform and Development Program of Beijing Institute of Respiratory Medicine (ysrh2022014).

Guarantor

LRL

ORCID iD

Shuilian Chu

Supplemental material

Supplemental material for this article is available online.

References

U.S. Department of Health and Human Services. The Health Benefits of Smoking Cessation. Surgeon General’s Report on Smoking and Health. Washington, DC: Superintendent of Documents, U.S. Government Printing Office, 1990.

Hughes

Keely

Naud

. Shape of the relapse curve and long-term abstinence among untreated smokers. Addiction 2004; 99: 29–38.

World Health Organization. WHO Report on the Global Tobacco Epidemic, 2008: The MPOWER package. Geneva: World Health Organization, 2009.

Clinical Practice Guideline Treating Tobacco Use and Dependence 2008 Update Panel, Liaisons, and Staff. A clinical practice guideline for treating tobacco use and dependence: 2008 update. A U.S. Public health service report. Am J Prev Med 2008; 35: 158–176.

Schroeder

. What to do with a patient who smokes. JAMA 2005; 294: 482–487.

World Health Organization. WHO Report on the Global Tobacco Epidemic, 2021: Addressing new and emerging products. Geneva: World Health Organization, 2021.

Chinese Center for Disease Control and Prevention. 2018 GATS China report. Beijing: Chinese Center for Disease Control and Prevention; 2018.

Lin

Xiao

Liu

, et al. National survey of smoking cessation provision in China. Tob Induc Dis 2019; 17: 25.

Wang

Nan

Yang

, et al. Quitline activity in China. Asian Pac J Cancer Prev 2016; 17: 7–9.

10.

Nan

Wang

, et al. Effectiveness evaluation on quitline services in four Chinese cities. Chin J Health Educ 2015; 31: 256–258. [Chinese].

11.

Jing

Zhang

Liang

, et al. Analysis of the changes in the number of calls and the characteristics of callers on China's tobacco quitline. Chinese Journal of Prevention and Control of Chronic Diseases 2020; 28: 911–916. [Chinese].

12.

Chen

Xiao

Henderson

, et al. Characteristics of callers accessing the tobacco cessation quitline in mainland China. Biomed Environ Sci 2013; 26: 697–701.

13.

Tian

Pei

Zheng

, et al. Development and status analysis of 12320 health hotline in smoking cessation service. Chin J Health Educ 2021; 37: 737–739.[Chinese].

14.

Momin

Neri

McCausland

, et al. Traditional and innovative promotional strategies of tobacco cessation services: A review of the literature. J Community Health 2014; 39: 800–809.

15.

Weinstein

Lopez

Joseph

, et al. Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers. Am J Med 2014; 127: 183–187.

16.

El-Toukhy

. Insights from the SmokeFree.gov initiative regarding the use of smoking cessation digital platforms during the COVID-19 pandemic: cross-sectional trends analysis study. J Med Internet Res 2021; 23: e24593.

17.

Whittaker

McRobbie

Bullen

, et al. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database Syst Rev 2019; 10: CD006611.

18.

TechWeb. WeChat Chinese and International versions combined monthly active users over 1.2 billion, mini programme daily users over 400 million. https://tech.sina.cn/2020-05-13/detail-iircuyvi2911382.d.html (2020).

19.

Luo

Williams

, , et al. A WeChat-based smoking cessation intervention for Chinese smokers: A feasibility study. Transl Behav Med 2022; 12: 1018–1027.

20.

Luo

Williams

, et al. A WeChat-based smoking cessation intervention for Chinese smokers: A pilot study. Internet Interv 2022; 28: 100511.

21.

Lin

Liu

Zhang

, et al. Assessment of a text message-based smoking cessation intervention for adult smokers in China: A randomized clinical trial. JAMA Netw Open 2023; 6: e230301.

22.

Chu

Tong

Zhang

, et al. Usage, acceptability, and preliminary effectiveness of an mHealth-based integrated modality for smoking cessation interventions in western China. Tob Induc Dis 2023; 21: 07.

23.

Zeng

. Application of micro message group in smoking cessation of young and middle-aged patients with coronary intervention after discharge. Chinese Clinical Nursing 2014; 5: 389–390. 391[Chinese].

24.

May

West

. Do social support interventions (“buddy systems”) aid smoking cessation? A review. Tob Control 2000; 9: 415–422.

25.

Pechmann

Delucchi

Lakon

, et al. Randomised controlled trial evaluation of Tweet2Quit: A social network quit-smoking intervention. Tob Control 2017; 26: 188–194.

26.

Baskerville

Azagba

Norman

, et al. Effect of a digital social Media campaign on young adult smoking cessation. Nicotine Tob Res 2016; 18: 351–360.

27.

Naslund

Kim

Aschbrenner

, et al. Systematic review of social media interventions for smoking cessation. Addict Behav 2017; 73: 81–93.

28.

Cheng

Ren

Hong

, et al. An exploratory analysis of travel-related WeChat Mini program usage: affordance theory perspective. In: Information and communication technologies in tourism 2019. Cham: Springer International Publishing, 2019, pp.333–343.

29.

Chen

Jiang

, et al. A Mobile social network-based smoking cessation intervention for Chinese male smokers: protocol for a pilot randomized controlled trial. JMIR Res Protoc 2020; 9: e18071.

30.

Xie

Qiu

Zhu

, et al. Evaluation of the smoking cessation effects of QuitAction, a smartphone WeChat platform. Tob Induc Dis 2023; 21: 49.

31.

U.S. Department of Health and Human Services. Smoking cessation: A Report of the Surgeon General. Washington, DC: Superintendent of Documents, U.S. Government Printing Ofﬁce, 2020.

32.

National Health and Family Planning Commission of China. Clinical Guideline for Smoking Cessation in China. http://www.cfchina.org.cn/uploadfile/2016/0622/%E4%B8%AD%E5%9B%BD%E4%B8%B4%E5%BA%8A%E6%88%92%E7%83%9F%E6%8C%87%E5%8D%9720150515-final.pdf (2015).

33.

Michie

van Stralen

West

. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci 2011; 6: 42.

34.

Prochaska

DiClemente

Norcross

. In search of how people change. Applications to addictive behaviours. Am Psychol 1992; 47: 1102–1114.

35.

Cahill

Lancaster

Green

. Stage-based interventions for smoking cessation. Cochrane Database Syst Rev 2010; 11: CD004492.

36.

Butler

Chapman

Forman

, et al. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clin Psychol Rev 2006; 26: 17–31.

37.

Lindson-Hawley

Thompson

Ferrey

, et al. Motivational interviewing for smoking cessation. Cochrane Database Syst Rev 2019; 7: CD006936.

38.

Ainscough

Brose

Strang

, et al. Contingency management for tobacco smoking during opioid addiction treatment: a randomised pilot study. BMJ Open 2017; 7: 017467.

39.

Michie

Hyder

Walia

, et al. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav 2011; 36: 315–319.

40.

Kurniawan

. Interaction Design: Beyond Human-Computer Interaction. New York, USA: Universal Access in the Information Society, 2004.

41.

WeChat Mini programme Design Guideline. Tencent Inc. https://developers.weixin.qq.com/miniprogram/design/ (2019).

42.

Huang

Lin

Wang

. The psychometric properties of the Chinese version of the fagerstrӧm test for nicotine dependence. Addict Behav 2006; 31: 2324–2327.

43.

Borsci

Federici

Lauriola

. On the dimensionality of the system usability scale: A test of alternative measurement models. Cogn Process 2009; 10: 193–197.

44.

Stoyanov

Hides

Kavanagh

, et al. Mobile app rating scale: A new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015; 3: e27.

45.

Eveline Chao. How WeChat Became China's App For Everything. Fast Company. https://www.fastcompany.com/3065255/china-wechat-tencent-red-envelopes-and-social-money (2017).

46.

Hughes

Keely

Niaura

, et al. Measures of abstinence in clinical trials: issues and recommendations. Nicotine Tob Res 2003; 5: 13–26.

47.

Zhang

Vickerman

Malarcher

, et al. Intermediate cessation outcomes among quitline callers during a national tobacco education campaign. Nicotine Tob Res 2014; 16: 1478–1486.

48.

Braun

Clarke

. Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77–101.

49.

Chulasai

Chinwong

, et al. Feasibility of a smoking cessation smartphone app (quit with US) for young adult smokers: A single arm, Pre-post study. Int J Environ Res Public Health 2021; 18: 9376.

50.

Schwaninger

Berli

Scholz

, et al. Effectiveness of a dyadic buddy app for smoking cessation: randomized controlled trial. J Med Internet Res 2021; 23: e27162.

51.

Etter

Khazaal

. The stop-tabac smartphone application for smoking cessation: a randomized controlled trial. Addiction 2022; 117: 1406–1415.

52.

BinDhim

McGeechan

Trevena

. Smartphone smoking cessation application (SSC app) trial: A multicountry double-blind automated randomised controlled trial of a smoking cessation decision-aid ‘app’. BMJ Open 2018; 8: e017105.

53.

Bricker

Watson

Mull

, et al. Efficacy of smartphone applications for smoking cessation: A randomized clinical trial. JAMA Intern Med 2020; 180: 1472–1480.

54.

Webb

Peerbux

Smittenaar

, et al. Preliminary outcomes of a digital therapeutic intervention for smoking cessation in adult smokers: randomized controlled trial. JMIR Ment Health 2020; 7: e22833.

55.

Chulasai

Chinwong

Vientong

, et al. Smartphone application for smoking cessation (quit with US): A randomized controlled trial among young adult light smokers in Thailand. Int J Environ Res Public Health 2022; 19: 8265.

56.

Masaki

Tateno

Nomura

, et al. A randomized controlled trial of a smoking cessation smartphone application with a carbon monoxide checker. NPJ Digit Med 2020; 3: 35.

57.

Ahmed

Ahmad

Ali

, et al. Medication adherence apps: review and content analysis. JMIR Mhealth Uhealth 2018; 6: e62.

58.

Rajani

Weth

Mastellos

, et al. Adherence of popular smoking cessation mobile applications to evidence-based guidelines. BMC Public Health 2019; 19: 743.

59.

Seo

Cho

Yoon

, et al. Classification of smoking cessation apps: quality review and content analysis. JMIR Mhealth Uhealth 2022; 10: e17268.

60.

Connor Gorber

Schofield-Hurwitz

Hardt

, et al. The accuracy of self-reported smoking: A systematic review of the relationship between self-reported and cotinine-assessed smoking status. Nicotine Tob Res 2009; 11: 12–24.

61.

Daxueconsulting. Leveraging WeChat mini programs for enhanced brand engagement and independence. https://daxueconsulting.com/wechat-mini programs-2020-report/(2023).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB

0.18 MB

0.03 MB

0.02 MB