Sage Journals: Discover world-class research

Abstract

Background:

Limited data are available to help identify patients with schizophrenia who are most likely to benefit from long-acting injectable antipsychotics.

Aim:

To investigate the efficacy of long-acting injectable antipsychotic paliperidone palmitate one-month formulation for preventing relapses, factors influencing time to first relapse, and the effect of different antipsychotic adherence levels on time to first relapse in Chinese patients with schizophrenia.

Methods:

This was a post-hoc analysis from an open-label, single-arm study of stable patients (Positive and Negative Syndrome Scale total score <70; n=367) receiving paliperidone palmitate one-month formulation at the end of an acute 13-week treatment phase, who entered a naturalistic one-year follow-up period, either continuing with flexibly dosed paliperidone palmitate one-month formulation (75–150 mg eq.) or switching to another antipsychotic(s).

Results:

There were 362/367 patients (age=31.4±10.75 years) included in the analysis of time to first relapse (primary outcome) and 327/362 patients (39/327, poor antipsychotic adherence (<80%)) willing to receive antipsychotics were included in the exposure/adherence analysis. Overall, 84.6% (95% confidence interval=79.2–88.7) patients remained relapse-free. Poor adherence during follow-up (hazard ratio=2.97, 95% confidence interval=1.48–5.98, p=0.002) and frequent hospitalizations in the previous year (hazard ratio=1.29, 95% confidence interval=1.02–1.62, p=0.03) were associated with a significant risk of shorter time to first relapse in the univariate analysis. In patients with poor adherence, ‘no use’ (hazard ratio=13.13, 95% confidence interval=1.33–129.96, p=0.03) and ‘interrupted use’ (hazard ratio=11.04, 95% confidence interval=1.03–118.60, p=0.047) of paliperidone palmitate one-month formulation (vs continued use) showed a significantly higher risk of relapse; this was not observed in patients with good (≥80%) antipsychotic adherence. No new safety concerns were identified.

Conclusion:

Continued use of paliperidone palmitate one-month formulation/long-acting injectable antipsychotic was effective in preventing schizophrenia relapses, especially in patients with suboptimal antipsychotic adherence.

Keywords

Adherence long-acting injectable antipsychotic paliperidone palmitate Positive and Negative Syndrome Scale relapse prevention schizophrenia

Introduction

Symptom exacerbations or relapses in schizophrenia during maintenance treatment often lead to inferior outcomes and increased hospitalization risk (Carbon and Correll, 2014; Kane, 2013; Nasrallah and Lasser, 2006). The long-term consequences of relapses include increased risk of self-harm, decline in psychosocial and occupational abilities, increasing personal and healthcare burden, social stigmatization, and progressive clinical deterioration (Almond et al., 2004; Kane, 2007; Thornicroft et al., 2009). Thus, delaying the time to relapse and achieving optimal symptom control are fundamental objectives of schizophrenia management that are recommended by existing clinical practice guidelines (Kane and Garcia-Ribera, 2009; Malla et al., 2013).

Poor adherence (<70% to 80% of prescribed medications) or nonadherence to antipsychotics is recognized as one of the strongest risk factors for relapses in schizophrenia (Agid et al., 2010; Ascher-Svanum et al., 2009; Kane et al., 2013; Novick et al., 2010). Medication discontinuations substantially increase the risk of relapse during the critical early intervention period (first 2–5 years of illness) that determine the long-term course of treatment outcome in schizophrenia (Birchwood et al., 1998; Chien et al., 2016). In a survival analysis of relapses, stopping antipsychotic medications was the strongest predictor of relapses, increasing the risk by five-fold compared with patients continuing antipsychotic treatment (hazard ratio (HR) to first relapse=4.89, 99% confidence interval (CI)=2.49–9.60) (Robinson et al., 1999). Similar results were obtained in another study of first-episode patients with schizophrenia, where nonadherence to antipsychotic treatment was the only predictor of relapse, also increasing the relapse risk by about five-fold (HR=4.8, 95% CI=2.9–7.7) (Caseiro et al., 2012). Medication nonadherence was also the greatest risk factor associated with relapse (odds ratio (OR)= 4.6, 95% CI=3.4–6.2) in a one-year real-world study conducted in China (Xiao et al., 2015). Additionally, results from a survey of 1854 psychiatrists from the People’s Republic of China reported that nearly 56% (range: 30–71%) of patients with schizophrenia were either partially or fully non-adherent to medications (Olivares et al., 2013).

The use of long-acting injectable (LAI) antipsychotics at various stages of schizophrenia (patients with first-episode, insufficient response to oral antipsychotics, or history of relapse) has been shown to reduce nonadherence and subsequent episodes of relapses and hospitalizations (Correll et al., 2016; Heres et al., 2014). Extensive studies have demonstrated the therapeutic efficacy of paliperidone palmitate one-month formulation (PP1M), an atypical LAI antipsychotic (dopamine (D2) and serotonin (5-HT2A) receptor antagonist), along with practical advantages of extended durations of therapeutic plasma levels, simplified medication schedules, and efficient monitoring of treatment schedules (Agid et al., 2010; Alphs et al., 2013; Gonzalez-Rodriguez et al., 2015; Samtani et al., 2009). Evidence for effective relapse prevention with PP1M supporting maintenance treatment is accrued largely from studies of non-Asian patients (Alphs et al., 2015; Gopal et al., 2011; Hough et al., 2010; McEvoy et al., 2014).

This post-hoc analysis was conducted in the one-year observational follow-up phase following the 13-week, acute treatment phase of a large prospective, open-label study of flexibly dosed PP1M in Chinese patients with acute schizophrenia showing unsatisfactory response to prior antipsychotics (Si et al., 2015b). The role of PP1M in the prevention of schizophrenia relapse over a one-year period among patients who continued treatment with PP1M or any antipsychotic (PP1M switched to other antipsychotics) was explored in the follow-up phase analyses using a naturalistic design. Further, predictors of relapse prevention, including the relationship between the proportion of PP1M medication usage and time to first relapse (TFR) in patients with different degrees of antipsychotic adherence, was also characterized. Overall, the primary objective was to evaluate the impact of PP1M versus other antipsychotics, in preventing schizophrenia relapse in patients with different levels of adherence and determine factors influencing TFR.

Methods

Patients and study design

Data from an open-label, single-arm, multicenter phase 4 study (NCT01685931) were analyzed. The study was conducted between October 2012–December 2014 at 22 sites in the People’s Republic of China and consisted of three phases: a screening phase of up to seven days, a 13-week acute treatment phase, and a one-year observational follow-up phase that provided data for the current analyses.

Details of study design and primary results have been reported previously (Si et al., 2015b). Briefly, adults (18–65 years, inclusive) with a diagnosis of acute schizophrenia based on Diagnostic and Statistical Manual of Mental Disorders-IV-Text Revision (DSM-IV-TR; American Psychiatric Association, 2000) criteria who lacked a satisfactory therapeutic effect to previous oral antipsychotic treatment (i.e. Positive and Negative Syndrome Scale (PANSS) total score of 70–120 at screening and baseline) were eligible. Major exclusion criteria were: active, non-schizophrenia DSM-IV-TR axis 1 diagnosis; attempted suicide within 12 months before screening, or at imminent risk of suicidal or violent behavior; risk factors for prolonged QT interval, torsades de pointes, or sudden death; treatment with clozapine or monoamine oxidase inhibitor antidepressants within one month before screening; use of LAI antipsychotics including PP1M (within six injection intervals before screening); or use of electroconvulsive therapy within one month before screening. For the present post-hoc analysis, symptomatically stable patients (PANSS total score <70) at week 13 who continued in the follow-up phase and received PP1M or any antipsychotic in the first month of the maintenance period after the 13-week acute phase were included.

An independent ethics committee at each study center approved the study protocol. This study was conducted in accordance with the ethical principles that have their origin in the Declaration of Helsinki and in accordance with International Conference on Harmonization’s Good Clinical Practice guidelines and applicable regulatory requirements. All patients provided written informed consent prior to study enrollment.

Treatments

Eligible patients received (Invega Sustenna, Janssen, Division of Ortho-McNeil-Janssen Pharmaceuticals, Inc.) deltoid injections of PP1M at a dose of 150 mg eq. of paliperidone (PP) (equal to 234 mg of PP) on day 1 and 100 mg eq. (156 mg of PP) on day 8 followed by a monthly maintenance flexible dose between 75–150 mg eq. (117–234 mg of PP) of PP (deltoid or gluteal injections) during the acute 13-week treatment phase. All oral antipsychotics taken prior to study entry were gradually tapered and then withdrawn in the first two weeks after initiation of PP1M.

During the one-year observational follow-up phase (the focus of this study), patients continued PP1M injection, switched to other oral or LAI antipsychotics or continued without any treatment, based on patient and prescriber preference/judgment. PP1M was provided gratis to study patients during the 13-week acute treatment phase; patients needed to source and pay for PP1M or other AP medications during the one-year observational follow-up phase.

Assessments

The primary endpoint of the post-hoc analysis was estimation of TFR during the period of analysis (30 days after a 13-week acute treatment phase to date of the first relapse or end of study during the follow-up phase). All patients who did not relapse during the follow-up phase were censored at end of the study. Relapse was defined by occurrence of one or more of the following: (a) hospitalization due to symptoms of schizophrenia (involuntary or voluntary); (b) two consecutive assessments at which PANSS total score was increased by 25% in patients with PANSS total scores >40 at baseline, or by 10 points in patients whose PANSS total score was ≤40 at baseline; (c) intentional self-harm or aggressive behavior, suicide or homicide attempts, or aggressive behavior with clinical significance; and (d) pre-selected baseline PANSS item scores (P1, P2, P3, P6, P7, and G8) of ≤3 increased to ≥5 in two consecutive assessments, or item scores of four at baseline increased to ≥6 (Hough et al., 2010).

Antipsychotic exposure and adherence

Adherence to all antipsychotics (any prescribed LAI antipsychotic and/or oral antipsychotics) was calculated as the ratio of cumulative exposure duration of all antipsychotics (days) to TFR (days) or time to the study completion/withdrawal (days) (for patients who did not relapse). Details of antipsychotic use were identified by prescription. Patients treated with LAI antipsychotics were considered as still exposed to the drug during the interval between the last administration and the next regularly scheduled administration (for example, if patients were only treated with PP1M, the exposure duration of PP1M was the date of the last injection+30 days). Antipsychotic exposure was treated as both a continuous variable (percentage of time with adherence) and a categorical variable. For the categorical variable, patients adhering to <80% of all prescribed antipsychotics were classified as patients with poor adherence, while those adhering to ≥80% of all prescribed antipsychotics were considered as patients with good adherence.

Proportion of PP1M usage

The ratio of PP1M to all antipsychotics was also examined as a factor influencing TFR. Therefore, the proportion of PP1M usage was estimated to investigate the impact of PP1M usage as compared with other antipsychotics in preventing relapses. As the impact of PP1M was to be estimated in patients who were receiving at least some antipsychotic therapy, patients exposed to antipsychotics for at least one day (antipsychotic adherence >0%) were included in the analysis. As part of the sensitivity analysis, the impact of usage of all LAI antipsychotics (includes PP1M and other LAI antipsychotics) on TFR was also investigated. The ratio of cumulative exposure duration of PP1M/all LAI antipsychotics (days) to cumulative exposure duration of all antipsychotics (days) was calculated. Similarly to the analysis of antipsychotic adherence, patients treated with LAI antipsychotics were considered as still exposed to the medication during the interval between the last administration and the next regularly scheduled administration. Likewise, the proportion of PP1M/all LAI antipsychotic usage was estimated as both a continuous variable and a categorical variable. As a categorical variable, this parameter was divided into three types: (a) no use PP1M/all LAI antipsychotics (proportion of PP1M/all LAI antipsychotic usage=0), patients were not treated by PP1M/all LAI antipsychotics before relapse or the study completion/withdrawal; (b) interrupted use PP1M/all LAI antipsychotics (proportion of PP1M/all LAI antipsychotic usage >0 but <100%), patients received at least one injection of PP1M/all LAI antipsychotics before relapse or the study completion/withdrawal; (c) continued use PP1M/all LAI antipsychotics (proportion of PP1M/all LAI antipsychotic usage=100%), patients regularly received PP1M/all LAI antipsychotic treatment before occurrence of relapse or the study completion/withdrawal.

Efficacy

The following efficacy measures were assessed during the follow-up visits conducted every three months: PANSS total score (range: 30–210; higher score indicates more extreme schizophrenia psychopathology) (Marder et al., 1997), Clinical Global Impressions-Severity (CGI-S) (scores range from 1= normal to 7=extreme) (Busner and Targum, 2007), Personal and Social Performance (PSP) scale (71–100=not more than mild degree of difficulty; 31–70=moderate degree of dysfunction; ≤30=functioning so poor that the patient required intensive supervision) (Morosini et al., 2000), Medication Satisfaction Questionnaire (MSQ; scores range from 1=extremely dissatisfied to 7=extremely satisfied) (Gharabawi et al., 2006; Vernon et al., 2010).

Safety

Safety evaluations included recording of treatment-emergent adverse events (TEAEs), vital signs, laboratory values and electrocardiogram.

Statistical analysis

Since the exposure analysis was intended to assess the influence of antipsychotic adherence and usage of PP1M on TFR, only patients willing to take antipsychotics during the evaluation period (adherence to all antipsychotics >0%) were included in this post-hoc analysis. The primary endpoint (TFR) was estimated using the Kaplan-Meier method, and descriptive statistics (number of relapses, number of censored patients, and median, 25th, and 75th percentile of time to relapse, if estimable) were provided.

A Cox regression model was used to analyze the association of specific variables with TFR and HR along with 95% CI. Univariate association of individual variables with TFR was calculated separately for patients who used PP1M or all antipsychotics during the study period. The following covariables were entered in the univariate analysis: age, sex, monthly income, monthly income level, number of hospitalizations in the previous year, disease duration, adherence to all prescribed antipsychotics (0%, >0% to <80%, >80%), proportion of PP1M usage (no use, interrupted use versus continued use (reference group)) and MSQ, PSP, CGI-S and total PANSS scores at baseline and endpoint during acute phase. Independent variables with p≤0.15 in the univariate analysis and their potential interactions were examined in a stepwise screening approach to construct the multivariate Cox regression model. A p=0.05 level of significance was applied to determine whether variables were added or removed from the multivariate model. The multivariate model further explored associations in two subgroups separately (poor antipsychotic adherence versus good antipsychotic adherence).

Results

Patient disposition and characteristics

Details of patient disposition and demographics for the acute treatment phase are detailed in the primary study publication (Si et al., 2015b). Of the 477 patients who completed the acute treatment phase, 367 patients who achieved PANSS<70 at week 13 were included in the analysis of relapse (Figure 1). The percentage of patients using PP1M during the one-year follow-up phase ranged from 18.10% (57/315) to 29.01% (105/362). The number of patients using other LAI antipsychotics was low and ranged from 2.45% (9/367) to 3.59% (13/362). A total of 362 patients continued the study for ≥30 days after the last PP1M injection of the acute phase (Figure 1). The mean (standard deviation (SD)) patient age was 31.4 (10.75) years, with a slight preponderance of men (196 (54.14%)). During the one-year follow-up period, a majority (288 (79.59%)) of patients had good adherence (≥80%), 39 (10.77%) patients had poor adherence (<80%) while 35 (9.67%) refused any treatment. Thus, a total of 327 patients with >0% adherence to all antipsychotics were included in the Cox regression analysis (Table 1).

Figure 1.

Patient disposition. PANSS: Positive and Negative Syndrome Scale.

Table 1.

Demographics and key baseline characteristics (13-week acute treatment phase) and medication adherence distribution and summary of long-acting injectable antipsychotic (LAI) antipsychotic usage (one-year observational follow-up phase).

Characteristics	Total(n=362)
13-week acute treatment phase
Age, mean (SD), years	31.4 (10.75)
Sex, n (%)
Men	196 (54.14)
Body weight, mean (SD), kg	64.4 (12.56)
BMI, mean (SD), kg/m²	23.3 (3.76)
Duration of schizophrenia, mean (SD), years	4.9 (5.25)
Duration distribution of disease, n (%)
≤5 years	237 (65.47)
>5 years	125 (34.53)
PANSS total score, mean (SD)
Baseline	90.5 (12.04)
Week-13 acute phase	48.9 (10.22)
CGI-S score, mean (SD)
Baseline	5.2 (0.73)
Week-13 acute phase	2.7 (0.84)
PSP score, mean (SD)
Baseline	46.0 (13.53)
Week-13 acute phase	70.7 (8.21)
One-year observational follow-up phase
Degree of adherence to antipsychotics^a, n (%)
0%	35 (9.7)
<80%	39 (10.8)
≥80%	288 (79.6)
LAI usage^b in patients with >0% adherence, n (%)	n=327
PP1M
No use	205 (62.7)
Interrupted use	35 (10.7)
Continued use	87 (26.6)
All LAI^c
No use	193 (59.0)
Interrupted use	36 (11.0)
Continued use	98 (30.0)

BMI: body mass index; CGI-S: Clinical Global Impressions-Severity; PANSS: Positive and Negative Syndrome Scale; PSP: Personal and Social Performance Scale; SD: standard deviation.

Adherence to all antipsychotics (%)=exposure duration of all antipsychotics (days)/time to the first relapse (days) or study completion/withdrawal (days)×100. For patients on LAI antipsychotics, the last dose in the acute phase was considered to provide continued exposure during the interval between last dose and the next scheduled dose. For example, the actual exposure duration for PP1M in this analysis was calculated as the date of last injection in the acute phase+30 days. Time to the first relapse was determined as the time from the start date to the first relapse recorded.

The usage of PP1M/all LAI antipsychotics was categorized according to the percent of PP1M/all LAI antipsychotic exposure duration in all antipsychotics exposure duration during the observational follow-up phase; 0% usage was defined as “no use;” 0–100% usage as “interrupted use;” and 100% usage as “continued use.”

PP1M and/or other LAI antipsychotics.

Analysis of time to first relapse

Of 367 patients who entered the one-year observational period, 362 were included in the analysis of relapse (n=5, excluded (n=1, relapsed and n=4, discontinued within one month after 13-week phase)). A total of 45/362 (12.4%) patients relapsed during the one-year observational period. The estimated percentage of patients remaining relapse-free was 84.6% (95% CI=79.2–88.7) using the Kaplan-Meier method. The 25^th percentile and median TFR (estimated time-point at which 25% or 50% (median) of patients experienced a relapse) could not be estimated since <25% patients relapsed during the observational period.

Univariate analysis

In the univariate analysis, poor antipsychotic adherence (vs good) (HR=2.97, 95% CI=1.48–5.98, p=0.002) and number of hospitalizations in the previous one year (HR=1.29, 95% CI=1.02–1.62, p=0.03) were associated with a significantly shorter TFR both for usage of PP1M, and all LAI antipsychotics. Adherence to all antipsychotics, usage of PP1M (interrupted use vs continued use), MSQ scores of patients and caregivers at the baseline of the acute phase and total PANSS scores at baseline in the acute phase were potentially important contributors to TFR (p≤0.15) and thus were also included in the multivariate Cox regression analysis (Table 2).

Table 2.

Univariate analysis of time to first relapse.

	PP1M (n=327)			All LAIs (n=327)
	Hazard ratio	95% CI	p-Value	Hazard ratio	95% CI	p-Value
Adherence to all antipsychotics (%)	0.99	0.98–1.00	0.06	0.99	0.98–1.00	0.06
Adherence distribution to all antipsychotics (poor adherence vs good adherence)^a	2.97	1.48–5.98	0.002	2.97	1.48–5.98	0.002
Usage of PP1M/all LAI antipsychotics: interrupted antipsychotic use versus continued use	2.52	0.88–7.18	0.08	2.74	0.96–7.81	0.06
The number of hospitalizations in the previous one year (times)	1.29	1.02–1.62	0.03	1.29	1.02–1.62	0.03
MSQ scores of patients at baseline of acute phase	0.80	0.62–1.03	0.08	0.80	0.62–1.03	0.08
MSQ scores of caregivers at baseline of acute phase	0.80	0.61–1.03	0.09	0.80	0.61–1.03	0.09
Total PANSS scores at baseline in the acute phase	1.02	1.00–1.05	0.10	1.02	1.00–1.05	0.10

CI: confidence interval; LAI: long-acting injectable; MSQ: Medication Satisfaction Questionnaire; PANSS: Positive and Negative Syndrome Scale; PP1M: paliperidone palmitate one-month formulation; PSP: Personal and Social Performance.

Poor adherence: adherence distribution to antipsychotics, (>0 to <80%); good adherence: adherence distribution to antipsychotics, ≥80%.

Variables tested as p≤0.15 are presented. Other variables tested: proportion of PP1M/all LAI antipsychotic exposure, usage of PP1M/all LAI antipsychotics (no use vs continued use); sex (women vs men), age (years), monthly income (income/month; yes vs no; income levels), disease duration (years), disease duration distribution, MSQ scores of patients at endpoint of acute phase, MSQ scores of caregivers at endpoint of acute phase, total PSP scores at baseline and endpoint of acute phase, CGI-S scores at baseline and endpoint of acute phase, total PANSS scores at baseline and endpoint of acute phase.

Multivariate analysis

In the multivariate Cox regression analysis, proportion of PP1M exposure (%), adherence to all antipsychotics (%) and the interaction of adherence to all antipsychotics to proportion of PP1M (p=0.03) and all LAI antipsychotics (p=0.04)) exposure had a significant association with TFR (Table 3). The significant influence of this interaction on TFR, steered the multivariate analysis to further determine any clinically important difference in the treatment effect of PP1M and all LAI antipsychotics in delaying TFR due to differential adherence among patients with varied LAI antipsychotic usage patterns. Thus, the patients were categorized as having good and poor adherence to antipsychotics, and the effect of LAI antipsychotic usage pattern on TFR was analyzed. Other variables with a significant association with TFR included: number of hospitalizations in the previous one year, and total PANSS score at baseline in the acute phase. Interaction between proportion of LAI antipsychotic usage (no use vs continuous use) and adherence to all antipsychotics (poor vs good) were also identified as relevant contributors to TFR (PP1M: p=0.04; all LAI antipsychotics: p=0.06) (Table 3).

Table 3.

Multivariate Cox regression analysis of first time to relapse (day) to investigate interaction between adherence to all antipsychotic medications and proportion of paliperidone palmitate one-month formulation (PP1M)/all long-acting injectable (LAI) antipsychotic usage.

	PP1M (n=327)				All LAIs (n=327)
	Estimate	SE	Chi-square	p-Value	Estimate	SE	Chi-square	p-Value
Model 1: variables of interest included in continuous format
Proportion of PP1M or all LAI antipsychotics exposure (%)	−0.04	0.02	5.44	0.02	−0.03	0.01	5.18	0.02
Adherence to all antipsychotics (%)	−0.02	0.01	10.31	0.001	−0.02	0.01	9.85	0.002
Proportion of PP1M or all LAI andtipsychotics exposure (%) × adherence to all antipsychotics (%)	0.0004	0.0002	4.73	0.03	0.0003	0.0002	4.08	0.04
Total PANSS scores at baseline in the acute phase	0.03	0.01	4.38	0.04	0.03	0.01	4.38	0.04
The number of hospitalizations in the previous one year (times)	0.34	0.14	5.80	0.02	0.33	0.14	5.47	0.02
Model 2: variables of interest included in classified format
Usage of PP1M or all LAI antipsychotics
No use versus continued use	−0.04	0.53	0.00	0.94	0.14	0.53	0.07	0.79
Interrupted use versus continued use	0.34	0.74	0.21	0.65	0.44	0.74	0.35	0.55
Adherence distribution of all antipsychotics (poor adherence vs good adherence)^a	−0.65	1.14	0.33	0.57	−0.51	1.14	0.21	0.65
Interaction between adherence to all antipsychotics and usage of PP1M or all LAI antipsychotics
Usage of PP1M or all LAI antipsychotics (no use vs continued use) × adherence distribution to all antipsychotics (>0−80% vs ≥80%)	2.47	1.23	4.08	0.04	2.31	1.22	3.59	0.06
Usage of PP1M or all LAI antipsychotics (interrupted use vs continued use) × adherence distribution to all antipsychotics (>0−80% vs ≥80%)	1.87	1.42	1.74	0.19	1.76	1.42	1.54	0.22
The number of hospitalizations in the previous one year (times)	0.25	0.14	3.47	0.06	0.25	0.14	3.23	0.07
Total PANSS scores at baseline in the acute phase	0.03	0.02	3.62	0.06	0.03	0.02	3.62	0.06

PANSS: Positive and Negative Syndrome Scale; SE: standard error.

Poor adherence: adherence distribution to antipsychotics, (>0 to <80%); good adherence: adherence distribution to antipsychotics, ≥80%.

Independent variables tested as p≤0.15 in univariate analysis were screened automatically by using stepwise methods and analyzed in the multivariate Cox regression model.

Stratified analysis by good versus poor adherence

In the largest LAI antipsychotic subgroup (all LAI antipsychotics), the usage pattern of LAI antipsychotics did not show any significant influence on TFR in patients with good antipsychotic adherence (Table 4). The continued use of all LAI antipsychotics (vs no use (log rank test: p=0.45) and interrupted use (log rank test: p=0.36) of all LAI antipsychotics) did not have a significant effect on relapse-free survival time in these patients and the Kaplan-Meier plots supported this finding (Figure 2(a) and (b)). However, in patients with poor antipsychotic adherence, the benefits of uninterrupted LAI antipsychotic usage were demonstrated from the multivariate analysis (Table 4) as well as the Kaplan-Meier plots (Figure 2(c) and (d)). The continued use of all LAI antipsychotics (vs no use (log rank test: p=0.028) and interrupted use (log rank test: p=0.014)) was associated with a significantly longer relapse-free survival period in patients with poor antipsychotic adherence.

Table 4.

Multivariate Cox regression analysis of time to first relapse with usage of paliperidone palmitate one-month formulation (PP1M)/all long-acting injectable (LAI) antipsychotics.

	PP1M			All LAI antipsychotics
	Hazard ratio	95% CI	p-Value	Hazard ratio	95% CI	p-Value
Patients with poor adherence to antipsychotics (adherence distribution of all antipsychotics (>0, <80%), n=39
Usage of PP1M/all LAI antipsychotics^a
No use versus continued use	13.13	1.33–129.96	0.0277	13.37	1.37–130.14	0.0255
Interrupted use versus continued use	11.04	1.03–118.60	0.0474	11.08	1.04–118.18	0.0465
Total PANSS scores at baseline in the acute phase	1.06	1.01–1.11	0.0285	1.06	1.01–1.11	0.0299
Patients with good adherence to antipsychotics (adherence distribution of all antipsychotics (≥80%), n=288
Usage of PP1M/all LAI antipsychotics^b
No use versus continued use	1.18	0.47–2.97	0.7308	1.43	0.57–3.60	0.4485
Interrupted use versus continued use	1.59	0.45–5.65	0.4709	1.74	0.49–6.18	0.3900
Total PANSS scores at baseline in the acute phase	1.01	0.98–1.05	0.3516	1.01	0.98–1.05	0.3561

CI: confidence interval; PANSS: Positive and Negative Syndrome Scale.

PP1M: no use, n=20; continued use, n=13; interrupted use, n=6; All LAI antipsychotics: no use, 19; continued use, n=14; interrupted use, n=6.

PP1M: no use, n=185; continued use, n=74; interrupted use, n=29; All LAI antipsychotics: no use, 174; continued use, n=84; interrupted use, n=30.

Figure 2.

Kaplan–Meier plots for time to first relapse. (a) Relapse-free survival in patients with good adherence to antipsychotics and on continued use versus no use of all long-acting injectable (LAI) antipsychotics; (b) relapse-free survival in patients with good adherence to antipsychotics and on continued use versus interrupted use of all LAIs; (c) relapse-free survival in patients with poor adherence to antipsychotics and on continued use versus no use of all LAIs; (d) relapse-free survival in patients with poor adherence to antipsychotics and on continued use versus interrupted use of all LAIs; (e) relapse-free survival in patients with good adherence to antipsychotics and on continued use versus no use of paliperidone palmitate one-month formulation (PP1M); (f) relapse-free survival in patients with good adherence to antipsychotics and on continued use versus interrupted use of PP1M; (g) relapse-free survival in patients with poor adherence to antipsychotics and on continued use vs interrupted use of PP1M; (h) relapse-free survival in patients with poor adherence to antipsychotics and on continued use versus interrupted use of PP1M.

Regarding the use of PP1M (patients with continued or interrupted use of PP1M among all LAI antipsychotics, 122/134 (91.0%)), the findings were generally similar to those noted in the all LAI antipsychotic subgroup. In patients with good antipsychotic adherence, no use and interrupted use of PP1M were not associated with any significant effect on TFR, although numerically, the results indicated higher risk of relapse compared with continued use of PP1M (continued vs no use: HR=1.18, 95% CI=0.47–2.97, p=0.7308; continued vs interrupted PP1M use: HR=1.59, 95% CI=0.45–5.65, p=0.4709) (Table 4). Similarly, continued use of PP1M (vs no use (log rank test: p=0.75) and interrupted use (log rank test: p=0.451)) did not demonstrate significant relapse-prevention benefit in patients with good antipsychotic adherence (Figure 2(e) and (f) shows the Kaplan-Meier plot of time to relapse). Among patients with poor antipsychotic adherence, no use (HR=13.13, 95% CI=1.33–129.96, p=0.0277) and interrupted use (HR=11.04, 95% CI=1.03–118.60, p=0.0474) of PP1M were associated with a significantly increased risk of relapse vs continued use of PP1M (Table 4). A significantly longer relapse-free survival time with continued use of PP1M (vs no use (log rank test: p=0.035) and interrupted use (log rank test: p=0.016)) was observed among patients with poor antipsychotic adherence, suggesting the efficacy of PP1M maintenance therapy in schizophrenia. The Kaplan-Meier plots corroborated this observation (Figure 2(g) and (h)).

Safety

Safety was evaluated in 362 patients who continued the study for ≥30 days after the last PP1M injection of acute phase. During the one-year observational period, 50 (13.8%) patients experienced TEAEs. The most commonly reported TEAEs (in ≥1% patients) were schizophrenia (2.8%), extrapyramidal disorder (1.9%), weight gain (1.7%), nasopharyngitis, abnormal liver function (1.4% each), and upper respiratory tract infection (1.1%). Only one patient experienced schizophrenia that led to treatment discontinuation, and serious TEAEs were reported in 12 (3.3%) patients. One death (cause unknown) was reported during the observational follow-up phase (Table 5).

Table 5.

Summary of treatment-emergent adverse events (TEAEs) reported for patients who continued the study for ≥30 days during the one-year observational follow-up phase (safety analysis set).

	Total(n=362)
Patients with TEAEs, n (%)	50 (13.8)
TEAE leading to treatment discontinuation, n (%)	1 (0.3)
Patients with ≥1 serious TEAE, n (%)	12 (3.3)
TEAEs leading to death, n (%)	1 (0.3)
TEAEs reported in ≥1% patients, n (%)
Schizophrenia	10 (2.8)
Extrapyramidal disorder	7 (1.9)
Weight gain	6 (1.7)
Nasopharyngitis	5 (1.4)
Abnormal liver function	5 (1.4)
Upper respiratory tract infection	4 (1.1)

Discussion

Prevention of symptomatic relapses during maintenance treatment in schizophrenia is an important clinical outcome with several short- and long-term benefits. In the current study in Chinese patients, a large number of patients who were maintained on antipsychotic therapy after the 13-week acute PP1M treatment remained relapse-free during the one-year follow-up period. This finding was consistent with the low relapse rates observed in the global PP1M study (Hough et al., 2010). Findings from the exposure analysis were suggestive of variability in the effect of PP1M in delaying TFR. The efficacy of relapse prevention was largely dependent on the continuity of PP1M usage and this influence had a greater clinical significance in patients with suboptimal antipsychotic adherence.

Among the variables examined in the study, poor antipsychotic adherence and higher numbers of hospitalizations in the previous year were significant risk factors for relapses in the univariate analysis. This finding is consistent with data from prior studies identifying suboptimal adherence/nonadherence as one of the key and modifiable factors influencing symptom exacerbations in patients receiving antipsychotics (Agid et al., 2010; Ascher-Svanum et al., 2006; Gharabawi et al., 2006; Kaplan et al., 2013; Nasrallah and Lasser, 2006; Robinson et al., 1999; Xiao et al., 2015).

Patients with suboptimal antipsychotic adherence and who discontinued or sporadically used PP1M/all LAI antipsychotics after the acute PP1M treatment period had a significantly increased risk of relapse risk when compared with patients who continued treatment with PP1M during the one-year observational period. The similarities in HR for no use and interrupted use of PP1M/all LAI antipsychotics suggest that the benefits of LAI antipsychotic treatment are undermined by any disruption in treatment continuity. These results suggest that continued LAI antipsychotic use is effective in minimizing the gap between any level of antipsychotic nonadherence that is greater than 20% and the associated downstream symptomatic relapses. The convenience of reduced dosing frequency and protracted dosing interval with LAI antipsychotics offers a distinct advantage over oral antipsychotics in patients with antipsychotic nonadherence or partial adherence (Berwaerts et al., 2015; Samtani et al., 2009). In patients with ≥80% antipsychotic adherence (oral/ PP1M and other LAI antipsychotics) the relapses were not influenced by the usage of PP1M during the one-year follow-up period. The number of patients with good adherence (nearly 80%) in this study was generally higher than those commonly observed in clinical practice. Potential reasons for this observation include an initial selection bias toward patients with greater illness insight and readiness to adhere to treatments (as they were part of the 13-week, acute, controlled study phase), reminders to patients for their return to follow-up visits, and patients’ awareness that their adherence was being measured during the naturalistic follow-up period.

Overall, the TEAEs reported during the one-year naturalistic treatment period were consistent with the known safety and tolerability profile of PP1M (Gopal et al., 2011; Hough et al., 2010; Pandina et al., 2010; Pandina et al., 2011). No new safety concerns pertaining to use of LAI antipsychotics emerged. Previous studies of PP1M conducted in patients from the Asia-Pacific region have reported improvement in patients’ psychotic symptoms and functioning as well as reduction in schizophrenia-related hospitalizations during long-term PP1M maintenance treatment (Li et al., 2011; Li et al., 2016; Takahashi et al., 2013; Zhang et al., 2015). These positive findings extend not only to patients with chronic, multi-episode illness, but also to first-episode and early phase patients with illness onset within the past five years (Si et al., 2015a; Zhang et al., 2015). In the primary 13-week clinical study, patients with suboptimal responses to prior oral antipsychotics responded favorably to PP1M treatment, with 73% patients achieving the primary efficacy endpoint (≥30% improvement in the PANSS total score) (Si et al., 2015b). Thus, these results of this study combined with observations from the primary study and available evidence, lend support to the potential of uninterrupted LAI antipsychotic treatment over time in maintaining symptom remission and prevention of relapses in schizophrenia (Correll et al., 2016; Robinson et al., 1999).

Although the naturalistic treatment selection and observational setting of the study are more aligned with everyday clinical practice as compared to randomized controlled and comparative studies, these results should be interpreted within the context of certain limitations of the study. First, the open-label design may have introduced a potential bias affecting patient-related and physician-related outcomes, and the non-comparative design of the study may limit the interpretation of these results. Second, the study was designed to evaluate relapse prevention in patients with stable schizophrenia (following treatment with PP1M in the acute phase). Although this enrichment strategy minimizes the influence of confounding factors, it potentially limits the generalizability of the study results. Nevertheless, the study intended to evaluate the potential longer-term benefits of PP1M and LAI antipsychotic treatment in general, and, clinically, treatment is continued in patients who have demonstrated acute treatment benefits. The assessment of adherence was based on monitoring of prescription practice during the one-year follow-up phase and did not account for potential discrepancies between prescription and actual use. Lastly, the effect of concomitant medications or non-pharmacologic interventions was neither controlled nor systematically assessed, precluding the assessment of a potential influence on the findings.

In summary, >30% of patients continued on LAI antipsychotic therapy (of which >90% continued PP1M) during the one-year naturalistic treatment period following acute stabilization with PP1M. Treatment with PP1M also showed acceptable tolerability during long-term disease management. The efficacy of uninterrupted PP1M and all LAI antipsychotics in delaying TFR during the follow-up period was more pronounced in patients with risk factors for relapse. These risk factors included, partial or poor antipsychotic adherence (during the one-year study period), a history of hospitalizations in the past year, and greater illness severity at the acute treatment baseline. Collectively, these data may assist clinicians in selecting patients who may benefit the most from maintenance treatment with LAI antipsychotics.

Footnotes

Acknowledgements

The authors thank Priya Ganpathy, MPharm, ISMPP CMPP^TM (SIRO Clinpharm Pvt Ltd, Thane, India) for writing assistance and Ellen Baum, PhD (Janssen Research and Development, LLC) for additional editorial assistance. The authors also thank the study participants, without whom this study would not have been accomplished, as well as the following investigators (in alphabetical order by surname) for their participation in this study: Y Cao, H Deng, J Dong, J Fan, M Fang, B Feng, H Li, K Li, C Liang, Q Mei, H Meng, Z Qu, J Tang, Hn Tian, B Wang, F Yang, J Zhang, K Zhang, H Zhang, J Zheng, and J Zhou. Author contributions were as follows: T Si was the principal investigator for this study and contributed to study design and data interpretation. N Li proposed the research idea and contributed to the design, analysis and interpretation. CU Correll contributed to the post-hoc analysis, data evaluation and interpretation, and made critical revisions to the manuscript. H Lu, S Cai, Y Feng, and L Zhang were responsible for the design, conduct and data analysis for the study. J Zhuo conducted the statistical analyses and contributed to design and interpretation of study results. All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors meet the International Committee of Medical Journal Editors criteria and all those who fulfilled those criteria are listed as authors. All authors provided direction and comments on the manuscript, made the final decision about where to publish these data, and approved submission to the journal. The data reported in this manuscript have not been presented elsewhere.

Declaration of conflicting interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: H Lu and L Zhang are employees of Xi’an-Janssen Pharmaceutical, Ltd, China, Y Feng is an employee of Janssen Pharmaceutical Companies of Johnson and Johnson, Singapore and J Zhuo is an employee of Johnson & Johnson (China) Investment Ltd, which are families of Johnson & Johnson Company. S Cai was an employee of Xi’an-Janssen Pharmaceutical, Ltd, China, at the time of study. T Si has been a consultant and/or advisor to or has received honoraria and research grant/support from: Xi’an-Janssen, Pfizer, Lundbeck, and Otsuka. N Li has been a consultant and/or advisor to or has received honoraria from: Xi’an-Janssen, Lundbeck, Otsuka, Astellas, Sunovion, Hisun-Pfizer, and Merck Serono. CU Correll has been a consultant and/or advisor to or has received honoraria from: Alkermes, Allergan, Forum, Gerson Lehrman Group, IntraCellular Therapies, Janssen/J&J, LB Pharma, Lundbeck, Medavante, Medscape, Neurocrine, Otsuka, Pfizer, ProPhase, Sunovion, Supernus, Takeda, and Teva, and received research grant/support from Takeda. He has provided expert testimony for Bristol-Myers Squibb, Janssen, and Otsuka. He also served on a Data Safety Monitoring Board for Lundbeck and Pfizer.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was sponsored by Xian Janssen Pharmaceutical Ltd, Beijing, People’s Republic of China.

ORCID iD

Yu Feng

References

American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders (4th ed, text rev). Arlington, VA: American Psychiatric Press Inc.

Agid

Foussias

Remington

(2010) Long-acting injectable antipsychotics in the treatment of schizophrenia: Their role in relapse prevention. Expert Opin Pharmacother 11: 2301–2317.

Almond

Knapp

Francois

et al . (2004) Relapse in schizophrenia: Costs, clinical outcomes and quality of life. Br J Psychiatry 184: 346–351.

Alphs

Benson

Cheshire-Kinney

et al . (2015) Real-world outcomes of paliperidone palmitate compared to daily oral antipsychotic therapy in schizophrenia: A randomized, open-label, review board-blinded 15-month study. J Clin Psychiatry 76: 554–561.

Alphs

Bossie

Sliwa

et al . (2013) Paliperidone palmitate and risperidone long-acting injectable in subjects with schizophrenia recently treated with oral risperidone or other oral antipsychotics. Neuropsychiatr Dis Treat 9: 341–350.

Ascher-Svanum

Faries

Zhu

et al . (2006) Medication adherence and long-term functional outcomes in the treatment of schizophrenia in usual care. J Clin Psychiatry 67: 453–460.

Ascher-Svanum

Zhu

Faries

et al . (2009) Medication adherence levels and differential use of mental-health services in the treatment of schizophrenia. BMC Res Notes 2: 6.

Berwaerts

Liu

Gopal

et al . (2015) Efficacy and safety of the 3-month formulation of paliperidone palmitate vs placebo for relapse prevention of schizophrenia: A randomized clinical trial. JAMA Psychiatry 72: 830–839.

Birchwood

Todd

Jackson

(1998) Early intervention in psychosis. The critical period hypothesis. Br J Psychiatry Suppl 172: 53–59.

10.

Busner

Targum

(2007) The clinical global impressions scale: Applying a research tool in clinical practice. Psychiatry (Edgmont) 4: 28–37.

11.

Carbon

Correll

(2014) Clinical predictors of therapeutic response to antipsychotics in schizophrenia. Dialogues Clin Neurosci 16: 505–524.

12.

Caseiro

Perez-Iglesias

Mata

et al . (2012) Predicting relapse after a first episode of non-affective psychosis: A three-year follow-up study. J Psychiatr Res 46: 1099–1105.

13.

Chien

Mui

Gray

et al . (2016) Adherence therapy versus routine psychiatric care for people with schizophrenia spectrum disorders: A randomised controlled trial. BMC Psychiatry 16: 42.

14.

Correll

Citrome

Haddad

et al . (2016) The Use of Long-Acting Injectable Antipsychotics in Schizophrenia: Evaluating the Evidence. J Clin Psychiatry 77(Suppl 3): 1–24.

15.

Gharabawi

Greenspan

Rupnow

et al . (2006) Reduction in psychotic symptoms as a predictor of patient satisfaction with antipsychotic medication in schizophrenia: Data from a randomized double-blind trial. BMC Psychiatry 6: 45.

16.

Gonzalez-Rodriguez

Catalan

Penades

et al . (2015) Profile of paliperidone palmitate once-monthly long-acting injectable in the management of schizophrenia: Long-term safety, efficacy, and patient acceptability - a review. Patient Prefer Adherence 9: 695–706.

17.

Gopal

Vijapurkar

Lim

et al . (2011) A 52-week open-label study of the safety and tolerability of paliperidone palmitate in patients with schizophrenia. J Psychopharmacol 25: 685–697.

18.

Heres

Lambert

Vauth

(2014) Treatment of early episode in patients with schizophrenia: The role of long acting antipsychotics. Eur Psychiatry 29(Suppl 2): 1409–1413.

19.

Hough

Gopal

Vijapurkar

et al . (2010) Paliperidone palmitate maintenance treatment in delaying the time-to-relapse in patients with schizophrenia: A randomized, double-blind, placebo-controlled study. Schizophr Res 116: 107–117.

20.

Kane

(2007) Treatment strategies to prevent relapse and encourage remission. J Clin Psychiatry 68(Suppl 14): 27–30.

21.

Kane

(2013) Improving patient outcomes in schizophrenia: Achieving remission, preventing relapse, and measuring success. J Clin Psychiatry 74: e18.

22.

Kane

Garcia-Ribera

(2009) Clinical guideline recommendations for antipsychotic long-acting injections. Br J Psychiatry Suppl 52: S63–S67.

23.

Kane

Kishimoto

Correll

(2013) Non-adherence to medication in patients with psychotic disorders: Epidemiology, contributing factors and management strategies. World Psychiatry 12: 216–226.

24.

Kaplan

Casoy

Zummo

(2013) Impact of long-acting injectable antipsychotics on medication adherence and clinical, functional, and economic outcomes of schizophrenia. Patient Prefer Adherence 7: 1171–1180.

25.

Rui

Ning

et al . (2011) A comparative study of paliperidone palmitate and risperidone long-acting injectable therapy in schizophrenia. Prog Neuropsychopharmacol Biol Psychiatry 35: 1002–1008.

26.

Turkoz

Zhang

(2016) Efficacy and safety of once-monthly injection of paliperidone palmitate in hospitalized Asian patients with acute exacerbated schizophrenia: An open-label, prospective, noncomparative study. Neuropsychiatr Dis Treat 12: 15–24.

27.

Malla

Tibbo

Chue

et al . (2013) Long-acting injectable antipsychotics: Recommendations for clinicians. Can J Psychiatry 58(5 Suppl 1): 30S-35S.

28.

Marder

Davis

Chouinard

(1997) The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: Combined results of the North American trials. J Clin Psychiatry 58: 538–546.

29.

McEvoy

Byerly

Hamer

et al . (2014) Effectiveness of paliperidone palmitate vs haloperidol decanoate for maintenance treatment of schizophrenia: A randomized clinical trial. JAMA 311: 1978–1987.

30.

Morosini

Magliano

Brambilla

et al . (2000) Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social functioning. Acta Psychiatr Scand 101: 323–329.

31.

Nasrallah

Lasser

(2006) Improving patient outcomes in schizophrenia: Achieving remission. J Psychopharmacol 20(6 Suppl): 57–61.

32.

Novick

Haro

Suarez

et al . (2010) Predictors and clinical consequences of non-adherence with antipsychotic medication in the outpatient treatment of schizophrenia. Psychiatry Res 176: 109–113.

33.

Olivares

Thirunavukarasu

Kulkarni

et al . (2013) Psychiatrists’ awareness of partial and nonadherence to antipsychotic medication in schizophrenia: Results from an Asia-Pacific survey. Neuropsychiatr Dis Treat 9: 1163–1170.

34.

Pandina

Lane

Gopal

et al . (2011) A double-blind study of paliperidone palmitate and risperidone long-acting injectable in adults with schizophrenia. Prog Neuropsychopharmacol Biol Psychiatry 35: 218–226.

35.

Pandina

Lindenmayer

Lull

et al . (2010) A randomized, placebo-controlled study to assess the efficacy and safety of 3 doses of paliperidone palmitate in adults with acutely exacerbated schizophrenia. J Clin Psychopharmacol 30: 235–244.

36.

Robinson

Woerner

Alvir

et al . (1999) Predictors of relapse following response from a first episode of schizophrenia or schizoaffective disorder. Arch Gen Psychiatry 56: 241–247.

37.

Samtani

Vermeulen

Stuyckens

(2009) Population pharmacokinetics of intramuscular paliperidone palmitate in patients with schizophrenia: A novel once-monthly, long-acting formulation of an atypical antipsychotic. Clin Pharmacokinet 48: 585–600.

38.

Tan

Zhang

et al . (2015a) An open-label, flexible-dose study of paliperidone extended-release in Chinese patients with first-onset psychosis. Neuropsychiatr Dis Treat 11: 87–95.

39.

Zhang

Tang

et al . (2015b) Efficacy and safety of flexibly dosed paliperidone palmitate in Chinese patients with acute schizophrenia: An open-label, single-arm, prospective, interventional study. Neuropsychiatr Dis Treat 11: 1483–1492.

40.

Takahashi

Saito

et al . (2013) Randomized, placebo-controlled, double-blind study assessing the efficacy and safety of paliperidone palmitate in Asian patients with schizophrenia. Neuropsychiatr Dis Treat 8: 1889–1898.

41.

Thornicroft

Brohan

Rose

et al . (2009) Global pattern of experienced and anticipated discrimination against people with schizophrenia: A cross-sectional survey. Lancet 373: 408–415.

42.

Vernon

Revicki

Awad

et al . (2010) Psychometric evaluation of the Medication Satisfaction Questionnaire (MSQ) to assess satisfaction with antipsychotic medication among schizophrenia patients. Schizophr Res 118: 271–278.

43.

Xiao

et al . (2015) High relapse rate and poor medication adherence in the Chinese population with schizophrenia: Results from an observational survey in the People’s Republic of China. Neuropsychiatr Dis Treat 11: 1161–1167.

44.

Zhang

Chiou

et al . (2015) Efficacy, safety, and impact on hospitalizations of paliperidone palmitate in recent-onset schizophrenia. Neuropsychiatr Dis Treat 11: 657–668.

Impact of paliperidone palmitate one-month formulation on relapse prevention in patients with schizophrenia: A post-hoc analysis of a one-year,open-label study stratified by medication adherence

Abstract

Background:

Aim:

Methods:

Results:

Conclusion:

Keywords

Introduction

Methods

Patients and study design

Treatments

Assessments

Antipsychotic exposure and adherence

Proportion of PP1M usage

Efficacy

Safety

Statistical analysis

Results

Patient disposition and characteristics

Analysis of time to first relapse

Univariate analysis

Multivariate analysis

Stratified analysis by good versus poor adherence

Safety

Discussion

Footnotes

Acknowledgements

Declaration of conflicting interest

Funding

ORCID iD

References