Sage Journals: Discover world-class research

Abstract

Background

Electronic patient-reported outcomes (ePRO) are valuable for cancer symptom monitoring, but implementation in resource-limited primary health care remains challenging. Identifying stakeholder preferences regarding implementation strategies is crucial for facilitating the adoption and effectiveness of ePRO-based cancer symptom management.

Objective

This study aimed to prioritize optimal implementation strategies for ePRO-based cancer symptom management in Chinese primary health care, considering stakeholder preferences.

Methods

A best–worst scaling was conducted among stakeholders. Thirteen implementation strategies, derived through qualitative research and implementation science frameworks, were assessed. Preferences were analyzed using count analysis, conditional logit model (CLM), mixed logit model (MLM), and latent class analysis (LCA).

Results

A total of 169 respondents were included in the final analysis. Count analysis identified the top three preferred strategies as developing tiered cancer management through ePRO (standardized BW = 0.32), providing evidence-based practice (EBP) incentives (standardized BW = 0.20), and engaging stakeholders (standardized BW = 0.14). Least preferred were disseminating paper/web-based ePRO-related materials (standardized BW = –0.28) and developing ePRO toolkits (standardized BW = –0.25). CLM and MLM results aligned with count analysis, with tiered cancer management through ePRO having the highest odds ratios (OR = 3.20 for CLM, 4.37 for MLM), followed by EBP incentives (OR = 2.55 for CLM, 3.12 for MLM) and stakeholder engagement (OR = 2.25 for CLM, 3.13 for MLM). LCA identified five stakeholder subgroups with distinct preference patterns based on professional roles and settings, including: favorability towards tiered cancer management through ePRO (class 1), preferences for disseminating paper/web-based ePRO-related materials (class 2), financial incentives among younger practitioners and healthcare administrators (classes 3 and 4), and prioritization of stakeholder engagement (class 5). However, tiered cancer management through ePRO were relatively highly preferred across all groups.

Conclusions

Tailoring ePRO implementation to stakeholder preferences is crucial. Tiered cancer management was universally preferred, while other strategies varied by professional role. Future research should focus on longitudinal evaluations and economic assessments.

Keywords

Best–worst scaling cancer management electronic patient-reported outcome preference primary health care

Introduction

Electronic patient-reported outcomes (ePRO) systems enable real-time symptom monitoring and tailored intervention, and have been shown to improve quality of life, and reduce hospital admissions and emergency visits in large-scale community practices.^1–3 The integration of ePRO system into primary care addresses a critical need: effective monitoring of cancer-related symptom clusters that,⁴ when poorly controlled, impair functional capacity and treatment adherence.^5–7 These benefits are particularly salient as cancer evolves from a terminal diagnosis to a chronic condition requiring long-term management,^8,9 with primary health care playing an expanding role in prevention, survivorship care, and end-of-life support.^10–12 The adoption of such digital tools is further encouraged by policy initiatives focused on health promotion and chronic disease management, underscoring the alignment of ePRO with broader digital health objectives.¹³ Consequently, when effectively implemented, ePRO offers a scalable and cost-effective means of delivering patient-centered care, particularly important in primary health care settings where resources may be constrained.^14–16

However, despite the mounting value, ePROs are still not widely implemented or reimbursed in cancer care.¹⁷ The effective implementation of ePRO-based symptom management for cancer patients in primary health care settings presents complexities.¹⁸ First, the effective implementation of ePRO-based symptom management in resource-limited primary health care faces multi-dimensional barriers. Technologically, primary care facilities often lack adequate digital infrastructure, including reliable internet access, compatible electronic health record systems, and device support for both providers and patients, thus hindering seamless ePRO integration.^16,19 Organizational challenges include fragmented workflow integration, insufficient staffing for ePRO administration and follow-up, and limited institutional support.^18,20–22 At the patient level, barriers persist such as low digital health literacy among elderly cancer patients, data privacy concerns, and resistance to technology adoption due to unfamiliarity with electronic reporting.^19,23 Furthermore, as ePRO systems evolve to incorporate more advanced analytics and clinical decision support features, challenges related to the explainability and usability of these algorithms become increasingly relevant for user trust and successful implementation.²⁴ To address these multi-faceted challenges, implementation strategies such as increasing staff engagement, integrating ePROs into existing systems to optimize clinic workflow, and securing funding have been recommended.^15,18,21 However, different countries must tailor solutions to their unique contexts, considering factors like the political and cultural environment and healthcare system structure. Therefore, there is no one-size-fits-all solution to address the global divide.²⁵ Second, empirical evidence indicates that the translation of clinical innovations into routine practice is often protracted, requiring 17–20 years for widespread integration, with fewer than half achieving broad adoption.^26,27 Contextual tailoring of implementation strategies has been shown to outperform generic approaches in improving uptake.^28–31 As such, tailoring strategies is crucial to accelerate and ensure the successful implementation of ePRO system.

To address these challenges, implementation science provides a structured, context-sensitive framework for promoting the adoption, integration, and sustainability of evidence-based practices (EBPs).³² Engaging stakeholder preferences in implementation strategies is imperative to accelerate and bolster knowledge translation and enhance the efficiency of EBPs by³³: (1) increasing engagement and buy-in,^34,35 (2) informing context-specific adaptations,²⁹ and (3) identifying optimal starting points for intervention when preferences and evidence diverge.³¹

However, conventional methods for eliciting stakeholder preferences, including qualitative (e.g. interviews, focus groups) and quantitative (e.g. Likert-type scales) approaches, exhibit notable limitations. Qualitative methods provide in-depth insights among a small group but are resource-intensive and lack generalizability.³¹ Quantitative approaches, while cost-effective and scalable, fail to capture trade-offs, leading to ceiling effects and biases such as social desirability, acquiescence, and extreme response bias.^31,33,36,37 In contrast, stated preference choice experiments, rooted in health economics, overcome these constraints by simulating real-world decision-making through structured choice tasks, enabling efficient and generalizable preference estimation while accounting for contextual factors.^38–40 Best–worst scaling (BWS) is a type of choice experiment well-suited for eliciting preferences.⁴¹ Grounded in random utility theory, BWS asks respondents to identify the most and least preferred items, forcing trade-offs and reducing common biases.⁴⁰ For this study, the BWS object case was chosen over other preference-elicitation methods, such as discrete choice experiments, because our primary objective was to rank and quantify the relative importance of a finite set of implementation strategies (attributes) themselves, rather than to evaluate trade-offs between different levels of strategy characteristics.⁴⁰ The BWS object case is therefore the most direct and efficient method for this purpose. This approach proves particularly valuable in implementation science, where it can identify optimal strategies from complex sets or determine preferred levels of shared attributes, ultimately supporting more effective strategy selection and deployment.³¹

Therefore, this study applies BWS methodology to elicit stakeholder preferences for ePRO implementation strategies in primary cancer care in China, aiming to inform more effective, context-sensitive approaches to improving symptom management at the primary care level.

Methods

The entire study process from strategy development to analysis is shown in Figure 1.

Figure 1.

Study process flow diagram.

Identifying implementation strategies

To well-design and -implement the ePRO-based cancer symptom management in Chinese primary health care settings, we had conducted a semi-structured interview and focus group discussions with 72 stakeholders, aiming at identifying potential implementation barriers in Yangzhou city, Jiangsu Province. Guided by implementation science principles, such barriers were synthesized within the Consolidated Framework for Implementation Research (CFIR).⁴² Theory-informed implementation strategies were then derived through the CFIR-Expert Recommendations for Implementing Change (CFIR-ERIC) matching tool.⁴³ Among these, 13 strategies were selected and further adapted to address the practical challenges of ePRO integration in primary care. The finalized strategies for ePRO symptom monitoring are presented in Figure 2. A comprehensive description of barriers and strategies is available in our companion paper,¹⁹ with key summaries provided in the Appendix 1 of this study.

Figure 2.

The adapted strategies for electronic patient-reported outcome in cancer management.

BWS experiment design

BWS has been classified into three categories: the object case (case 1), the profile case (case 2), and the multi-profile case (case 3).⁴¹ The object case involves the ranking of attribute importance through comparative best–worst selection tasks, where respondents evaluate a set of three or more attributes without levels by repeatedly selecting the most and least preferred options. This approach enables the derivation of relative preference weights across attributes.⁴⁴ In contrast, the profile case extends this framework by incorporating attributes with varying levels, requiring respondents to assess attribute-level combinations rather than standalone attributes.⁴⁵ As for the multi-profile case, it presents complete profiles composed of multiple characteristics, requiring respondents to identify the best and worst profiles within each choice set.^46,47

In this study, an object case BWS instrument was employed to assess respondents’ relative preferences for 13 implementation strategies referred to as attributes. The BWS questionnaire presented these attributes in subsets of four at a time. A full factorial design would require evaluation of all possible combinations of four attributes selected from 13, resulting in 715 unique question sets $(∁_{13}^{4} = \frac{13 \times 12 \times 11 \times 10}{4 \times 3 \times 2 \times 1} = 715)$ . However, such a large number of questions was anticipated to induce respondent fatigue and confusion, potentially compromising the reliability of the collected data.^48,49 To mitigate the respondent burden while maintaining the integrity of the data, a balanced incomplete block design (BIBD) was employed as the experimental design methodology for this object case study.^50,51 BIBD is a well-established and efficient experimental design approach that ensures: (1) equal occurrence frequency for each attribute, and (2) balanced pairwise comparisons across the entire design. The BIBD employed in this study was characterized by a set of design parameters: $ν = 13$ , $r = 4$ , $k = 4$ , and $b = 13$ . Here, $ν$ represents the total number of implementation strategies, r indicates the number of times each element appears across the entire design set, k denotes the number of attributes included in each scenario, and b denotes the quantity of choice sets presented to each respondent within the survey. The detailed BIBD design is presented in Table 1, and an illustrative example of the choice task is depicted in Figure 3.

Figure 3.

An example of the choice tasks.

Table 1.

The balanced incomplete block design for this study.

Choice task	Implementation strategies
Choice task	S1	S2	S3	S4	S5	S6	S7	S8	S9	S10	S11	S12	S13
1		√		√					√			√
2	√			√	√	√
3					√				√	√	√
4						√					√	√	√
5	√							√		√		√
6		√			√			√					√
7	√	√					√				√
8				√			√			√			√
9		√	√			√				√
10			√		√		√					√
11			√	√				√			√
12						√	√	√	√
13	√		√						√				√

Data collection

Stakeholders, including general practitioners, specialists, nurses, public health physicians, and health administrators in Yangzhou, were invited to participate in an online survey. The online survey was conducted via the Questionnaire Star platform. The survey has three sections: (1) introduction and informed consent, (2) demographic questions: age, gender, education level, occupation, work year, etc., and (3) the BWS choice scenarios.

A minimum sample size of 125 respondents was required as it is optimal to ensure that each attribute appears no fewer than 500 times in one survey.⁵² The number of respondents n could be calculated as

n \geq \frac{500 ν}{k \times b}

where the meanings of the characters in the inequality can be found in the section “BWS experiment design.”

Data analysis

Questionnaires that were fully completed were included in the final analysis. For the BWS section, responses in which the same attribute was selected as both the most and least important were excluded. Descriptive statistics were computed to summarize the demographic characteristics of the study participants. For the statistical analysis of the BWS data, both count-based methods and model-based methods were employed. Finally, the highest-ranked strategies were translated into a set of potentially feasible actions and corresponding resource requirements for Chinese primary health care settings.

Count analysis

The count analysis commonly applied for BWS design is limited to examining choice frequencies.^53,54 In this study, the best–worst score (BW score), mean BW score, and standardized BW score for each item were reported. The BW score was derived by subtracting the number of times an attribute was selected as the least important from the number of times it was chosen as the most important (equation (1)). A positive BW score indicates that the attribute was more frequently prioritized as most important, whereas a negative BW score suggests the opposite. The mean BW score was calculated by dividing the total BW score for each attribute by the number of respondents who evaluated it (equation (2)). The standardized BW score was derived by dividing the BW score by the product of the total number of times the item appeared in the survey design and the sample size (equation (3)). This standardized BW score ranges between −1 and 1. In the context of this study, standardized BW scores approaching 1 denote that the item is a highly important strategy, while scores approaching −1 suggest that the item is of less importance.

B W = B - W

(1)

m e a n B W = \frac{B W}{n}

(2)

s t a n d a r d i z e d B W = \frac{B W}{n \times m}

(3)

where B denotes the frequency of responding to the attribute as the most important strategy, W denotes the frequency of responding to attribute as the least important strategy, n denotes the respondents, and m denotes the number of times the strategy was presented to each respondent = 4.

Model-based analysis

Count methods failed to quantify the probability of an attribute being ranked as most or least important.⁵⁴ To analyze the likelihood of relative preferences among 13 strategies, three model-based analysis were applied: the conditional logit model (CLM), mixed logit model (MLM), and latent class analysis (LCA).⁵⁵ In these models, each attribute was dual-coded to represent preference polarity. The best variable was assigned a value of 1 when an attribute was selected as the most important and 0 otherwise; the worst variable received a value of −1 if the attribute was identified as the least important and 0 in other cases.⁵⁶ These variables were used to estimate propensity scores, indicating the probability of an attribute being favored or disfavored within a given combination.

The CLM assumed homogeneous preferences across respondents, estimating fixed coefficients for each attribute. It was used to calculate the average marginal utility of attributes, with the estimated attribute coefficients indicating the relative preference of an attribute compared to others.⁵⁷ The MLM extends the CLM by allowing for random parameters, which can capture the heterogeneity in preferences across individuals. This model assumes that the coefficients of the attributes can vary across individuals, following a specific distribution.^58,59 The MLM also generates standard deviation (SD) statistics for each attribute, indicating unexplained variation around the mean. Significant SD terms suggest substantial heterogeneity. Model fit was evaluated using the Akaike information criterion (AIC), Bayesian Information Criteria (BIC), and log-likelihood values.

Preference heterogeneity was further explored using a panel-based latent class regression model. The LCA takes a different approach by segmenting the population into distinct latent classes based on their unobserved preference patterns. Each latent class consisted of individuals with similar preferences, and the LCA estimated coefficients for each of these classes.^60,61 Models with 2 to 6 classes were estimated, and the optimal number of classes was determined based on the BIC and Consistent AIC (CAIC).

The estimated coefficients $(β)$ , odds ratio (OR), rank order, and 95% confidence interval (CI) were reported for each attribute's preference weight. The relative importance of each attribute was calculated by the following formula^62,63:

R I_{k} = \frac{\exp (β_{k})}{\sum_{i = 1}^{13} \exp (β_{i})}

Data analyses were conducted in RStudio (using support.BWS and mlogit packages) and STATA. Statistical significance was determined by $P < 0.05$ .

Ethical considerations

Ethical approval for the process evaluation was obtained from the Yangzhou Hospital of Traditional Chinese Medicine Ethics Committee (ChiCTR2200065569). Written informed consent was obtained from all the subjects prior to our study initiation.

Results

Participant demographics

A total of 222 participants completed the survey. After excluding 31 respondents who chose the same term for both the most and least important attribute and 22 who did not meet the time response threshold, 169 (76.1%) were included in the final analysis. The mean age of the participants was 49.5 years (SD = 9.8); 52.7% (n = 89) were male. Nearly half held an undergraduate or above degree (n = 88, 52.1%). The majority worked in primary care facilities (n = 132, 78.1%), with 54.4% (n = 92) holding junior or below qualifications. The average work experience was 27.6 years (SD = 10.9), including 40.2% (n = 68) with over 30 years of service (Table 2).

Table 2.

Demographic characteristics of the participants (N = 169).

Characteristics	Value
Gender, n (%)
Male	89 (52.7)
Age (years), mean ± SD	49.5 ± 9.8
Age (years), n (%)
≥50	95 (56.2)
30–49	61 (36.1)
＜30	13 (7.7)
Education level, n (%)
Junior college and below	81 (47.9)
Undergraduate	76 (45.0)
Master and above	12 (7.1)
Vocation, n (%)
General practitioner	78 (46.2)
Medical specialist	54 (32.0)
Public health physician	14 (8.3)
Health administrators	12 (7.1)
Nurse	11 (6.4)
Workplace, n (%)
Primary care facilities	132 (78.1)
Secondary hospital	19 (11.2)
Tertiary hospital	11 (6.5)
Health administration	7 (4.2)
Professional qualifications, n (%)
Junior and below	92 (54.4)
Senior	58 (34.3)
Intermediate	19 (11.3)
Work years, mean ± SD	27.6 ± 10.9
Work years, n (%)
≥40	68 (40.2)
20–29	51 (30.2)
30–39	18 (10.7)
10–19	17 (10.1)
1–9	15 (8.8)

SD: standard deviation.

Results of count analysis

Figure 4 and Appendix 2 (Table 1) illustrate the distribution of individual BW scores across implementation strategy attributes. As summarized in Table 3, the highest-ranked strategy was “(S6) Develop tiered cancer management through ePRO” (standardized BW score = 0.32), indicating strong stakeholder preference. Subsequent strategies, in descending order of importance, included “(S3) EBP incentives” (0.20), “(S1) Engage EBP stakeholders” (0.14), “(S2) Peer-led ePRO sessions” (0.07), and “(S7) Access new funding” (0.04). Conversely, “(S12) Disseminate Paper/Web-based ePRO-related materials” (−0.28), “(S10) Develop ePRO-related toolkits” (−0.25), and “(S8) Expert-led ePRO consultation” (−0.15) were perceived of relatively low importance.

Figure 4.

The distribution of individual best–worst scores across 13 implementation strategies.

Table 3.

The aggregated results of count analysis.

Strategy	B	W	BW	Mean BW (95% CI)	Standardized BW (95% CI)	Rank
S1	227	132	95	0.56 (0.27, 0.86)	0.14 (0.07, 0.21)	3
S2	198	153	45	0.27 (0.01, 0.52)	0.07 (0.002, 0.13)	4
S3	256	118	138	0.82 (0.53, 1.11)	0.20 (0.13, 0.28)	2
S4	171	147	24	0.14 (−0.10, 0.38)	0.04 (−0.02, 0.10)	6
S5	148	165	−17	−0.10 (−0.33, 0.13)	−0.03 (−0.08, 0.03)	8
S6	289	75	214	1.27 (1.00, 1.53)	0.32 (0.25, 0.38)	1
S7	181	151	30	0.18 (−0.13, 0.48)	0.04 (−0.03, 0.12)	5
S8	112	210	−98	−0.58 (−0.84, −0.32)	−0.15 (−0.21, −0.08)	11
S9	160	153	7	0.04 (−0.25, 0.33)	0.01 (−0.06, 0.08)	7
S10	73	245	−172	−1.02 (−1.26, −0.78)	−0.25 (−0.31, −0.20)	12
S11	146	177	−31	−0.18 (−0.47, 0.10)	−0.05 (−0.12, 0.03)	9
S12	149	197	−48	−0.28 (−0.58, 0.01)	−0.07 (−0.14, 0.002)	10
S13	87	274	−187	−1.11 (−1.40, −0.81)	−0.28 (−0.35, −0.20)	13

B: best; BW: best–worst; W: worst.

Results of model-based analysis

The conditional and mixed logit model estimates

The CLM and MLM analyses identified statistically significant disparities in stakeholder preferences for specific implementation strategies. Tables 4 and 5 illustrate the estimated coefficients, OR, 95% CI, and corresponding P values for each attribute.

Table 4.

The estimates of conditional logit model.

Strategy	Coefficient (95% CI)	OR (95% CI)	Marginal probability	RI (%)	Rank
S1	0.81 (0.66, 0.96)***	2.25 (1.94, 2.62)	0.19	9.59	3
S2	0.66 (0.51, 0.82)***	1.94 (1.61, 2.26)	0.16	8.28	4
S3	0.94 (0.78, 1.09)***	2.55 (2.13, 2.97)	0.22	10.88	2
S4	0.61 (0.46, 0.76)***	1.84 (1.58, 2.14)	0.15	7.85	6
S5	0.49 (0.34, 0.64)***	1.64 (1.41, 1.91)	0.14	6.98	8
S6	1.16 (1.01, 1.32)***	3.20 (2.74, 3.74)	0.28	13.64	1
S7	0.63 (0.48, 0.78)***	1.88 (1.61, 2.18)	0.16	7.99	5
S8	0.26 (0.11, 0.41)***	1.30 (1.12, 1.51)	0.11	5.53	11
S9	0.56 (0.41, 0.71)***	1.75 (1.50, 2.03)	0.15	7.44	7
S10	0.05 (−0.11, 0.20)	1.05 (0.90, 1.22)	0.09	4.46	12
S11	0.46 (0.30, 0.61)***	1.58 (1.36, 1.83)	0.13	6.71	9
S12	0.41 (0.26, 0.56)***	1.50 (1.29, 1.75)	0.12	6.40	10
S13	Reference		0.04	4.26	13

Akaike information criterion: 10507; Bayesian information criteria: 10575; log-likelihood: −5242.

CI: confidence interval; OR: odds ratio; RI: relative importance.

***P < 0.001.

Table 5.

The estimates of mixed logit model.

Strategy	Coefficient (95% CI)	SD (95% CI)***	OR (95% CI)	Marginal probability	RI (%)	Rank
S1	1.14 (0.96, 1.33)***	1.18 (0.95, 1.40)	3.13 (2.60, 3.78)	0.20	11.12	2
S2	0.85 (0.68, 1.02)***	0.84 (0.62, 1.06)	2.34 (1.97, 2.78)	0.17	8.29	4
S3	1.14 (0.96, 1.32)***	1.19 (0.97, 1.41)	3.12 (2.61, 3.74)	0.24	11.07	3
S4	0.76 (0.59, 0.93)***	0.57 (0.36, 0.79)	2.14 (1.80, 2.54)	0.16	7.58	6
S5	0.62 (0.45, 0.78)***	−0.45 (−0.68, −0.22)	1.85 (1.57, 2.19)	0.15	6.56	8
S6	1.47 (1.28, 1.67)***	1.00 (0.76, 1.23)	4.37 (3.60, 5.30)	0.30	15.50	1
S7	0.81 (0.63, 1.00)***	1.24 (1.01, 1.47)	2.25 (1.87, 2.71)	0.17	7.99	5
S8	0.33 (0.15, 0.51)***	0.82 (0.61, 1.03)	1.39 (1.16, 1.66)	0.12	4.92	11
S9	0.74 (0.55, 0.93)***	1.09 (0.85, 1.32)	2.09 (1.74, 2.52)	0.15	7.43	7
S10	0.03 (−0.14, 0.21)	0.60 (0.37, 0.84)	1.03 (0.87, 1.23)	0.10	3.66	12
S11	0.56 (0.38, 0.73)***	0.95 (0.75, 1.16)	1.74 (1.46, 2.08)	0.14	6.19	9
S12	0.55 (0.37, 0.72)***	1.00 (0.80, 1.20)	1.73 (1.45, 2.06)	0.13	6.14	10
S13	Reference			0.05	3.55	13

Akaike information criterion: 9977; Bayesian information criteria: 10114; log-likelihood: −4965.

CI: confidence interval; OR: odds ratio; RI: relative importance; SD: standard deviation.

***P < 0.001.

Taking “(S13) Disseminate Paper/Web-based ePRO-related materials” as the reference category, which was previously determined to be the least important attribute in the count analysis, the CLM quantified the relative preferences for each implementation strategy. The findings showed that “(S6) Develop tiered cancer management through ePRO” (OR = 3.20, 95% CI: 2.74–3.74, P < 0.001) was the most favored choice among the participants. Subsequently, “(S3) EBP incentives,” “(S1) Engage EBP stakeholders,” “(S2) Peer-led ePRO sessions,” and “(S7) Access new funding” were found to be relatively high important, with ORs of 2.55 (95% CI: 2.19–2.98, P < 0.001), 2.25 (95% CI: 1.94–2.62, P < 0.001), 1.94 (95% CI: 1.67–2.26, P < 0.001), and 1.88 (95% CI: 1.61–2.18, P < 0.001), respectively. In contrast, “(S10) Develop ePRO-related toolkits” (OR = 1.05, 95% CI: 0.90–1.22, P = 0.55), “(S8) Expert-led ePRO consultation” (OR = 1.30, 95% CI: 1.12–1.51, P < 0.001), and “(S12) ePRO-based continuing medical education” (OR = 1.50, 95% CI: 1.29–1.75, P < 0.001) were associated with relatively lower importance. Compared with the reference attribute, all other attributes exhibited significant preferences, with the exception of “(S10) Develop ePRO-related toolkits.” The results of the MLM were generally consistent with those of the CLM and the count analysis, although the ranking of the preferences for “(S3) EBP incentives” and “(S1) Engage EBP stakeholders” differed. In the CLM and count analysis, their ranks were 2 and 3, while in the MLM, they were 3 and 2. Moreover, the MLM model provided the SD estimates for each attribute. The significant results indicated substantial heterogeneity in the preference of all items.

Figure 5 illustrates the relative importance of each preference, re-scaled by proportions. In the CLM, the relative importance of the attributes ranged from a maximum of 13.64% for “(S6) Develop tiered cancer management through ePRO” to a minimum of 4.26% for “(S13) Disseminate Paper/Web-based ePRO-related materials.” In the MLM, these values fell from 15.50 to 3.55. The relative importance analysis indicated that “(S6) Develop tiered cancer management through ePRO,” the most preferred strategy, was 3.20 times (13.64/4.26 = 3.20) as important as “(S13) Disseminate Paper/Web-based ePRO-related materials,” the least preferred strategy, as estimated by the CLM, and 4.37 times (15.50/3.55 = 4.37) as important as the least preferred strategy, as estimated by the MLM.

Figure 5.

The relative importance of 13 implementation strategies estimated by (A) conditional logit model and (B) mixed logit model.

Latent class analysis findings

The findings from the LCA indicated that the five-class model demonstrated the optimal fit for the dataset, as evidenced by the lowest values of CAIC and BIC among all class models (Appendix 2 (Table 2)). The disparities in preferences among the five identified groups were graphically presented in Figure 6 and detailed in Appendix 2 (Table 3), with their relative importance illustrated in Figure 7 and Appendix 2 (Table 4). A mapping of top-ranked implementation strategies to potential actions and resource requirements in Chinese primary health care could be found in Appendix 2 (Table 5).

Figure 6.

The estimates of latent class analysis for five groups.

Figure 7.

The relative importance of 13 implementation strategies estimated latent class analysis with five groups.

All statistical analyses were executed with “(S13) Disseminate Paper/Web-based ePRO-related materials” serving as the reference level. For respondents assigned to class 1, the most favored strategy was “(S6) Develop tiered cancer management through ePRO” (OR = 18.68, 95% CI: 11.18–31.21, P < 0.001), which was 18.7 times more likely to be implemented than the reference category. Subsequently, “(S8) Expert-led ePRO consultation” (OR = 6.08, 95% CI: 3.75–9.84, P < 0.001) and “(S9) Establish clinical support partnerships” (OR = 5.92, 95% CI: 3.70–9.47, P < 0.001) ranked second and third in preference among class 1 participants. Conversely, “(S10) Develop ePRO-related toolkits” was relatively less favored (OR = 1.26, 95% CI: 0.80–2.00, P = 0.032).

In class 2, the reference strategy “(S13) Disseminate Paper/Web-based ePRO-related materials” emerged as the most preferred option, featuring a relative importance of 15.06%. “(S6) Develop tiered cancer management through ePRO” (OR = 0.91, 95% CI: 0.56–1.48, P = 0.71) and “(S12) ePRO-based continuing medical education” (OR = 0.73, 95% CI: 0.49–1.07, P = 0.11) followed. The non-significant P values suggested that these strategies were also highly desired. In contrast, “(S8) Expert-led ePRO consultation” (OR = 0.20, 95% CI: 0.13–0.30, P < 0.001) and “(S7) Access new funding” (OR = 0.24, 95% CI: 0.15–0.39, P < 0.001) were less likely to be chosen.

For class 3, the top three preferred strategies were “(S3) EBP incentives” (OR = 22.85, 95% CI: 13.80–37.86, P < 0.001), “(S7) Access new funding” (OR = 20.63, 95% CI: 12.62–33.71, P < 0.001), and “(S9) Establish clinical support partnerships” (OR = 13.10, 95% CI: 7.81–21.98, P < 0.001). The most preferred strategy, S3, was 22.76 times more likely to be implemented compared to the reference category. Additionally, “(S6) Develop tiered cancer management through ePRO” ranked fourth in this group (OR = 9.39, 95% CI: 5.62–15.68, P < 0.001). Analogously to class 1, “(S10) Develop ePRO-related toolkits” was relatively less favored in class 3 (OR = 1.42, 95% CI: 0.91–2.23, P = 0.13).

In class 4, the top three preferred strategies were “(S3) EBP incentives” (OR = 27.19, 95% CI: 11.79–62.71, P < 0.001), “(S7) Access new funding” (OR = 23.77, 95% CI: 11.95–47.31, P < 0.001), and “(S6) Develop tiered cancer management through ePRO” (OR = 5.42, 95% CI: 2.80–10.51, P < 0.001). In this class, “(S9) Establish clinical support partnerships” was less preferred than the reference strategy S13 (OR = 0.99, 95% CI: 0.54–1.80, P = 0.96). Moreover, “(S10) Develop ePRO-related toolkits” (OR = 1.95, 95% CI: 1.00–3.82, P = 0.05) and “(S12) ePRO-based continuing medical education” (OR = 1.47, 95% CI: 0.80–2.71, P = 0.22) were also among the less favored strategies.

Class 5 participants predominantly favored “(S1) Engage EBP stakeholders” (OR = 19.33, 95% CI: 12.91–28.95, P < 0.001), “(S2) Peer-led ePRO sessions” (OR = 9.87, 95% CI: 6.60–14.75, P < 0.001), and “(S6) Develop tiered cancer management through ePRO” (OR = 8.64, 95% CI: 5.93–12.59, P < 0.001). The most preferred strategy, S1, was 19.38 times more likely to be selected for implementation compared to the reference category. In class 5, “(S10) Develop ePRO-related toolkits” (OR = 1.06, 95% CI: 0.75–1.50, P = 0.73) and “(S12) ePRO-based continuing medical education” (OR = 1.25, 95% CI: 0.88–1.78, P = 0.21) were less preferred.

The five classes manifested distinct demographic profiles, which we descriptively labeled for ease of interpretation as: Experienced Primary Care Advocates, Junior Community Practitioners, Young Innovators, Healthcare Administrators, and Hospital-based Specialists. Class 1 participants were characterized as experienced primary care physicians with advanced professional titles (93.7%) and relatively higher education levels, mainly concentrated at the bachelor's degree (93.7%). In contrast, class 2 participants were all at the associate degree level or below, with professional titles mainly at the junior level or below (98.5%). Class 3 participants were relatively younger (36.2 ± 9.6), resulting in a shorter average work duration (12.7 ± 8.9). Class 4 was predominantly made up of healthcare administrators (72.7%), and had a higher percentage of individuals with senior professional titles (45.4%) compared to the previous three classes. Class 5 primarily included professionals working in secondary and tertiary hospitals, with the majority being specialists (62.7%) and the highest proportion of advanced professional titles among the five classes (68.6%). Detailed demographic information of the five classes are shown in Table 6.

Table 6.

Demographic characteristics of each class.

Characteristics	Class 1	Class 2	Class 3	Class 4	Class 5
Gender, n (%)
Male	62.5%	44.6%	46.2%	90.9%	54.9%
Female	37.5%	55.4%	53.8%	9.1%	45.1%
Age (years), mean ± SD	53.1 ± 3.6	55.1 ± 8.0	36.2 ± 9.6	48.7 ± 10.6	48.2 ± 5.7
Age (years), n (%)
＜30	0%	1.5%	42.3%	9.1%	0%
30–49	6.3%	21.6%	50.0%	27.3%	58.8%
≥50	93.7%	76.9%	7.7%	63.6%	41.2%
Education level, n (%)
Junior college and below	0%	100%	26.9%	0%	17.6%
Undergraduate	93.7%	0%	65.4%	63.6%	72.5%
Master and above	6.3%	0%	7.7%	36.4%	9.9%
Vocation, n (%)
Medical specialist	56.3%	4.6%	30.8%	18.2%	62.7%
General practitioner	31.3%	78.5%	53.8%	0%	15.7%
Nurse	6.2%	3.1%	7.8%	0%	11.8%
Public health physician	6.2%	13.8%	3.8%	9.1%	3.9%
Health administrators	0%	0%	3.8%	72.7%	5.9%
Workplace, n (%)
Primary care facilities	93.7%	100%	84.6%	36.4%	51.0%
Secondary hospital	0%	0%	11.5%	9.1%	29.4%
Tertiary hospital	6.3%	0%	3.9%	9.1%	15.7%
Health administration	0%	0%	0%	45.4%	3.9%
Professional qualifications, n (%)
Junior and below	6.3%	98.5%	69.2%	27.3%	11.8%
Intermediate	0%	1.5%	19.2%	27.3%	19.6%
Senior	93.7%	0%	11.6%	45.4%	68.6%
Work years, mean ± SD	32.4 ± 3.3	34.3 ± 8.4	12.7 ± 8.9	27.7 ± 11.9	25.1 ± 7.4
Work years, n (%)
1–9	0%	1.5%	42.3%	9.1%	3.9%
10–19	0%	0%	30.8%	9.1%	15.7%
20–29	6.3%	18.5%	23.1%	27.3%	56.9%
30–39	93.7%	53.8%	3.8%	45.4%	23.5%
≥40	0%	26.2%	0%	9.1%	0%

SD: standard deviation.

Discussion

Principal findings

This study examined stakeholder preferences for implementation strategies of ePRO-based cancer symptom management in Chinese primary health care settings. Employing a BWS approach allowed detailed exploration of priorities among 13 implementation strategies identified through qualitative research and the CFIR-ERIC matching process. Stakeholders, including primary healthcare providers, specialists, nurses, public health physicians, and health administrators, demonstrated clear and differentiated preferences for certain strategies, emphasizing the need for context-specific approaches to implementation.

The primary finding was the high prioritization stakeholders placed on strategies supporting structured and coordinated cancer management frameworks, specifically the establishment of tiered cancer management through ePRO (ranked highest by both count analysis and logit models). Incentivizing EBP and engaging stakeholders through tailored and participative approaches were also strongly favored. Conversely, traditional strategies such as disseminating paper/web-based ePRO-related materials or developing standardized toolkits without active stakeholder involvement were least preferred, reflecting a clear preference shift towards proactive engagement and structured frameworks rather than passive dissemination of information. Policymakers and implementation leaders should, therefore, emphasize co-design and participatory mechanisms when promoting EBP in this context. Additionally, although all strategies except one demonstrated statistically significant preferences relative to the reference category in the conditional and mixed models, the practical significance of differences among middle-ranked strategies is modest. Their overlapping intervals reflect limited stakeholder differentiation, with the primary actionable insights lying in the clear preference for top-tier strategies and consistent rejection of bottom-tier strategies. Implementation efforts should therefore concentrate resources on top-tier strategies rather than dispersing them across the middle tier. Notably, the overlapping CIs among middle-ranked strategies offer flexibility in implementation: for resource-constrained settings, prioritizing lower-cost over higher-cost mid-tier strategies may be feasible without compromising stakeholder acceptance. This flexibility enhances the scalability of ePRO implementation across diverse primary care environments.

Furthermore, LCA unveiled considerable heterogeneity in stakeholder preferences, discerning five distinct subgroups: Experienced Primary Care Advocates, Junior Community Practitioners, Young Innovators, Healthcare Administrators, and Hospital-based Specialists. Each subgroup exhibited unique, and at times conflicting, priorities, which can be attributed to their diverse professional roles, demographic characteristics, and practical contexts. For example, experienced primary care providers showed a preference for expert-led consultations and structured management protocols, while younger healthcare professionals and administrators demonstrated a strong inclination towards financial incentives, novel funding opportunities, and collaborative partnerships. This suggests that the successful implementation of ePRO-based cancer symptom management needs a diversified and flexible strategy, rather than a one-size-fits-all approach. Additionally, class 2 (Junior Community Practitioners) uniquely prioritized disseminating paper/web-based EBP materials, possibly due to familiarity with traditional health education methods. Despite the differences among the five subgroups, the establishment of tiered cancer management through ePRO ranked highly across all groups, highlighting its critical urgency in cancer management in China.

Comparison to similar studies

Previous research has systematically reviewed implementation strategies in ePRO-based cancer care, identifying frequently used approaches such as conducting educational meetings, distributing educational materials, reforming record systems, enhancing patient adherence through interventions, modifying physical infrastructure, and providing localized technical assistance.⁶⁴ Additional research identified five core implementation elements in community oncology: staff engagement, survey design, alerts, reporting, and workflow integration.¹⁸ However, the applicability of specific strategies depends on the context of implementations, which may exhibit substantial variability across diverse clinical settings.⁶⁵ A type II hybrid implementation study emphasized that strategy effectiveness is highly context-dependent, with universal strategies (e.g. clinician engagement, iterative feedback) demonstrating high impact across diverse settings, while adaptive strategies were needed to address site-specific barriers.⁶⁶ This mirrors our observation that stakeholder preferences varied significantly among the 13 strategies.

Besides, prior research using BWS experiment revealed that behavioral health stakeholders (clinicians, supervisors, and administrators) strongly preferred financial incentives over feedback strategies.³¹ This finding aligns closely with our study, which also identified financial incentives as a highly preferred implementation strategy. Although the increasing recognition in implementation science that strategies should be contextually tailored to match the specific characteristics of the EBP, setting, and individuals characteristics,^33,67,68 the preferences of stakeholders are often overlooked when implementing EBPs. However, understanding how stakeholder preferences influence EBP implementation is crucial for tailoring effective strategies.³¹

Implications for practice and future research

The results from this study provide practical guidance and a clear roadmap for policymakers, healthcare administrators, and primary health care providers seeking to integrate ePRO-based cancer symptom management systems within primary healthcare settings. While general practitioner-led models hold promise given their established patient relationships and greater accessibility compared to specialists,^10,11,69 their adoption remains underdeveloped and widespread implementation faces challenges. Such models have proven unsustainable in high-income countries when addressing the complex, growing needs of primary care patients,^12,70 and similar issues persist in China, demanding urgent attention.

This study highlights the need to prioritize the establishment of tiered cancer management frameworks in China. In recent years, under the progressive development of tiered healthcare system of China, primary care hospitals have achieved notable advancements in the management of cardiovascular diseases and diabetes.^71–73 However, as one of the four major chronic diseases emphasized in the Healthy China 2030 Action Plan issued by the State Council, cancer symptom management still has obvious shortcomings in community-level medical services.⁷⁴ To bridge this gap, efforts should focus on enhancing integration and coordination across different levels of the healthcare delivery system.⁷⁵ Through bidirectional referral mechanisms, the fundamental functions of primary care hospitals in cancer care can be established and consolidated.

Financial incentives must also be institutionalized through reimbursement mechanisms or quality-based bonuses to drive sustained provider engagement, with local health commissions encouraged to pilot ePRO-specific incentive schemes. Continuous stakeholder engagement is critical, involving providers in co-designing ePRO workflows and feedback systems to ensure contextual fit. Additionally, resource allocation should be differentiated: experienced primary care physicians require expert consultation pathways, while junior practitioners benefit from educational materials and peer-led training.

The considerable variability highlighted through LCA further suggests that one-size-fits-all approaches to implementation are unlikely to be effective. Instead, tailored strategies addressing specific subgroup needs and preferences should be developed. Targeting strategies to stakeholders’ preference may have a general accelerator effect due to increased engagement or buy-in.³¹ Our study identified five distinct subgroups. While class 1 (Experienced Primary Care Advocates) showed a clear preference for tiered cancer management, the other four groups—Junior Community Practitioners, Young Innovators, Healthcare Administrators, and Hospital-based Specialists—were more responsive to strategies involving dissemination, financial incentives, and broader participation. These preferences likely reflect the specific demands and working contexts of each group. Community Practitioners with lower educational levels are accustomed to passively receiving information and likely to be intrinsically motivated.⁷⁶ Young Innovators, typically early in their careers, may be more motivated by financial incentives, as they tend to prioritize economic needs consistent with the lower tiers of Maslow's hierarchy.^77,78 Meanwhile, both Young Innovators and Healthcare Administrators are generally well-educated. In the labor market, individuals with better educational backgrounds are often more motivated by external rewards, such as competitive remuneration, which may influence their engagement in program implementation and innovation adoption.⁷⁹ Hospital-based specialists preferred to involve more participants in the program, possibly due to staffing shortages in their work environments.^80,81 These findings highlight the importance of designing subgroup-specific strategies to enhance the effectiveness and sustainability of program implementation across diverse healthcare settings.

Future research will focus on longitudinal evaluations of these tailored strategies within actual primary care settings to measure their impact on adoption rates, sustainability, and patient outcomes. Building on the Multiphase Optimization Strategy framework, such studies could employ factorial designs to evaluate the independent and combined effects of discrete implementation strategies on key implementation outcomes like acceptability, feasibility, and fidelity.^82,83 This approach would not only identify the most effective strategies but also reveal potential synergistic or antagonistic interactions between them.⁸⁴ Moreover, research on economic evaluations of implementing preferred strategies is warranted, particularly given the resource-limited context of community healthcare in China. The large preference disparity between highly ranked strategies (such as tiered cancer management) and other approaches strategies highlights a critical need to assess the relative effectiveness of high-cost tiered cancer management versus lower-cost alternatives. Cost-effectiveness analyses could further support decision-making processes by identifying strategies that offer the greatest value for investment, thereby facilitating more widespread adoption.^85,86 Furthermore, emerging technologies offer new frontiers for ePRO implementation. Artificial intelligence could enhance ePRO systems by automating symptom triage, personalizing patient feedback, and predicting deterioration.⁸⁷ Federated learning frameworks may enable secure, decentralized data analysis across tiered healthcare institutions without compromising patient privacy, critical for scaling ePRO in China's distributed health system.⁸⁸ Future research should explore how these innovations integrate with stakeholder-preferred strategies.

Strengths and limitations

This study possesses several important strengths. The use of BWS methodology with analytical methods, including conditional logit, mixed logit, and latent class analyses, allowed detailed exploration of both overall and subgroup-specific preferences, providing nuanced insights critical for practical implementation. In addition, involving diverse stakeholders from multiple professional backgrounds and care settings strengthened the generalizability and practical relevance of findings, reflecting the complexity of primary care environments. By simultaneously assessing multiple stakeholders’ preferences, our results can better inform tailored implementation strategies that align with the “Quadruple Aim”: improving experience and satisfaction for patient, improving well-being of healthcare workers, improving population health and reduced costs.^33,34

Several limitations must also be acknowledged. First, the strategies used in our study were generated based on the CFIR-ERIC Matching Tool recommendations, which may not fully capture all actual implementation strategies. Although prior research has demonstrated strong connections between strategies recommended by the CFIR-ERIC Matching Tool and their selection and effectiveness in real-world settings,⁸⁹ the validity and applicability of this expert-informed tool in our specific context require further investigation. Second, while we recognized that one-size-fits-all approaches are ineffective and thus involved multiple stakeholders with diverse backgrounds, a one-size-fits-all issue still exists in the temporal aspect due to the cross-sectional nature of our study. Since strategy preferences may change over time, longitudinal tracking of implementation strategies is essential.⁹⁰ Future longitudinal designs would be beneficial to monitor evolving stakeholder preferences and adapt implementation strategies accordingly. Third, the labels assigned to the latent classes, while based on clear demographic and professional patterns within the sample, are descriptive interpretations rather than theoretically derived constructs. This practice, common in LCA, carries a risk of overfitting the interpretation to the specific sample characteristics, and the labels should be understood as heuristic summaries rather than definitive categories. Fourth, this study relied on self-reported stakeholder preferences through hypothetical choice tasks. Real-world implementation behaviors involve numerous uncontrolled factors absent from our experiment, so actual behavior may differ from our predictions. However, the study design followed best practices to limit the potential for bias. For instance, we limited the number of questions per respondent to reduce cognitive burden.³¹ Finally, while our study sample demonstrated diversity, its composition may impact generalizability and influence the LCA proportions. The majority of respondents were primary care practitioners, which may be attributed to the fact that ePRO-based symptom management is primarily implemented within primary care settings. Consequently, perspectives from secondary and tertiary hospitals, as well as health administrators, are less represented. Given that successful ePRO implementation requires coordination across care levels, this underrepresentation is a limitation. These sampling characteristics suggest our findings may be most applicable to primary care contexts in China, while their relevance to specialty care settings remains to be determined.

Conclusions

This study employs BWS methodology to analyze stakeholder preferences for implementing ePRO-based cancer symptom management in Chinese primary health care. The results of count analysis, CLM, and MLM highlighted a clear consensus on the importance of establishing tiered cancer management through ePRO systems, incentivizing EBPs, and actively engaging stakeholders, while traditional dissemination strategies were viewed as less effective. Importantly, although LCA revealed substantial heterogeneity in stakeholder preferences, tiered cancer management through ePRO emerged as a universally valued strategy. These findings provide critical insights for policymakers and healthcare leaders aiming to enhance the adoption and sustainability of ePRO interventions. Future efforts should focus on longitudinal validation of tailored strategies and economic evaluations to ensure scalable and cost-effectiveness implementation across diverse primary health care environments in China.

Supplemental Material

sj-pdf-1-dhj-10.1177_20552076261431856 - Supplemental material for Prioritizing implementation strategies of electronic patient-reported outcome in cancer management in primary health care: A best–worst scaling study among stakeholders

Supplemental material, sj-pdf-1-dhj-10.1177_20552076261431856 for Prioritizing implementation strategies of electronic patient-reported outcome in cancer management in primary health care: A best–worst scaling study among stakeholders by Changjin Wu, Jingyu Zhang, Xuqing Wang, Jundi Zheng, Min Li, Lu Xu, Maoyu Wei, Zhongqiang Shen, Ping Chen, Yuxian Nie, Qiuling Shi and Xiaojun Dai in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076261431856 - Supplemental material for Prioritizing implementation strategies of electronic patient-reported outcome in cancer management in primary health care: A best–worst scaling study among stakeholders

Supplemental material, sj-pdf-2-dhj-10.1177_20552076261431856 for Prioritizing implementation strategies of electronic patient-reported outcome in cancer management in primary health care: A best–worst scaling study among stakeholders by Changjin Wu, Jingyu Zhang, Xuqing Wang, Jundi Zheng, Min Li, Lu Xu, Maoyu Wei, Zhongqiang Shen, Ping Chen, Yuxian Nie, Qiuling Shi and Xiaojun Dai in DIGITAL HEALTH

Footnotes

Acknowledgements

We are grateful to all participants for sharing their insights in this survey.

ORCID iDs

Changjin Wu

Jingyu Zhang

Jundi Zheng

Min Li

Lu Xu

Qiuling Shi

Xiaojun Dai

Ethical considerations

Ethical approval for the process evaluation was obtained from the Yangzhou Hospital of Traditional Chinese Medicine Ethics Committee (ChiCTR2200065569). All individuals provided consent to participate in the survey.

Consent to participate

All individuals provided written consent to participate in the survey.

Author contributions

Changjin Wu, Jingyu Zhang, and Xuqing Wang contributed equally to this manuscript as joint first authors. Qiuling Shi and Xiaojun Dai contributed equally to this manuscript as joint corresponding authors. Changjin Wu: conceptualization, investigation, methodology, data curation, formal analysis, writing-original draft, writing-review and editing. Jingyu Zhang: investigation, methodology, data curation, writing-review and editing. Xuqing Wang: conceptualization, investigation, methodology, writing-review and editing. Jundi Zheng: investigation, methodology, data curation. Min Li: conceptualization, investigation, methodology, data curation. Lu Xu: investigation, methodology, data curation. Maoyu Wei: investigation, methodology, data curation. Zhongqiang Shen: investigation, methodology, data curation. Ping Chen: investigation, methodology, data curation, funding acquisition. Yuxian Nie: investigation, methodology, data curation. Qiuling Shi: conceptualization, writing-review and editing, funding acquisition, supervision. Xiaojun Dai: conceptualization, writing-review and editing, funding acquisition, supervision.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by Key Research and Development Program of Yangzhou (YZ2023091), Traditional Chinese Medicine Science and Technology Development Project of Jiangsu Province (ZD202426), National Traditional Chinese Medicine Advantageous Specialty Construction Unit Project ([2024]90), National Inheritance Studio Construction Project for Renowned Senior Traditional Chinese Medicine Experts ([2020]75), Health Commission of Sichuan Province Medical Science and Technology Program (24QNMP007), and Medical Research Program of Health Commission of Chengdu (2023535).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data generated or analyzed during this study can be shared by the corresponding author when a reasonable request is made.

Guarantor

Qiuling Shi and Xiaojun Dai.

Supplemental material

Supplemental material for this article is available online.

References

Basch

Schrag

Jansen

, et al. Symptom monitoring with electronic patient-reported outcomes during cancer treatment: final results of the PRO-TECT cluster-randomized trial. Nat Med 2025; 31: 1225–1232.

Rocque

Franks

Deng

, et al. Remote symptom monitoring with electronic patient-reported outcomes in clinical cancer populations. JAMA Netw Open 2025; 8: e259852.

Patt

Patel

Bhardwaj

, et al. Impact of remote symptom monitoring with electronic patient-reported outcomes on hospitalization, survival, and cost in community oncology practice: the Texas two-step study. JCO Clin Cancer Inform 2023; 7: e2300182.

Henson

Maddocks

Evans

, et al. Palliative care and the management of common distressing symptoms in advanced cancer: pain, breathlessness, nausea and vomiting, and fatigue. J Clin Oncol 2020; 38: 905–914.

Efficace

Cottone

Yanez

, et al. Patient-reported symptom monitoring and adherence to therapy in patients with newly diagnosed chronic myeloid leukemia. Cancer 2024; 130: 287–299.

Henry

Azzouz

Desta

, et al. Predictors of aromatase inhibitor discontinuation as a result of treatment-emergent symptoms in early-stage breast cancer. J Clin Oncol 2012; 30: 936–942.

Hiyoshi

Alexanderson

Tinghög

, et al. Future sick leave, disability pension, and unemployment among patients with cancer after returning to work: Swedish register-based matched prospective cohort study. Cancer 2025; 131: e35580.

Glare

Aubrey

Gulati

, et al. Pharmacologic management of persistent pain in cancer survivors. Drugs 2022; 82: 275–291.

Miller

Nogueira

Devasia

, et al. Cancer treatment and survivorship statistics, 2022. CA Cancer J Clin 2022; 72: 409–436.

10.

Rubin

Berendsen

Crawford

, et al. The expanding role of primary care in cancer control. Lancet Oncol 2015; 16: 1231–1272.

11.

Jefford

Howell

, et al. Improved models of care for cancer survivors. Lancet 2022; 399: 1551–1560.

12.

Hudson

Mollica

Reed

, et al. Enhancing capacity for primary care research in cancer survivorship: national cancer institute meeting report. J Natl Cancer Inst 2025; 117: 827–832.

13.

Faizan

Han

Lee

. Policy-driven digital health interventions for health promotion and disease prevention: a systematic review of clinical and environmental outcomes. Healthcare (Basel) 2025; 13: 2319.

14.

Schwartzberg

. Electronic patient-reported outcomes: the time is ripe for integration into patient care and clinical research. Am Soc Clin Oncol Educ Book 2016; 35: e89–e96.

15.

Basch

Schrag

Henson

, et al. Effect of electronic symptom monitoring on patient-reported outcomes among patients with metastatic cancer: a randomized clinical trial. JAMA 2022; 327: 2413–2422.

16.

Hassett

Cronin

Tsou

, et al. Esym: an electronic health record-integrated patient-reported outcomes-based cancer symptom management program used by six diverse health systems. JCO Clin Cancer Inform 2022; 6: e2100137.

17.

Basch

Abernethy

. Commentary: encouraging clinicians to incorporate longitudinal patient-reported symptoms in routine clinical practice. J Oncol Pract 2011; 7: 23–25.

18.

Mody

Bennett

Stover

, et al. Implementation of symptom monitoring with electronic patient-reported outcomes: perspectives and recommendations from community oncology practices (Alliance AFT-39). JCO Oncol Pract 2025; 21: 1830–1837.

19.

Zhang

Zheng

, et al. Implementation challenges of remote cancer symptom management with electronic patient-reported outcomes in China's primary health care settings: qualitative study. J Med Internet Res 2025; 27: e78333.

20.

Patt

Wilfong

Hudson

, et al. Implementation of electronic patient-reported outcomes for symptom monitoring in a large multisite community oncology practice: dancing the Texas two-step through a pandemic. JCO Clin Cancer Inform 2021; 5: 615–621.

21.

Basch

Rocque

Mody

, et al. Tenets for implementing electronic patient-reported outcomes for remote symptom monitoring during cancer treatment. JCO Clin Cancer Inform 2023; 7: e2200187.

22.

Flagg

Bates

Payne

. QIM25-238: navigating barriers to ePRO implementation in community-based oncology practices. J Natl Compr Canc Netw 2025; 23: QIM25-238.

23.

Lai-Kwon

Rutherford

Jefford

, et al. Using implementation science frameworks to guide the use of electronic patient-reported outcome symptom monitoring in routine cancer care. JCO Oncol Pract 2024; 20: 335–349.

24.

Abbas

Jeong

Lee

. Explainable AI in clinical decision support systems: a meta-analysis of methods, applications, and usability challenges. Healthcare (Basel) 2025; 13: 2154.

25.

Gong

Wang

Zhu

, et al. Bridging the digital divide to promote prevention and control of non-communicable diseases for all in China and beyond. Br Med J 2024; 387: e076768.

26.

Kirchner

Smith

Powell

, et al. Getting a clinical innovation into practice: an introduction to implementation strategies. Psychiatry Res 2020; 283: 112467.

27.

Bauer

Kirchner

. Implementation science: what is it and why should I care? Psychiatry Res 2020; 283: 112376.

28.

Proctor

Powell

McMillen

. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013; 8: 139.

29.

Baker

Camosso-Stefinovic

Gillies

, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev 2015; 2015: Cd005470.

30.

Cunningham

Henderson

Niccols

, et al. Preferences for evidence-based practice dissemination in addiction agencies serving women: a discrete-choice conjoint experiment. Addiction 2012; 107: 1512–1524.

31.

Williams

Candon

Stewart

, et al. Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment. BMC Psychiatry 2021; 21: 74.

32.

Chambers

. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press, 2023.

33.

Irie

Kerkhoff

Kim

, et al. Using stated preference methods to facilitate knowledge translation in implementation science. Implement Sci Commun 2024; 5: 32.

34.

Salloum

Shenkman

Louviere

, et al. Application of discrete choice experiments to enhance stakeholder engagement as a strategy for advancing implementation: a systematic review. Implement Sci 2017; 12: 140.

35.

Cunningham

Barwick

Rimas

, et al. Modeling the decision of mental health providers to implement evidence-based children's mental health services: a discrete choice conjoint experiment. Adm Policy Ment Health 2018; 45: 302–317.

36.

Lee

Soutar

Louviere

. The best-worst scaling approach: an alternative to Schwartz's Values Survey. J Pers Assess 2008; 90: 335–347.

37.

Baumgartner

Steenkamp

J-B

. Response styles in marketing research: a cross-national investigation. J Mark Res 2001; 38: 143–156.

38.

Whitty

Oliveira Gonçalves

. A systematic review comparing the acceptability, validity and concordance of discrete choice experiments and best-worst scaling for eliciting preferences in healthcare. Patient 2018; 11: 301–317.

39.

Van Houtven

Johnson

Kilambi

, et al. Eliciting benefit-risk preferences and probability-weighted utility using choice-format conjoint analysis. Med Decis Making 2011; 31: 469–480.

40.

Cheung

Wijnen

Hollin

, et al. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics 2016; 34: 1195–1209.

41.

Flynn

Louviere

Peters

, et al. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ 2007; 26: 171–189.

42.

Damschroder

Aron

Keith

, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4: 50.

43.

Waltz

Powell

Fernández

, et al. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci 2019; 14: 42.

44.

Finn

Louviere

. Determining the appropriate response to evidence of public concern: the case of food safety. J Public Pol Market 1992; 11: 12–25.

45.

Marley

AAJ

Flynn

Louviere

. Probabilistic models of set-dependent and attribute-level best–worst choice. J Math Psychol 2008; 52: 281–296.

46.

Louviere

Flynn

. Using best-worst scaling choice experiments to measure public perceptions and preferences for healthcare reform in Australia. Patient 2010; 3: 275–283.

47.

Marti

. A best-worst scaling survey of adolescents’ level of concern for health and non-health consequences of smoking. Soc Sci Med 2012; 75: 87–97.

48.

Min

Baek

Choi

. What are the key factors influencing newly graduated nurses’ preference for choosing their workplace? A best-worst scaling approach. J Nurs Scholarsh 2023; 55: 1204–1214.

49.

Lee

Kim

Han

. Most and least important attributes for domestic travel: a best-worst scaling approach. Front Psychol 2022; 13: 987384.

50.

Smith

Street

. The use of balanced incomplete block designs in designing randomized response surveys. Aust N Z J Stat 2003; 45: 181–194.

51.

Kim

Synnott

, et al. Developing criteria for health economic quality evaluation tool. Value Health 2023; 26: 1225–1234.

52.

Orme

. Sample size issues for conjoint analysis. WA, USA: Sawthooth Software Inc, 1998.

53.

Coltman

Devinney

Keating

. Best–worst scaling approach to predict customer choice for 3PL services. J Bus Logist 2011; 32: 139–152.

54.

Mühlbacher

Kaczynski

Zweifel

, et al. Experimental measurement of preferences in health and healthcare using best-worst scaling: an overview. Health Econ Rev 2016; 6: 2.

55.

Flynn

Marley

AAJ

. Best-worst scaling: theory and methods. UK: Edward Elgar, 2014.

56.

Flynn

Huynh

Peters

, et al. Scoring the Icecap-a capability instrument. Estimation of a UK general population tariff. Health Econ 2015; 24: 258–269.

57.

Mcfadden

. Condition logit analysis of qualitative choice behavior. Front Econometr 1974: 105–142.

58.

McFadden

Train

. Mixed MNL models for discrete response. J Appl Econ 2000; 15: 447–470.

59.

Rischatsch

Zweifel

. What do physicians dislike about managed care? Evidence from a choice experiment. Eur J Health Econ 2013; 14: 601–613.

60.

Greene

Hensher

. A latent class model for discrete choice analysis: contrasts with mixed logit. Transp Res B Methodol 2003; 37: 681–698.

61.

Cohen

. Maximum difference scaling: improved measures of importance and preference for segmentation. In: Sawtooth software conference proceedings, 2003, pp.61–74.

62.

Gonzalez

. A guide to measuring and interpreting attribute importance. Patient 2019; 12: 287–295.

63.

Osman

AMY

, et al. Eliciting SF-6Dv2 health state utilities using an anchored best-worst scaling technique. Soc Sci Med 2021; 279: 114018.

64.

Lopez

Teggart

Ahmed

, et al. Implementation of electronic prospective surveillance models in cancer care: a scoping review. Implement Sci 2023; 18: 11.

65.

Crossnohere

Anderson

Baumhauer

, et al. A framework for implementing patient-reported outcomes in clinical care: the PROTEUS-practice guide. Nat Med 2024; 30: 1519–1520.

66.

Hassett

Dias

Cronin

, et al. Strategies for implementing an electronic patient-reported outcomes-based symptom management program across six cancer centers. BMC Health Serv Res 2024; 24: 1386.

67.

Powell

Beidas

Lewis

, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res 2017; 44: 177–194.

68.

Powell

Fernandez

Williams

, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health 2019; 7: 3.

69.

Brandenbarg

Roorda

Groenhof

, et al. Primary healthcare use during follow-up after curative treatment for colorectal cancer. Eur J Cancer Care (Engl) 2017; 26: e12581.

70.

Chan

Crawford-Williams

Crichton

, et al. Effectiveness and implementation of models of cancer survivorship care: an overview of systematic reviews. J Cancer Surviv 2023; 17: 197–221.

71.

, et al. Diabetes in China part 2: prevention, challenges, and progress. Lancet Public Health 2024; 9: e1098–e1104.

72.

Liang

Wang

. Longitudinal study of the earliest pilot of tiered healthcare system reforms in China: will the new type of chronic disease management be effective? Soc Sci Med 2021; 285: 114284.

73.

Zhou

, et al. Inequalities in non-communicable disease management in China and progress toward universal health coverage: an analysis of nationwide household survey data from 2004 to 2018. Lancet Reg Health West Pac 2024; 44: 100989.

74.

Wang

, et al. National and subnational trends in cancer burden in China, 2005-20: an analysis of national mortality surveillance data. Lancet Public Health 2023; 8: e943–e955.

75.

Meiklejohn

Mimery

Martin

, et al. The role of the GP in follow-up cancer care: a systematic literature review. J Cancer Surviv 2016; 10: 990–1011.

76.

Arora

Crastes Dit Sourd

Hanson

, et al. Linking health worker motivation with their stated job preferences: a hybrid choice analysis in Ethiopia. Soc Sci Med 2022; 307: 115151.

77.

Afolabi

Fernando

Bottiglieri

. The effect of organisational factors in motivating healthcare employees: a systematic review. Br J Healthc Manage 2018; 24: 603–610.

78.

Maslow

. A theory of human motivation. Psychol Rev 1943; 50: 370.

79.

Schweri

Hartog

. Do wage expectations predict college enrollment? Evidence from healthcare. J Econ Behav Organ 2017; 141: 135–150.

80.

Mathauer

Imhoff

. Health worker motivation in Africa: the role of non-financial incentives and human resource management tools. Hum Resour Health 2006; 4: 24.

81.

Zhao

. Shortage of healthcare professionals in China. Br Med J 2016; 354: i4860.

82.

Collins

Murphy

Strecher

. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med 2007; 32: S112–S118.

83.

Collins

. Optimization of Behavioral, Biobehavioral, and Biomedical Interventions: The Multiphase Optimization Strategy (MOST). Switzerland: Springer Cham, 2018.

84.

Szeszulski

Guastaferro

. Optimization of implementation strategies using the multiphase optimization STratgey (MOST) framework: practical guidance using the factorial design. Transl Behav Med 2024; 14: 505–513.

85.

Dopp

Kerns

SEU

Panattoni

, et al. Translating economic evaluations into financing strategies for implementing evidence-based practices. Implement Sci 2021; 16: 66.

86.

Hoomans

Severens

. Economic evaluation of implementation strategies in health care. Implement Sci 2014; 9: 168.

87.

Abbas

Seol

Abbas

, et al. Exploring the role of artificial intelligence in smart healthcare: a capability and function-oriented review. Healthcare (Basel) 2025; 13: 1642.

88.

Abbas

Zahir

, et al. Federated learning in smart healthcare: a comprehensive review on privacy, security, and predictive analytics with IoT integration. Healthcare (Basel) 2024; 12: 2587.

89.

Yakovchenko

Lamorte

Chinman

, et al. Comparing the CFIR-ERIC matching tool recommendations to real-world strategy effectiveness data: a mixed-methods study in the Veterans Health Administration. Implement Sci 2023; 18: 49.

90.

Smith

Merle

Webster

, et al. Tracking dynamic changes in implementation strategies over time within a hybrid type 2 trial of an electronic patient-reported oncology symptom and needs monitoring program. Front Health Serv 2022; 2: 983217.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.44 MB

0.10 MB