Developing a Novel Values Elicitation Measure for Clinical Use in Older Adults with Hematologic Malignancies: Values-HM

Abstract

Introduction. The recent expansion of treatment options for older adults with blood cancers has increased the necessity for understanding patient preferences and values to inform treatment decision making. Currently, no values elicitation measures have been validated for clinical use in this population. Values-HM (Values elicitation measure for Hematologic Malignancies) is a novel values elicitation measure based on best–worst scaling. It was developed specifically for older adults with blood cancers involving multiple stakeholders including patients, caregivers, and clinicians. Objective. The objective of this study was to test the preliminary validity, reliability, and acceptability of this measure for clinical use based on US Food and Drug Administration and International Society for Pharmacoeconomics and Outcomes Research guidance. Results. Twenty-nine adults aged ≥60 y with newly diagnosed lymphoma, leukemia, myelodysplastic syndrome, and multiple myeloma were enrolled. Most patients were White (97%). Most patients (88%) felt that the measure was relevant to them, showed their real preferences (73%), and was acceptable to clarify their preferences (68%). The measure demonstrated discriminant validity of importance scores and convergent validity for most patients (77%) with a ranking exercise of the treatment values. Cognitive interviews with 10 participants suggested question comprehension, outcome understanding, and appropriate judgment. Data from 18 participants who completed the measure more than once provide some evidence for the reliability of the measure to capture repeated values over time. Conclusions. These initial data suggest that Values-HM may be acceptable to patients and valid and reliable to capture patient values. Additional data are needed to confirm these findings in a larger and more diverse sample. This trial was registered at www.clinicaltrials.gov as #NCT05061095.

Highlights

Values-HM is a novel values elicitation measure designed based on best–worst scaling for adults with blood cancers.

Values-HM appears to be acceptable to patients and may be valid and reliable to capture patient values in clinical care.

Graphical abstract

Keywords

Patient Preference Decision Making,Shared Hematologic Neoplasms Patient-Reported Outcome Measures best-worst scaling values elicitation treatment decision making older adults

Introduction

Hematologic malignancies are a group of heterogeneous diseases including leukemia, lymphoma, and multiple myeloma. Roughly 1.3 million people are currently living with blood cancers in the United States, with more than 180,000 new diagnoses made each year.¹ Blood cancers primarily affect older adults, with most patients being diagnosed over the age of 60 y.² In the last decade, more than 50 new drug therapies have been approved for blood cancers in the United States, rapidly expanding treatment options and fundamentally changing treatment paradigms.³ Increasingly, these new drug approvals have enabled oncologists to personalize chemotherapy recommendations to achieve what matters most to individual patients.

Patients with blood cancers differ substantially in what matters most to them.^4–8 Some patients prefer to maximize their chances of extended survival time while others prefer to minimize treatment impacts as a means of maintaining their quality of life. This preference heterogeneity leads to differences in ideal treatments. Eliciting patient treatment values is an essential aspect of quality shared decision making and a standard of care.^9–11 However, this process is challenging and often poorly executed¹²; oncologists fail to ask patients about their treatment values in up to 60% of new encounters.¹³ Patients frequently report dissatisfaction with communication and report feeling left out and uninformed about treatment options.^14–18 Current practice often results in physician perceptions of patient treatment values that are discordant with patient values.^19–22 This discordance undermines patient autonomy and results in downstream harm to patients including increased patient distress, lower quality of life, and wasted health care resources.^23–26

Values elicitation measures using validated theory-driven methods are increasingly being applied to health care to understand patient values.^27–29 Multiple stakeholders have called for the development of validated tools to elicit patient treatment values to improve shared decision making in oncology.^11,30–33 We developed and refined a values elicitation measure for patients with blood cancers called Values-HM (Values elicitation measure for Hematologic Malignancies) that uses a validated theory-driven preference method called best–worst scaling (BWS).^4,34–39 BWS is commonly used to assess the priorities of patients and other stakeholders in health and has been used in other clinical settings in oncology.^40,41 The development of Values-HM was an iterative process that included data from more than 800 patients, 200 caregivers, and clinical and methodological experts in addition to formal heuristic testing using human-centered design principles.^4,34–39

This study sought to test the preliminary construct validity of the Values-HM measure and evaluate the initial feasibility, acceptability, and reliability of using the measure with newly diagnosed older adults with blood cancers in a clinical setting. The findings of this study may provide some support for the use of Values-HM in clinical and research settings and provide additional evidence for the development and testing of similar measures for different clinical contexts.

Methods

Study Design

The US Food and Drug Administration (FDA) released a statement on evaluating the quality of patient preference studies to inform clinical care and patient-focused drug development that includes 14 recommended qualities of patient preference studies.⁴² This statement is relevant because we developed a patient preference measure to capture patient values. We summarize these 14 qualities and how we applied them to the development of this measure in Table 1. We followed published guidance on the development and testing of values elicitation measures and published guidance on evaluating the content validity of patient-reported outcome measures to design the study.^29,43,44 The study was approved by the Institutional Review Board (#21-2282) at the University of North Carolina at Chapel Hill. This trial was registered at www.clinicaltrials.gov as #NCT05061095. All participants provided written informed consent in compliance with regulatory requirements.

Table 1

Application of Quality Indicators from the FDA Guidance on Use of Patient Preference Information

Domain	Quality Indicator	Summary of FDA Description	Application to this Study
Patient focused	Patient centeredness	Patient preference studies should ensure that the patient, not the health care professional, is the central focus of the study.	All testing and evaluation of the measure come directly from patients including patient-reported outcomes, interviews, and feedback sessions.
	Representativeness of the sample and generalizability	A study should measure the preferences of a representative sample of adequate size so that the study can be generalizable and include participants of the population of interest.	This study followed published guidance for sample size for evaluating relevant outcomes. Outcomes were derived from large population samples to ensure generalizability. All participants included were from the population of interest.
	Relevance	Preferences should be measured over relevant clinical domains. Critical aspects of harm, risk, benefit, and uncertainty should be included in the measure.	Prior work to develop the measure included iterative improvement of outcomes to ensure relevance. Relevance to the population of interest was specifically measured in the current study both in surveys and in interviews.
Choice task design	Comprehension by study participants	Study participants should fully understand all the information being presented to them.	In prior studies, human-centered design was used to improve understandability of the outcomes. In this study, cognitive interviews were completed with patients to ensure understandability.
	Minimal cognitive bias	Study design should minimize potential cognitive biases such as framing, anchoring, or ordering effect.	The choice task design limits anchoring and simplifying heuristics by not presenting numerical values. The design limits ordering effects by varying the position of each outcome in question sequence.
	Effective communication of benefit, harm, risk, and uncertainty	Studies should define the context of the decision, explain outcomes, and help patients conceptualize probabilities/uncertainty.	The context of the decision was explicitly addressed with participants by the measure. Using best–worst scaling eliminates problems with numeracy by presenting only objects, not numerical levels. Cognitive interviews were completed demonstrating that patients appropriately completed tradeoffs between outcomes in the choice tasks.
Data quality	Established good research practices	Studies should follow guidelines for good research practices established by a recognized professional organization such as ISPOR.	Development of the tool followed ISPOR conjoint analysis taskforce checklist recommendations and ISPOR PRO good research practices.
	Study conduct	Studies should be administered by trained research staff in compliance with the study protocol.	Trained study staff followed a peer-reviewed protocol in accordance with cancer center policies.
	Logical soundness	Data should include internal validity tests of logic and consistency and should be verified for conformity with logic and consistency.	We compared the consistency of data from the measure with ranking exercises. These data are reported along with data on patient preferences.
Analysis	Robustness of analysis of results	Data analyses should ensure appropriate interpretation of the collected evidence including estimates of uncertainty of the parameters of interest.	Analyses followed ISPOR guidance including reporting of confidence intervals around population-level estimates of importance scores.
Analysis	Capturing heterogeneity of patients’ preferences	Data analyses should report preference heterogeneity that may be population, condition, treatment, and study specific.	Data were analyzed using frequency (count) analyses, which allows for reporting of heterogeneity through comparison between individual participant preferences and cohort preferences.

FDA, US Food and Drug Administration; ISPOR, International Society for Pharmacoeconomics and Outcomes Research.

Development of Values-HM

The initial development of the measure has been previously published.^4,34–36 In sum, we developed the outcomes (objects) of the choice experiment through multistakeholder engagement including clinical experts, patients, and caregivers.³⁴ Following, we tested a version of the measure in a study with more than 800 patients with acute myeloid leukemia and 200 caregivers.³⁶ We then tailored the measure for clinical use with older adults with blood cancers by completing a mixed-methods study involving patients, clinicians, and caregivers including qualitative assessments (think-aloud sessions and semi-structured interviews) and questionnaires to iteratively improve the perceived usability and the cognitive workload required to complete the measure.³⁷ This process led to the refinement of the 7 outcomes included in Values-HM.

Values-HM is a type of BWS measure, an analytically efficient, validated theory-driven preference method that allows for the quantification of the tradeoff importance of objects based on a series of questions (choice tasks).⁴⁰ Providing relative importance, or tradeoff preference, is a major strength of BWS compared with Likert scales, in which all outcomes may be rated as equally important.²⁷ In designing Values-HM, we sought to develop a values elicitation measure that would be valid and reliable to prioritize the values of individual patients in a clinical setting. Values-HM is suitable for analysis on both the cohort and individual level.^37,45 The importance scores can be calculated by count analysis, which is simple compared with the regression analysis needed for discrete choice experiments.⁴⁶ This analysis results in individual importance scores for each outcome for each individual patient. This output gives clinical teams knowledge as to which outcomes are most important and the relative value that the patient put on each individual outcome compared with other included outcomes.

Participants complete Values-HM choice tasks by selecting 1 outcome that is most important and 1 that is least important. Each choice task contains a subset of 4 of the 7 included outcomes: maintain day-to-day activities, living longer, avoid becoming dependent upon others, avoid short-term side effects, avoid long-term side effects, avoid hospitalizations, and avoid high financial costs. See the example choice task in Figure 1; the full measure is included in Appendix Figure 1. Participants complete a total of 7 choice tasks. A balanced incomplete block design was used to construct the choice sets.²⁷

Figure 1

Sample choice task in Values-HM (Values elicitation measure for Hematologic Malignancies). Values-HM includes 7 choice tasks in which respondents choose the most important and least important outcome among a subset of 4 outcomes. A total of 7 outcomes are included in the choice tasks.

Data Collection

Older adults (aged ≥60 y) with newly diagnosed blood cancers were sequentially recruited to participate. Participants were eligible if they were within 28 d of initial diagnosis, had not yet started on initial definitive treatment, and were able to read and communicate in English. We specifically recruited patients with newly diagnosed blood cancers to ensure that the measure was able to capture values in the setting of significant psychological and physical distress, which is common among patients with newly diagnosed blood cancers.^14,47,48

Participants were recruited remotely and after completing consent were prompted to complete Values-HM, alongside patient-reported outcome measures and study-specific surveys through an online portal to evaluate acceptability. Baseline clinical and demographic information was also captured upon enrollment. After completing their initial consultation with their oncologist, participants were prompted to complete additional surveys and Values-HM. Participants were prompted to complete Values-HM as part of longitudinal assessments initially at a 2-wk interval (up to 5 times) then every 3 mo for up to 2 y.

A subset of participants was asked to complete interviews after the initial treatment decision to evaluate the content validity of the measure. Participants were contacted via phone to schedule a virtual interview at their convenience. Interviews were scheduled within 6 mo of enrollment. Cognitive interviews were conducted in accordance with FDA and International Society for Pharmacoeconomics and Outcomes Research (ISPOR) taskforce guidance on establishing the content validity of patient-reported outcomes measures.^42–44 To evaluate participant understanding of outcomes, participants were asked to describe each outcome in their own words. To evaluate the content validity of Values-HM, participants were asked to “think aloud” while completing choice tasks to provide the research team with an understanding of the cognitive process they went through to arrive at answers for each choice task. To assess for missing domains, participants were asked about additional areas that they felt important to include in a values elicitation measure that were not included in Values-HM. A member of our research team with experience in conducting qualitative interviews (C.J.M.) facilitated the interview, while 2 other members took field notes on the interview.

Acceptability, Feasibility, and Validity Assessment

Despite ISPOR recommendations to measure participant acceptability in the development of measures, there are not well-established methods by which to do so. Pilot and developmental studies typically assess acceptability through qualitative interviews. Larger studies have assessed acceptability by asking participants to indicate their level of agreement with several statements including, “I found it easy to understand the questions,”“I found it easy to answer the questions,” and “My answers showed my real preferences.”^28,34,49,50 We followed these studies in assessing participants’ level of agreement with all 3 of these statements after participants completed Values-HM for the first time. We stipulated a threshold of agreement of 70% for acceptability, which is similar to thresholds used in other similar studies.^49,51

We assessed the feasibility of clinical implementation (that is, the extent to which Values-HM could be successfully used or carried out within a clinical setting) through the evaluation of rates of consent and completion of the measure.⁵² There was no predetermined threshold for feasibility, although we anticipated using rates of completion to inform further work on identifying appropriate implementation strategies for individual contexts.

To begin to explore discriminant validity, the degree to which a measure can distinguish between different constructs, we compared importance weights and confidence intervals among the 7 outcomes. If the measure demonstrated discriminant validity, we would anticipate that the importance weights of each outcome would vary.⁵³ Conversely, we would expect all importance scores to be clustered around 0 if the measure lacked discriminant validity. Preliminary evidence of convergent validity was explored by comparing each participant’s most important outcome derived from the BWS measure to a ranking exercise.⁴⁴ Analysis of variance was used to evaluate differences in variations between within-subject importance scores (within-person variance, test–retest noise or error) and between-subject importance scores (between-subject variance, model variance) to evaluate for test–retest reliability.⁵⁴ This approach assumes that differences within persons are error; however, the importance of patient values may change over time due to many clinical and personal factors that are not model error. Thus, this approach is conservative and may overestimate error.

Data Analysis

Data from Values-HM were analyzed using count (frequency) analysis.⁴⁶ Descriptive statistics were used to describe the heterogeneity of values based age, sex, and income. Interviews were recorded and coded based on thematic deductive analysis. Cognitive interview question data were used to describe language in the BWS measure for understandability, concept relevance, necessary coverage, and degree of bother. These domains were analyzed qualitatively.⁵³

Results

Study Participants and Feasibility of Clinical Implementation

Potential participants were identified through a review of the electronic medical record at the University of North Carolina at Chapel Hill (UNC), an academic medical center. UNC serves patients from all counties in North Carolina with a 77% urban and 23% rural population. Most patients (71%) seen at UNC are White. A total of 185 potential participants were identified (see the CONSORT diagram in Figure 2). Of those, 164 were approached (88.6%). A total of 51 participants were consented (32% of approached, Appendix Table 1). Twenty-nine participants of the 51 consented (56.9%) completed at least 1 survey including Values-HM as part of the baseline surveys (Table 2). The median age was 70 y, with a slight male predominance. Most patients had lymphoma, leukemia, or myelodysplastic syndrome.

Figure 2

CONSORT diagram indicating eligibility, approach, and completion of Values-HM.

Table 2

Participant Characteristics

Characteristics (Total N = 29)	n (%)
Total	29
Age, y, median (range)	70 (62, 95)
Sex, % male	17 (58.6)
Race
White	28 (96.6)
African American	1 (3.4)
Diagnosis
Lymphoma	12 (43.1)
Leukemia	8 (27.5)
Myelodysplastic syndrome	8 (27.5)
Multiple myeloma	1 (3.4)
Income, yearly
$75,000 or less	12 (41.4)
More than $75,001	13 (44.8)
Missing	4 (13.7)
Education level
High school graduate, diploma, or the equivalent	6 (20.7)
Some college, no degree	2 (6.9)
Associate degree or higher	21 (72.4)

Values-HM, Values elicitation measure for Hematologic Malignancies.

Acceptability, Understandability, and Relevance

Twenty-six participants (89.7%) completed at least 1 survey to evaluate acceptability, understandability, and relevance. Eighty-five percent (22/26) of participants agreed or strongly agreed that the measure was understandable. Seventy-three percent (19/26) of participants agreed or strongly agreed that the measure showed their real preferences, 54% (14/26) felt the measure was easy to answer, and 88% (23/26) of participants felt the measure was relevant. Nineteen of the 26 participants (73%) completed additional study-associated surveys after making a treatment decision. Of these, 68% (13/19) agreed or strongly agreed that the measure was acceptable to clarify their preferences.

Convergent and Discriminant Validity

The measure demonstrated convergent validity with a simple ranking exercise of the outcomes: of the 26 participants who fully completed baseline ranking and the BWS measure, 77% (20) had convergence of the top priority on Values-HM with 1 of their top 3 ranked outcomes including 14 of 20 (70%), of which the top outcome was the same in both the ranking exercise and the BWS measure. In 72% (21) of participants, no conflicts were found comparing their BWS least important to the top 3 ranking. Of the participants who had conflicts, 2 had complete mismatches in which their BWS least important outcome was the outcome they chose in the ranking exercise as most important. The Values-HM measure resulted in disparate importance scores and confidence intervals for individual outcomes suggesting discriminant validity (see the “Most and Least Important Treatment Values” section below).

Test–Retest Reliability

Eighteen participants completed Values-HM more than once as part of repeated assessments. Of patients who completed Values-HM more than once, the median number was 8. The median time from the first completion to the second completion was less than 30 d. These data were used to compare differences in variations between within-subject importance scores (error) and between-subject importance scores (model) to evaluate for test–retest reliability. Differences were statistically significant for the following outcomes: avoid becoming dependent on others (mean square for model [MS_m] 3.60, mean square for error [MS_e] 1.41, P = 0.002), avoid high financial costs (MS_m 3.90, MS_e 1.17, P < 0.001), avoid long-term side effects (MS_m 4.35, MS_e 0.80, P < 0.001), and avoid hospitalizations (MS_m 4.89, MS_e 0.64, P < 0.001) (Appendix Table 2). Differences approached statistical significance for the other outcomes: avoid short-term side effects (MS_m 1.93, MS_e 1.21, P = 0.08, living longer (MS_m 2.56, MS_e 1.56, P = 0.06), and maintain day-to-day activities (MS_m 1.52, MS_e 1.13, P = 0.18). There was the most variability in the population (MS_m) for avoid hospitalizations, avoid long-term side effects, and avoid high financial costs. Maintain day-to-day activities and avoid short-term side effects had the lowest population variability. These data suggest that, in general, there was more variation in the population (model) than within participants (assumed error), which suggests reliability of the measure for capturing repeated measurements over time.

Content Validity

Ten participants completed cognitive interviews to assess several aspects of content validity of the included outcomes in Values-HM. Data saturation was reached at 10 interviews. Results are summarized in Table 3. All participants were able to accurately describe the instructions of Values-HM. All participants were able to correctly define each included outcome. All patients understood the choice task, although some found completing the choice task challenging due to difficulty choosing between or among multiple important factors. Patients frequently cited individual experiences that led them to choose specific outcomes in some choice tasks. Suggested improvements included addressing pain levels, enhancing communication with providers, and incorporating aspects of psychological support and individual circumstances into the tool. All patients reported it was beneficial. No patients reported missing domains of outcomes.

Table 3

Summary of Results from Cognitive Interviews

Domain	Specific Questions	Key Theme	Summary of Patient Responses
Question comprehension	• How well did you understand the BWS questions?• How easy was it to answer these questions?• Can you again describe in your own words what this question asked you to do?• Were the instructions at all confusing?	• Comprehension• Engagement	• Most patients understood the questions, although some found them challenging, especially when ranking multiple factors.• Patients had mixed experiences regarding the ease of answering the questions, with some finding the concept straightforward while struggling to weigh their choices meaningfully.• Strong grasp of the task and its requirements.
Outcome understanding	• Could you rephrase this in your own words or provide an example of what this would mean for you?	• Comprehension• Clarity	• Patients clearly understood and could rephrase outcomes.
Information retrieval	• What tricks or patterns did you use when selecting responses?• What recall period did you consider for what is most important?	• Various strategies• Varied time frames• Individual circumstance	• Maintaining consistency, referencing previous responses, using elimination methods, reflecting carefully, and drawing on past experiences.• Patients expressed recall periods typically of 1 to 5 y.• Responses were influenced by their current health status.
Judgment	• Walk me through how you made your choices on this question.• What do you think is missing from the survey?• Did the survey influence your discussion with the doctor?	• Mixed methods• Thoughtful engagement• Personalized and contextual considerations• Organization• Reinforcement of values	• Patients demonstrated a mix of reflective and analytical approaches, balancing personal insights with the outcomes presented in the choices.• Some reported more explicit considerations of pain, enhanced communication with providers, and a focus on psychological support and individual factors.• The measure was helpful in organizing their thoughts and reinforcing their values.
Response	• What factors are most important when considering treatment?	• Quality of life, independence, and longevity	• Patients prioritized quality of life and independence over merely living longer, influenced by personal experiences and the desire for normalcy in daily activities.

BWS, best–worst scaling.

Most and Least Important Treatment Values

The most and least important treatment values for the entire cohort at baseline are shown in Figure 3a. Maintaining day-to-day activities was the most important outcome for the cohort (mean importance score [mean best-minus-worst scores, max 4, min −4] 1.48, 95% confidence interval [CI] 0.70, 2.27), followed by living longer (0.97, 95% CI 0.00, 1.93), avoiding becoming dependent on others (0.55, 95% CI −0.21, 1.32), avoiding hospitalizations (0.38, 95% CI −0.24, 1.00), avoiding long-term side effects (−0.21, 95% CI −0.86, 0.45). Avoiding short-term side effects (−1.59, 95% CI −2.17, −1.00) and avoiding high financial costs were the least important (−1.59, 95% CI −2.35, −0.83).

Figure 3

(a) Cohort-level analysis of the mean importance score (mean best-minus-worst score) at baseline. Error bars are 95% confidence intervals. (b) Individual-level analysis of individual importance scores. Bars represent the number of patients choosing each treatment outcome value (attribute) as most or least important based on best-minus-worst scores. Four patients had a tie between treatment values for the most important treatment value, and 8 patients had a tie for the least important treatment value.

The individual-level analysis of the most and least important values at baseline is shown in Figure 3b. Results from best–minus worst scoring were the same (“tied”) for 4 patients for their most important value and 8 patients for their least important value. The most important values (including ties) for individual patients were living longer (n = 10, 34% of patients) and maintaining day-to-day activities (n = 10, 34%). The least important values were avoiding high financial costs (n = 13, 45%) and avoiding short term side effects (n = 8, 28%). While 10 patients (34%) chose living longer as most important, 5 patients (17%) chose it as least important. Only 1 patient chose maintaining day-to-day activities as least important. One patient chose avoiding high financial costs as most important, while no patients chose avoiding short-term side effects as most important.

We assessed the heterogeneity of values based on age, sex, and income, although the statistical analysis was limited by the relatively small sample size. For patients aged >70 y, 7 of 17 (41%) chose maintain day-to-day activities as most important, while 4 of 12 (33%) patients aged <70 y chose it as most important. Similarly, for patients aged >70 y, 7 of 17 (41%) chose living longer as most important, whereas 4 of 12 (33%) of patients aged <70 y chose it as most important. Sex did not appear to be associated with differences in values. For those making $75,000 or more, most chose avoid high financial costs as least important (10/13, 77%) while only 2 of 12 (16%) patients making less than $75,000 chose it as least important.

Discussion

A critical first step toward the routine integration of patient values into clinical care is the development of validated clinical measures. Values-HM is a theory-driven values elicitation measure developed for clinical use for patients with blood cancers using best practices for measure development. We followed guidance from the FDA (Table 1) and ISPOR in developing Values-HM and reporting results. This study builds on several prior studies involving patients, caregivers, and clinicians.^35–38,48 Data from quantitative surveys and cognitive interviews in this study expand on previous work demonstrating that most older adults with blood cancers found the measure relevant to them, acceptable to clarify their preferences, and understandable. Cognitive interviews with participants demonstrated that participants actively engaged in the process of completing each choice task using reflective and analytical approaches to weigh each of the outcomes and arrive at their response, demonstrating high engagement with the measure in the process of values clarification. The measure demonstrated some level of discriminant validity and convergent validity with other value elicitation methods. The measure also showed evidence of reliability in repeated measurement over time. Together, these data provide preliminary evidence that Values-HM may be valid for clinical use.

We designed Values-HM to be clinically actionable.^37–39,55 Therefore, in this study we sought to produce initial evidence of validity and reliability in a clinical setting with individual patients. Values-HM provides individual importance scores for 7 outcomes that are most important to older adults with cancer. Prior studies suggest that these importance scores may support clinical teams to align care to what matters most to each patient.³⁸ Values-HM is not a decision aid to determine therapy but rather a decision support tool intended to foster engagement between patients and clinicians (who are disease experts) to integrate patient values into many different treatment paradigms. We designed Values-HM to be treatment and disease agnostic so that it could be widely used across different disease and treatment decisions even as disease paradigms change over time or as new therapies are developed. This provides clinicians and researchers flexibility in using Values-HM depending on the context. Values-HM could be collected prior to a patient visit, for example, through a patient portal or in the waiting room, to support shared decision making or longitudinally to inform ongoing care.

Because Values-HM is built on discrete choice methodology, it functions as both a values elicitation and values clarification tool. When patients complete Values-HM, it is necessary for them to “tradeoff” what matters most to them among 7 important outcomes. This is not an easy task (as noted by nearly half of respondents) because it requires prioritizing between living longer and maintaining aspects of quality of life such as day-to-day activities or independence and carefully weighing the tradeoffs between quantity and quality of life. In the end, most treatment decision making for older adults is similar to this process—deciding a direction among many options with competing risks and benefits. This similarity makes us believe that Values-HM could be helpful in preparing patients to make a treatment decision among options with competing risks and benefits.

We foresee that Values-HM may be most useful for patients with newly diagnosed or newly relapsed disease as treatment decisions in these contexts are naturally contingent on patient values. We developed Values-HM with patients with newly diagnosed blood cancers because this specific treatment context is often the most challenging for patients. Patients experience high levels of distress, uncertainty, and fear and find making initial treatment decisions troubling.^14,48 As such, the use of the measure with patients with newly diagnosed disease is most appropriate, although it seems reasonable that the measure may be helpful in similar treatment decision scenarios such as relapse. While Values-HM was specifically designed for patients with blood cancers, further studies may demonstrate that it is useful beyond this population as well.

We believe that using Values-HM together with validated frailty measures—especially validated geriatric assessments—is the best way to properly inform treatment decisions. Assessing patient values and preferences alongside validated geriatric assessments is optimal to improve outcomes.^56–58 This is because understanding patient values together with functional/frailty data is essential for clinicians to adequately understand and discuss the individualized risk–benefit tradeoffs facing each older adult with cancer.^9,57–59 Using this measure alongside the Practical Geriatric Assessment would be an efficient way to capture values and other critical domains.⁵⁸ In this study, we did not provide the results of Values-HM to patients; however, in our subsequent work, we discovered that real-time feedback was valued by patients to inform shared decision making.^38,39,55 Therefore, we now routinely provide individual results from Values-HM to all patients in real time.

This study adds to prior work demonstrating that many older adults with cancer prioritize aspects of quality of life over survival.^4,60,61 Many therapeutic clinical trials, which are essential to test novel treatments, are not designed with quality of life as the primary endpoint but rather overall survival. This may result in discordance between patient values and trial endpoints. Using values elicitation measures such as Values-HM as part of screening for clinical trials may help identify patients whose values are aligned with clinical trial endpoints or to prompt a discussion of alternative treatments that are more appropriate. Recruiting more “value-aligned” patients may result in higher recruitment, more engaged participation, and a reduction in dropout. Given that recruitment of older adults to treatment trials has historically been poor,⁶² this would be a substantial improvement.

This work provides a proof of principle for those who desire to develop tools to capture patient preferences or values in real time to inform clinical care. However, more scientific advances are necessary to see routine values assessment be integrated into the care of all older adults. A critical next step is to begin to identify implementation strategies. We envision Values-HM being collected alongside other electronic patient-reported outcomes (ePROs) via a patient portal or in person via a tablet with results provided to clinicians prior to treatment decisions. Many cancer centers are investing in platforms to collect ePROs given the proven benefits to their routine collection during cancer treatment.^63,64 These efforts likely will identify several appropriate strategies for the implementation of values elicitation measures.^65,66 Despite the similarity between ePROs and values elicitation measures, unique questions about the implementation of values elicitation measures remain, including: What is the best values elicitation method for each context? How do we apply patient values to different clinical contexts? How frequently should values be elicited? Can we use screening measures to reduce survey burden? In addition, other studies are necessary to evaluate the effect of using values elicitation measures on treatment decision making and other downstream outcomes. We are currently testing the effect on shared decision making of Values-HM and a similar measure we designed in randomized clinical trials for older adults with blood cancers (NCT06296368, NCT06697600).⁶⁷ These studies will help demonstrate how these measures affect the quality of decision making, patient distress, and health care utilization. The use of other shared decision-making tools have shown downstream benefits with varying degrees in different settings.²⁵

Limitations

There is a lack of clear guidance for the development of values elicitation measures that are to be used in a clinical setting. Therefore, our methods in combining advising bodies’ recommendations are subject to error. There are no benchmarks to determine validity and reliability for values elicitation measures. This makes the interpretation of our data particularly challenging.

Based on findings from similar studies, we anticipated that it may be challenging to recruit older participants with newly diagnosed blood cancers within the necessary recruitment window between diagnosis and treatment.² Unfortunately, the recruitment rate and the completion rate of Values-HM in this study were still both lower than anticipated, prompting us to complete a post hoc analysis of barriers to completion. In this analysis, we found that patients found the online consent process inefficient and frequently did not return to complete subsequent measures due to the cumbersome electronic delivery platform of Values-HM.⁵⁵ We now have employed several implementation science strategies to reduce patient burden in consent and completing Values-HM, which we have employed as part of the ongoing randomized controlled trial.⁵⁵ With these changes, completion rates of the measure on that trial are greater than 80%. This improvement in completion rates demonstrates that the low completion rates in the current study were due to identified barriers, not inherent to Values-HM as a measure. While this demonstrates that it is feasible to use Values-HM in clinical care, it also highlights the importance of identifying local resources to support older adults in the completion of measures during this vulnerable time.

White participants were overrepresented in this study compared with our clinical population. Four Black participants and 1 Asian participant consented to the study, which together represented 9.7% of the total population consented. Unfortunately, of these 5 patients, only 1 patient completed Values-HM. This substantially limited the racial diversity of the sample completing Values-HM and reduces our confidence in the generalizability of our findings. Although there was adequate representation of patients from minority backgrounds in the developmental studies of Values-HM,^4,34–36 further work evaluating the feasibility, acceptability, and validity of using Values-HM in these populations would be valuable. We are assessing data regarding the engagement of minority populations in the follow-up RCT (NCT06296368).

Conclusion

Validated measures to assess the values of patients with blood cancers are needed to better align treatment decisions to what matters most to patients. Values-HM is a values elicitation measure developed specifically to support clinical decision making for older adults with blood cancers. In this initial validation study involving mostly older White patients, the measure was acceptable to patients and appeared valid and reliable to capture patient values to inform treatment decision making. Further work is needed to develop optimal implementation strategies for values elicitation measures such as Values-HM and to optimize the use patient preference data in different populations to improve clinical care.

Supplemental Material

sj-docx-1-mpp-10.1177_23814683261440935 – Supplemental material for Developing a Novel Values Elicitation Measure for Clinical Use in Older Adults with Hematologic Malignancies: Values-HM

Supplemental material, sj-docx-1-mpp-10.1177_23814683261440935 for Developing a Novel Values Elicitation Measure for Clinical Use in Older Adults with Hematologic Malignancies: Values-HM by Daniel R. Richardson, Allison M. Deal, Carl J. Mhina, Matthew Washko, Amy C. Cole, Norah L. Crossnohere, Kah Poh Loh, Ethan Basch, Stephanie B. Wheeler, William A. Wood, John F. P. Bridges, Thomas W. LeBlanc and Antonia V. Bennett in MDM Policy & Practice

Supplemental Material

sj-pdf-2-mpp-10.1177_23814683261440935 – Supplemental material for Developing a Novel Values Elicitation Measure for Clinical Use in Older Adults with Hematologic Malignancies: Values-HM

Supplemental material, sj-pdf-2-mpp-10.1177_23814683261440935 for Developing a Novel Values Elicitation Measure for Clinical Use in Older Adults with Hematologic Malignancies: Values-HM by Daniel R. Richardson, Allison M. Deal, Carl J. Mhina, Matthew Washko, Amy C. Cole, Norah L. Crossnohere, Kah Poh Loh, Ethan Basch, Stephanie B. Wheeler, William A. Wood, John F. P. Bridges, Thomas W. LeBlanc and Antonia V. Bennett in MDM Policy & Practice

Footnotes

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article. NLC reports consulting or advisory roles at Boehringer Ingelheim and travel, accommodations, and expenses from Bausch and Lomb. EB reports consulting or advisory roles at AstraZeneca, N-Power Medicine, Navigating Cancer, Resilience Care, Savor Health LLC, Thyme Care, and Verily and other relationships with the American Society of Clinical Oncology, Centers for Medicare and Medicaid Services, National Cancer Institute, and the Patient-Centered Outcomes Research Institute (PCORI). SBW has received research funding from AstraZeneca and Pfizer and travel, accommodations, and expenses from Pfizer. WAW reports stock and other ownership interests in Koneska Health; consulting or advisory roles at ASH Research Collaborative, Koneska Health, and Teledoc Health; research funding from Pfizer, Genetech. JFPB reports consulting or advisory roles at Boehringer Ingelheim; and travel, accommodations, and expenses from Boehringer Ingelheim. TL reports stock and other ownership interests at Dosentrx and Thyme Care; honoraria from Genentech and Lilly; consulting or advisory roles at Abbvie/Genentech, Agios/Servier, Apellis Pharmaceuticals, Astellas Pharma, Bristol-Myers Squibb/Celgene, Gilead Sciences, GlaxoSmithKline, Menarini Group, Novartis, and Pfizer; Speakers’ Bureau at Abbvie, Bristol-Myers Squibb/Celgene, GlaxoSmithKline, Incyte, Menarini, Rigel, and SERVIER; research funding from Abbvie, AstraZeneca, Bristol Myers Squibb, GlaxoSmithKline, Jazz Pharmaceuticals, Novartis; royalties from UpToDate Inc.; travel, accommodations, and expenses from Abbvie/Genentech, Astellas Pharma, Bristol-Myers Squibb/Celgene, Incyte, Menarini, Rigel, and SERVIER. All other authors have no conflicts to disclose. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Palliative Care Research Cooperative group (to DRR) and the Carolina Medical Student Research Program (to MFW). Financial support for this study was provided in part by a grant from the Palliative Care Research Cooperative Group through a larger grant from the National Institute of Nursing Research. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Author Contributions

DRR, NLC, WAW, JFPB, TWL, and AVB, designed the research; DRR, CJM, MW, and ACC performed the research; DRR, CJM, MFW, and ACC collected the data; DRR, AMD, CJM, MFW, ACC, NLC, KPL, EB, SBW, WAW, JFPB, and AVB analyzed and interpreted the data; DRR, AMD, CJM, and MFW performed the statistical analysis; and DRR, AMD, NLC, JFPB, TWL, and AVB wrote or edited the manuscript.

Ethical Considerations

The study was approved by the Institutional Review Board (#21-2282) at The University of North Carolina at Chapel Hill. This trial was registered at as #NCT05061095.

Consent to Participate

All participants provided written informed consent in compliance with regulatory requirements.

Consent for Publication

Not applicable.

ORCID iDs

Daniel R. Richardson

Amy C. Cole

Norah L. Crossnohere

Ethan Basch

Data-Sharing Statement

Data can be requested via e-mail to the corresponding author.

Clinical Trial Data Sharing

De-identified participant data will be transferred to the Palliative Care Research Cooperative Group (PCRC) repository for dissemination once published.

References

SEER. SEER cancer stat facts. Available from:http://seer.cancer.gov/statfacts/index.html. [Accessed 27 June, 2022].

Kanapuru

Singh

Kwitkowski

Blumenthal

Farrell

Pazdur

Older adults in hematologic malignancy trials: representation, barriers to participation and strategies for addressing underrepresentation. Blood Rev. 2020;43:100670. DOI:10.1016/j.blre.2020.100670

Sochacka-Ćwikła

Mączyński

Regiec

FDA-approved drugs for hematological malignancies—the last decade review. Cancers. 2022;14(1):1. DOI:10.3390/cancers14010087

Richardson

Crossnohere

Seo

, et al. Age at diagnosis and patient preferences for treatment outcomes in AML: a discrete choice experiment to explore meaningful benefits. Cancer Epidemiol Biomarkers Prev. 2020;29(5):942–8. DOI:10.1158/1055-9965.EPI-19-1277

Johnson

Quek

Bailey

, et al. Treatment preferences among patients with diffuse large B-cell lymphoma: a survey across western Europe and the United States. J Manag Care Spec Pharm. 2022;28(10):S34.

Tervonen

Duenas

Collacott

, et al. Current health state affected patient preferences more than disease status: a discrete choice experiment in multiple myeloma. Value Health. 2023;26(6):909–17. DOI:10.1016/j.jval.2023.01.016

Seghers

PAL (Nelleke)

Wiersma

Festen

, et al. Patient preferences for treatment outcomes in oncology with a focus on the older patient—a systematic review. Cancers. 2022;14(5):1147. DOI:10.3390/cancers14051147

Richardson

Washko

Beke

, et al. Patient preference studies in hematologic malignancies: a systematic review. Blood. 2023;142(suppl 1):7241. DOI:10.1182/blood-2023-178754

DuMontier

Loh

Soto-Perez-de-Celis

Dale

Decision making in older adults with cancer. J Clin Oncol. 2021;39(19):2164–74. DOI:10.1200/JCO.21.00165

10.

Elwyn

Frosch

Thomson

, et al. Shared decision making: a model for clinical practice. J Gen Intern Med. 2012;27(10):1361–7. DOI:10.1007/s11606-012-2077-6

11.

Sekeres

Guyatt

Abel

, et al. American Society of Hematology 2020 guidelines for treating newly diagnosed acute myeloid leukemia in older adults. Blood Adv. 2020;4(15):3528–49. DOI:10.1182/bloodadvances.2020001920

12.

Carey

Anderson

Sanson-Fisher

Lynagh

Paul

Tzelepis

How well are we meeting haematological cancer survivors’ preferences for involvement in treatment decision making?

Patient Educ Couns. 2012;88(1):87–92. DOI:10.1016/j.pec.2011.12.014

13.

Williams

Miller-Sonet

Nipp

Kamal

Love

Rocque

GB.

Importance of quality-of-life priorities and preferences surrounding treatment decision making in patients with cancer and oncology clinicians. Cancer. 2020;126(15):3534–41. DOI:10.1002/cncr.32961

14.

LeBlanc

Fish

Bloom

, et al. Patient experiences of acute myeloid leukemia: a qualitative study about diagnosis, illness understanding, and treatment decision-making. Psychooncology. 2017;26(12):2063–8. DOI:10.1002/pon.4309

15.

Sorror

Storer

Fathi

, et al. Multisite 11-year experience of less-intensive vs intensive therapies in acute myeloid leukemia. Blood. 2021;138(5):387–400. DOI:10.1182/blood.2020008812

16.

Rood

JAJ

Nauta

Witte

, et al. Shared decision-making and providing information among newly diagnosed patients with hematological malignancies and their informal caregivers: not “one-size-fits-all.” Psychooncology. 2017;26(12):2040–7. DOI:10.1002/pon.4414

17.

Bories

Lamy

Simand

, et al. Physician uncertainty aversion impacts medical decision making for older patients with acute myeloid leukemia: results of a national survey. Haematologica. 2018;103(12):2040–8. DOI:10.3324/haematol.2018.192468

18.

Loh

Abdallah

Kadambi

, et al. Treatment decision-making in acute myeloid leukemia: a qualitative study of older adults and community oncologists. Leuk Lymphoma. 2021;62(2):387–98. DOI:10.1080/10428194.2020.1832662

19.

Bröckelmann

McMullen

Wilson

, et al. Patient and physician preferences for first-line treatment of classical Hodgkin lymphoma in Germany, France and the United Kingdom. Br J Haematol. 2019;184(2):202–14. DOI:10.1111/bjh.15566

20.

Landfeldt

Eriksson

Ireland

, et al. Patient, physician, and general population preferences for treatment characteristics in relapsed or refractory chronic lymphocytic leukemia: a conjoint analysis. Leuk Res. 2016;40:17–23. DOI:10.1016/j.leukres.2015.11.006

21.

Mühlbacher

Nübling

Analysis of physicians’ perspectives versus patients’ preferences: direct assessment and discrete choice experiments in the therapy of multiple myeloma. Eur J Health Econ. 2011;12(3):193–203. DOI:10.1007/s10198-010-0218-6

22.

Button

Cardona

Huntley

, et al. Clinicians’ understanding of preferences and values of people with hematological malignancies at the end of life: concurrent surveys. J Palliat Med. 2022;25(9):1386–97. DOI:10.1089/jpm.2021.0490

23.

Levit

Balogh

Nass

Ganz

Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington (DC): National Academies Press; 2013.

24.

Mack

Weeks

Wright

Block

Prigerson

HG.

End-of-life discussions, goal attainment, and distress at the end of life: predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010;28(7):1203–8. DOI:10.1200/jco.2009.25.4672

25.

Montori

Kunneman

Brito

JP.

Shared decision making and improving health care: the answer is not in. JAMA. 2017;318(7):617–8. DOI:10.1001/jama.2017.10168

26.

Witteman

Ndjaboue

Vaisson

, et al. Clarifying values: an updated and expanded systematic review and meta-analysis. Med Decis Making. 2021;41(7):801–20. DOI:10.1177/0272989X211037946

27.

Hauber

González

Groothuis-Oudshoorn

CGM

, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19(4):300–15. DOI:10.1016/j.jval.2016.04.004

28.

Bridges

JFP

Hauber

Marshall

, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011;14(4):403–13. DOI:10.1016/j.jval.2010.11.013

29.

Janssen

Marshall

Hauber

Bridges

JFP

. Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability? Expert Rev Pharmacoecon Outcomes Res. 2017;17(6):531–42. DOI:10.1080/14737167.2017.1389648

30.

Hunter

O’Callaghan

Califf

RM.

Engaging patients across the spectrum of medical product development: view from the US food and drug administration. JAMA. 2015;314(23):2499. DOI:10.1001/jama.2015.15818

31.

Rocque

Miller-Sonnet

Balch

, et al. Engaging multidisciplinary stakeholders to drive shared decision-making in oncology. J Palliat Care. 2019;34(1):29–31. DOI:10.1177/0825859718810723

32.

Bonamici S. H.R.34, 114th Congress (2015-2016): 21st Century Cures Act. 2016. Available from: https://www.congress.gov/bill/114th-congress/house-bill/34. [Accessed 27 May, 2020].

33.

Wicker

RF.

S. 1597, 114th Congress (2015-2016): Patient-Focused Impact Assessment Act of 2016. Available from: https://www.congress.gov/bill/114th-congress/senate-bill/1597. [Accessed 27 May, 2020].

34.

Seo

Smith

Estey

Voyard

Donoghue

Bridges

JFP

. Developing an instrument to assess patient preferences for benefits and risks of treating acute myeloid leukemia to promote patient-focused drug development. Curr Med Res Opin. 2018;34(12):2031–9. DOI:10.1080/03007995.2018.1456414

35.

Bridges

Oakes

Reinhart

Voyard

O’Donoghue

Developing and piloting an instrument to prioritize the worries of patients with acute myeloid leukemia. Patient Prefer Adherence. 2018;12:647–55. DOI:10.2147/PPA.S151752

36.

Richardson

Oakes

Crossnohere

, et al. Prioritizing the worries of AML patients: quantifying patient experience using best-worst scaling. Psychooncology. 2021;30(7):1104–11. DOI:10.1002/pon.5652

37.

Cole

Khasawneh

Adapa

Mazur

Richardson

DR.

Development of an electronic healthcare tool to elicit patient preferences in older adults diagnosed with hematologic malignancies. (Lecture Notes in Computer Science). In: Gao

Zhou

, eds. Human Aspects of IT for the Aged Population. Technology in Everyday Living. HCII 2022: Human Aspects of IT for the Aged Population. Cham (Switzerland): Springer International; 2022. p 210–28. DOI:10.1007/978-3-031-05654-3_14

38.

Kwong

Cole

Khasawneh

, et al. Evaluation of data visualizations for an electronic patient preferences tool for older adults diagnosed with hematologic malignancies. Paper presented at: 2022 Workshop on Visual Analytics in Healthcare (VAHC); Washington, DC. IEEE; 2022. p 1–6. DOI:10.1109/VAHC57815.2022.10108523

39.

Cole

Kwong

Mhina

King

Mazur

Richardson

DR.

Perceptions on tailored messages from a values clarification tool: a mixed-methods study of older adults with cancer. Front Commun. 2024;9:1347310. DOI:10.3389/fcomm.2024.1347310

40.

Flynn

Louviere

Peters

Coast

Best–worst scaling: what it can do for health care research and how to do it. J Health Econ. 2007;26(1):171–89. DOI:10.1016/j.jhealeco.2006.04.002

41.

Hollin

Paskett

Schuster

ALR

Crossnohere

Bridges

JFP

. Best–worst scaling and the prioritization of objects in health: a systematic review. Pharmacoeconomics. 2022;40(9):883–99. DOI:10.1007/s40273-022-01167-1

42.

US Department of Health and Human Services Food and Drug Administration. Patient preference information – voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling: guidance for industry, food and drug administration staff, and other stakeholders. 2016. Available from: https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm446680.pdf. [Accessed 18 March, 2019].

43.

Patrick

Burke

Gwaltney

, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1—eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–77. DOI:10.1016/j.jval.2011.06.014

44.

Patrick

Burke

Gwaltney

, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2—assessing respondent understanding. Value Health. 2011;14(8):978–88. DOI:10.1016/j.jval.2011.06.013

45.

Watson

Sanapala

Cortes

, et al. Adapting a patient-centered communication tool for older patients with acute myeloid leukemia and their oncologist. Blood Adv. 2022;6(21):5707–10. DOI:10.1182/bloodadvances.2022008041

46.

Wittenberg

Bharel

Bridges

JFP

Ward

Weinreb

Using best-worst scaling to understand patient priorities: a case example of papanicolaou tests for homeless women. Ann Fam Med. 2016;14(4):359–64. DOI:10.1370/afm.1937

47.

Nissim

Zimmermann

Minden

, et al. Abducted by the illness: a qualitative study of traumatic stress in individuals with acute leukemia. Leuk Res. 2013;37(5):496–502. DOI:10.1016/j.leukres.2012.12.007

48.

Richardson

Mhina

Teal

, et al. Experiences of treatment decision-making among older newly diagnosed adults with acute myeloid leukemia: a qualitative descriptive study. Support Care Cancer. 2024;32(3):197. DOI:10.1007/s00520-024-08397-3

49.

Bridges

JFP

Tsai

Janssen

Crossnohere

Fischer

Peay

. How do members of the Duchenne and Becker muscular dystrophy community perceive a discrete-choice experiment incorporating uncertain treatment benefit? An application of research as an event. Patient. 2019;12(2):247–57. DOI:10.1007/s40271-018-0330-8

50.

Janssen

Hauber

Bridges

JFP

. Conducting a discrete-choice experiment study following recommendations for good research practices: an application for eliciting patient preferences for diabetes treatments. Value Health. 2018;21(1):59–68. DOI:10.1016/j.jval.2017.07.001

51.

Crossnohere

Weiss

Hyman

Bridges

JFP

. Patient preferences for health information technologies: a systematic review. J Hosp Manag Health Policy. 2021;5(0):0. DOI:10.21037/jhmhp-20-105

52.

Proctor

Silmere

Raghavan

, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. DOI:10.1007/s10488-010-0319-7

53.

Rothman

Burke

Erickson

Leidy

Patrick

Petrie

CD.

Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PRO task force report. Value Health. 2009;12(8):1075–83. DOI:10.1111/j.1524-4733.2009.00603.x

54.

Shrout

Fleiss

JL.

Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8. DOI:10.1037//0033-2909.86.2.420

55.

Richardson

Cole

Mhina

, et al. Creating a tool for measuring preferences for treating blood cancers in older adults using implementation science. Blood. 2025;146(suppl 1):2644. DOI:10.1182/blood-2025-2644

56.

Scheepers

ERM

Vondeling

Thielen

van der Griend

Stauder

Hamaker

. Geriatric assessment in older patients with a hematologic malignancy: a systematic review. Haematologica. 2020;105(6):1484–93. DOI:10.3324/haematol.2019.245803

57.

Mohile

Dale

Somerfield

, et al. Practical assessment and management of vulnerabilities in older patients receiving chemotherapy: ASCO guideline for geriatric oncology. J Clin Oncol. 2018;36(22):2326–47. DOI:10.1200/JCO.2018.78.8687

58.

Dale

Klepin

Williams

, et al. Practical assessment and management of vulnerabilities in older patients receiving systemic cancer therapy: ASCO guideline update. J Clin Oncol. 2023;41(26):4293–312. DOI:10.1200/JCO.23.00933

59.

Brown

Brummel-Smith

Burgess

, et al. National Institutes of Health consensus development conference statement: geriatric assessment methods for clinical decision-making. J Am Geriatr Soc. 1988;36(4):342–7. DOI:10.1111/j.1532-5415.1988.tb02362.x

60.

Soto-Perez-de-Celis

Dale

Katheria

, et al. Outcome prioritization and preferences among older adults with cancer starting chemotherapy in a randomized clinical trial. Cancer. 2024;130(17):3000–10. DOI:10.1002/cncr.35333

61.

LoCastro

Sanapala

Wang

, et al. Patient-centered communication tool for older patients with acute myeloid leukemia, their caregivers, and oncologists: a single-arm pilot study. Cancer Med. 2023;12(7):8581–93. DOI:10.1002/cam4.5547

62.

Forsat

Palmowski

Boers

Buttgereit

Recruitment and retention of older people in clinical research: a systematic literature review. J Am Geriatr Soc. 2020;68(12):2955–63. DOI:10.1111/jgs.16875

63.

Basch

Schrag

Henson

, et al. Effect of electronic symptom monitoring on patient-reported outcomes among patients with metastatic cancer: a randomized clinical trial. JAMA. 2022;327(24):2413–22. DOI:10.1001/jama.2022.9265

64.

Basch

Schrag

Jansen

, et al. Symptom monitoring with electronic patient-reported outcomes during cancer treatment: final results of the PRO-TECT cluster-randomized trial. Nat Med. 2025;31(4):1225–32. DOI:10.1038/s41591-025-03507-y

65.

Basch

Barbera

Kerrigan

Velikova

Implementation of patient-reported outcomes in routine medical care. Am Soc Clin Oncol Educ Book. 2018;38:122–34. DOI:10.1200/EDBK_200383

66.

Crossnohere

Anderson

Baumhauer

, et al. A framework for implementing patient-reported outcomes in clinical care: the PROTEUS-practice guide. Nat Med. 2024;30(6):1519–20. DOI:10.1038/s41591-024-02909-8

67.

Loh

QMR

Mohile

, et al. Protocol of a decisional intervention for older adults with newly diagnosed acute myeloid leukemia and their caregivers: UR-GOAL 3. J Geriatr Oncol. 2025;16(2):102187. DOI:10.1016/j.jgo.2025.102187

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.03 MB

0.00 MB

0.13 MB