Feasibility and psychometric quality of smartphone administered cognitive ecological momentary assessments in women with metastatic breast cancer

Abstract

Objective

Metastatic breast cancer (MBC) is associated with burdensome side effects, including cognitive changes that require ongoing monitoring. Cognitive ecological momentary assessments (EMAs) allow for assessment of individual cognitive functioning in natural environments and can be administered via smartphones. Accordingly, we sought to establish the feasibility, reliability, and validity of a commercially available cognitive EMA platform.

Methods

Using a prospective design, clinical cognitive and psychosocial assessments (cognitive batteries; patient reported outcomes) were collected at baseline, followed by a 28-day daily EMA protocol that included self-ratings for symptoms and mobile cognitive tests (memory, executive functioning, working memory, processing speed). Satisfaction and feedback questions were included in follow-up data collection. Feasibility data were analyzed using mixed descriptive methods. Test-retest reliability was examined using intraclass correlation coefficients (ICCs) for each EMA, and Pearson's correlation were used to evaluate convergent validity between cognitive EMAs and baseline clinical cognitive and psychosocial variables.

Results

Fifty-one women with MBC (n = 51) completed this EMA study. High satisfaction (median 90%), low burden (median 19%), high adherence rates (mean 94%), and 100% retention rate were observed. ICCs for cognitive tests of working memory, executive function, and processing speed were robust (>0.90) and ICC for memory tests acceptable (>0.66). Other correlational findings indicated strong convergent validity for all cognitive and psychosocial EMAs.

Conclusion

Cognitive EMA monitoring for 28 days is feasible and acceptable in women with MBC, with specific cognitive EMAs (mobile cognitive tests; cognitive function self-ratings) demonstrating strong reliability and validity.

Keywords

Feasibility reliability convergent validity metastatic breast cancer ecological momentary assessment mobile cognitive testing

Introduction

It is estimated that 168,000 people are living with metastatic breast cancer (MBC) in the United States.¹ Almost 30% of those diagnosed and treated with early stage breast cancer will experience recurrent metastatic disease later in their lives,^2,3 and 6% to 10% of all new breast cancer diagnoses are metastatic at diagnosis.^1,4 Treatment for MBC is complex, longitudinal, and requires ongoing monitoring. Treatments typically consist of chemo- or immunotherapy, endocrine therapy, and targeted agents depending on the cancer subtype and actionable mutations. The treatment goal for MBC is to optimize quality of life (QoL) and prolong survival while minimizing the side effects of living with incurable cancer. Patients with MBC can live for years with advanced disease and the side effects of treatment, which makes symptom control and QoL optimization of paramount importance.⁵

Treatment for MBC is associated with burdensome physical, psychological, and spiritual side effects that negatively impact QoL and interfere with participation in everyday life.^5–10 Notably, women with MBC describe debilitating neuropsychological symptoms including fatigue and “chemo-brain” and that these symptoms, in addition to other side effects of MBC treatment, decrease their functional status and limit how they can engage with their everyday activities and their ability to work.¹⁰ Cancer-related cognitive impairments (CRCI) are commonly found in early-stage breast cancer and typically present as deficits in attention, executive functioning, processing speed, and memory.^11–16 Almost all CRCI research to date has focused on early-stage breast cancer, methodologically excluding those with advanced or metastatic disease. We recently characterized CRCI in women with MBC using traditional subjective (i.e. a cognitive patient reported outcome [PRO]) and objective assessments (i.e. remote standardized cognitive test battery) in a cross-sectional analysis and found that the majority of our sample (n = 52) had clinically significant CRCI (primarily on PROs and tests of executive functioning and memory) negatively affected QoL and daily functioning.¹⁷ Considering that cognitive functioning in the context of everyday life is highly variable and subject to contextual demands (e.g. demanding, stressful, or distracting situations) and/or state dependent factors (e.g. sleep quality, mood),¹⁸ and that those with MBC may be especially vulnerable to CRCI due to complex treatment regimens, changes to treatment plans, potential for brain metastases, and high psychological burden of disease, alternative methods for assess CRCI in this population should be applied.

QoL, symptoms, and functional status are often assessed using retrospective PRO measures or functional performance tests^19,20 that can be influenced by recall bias, and may not adequately represent dynamic and evolving symptoms, function, and QoL of MBC patients across time. Ecological momentary assessments (EMAs) offer one way to capture individuals’ thoughts, feelings, behaviors, and symptoms in natural environments and can be easily administered via their smartphones. EMA methods (via paper/pencil diaries and digital tools) have been applied to oncology populations both during and after treatment, and have been used to assess primary and secondary physical, psychosocial, and behavioral outcomes (e.g. fatigue, sleep, exercise, pain, mindfulness, self-efficacy, expectations, mood, distress, and QoL).²⁰ Digital EMA's have been reported to be feasible and valuable in studies conducted with persons living with advanced cancer to study stress,²¹ loss and life engagement,²² pain, anxiety, depression and medication use.²³

Use of cognitive EMAs, including ecological mobile cognitive tests, is on the rise and evidence supports use of cognitive EMAs in cognitively vulnerable populations such as persons with mild cognitive impairment, serious mental illness, and mood disorders.^24–28 Cognitive EMAs have only recently been applied to study CRCI in early stage (i.e. nonmetastatic) breast cancer patients²⁹ and survivors,^30,31 with preliminary data showing feasibility, acceptability this methodology,^29–31 strong reliability and validity for objective cognitive EMAs, and that EMAs may be more sensitive to CRCI than traditional assessments.³¹ However, many questions remain regarding how specific cognitive EMAs reflect traditional cognitive measures, what protocols may be most appropriate for various cancer populations, and how interactions among context, behavior, and symptoms explain cognitive variability in cancer survivors with CRCI. To our knowledge, EMAs have not yet been used to study cognitive functioning and associated psychosocial symptoms in persons with MBC. Cognitive EMAs could provide a more accessible and less burdensome means for assessing CRCI in persons with MBC or other cancer patients/survivors and provide a rigorous way to assess outcomes of interventions aimed to improve CRCI, especially modern precision health-type intervention designs such as just-in-time adaptive interventions or ecological momentary interventions. There is also potential to add cognitive EMAs to improve precision in pharmacological trials for MBC.

To address these gaps in knowledge, this study establishes the feasibility, reliability, and validity of a commercially available cognitive EMA platform to assess cognitive functioning and related psychosocial functioning in women living with MBC and describes the psychometric characteristics (practice and fatigue effects, reliability, and convergent validity metrics) of the specific EMAs.

Methods

All study-related procedures were conducted in accordance with the Declaration of Helsinki and approved by the University of Texas at Austin Institutional Review Board (STUDY00002393). All participants provided informed consent.

Sample description

A convenience sample of women who had been diagnosed with MBC participated in the study. To be eligible, participants were required to be 21 years of age or older; physically (i.e. able to use a computer and smartphone; and a Karnofsky Performance Scale result of >=70) and cognitively (MiniMoca Telephone screening result of >=11) able to participate^32,33; had access to a computer with internet and a smartphone with cellular or wireless internet connectivity; proficient in reading, speaking, and writing in English; and resided in the United States. Those with major sensory deficits (e.g. deafness or blindness), a prior history of cancer with systemic treatment for cancer (other than breast cancer), or neurological or cognitive comorbidities (e.g. self-reported dementia, substance abuse, unmanaged psychiatric conditions) were excluded.

Data collection procedures

All study procedures were conducted remotely from the University of Texas at Austin. Study information was distributed via social media communications, including administrators of METAvivor^©—a national social media peer support group for those affected by MBC. People interested to participate contacted the study team and completed a pre-screening and enrollment telephone appointment.

Data were collected and managed primarily using Research Electronic Data Capture (REDCap) hosted at the University of Texas at Austin.^34,35 Baseline data collection included an intake survey (i.e. written consent, sociodemographic data, clinical variables, cognitive variables, psychosocial variables) and remote administration of a cognitive test battery via BrainCheck (BrainCheck Inc., Austin, TX, USA; a digital cognitive testing and care provision platform). Participants then started their cognitive EMA protocols, which were administered daily for 4 weeks (28 assessments in total per participant). After the cognitive EMA protocol was completed, participants were sent follow-up data collection surveys (cognitive and psychosocial variables, feasibility, satisfaction, utility questions) and a second cognitive BrainCheck testing battery. Participants were given a $78 gift card, a report of their NeuroUX performance across the study (generated by NeuroUX and sent to our study team; see Supplemental Figure S1) and an educational handout “Optimizing Cognitive Function During and After Breast Cancer” (available on our lab website³⁶) as tokens of appreciation for participating.

Measures

Sociodemographic and clinical variables (baseline)

Sociodemographic characteristics (e.g. age, education, race, ethnicity, marital status, children/dependents, income, employment), health history (e.g. co-morbidities, menstrual history, current medications), and cancer history (e.g. breast cancer type/stage, cancer treatment details, end date of chemotherapy) were collected via survey to describe the sample and for use in convergent validity analyses (i.e. age and education).

Clinical cognitive function (baseline and follow-up)

The Functional Assessment of Cancer Treatment Cognitive Function, Perceived Cognitive Impairments 20 item subscale (FACT-Cog PCI, v3.0) was used to assess self-reported cognitive function in the previous 7 days.³⁷ A computerized battery of standardized neuropsychological tests via the BrainCheck platform was administered remotely to assess cognitive performance, and included: the Trail Making Tests for attention and processing speed, the Digit Symbol Substitution Test and the Stroop Test for executive functioning, and the Recall Test (list learning) for immediate and delayed verbal memory.³⁸ Alternative forms of the tests are used with every administration. Participants were sent detailed instructions for accessing and using the platform, including an introductory video, and study staff were available to assist via video conference as needed. Raw test scores from baseline were used in the psychometric analyses (median time between clicks for Trails A and B, Stroop median reaction times, Digit Symbol median reaction time between clicks for all trials and number correct per second, number of correct responses for immediate and delayed memory, and the raw combined score).

Psychosocial variables (baseline and follow-up)

Depressive symptoms, anxiety, and fatigue correlate with self-reported cognitive function in people affected by cancer,^14,39–42 thus we examined convergent validity of the EMAs by using the Patient-Reported Outcomes Measurement Information System (PROMIS) scales—Emotional Distress—Anxiety Short Form 8a (“PROMIS Anxiety”), Emotional Distress—Depression Short Form 8a (“PROMIS Depression”), PROMIS Fatigue Short Form 8a (“PROMIS Fatigue”),⁴³ PROMIS Sleep Disturbance 4a (“PROMIS Sleep”),⁴⁴ PROMIS Satisfaction with Social Roles & Activities Short Form 8a (“PROMIS Social”) and the UCLA Loneliness 3-item scale.⁴⁵ All PROMIS Scales were transformed to T-scores per their scoring manuals and used in the convergent validity analyses. The UCLA loneliness scale was summed to obtain a raw score (range: 0–9; higher scores indicating more loneliness). Completion of follow-up cognitive and psychosocial data was used to determine study retention rates, but not used in psychometric analyses.

Cognitive EMA protocol

EMA surveys and gamified mobile cognitive tests were administered via the NeuroUX platform (NeuroUX, Inc.) See Figure 1. NeuroUX was chosen to deliver the EMA protocol because this commercially available platform allows for integrated assessment of both traditional EMA surveys (e.g. assessments of mood, context, symptoms, behaviors) and cognitive tests. NeuroUX is a beautifully designed, gamified, and highly customizable platform that significantly improves participant engagement and adherence. The platform offers customization of surveys and has a menu of validated cognitive tests for various cognitive domains including memory, psychomotor speed, executive function, social cognition, and attention. The cognitive EMA assessments were pushed out via weblinks to participants via text messaging. This eliminated the need for participants to have to install and manage a native app. Weblinks were sent to participants at pseudorandom times between participant's preferred times in the morning and evening, and varied across morning, midday, and evening throughout the protocol. If participants requested to adjust their “start” and “stop” times for receiving EMA texts during their protocol, our study team made the requested adjustments. Once participants received their text messages, they were given 6 hours to complete the assessment session. Reminder texts were automatically sent if sessions were not completed after 3 hours and then again with 1 hour remaining. In addition to reminder text messages, the study coordinator was available as needed to assist participants. This assistance included providing more instructions for how to complete the specific cognitive tests and practice links for tests if participants requested them

Figure 1.

Mobile cognitive tests used in cognitive ecological momentary assessment protocol. (A) N-back; (B) CopyKat; (C) Color Trick; (D) Hand Swype; (E) Matching Pair; (F) Quick Tap 1; (G) Memory Matrix; and (H) Memory List-12.

Each assessment collected single-item Likert-type scale ratings for depressed/sadness, anxiety, fatigue, loneliness, cognitive symptoms, cognitive abilities, sleep quality (last night), social engagement, relational quality, social satisfaction, and overall wellbeing (see Supplemental Table S1 for exact items). Each item assessed how bad or good each symptom is ‘right now’. Response options could range from ‘Not at all’ (0) to ‘Extremely’ (7), consistent with our previous cognitive EMA studies.^26,46 After symptoms were assessed, 4 mobile cognitive tests were administered, each 2-minute long, in the domains of working memory, executive functioning, processing speed, and verbal/spatial memory. These domains were chosen as they are the most common impacted by CRCI.¹¹ Two tests were issued for each cognitive domain and balanced throughout the protocol (14 times each, Table 1).

Table 1.

Cognitive ecological momentary assessment protocol across 28 assessments.

Study day	Mobile cognitive test
	EMA self-report items	N-back	Copykat	Color trick	Matching pair	Hand Swype	Quick tap	Memory list	Memory matrix
1	X	X	—	X¹	X	—	—	X³	—
2	X	—	X	—	—	X	X	—	X
3	X	X	—	X²	—	—	X	X³	—
4	X	X	—	—	X	X	—	—	X
5	X	—	X	X¹	X	—	—	X⁴	—
6	X	—	X	—	—	X	X	X³	—
7	X	X	—	—	X	X	—	—	X
8	X	—	X	X²	—	—	X	—	X
9	X	X	—	X¹	—	—	X	—	X
10	X	X	—	X²	X	—	—	X³	—
11	X	X	—	—	X	X	—	X⁴	—
12	X	—	X	—	X	X	—	—	X
13	X	—	X	X¹	—	—	X	X³	—
14	X	X	—	—	X	X	—	—	X
15	X	—	X	X²	—	—	X	—	X
16	X	—	X	X¹	—	—	X	X⁴	—
17	X	—	X	—	X	X	—	—	X
18	X	X	—	—	—	X	X	—	X
19	X	—	X	—	X	X	—	X³	—
20	X	X	—	X²	—	—	X	X⁴	—
21	X	X	—	—	—	X	X	—	X
22	X	—	X	X¹	X	—	—	X³	—
23	X	—	X	X²	X	—	—	—	X
24	X	X	—	—	—	X	X	—	X
25	X	—	X	X¹	X	—	—	X³	—
26	X	X	—	X²	—	—	X	—	X
27	X	X	—	—	—	X	X	X⁴	—
28	X	—	X	—	X	X	—	X³	—
TOTAL	28	14	14	14⁵	14	14	14	14⁶	14

1: Color trick color-to-meaning version.

2: Color trick yes/no version.

3: Memory List 12 words.

4: Memory List 18 words.

5: 7 color to meaning version, 7 yes/no version.

6: 9 administrations of the 12-word list, 5 administrations of the 18 word list.

For working memory, we used the N-Back with a 2-back design (12 trials each administration) and the CopyKat tests. The N-Back is a common task that requires the participant to remember if a certain number was displayed 2 trials back (total score was used in the analyses). Participants are initially presented with a 2 × 2 matrix of colored tiles in a fixed position, which briefly light up in a random order. Participants are asked to replicate the pattern by pressing on the colored tiles in the correct order. Another feature of the task is that each color plays a distinct tone when highlighted, providing the phone's volume is turned on. The time to complete is variable, but 3 trials take approximately 2 to 3 min (total score was used in the analyses).

For executive functioning, the Color Trick and Hand Swype tests were used. The Color Trick test asks the participant to match the color of the word with its meaning, with 15 trials each test (total score and median reaction time were used in analyses). The Hand Swype test asks the participant to swipe in the direction of a hand symbol or in the direction that matches the way the symbols are moving across the screen. Hand Swype is a time-based task (1 minute every administration; total score and median reaction time were used in analyses).

For processing speed, we used the Matching Pair and Quick Tap 1 tests. For Matching Pair, the participant was asked to quickly identify the matching pair of tiles out of 6 or more tiles. Matching Pair is a time-based task for 90 seconds. The grid size increases based on correct responses and the max grid size is 4 × 4 tiles (total score was used in the analyses). For Quick Tap 1, the participant is asked to wait and tap the symbol when it is displayed (12 trials/administration; median reaction time were used in the analyses).

For memory we used the Variable Difficulty List Memory Test (VLMT) and Memory Matrix tests. For the VLMT, participants were provided with a list of 12 or 18 random (i.e. not semantically related) words and given 30 seconds to memorize the list. They were then asked yes/no questions to determine if words were on the list or not, immediately following the memorization time (total words correct was used for analyses). For Memory Matrix, patterns of tiles are quickly displayed to the participant, then they are asked to indicate the pattern that was displayed by touching the tiles that were in the pattern. This test gets progressively harder if responses are correct: the first trial starts with a 2 × 2 grid size, and the size can go up to 7 × 7 tiles until the participant gives three incorrect responses (total score was used in the analyses).

Feasibility and acceptability outcomes

Accrual rates (% of those who contacted our team who were eligible that enrolled), adherence rates (number of partial/completed cognitive EMAs across the 28 administrations), and retention rates (proportion of participants who completed baseline and follow-up data collection) were calculated. To assess acceptability of cognitive EMAs, the following questions were asked:

“Overall, how satisfied are you with your experience participating in this study?” Responses ranged: 0–100 (0 = unsatisfied, 50 = neutral, and 100 = very satisfied).

“How challenging was it for you to answer the survey questions and do the brain games on your smartphone during the protocol?” Responses ranged: 0–100 (0 = not challenging, 50 = neutral, and 100 = very challenging).

“Would you be open to incorporating smartphone based cognitive tasks as part of your ongoing care to monitor your cognitive functioning?” Responses were ‘Yes’, or ‘No’.

“If you have any suggestions or feedback about the study, please provide them here”, with an open text box for responses.

Data analysis plan

Preprocessing

Consistent with previously published methods^24,28,47,48 NeuroUX data were cleaned to remove instances of suspected low effort/engagement. N-Back scores of 0 were excluded due to suspected low effort/engagement. CopyKat scores less than 4 were excluded due to suspected low effort/engagement. Color Trick total scores < 3 and reaction times > 10 seconds were excluded due to suspected low effort/engagement. Matching Pair scores less than 100 were excluded due to suspected low effort/engagement. Quick Tap 1 median reaction times associated with a session in which < 8 trials were correct were excluded due to suspected low effort/engagement; only 2 Quick Tap sessions were excluded based on these criteria. Memory List scores corresponding to below chance (i.e. total score <12 and total score <18 on the 12-word and 18-word memory lists, respectively) performance were excluded due to suspected low effort/engagement; only 4 Memory List sessions were excluded based on these criteria. Memory Matrix scores for sessions with fewer than 5 correct trials were excluded due to suspected low effort/engagement. In addition to a priori effort thresholds for each test described above, we also excluded scores identified as person-specific outliers (i.e. any score >3 standard deviations [SD] away from a person's own mean performance on that task); 10 total instances across all NeuroUX tests were identified as person-specific mobile cognitive test outliers.

Feasibility and acceptability outcomes

Descriptive statistics (mean, SD, minimum, maximum, and frequencies) were calculated for adherence and retention rates and satisfaction questions, depending on the level of measurement and normality. A priori rates of 80% for retention and 75% for adherence were used to determine feasibility, consistent with previous rates reported in an EMA study of people with metastatic cancer.²³ EMA session fatigue effects (i.e. the likelihood of a missed cognitive EMA session over time) were evaluated using mixed effects logistic regression. Person-specific random intercepts and effects of time (i.e. study day) were modeled.

The open-text box responses to “If you have any suggestions about how the study could be improved or any other feedback, please provide that here”, were independently reviewed by 2 co-authors (OFR; SB) using a qualitative content analysis approach.^49,50 This approach allows for the distillation of words into fewer content-related categories, which is done manually.⁵¹ Each co-author first read through the full narrative to become familiar with the data, then initiated the line-by-line coding process. Codes were inductively grouped into larger categories that emerged directly from the data, without an organizing framework.⁵¹ Co-authors met to compare and collapse categories and complete the abstraction process which involves forming general descriptions and meanings of the final categories.⁵¹

Psychometric characteristics

Test-retest reliability was examined by calculating intraclass correlation coefficients (ICCs) for administrations of each test. ICC interpretation followed published guidelines for poor (<0.50), moderate (0.50–0.75), good (0.76–0.90), and excellent (>0.90) reliability.⁵²

Linear mixed effects models were used to evaluate practice effects. First, linear and quadratic effects of time (defined as study day) on each test score were examined. Person-specific random intercepts and effects of time were modeled. For scores with significant quadratic practice effects, mixed effects models with linear splines tested whether there was a changepoint at which improvements in performance level off.

Correlations evaluated convergent validity between cognitive EMA performance and EMA self-report symptoms (mean, and within-person variability) and baseline clinical cognitive (BrainCheck for objective cognitive function and FACT-Cog PCI for subjective), and psychosocial variables. P values were adjusted using a FDR correction. All analyses were conducted in R Studio (Posit Software, version 2024.04.1 + 748; ‘ggplot2’, ‘ggcorrplot’, ‘dplyer’, ‘tidyverse’, ‘magrittr’ libraries).

Results

This study included 51 women who were on average 4.2 years post MBC diagnosis, with most diagnosed as de novo MBC (58.8%). Participants mean age was 51.1 years old, the majority were White non-Hispanic race and ethnicity (84.6%), and 67.3% received 16 years or less of formal education. See Table 2 for demographic and clinical characteristics.

Table 2.

Demographic and clinical characteristics of the sample (N = 51).

Demographic characteristic	Mean (SD)	Frequency (percentage)
Age	51.1 (11.1)	–
Age groups
Ages 22–39	–	6 (11.8%)
Ages 40–60	–	34 (66.7%)
Ages 61–74	–	11 (21.6%)
Race/ethnicity
White	–	43 (84.3%)
Asian/Asian American	–	2 (3.9%)
Black/African American	–	2 (3.9%)
Hispanic	–	4 (7.8%)
Employment
Working full or part time	–	21 (41.2%)
Unemployed due to disability	–	19 (37.3%)
Laid off/fired/retired	–	7 (13.7%)
Other		4 (7.8%)
Years of education	16.1 (2.2)	–
<= 16 years of education	–	34 (66.7%)
>16 years of education	–	17 (33.3%)

Clinical characteristic	Mean (SD) or frequency (percentage)	Frequency (percentage)
Years since diagnosis with stage IV	4.2 (3.4)	–
MBC Diagnosis Type
De novo stage IV	–	30 (58.8%)
Recurrent stage IV	–	21 (41.2%)
MBC Pathological Type
HR + HER2-	–	32 (62.8%)
HER2+	–	6 (11.8%)
HR + HER2+		7 (13.7%)
Triple Negative/Other	–	6 (11.8%)
Post-menopausal	–	43 (84.3%)

Abbreviations: HER2: human epidermal growth factor receptor 2; SD: standard deviation.

Feasibility and satisfaction

One hundred and sixteen (n = 116) people completed study interest forms, of which 54 were lost to contact during prescreening scheduling communications. Sixty-two (n = 62) people were screened for enrollment of which 10 were ineligible (n = 6 resided outside the United States; n = 2 had Mini Moca scores <11; n = 2 had medical histories that were exclusion criteria), 52 women were consented and 51 enrolled in the cognitive EMA protocol, thus accrual was 84% (51/62). Mean adherence rates for the 28-day cognitive EMA protocol was 94% (SD = 8.5%, range = 61–100%). Participants expressed high satisfaction with the cognitive EMA protocol (mean rating = 88%, SD = 12.5%, median = 90%, minimum = 50%, maximum = 100%). On average, participants reported the EMA protocol as not challenging (mean rating = 32.2%, SD = 30.9%, median = 19%, range = 0–100%). Retention rates were very high, with all participants (100%) completing baseline and follow-up data collection. In response to the question about incorporating smartphone based cognitive tasks as part of ongoing care to monitor cognitive functioning, 98% of the sample responded ‘Yes’. Non-significant EMA session fatigue effects suggest that participants were not significantly more likely to miss an EMA session over time (OR = 1.02, 95%CI = 0.99–1.05, p = 0.144). The mean time to complete the daily EMA was 11 min 53 s (SD 1 min 36 s).

Qualitative content analyses revealed three categories for the narrative responses to the open-ended questions related to aspects of feasibility and acceptability of the cognitive EMA protocol: (a) tasks at inconvenient times, (b) reminders and time to complete tasks, and (c) unclear instructions. For ‘tasks at inconvenient times’, participants indicated frustration with the pseudorandom timing of the EMA texts across the protocol. For example, “the timing of the tests was all over the map…,” and “the hours I received the text messages were not consistent.” Additionally, participants noted that if they received the text closer to their bedtime, they did not have a full 6 hours to complete their tests before time was up. For ‘reminders and time to complete tasks’, participants noted that they appreciated the 6 hours to complete their EMA and that this was enough time to complete assessments each day. For ‘unclear instructions’, participants noted the instructions for the n-back were not clear and they did not know if they did it correctly, stating “I could not figure out the 2-back letter game.”

Psychometric characteristics

Variability and reliability

Distributions of cognitive EMA tests and self-report questions are presented in Table 3. Within-person variability (SD) and test-retest reliability estimates across administrations are presented in Table 4. Reliability estimates suggest excellent reliability for nearly all tests (ICCs > 0.90), except for the Memory List 12 and Memory List 18, which showed moderate-to-good test-retest reliability (ICCs > 0.66).

Table 3.

Raw scores distributions of each EMA cognitive test and self-report items (N = 51).

	Mean	SD	Min	Max
Average Cognitive Test Scores
N-back score	5.53	1.56	3	7.92
Color Trick median reaction time	2464.08	861.5	1408.89	5022.39
Matching Pair score	299.16	62.62	174.31	416.15
Memory Matrix score	40.23	7.35	24.92	60.14
Memory List 12 score	21.59	1.18	19.29	23.75
Memory List 18 score	30.5	2.12	24.75	34
CopyKat score	10.68	2.71	6.38	20.36
Hand Swype median reaction time	1921.42	546.13	787.64	3658.32
Hand Swype score	27.57	7.3	13.93	41.43
Quick Tap 1 median reaction time	430.2	86.35	277.85	734.57
Average Self-Report Item Ratings (possible range = 0–7)
Depressed	1.19	1.08	0	3.9
Anxiety	1.56	1.18	0	4.56
Fatigue	3.08	1.19	0	5.88
Loneliness	0.94	1.07	0	3.71
CRCI symptoms	2.16	1.39	0	4.75
Sleep quality	3.46	1.21	1.21	6.48
Social engagement	4.3	1.25	1.57	6.74
Cognitive abilities	3.75	1.31	1.46	7
Social satisfaction	4.25	1.44	1.11	7
Relationship quality	6.06	0.72	4.54	7
Overall well-being	5.03	0.87	1.71	7

Note: For self-report items for Depressed, Anxiety, Fatigue, Loneliness, CRCI symptoms, higher scores indicate worse symptoms. For self-report items for Sleep quality, Social engagement, Cognitive Abilities, Social Satisfaction, Relationship quality and overall Well-being, higher scores indicate better quality/satisfaction.

Table 4.

Within-person variability and test-retest reliability across administrations of each EMA cognitive test and self-report questions (N = 51).

	Within-Person Variability			Test-Retest Reliability
	Average within- person SD	Minimum within- person SD	Maximum within- person SD	ICC
Cognitive Tests
N-back score	1.38	0	2.39	0.934
Color Trick median reaction time	765.76	182.47	2136.72	0.930
Matching Pair score	53.74	28.56	84.94	0.939
Memory Matrix score	6.86	2.39	11.44	0.926
Memory List 12 score	1.68	0.46	3.18	0.770
Memory List 18 score	2.43	0.45	5.57	0.670
CopyKat score	3.02	1.26	5.53	0.900
Hand Swype median reaction time	369.39	116.79	907.24	0.973
Hand Swype score	5.73	1.85	9.77	0.978
Quick Tap 1 median reaction time	55.89	14.21	150.29	0.959
Self-Report Items
Depressed	0.91	0	2.47	0.962
Anxiety	0.94	0	2.24	0.968
Fatigue	1.36	0	2.49	0.947
Loneliness	0.67	0	2.34	0.972
CRCI symptoms	0.83	0	2.13	0.980
Sleep quality	1.46	0.65	2.59	0.945
Social engagement	1.24	0.55	2.18	0.962
Cognitive abilities	0.97	0	2.55	0.975
Social satisfaction	0.97	0	2.26	0.980
Relationship quality	0.34	0	1.14	0.983
Overall well-being	0.66	0	1.62	0.973

Practice effects

Linear or non-linear trends over time for each test are shown in Figure 2. Statistically significant linear practice effects (i.e. consistent improvements over time) were observed for Color Trick Reaction Time (day: b = -25.30, SE = 4.18, p < 0.001), Matching Pair Score (day: b = 1.46, SE = 0.26, p < 0.001), and Hand Swype Reaction Time (day: b = -19.16, SE = 1.75, p < 0.001). Non-linear practice effects (i.e. a quadratic effect of time) were observed for N-back, Memory Matrix, Hand Swype Score, and Quick Tap 1. Subsequent mixed effects regression models with linear splines showed that N-back practice effects leveled off after study day 14 (days 1–14: b = 0.13, SE = 0.02, p < 0.001; days 14–28: b = 0.01, SE = 0.01, p = 0.411), Memory Matrix practice effects leveled off after study day 12 (days 1–12: b = 0.22, SE = 0.10, p = 0.036; days 12–28: b = -0.07, SE = 0.07, p = 0.324), Hand Swype Score practice effects leveled off after day 21 (days 1–21: b = 0.61, SE = 0.03, p < 0.001; days 21–28: b = 0.09, SE = 0.09, p = 0.277), and Quick Tap 1 practice effects leveled off after day 6 (days 1–6: b = -10.81, SE = 2.22, p < 0.001; days 6–28: b = -0.30, SE = 0.38, p = 0.435). In contrast, there were no consistent trends in performance over time for Memory List 12 (day: b = -0.003, SE = 0.010, p = 0.735), Memory List 18 (day: b = -0.036, SE = 0.023, p = 0.117), or CopyKat (day: b = -0.020, SE = 0.016, p = 0.220).

Figure 2.

Practice effects (linear or non-linear trends) for each EMA cognitive test.

Cognitive and psychosocial EMA convergent validity

Correlations among EMA cognitive tests (mean and within-person variability in performance) with age, education, BrainCheck raw scores, and self-reported cognitive function (FACT-Cog PCI) are shown in Table 5 (mean scores) and Table 6 (SD in scores). Mean performance on Memory List 12 & 18 were most strongly associated with BrainCheck Immediate Recall and Delayed Recall scores. Cognitive EMA tests of processing speed (e.g. Matching Pair, Quick Tap) were most strongly associated with all speeded BrainCheck tests (Trails A, Trails B, Stroop, Digit Symbol). Almost all cognitive EMA tests of executive/working memory (i.e. Color Trick, Memory Matrix, CopyKat, Hand Swype) were associated with BrainCheck executive tests (Trails B, Stroop, Digit Symbol), correlations among N-back and BrainCheck tests did not remain significant with FDR p value adjustments. We included correlations among variability in cognitive EMA scores using root mean squared successive differences and BrainCheck scores in Supplemental Table S2. There were also some significant correlations between self-report and objective cognitive EMAs of processing speed (r's = 0.35 and −0.34, p's < 0.05, see Supplemental Figure S2). In addition, psychosocial EMA items were strongly related to validated measures of psychosocial function and cognitive symptoms (e.g. PROMIS scales and FACT-Cog; see Figure 3).

Figure 3.

Pearson r correlations between cognitive and psychosocial self-report EMA items (mean across 28 days), and clinical assessments of psychosocial and cognitive function. Colors represent the strength of the correlation for statistically significant associations using FDR adjusted p values; blank colors have p > 0.05.

Table 5.

Pearson r correlations testing convergent validity between average cognitive EMAs and baseline age, education, BrainCheck cognitive scores, and FACT Cog PCI (N = 51).

			BrainCheck Raw Scores
Average Score	Age	Education	Trails A Time	Trails B Time	Stroop Median Reaction Time	Digit Symbol Correct Per Second	Digit Symbol Median Reaction Time (all trials)	Immediate Recall Correct	Delayed Recall Correct	Raw Combined Score	FACT-Cog PCI
N-back score	−0.08	0.15	−0.07	−0.22	−0.21	0.26	−0.10	0.18	0.40*	0.31	−0.16
Color Trick median reaction time	0.43**	−0.02	0.44**	0.67***	0.65***	−0.54***	0.45**	−0.13	−0.25	−0.57***	−0.08
Matching Pair score	−0.39*	0.13	−0.54***	−0.55***	−0.61***	0.63***	−0.43**	0.36	0.27	0.68***	−0.13
Memory Matrix score	−0.44**	0.11	−0.51***	−0.59***	−0.64***	0.63***	−0.47**	0.22	0.31*	0.66***	0.02
Memory List 12 score	−0.11	0.29	−0.23	−0.17	−0.15	0.17	−0.23	0.37*	0.46**	0.35*	−0.11
Memory List 18 score	−0.10	0.07	−0.32	−0.36*	−0.22	0.36*	−0.39*	0.45**	0.43**	0.48**	0.01
CopyKat score	−0.21	−0.01	−0.35*	−0.47**	−0.45**	0.53***	−0.37*	0.41*	0.46**	0.61***	0.06
Hand Swype median reaction time	0.49***	0.11	0.48***	0.55***	0.69***	−0.52***	0.57***	−0.21	−0.31	−0.61***	−0.16
Hand Swype score	−0.39*	−0.08	−0.32	−0.45**	−0.51***	0.43**	−0.38*	0.19	0.34*	0.50***	0.12
Quick Tap 1 median reaction time	0.44**	−0.11	0.57***	0.66***	0.59***	−0.55***	0.62***	−0.29	−0.43**	−0.68***	−0.22
CRCI Symptoms (EMA)	0.14	−0.12	0.32	0.31	0.29	−0.17	0.26	0.03	−0.10	−0.24	−0.62***
Cognitive Abilities (EMA)	0.06	0.01	−0.16	−0.21	−0.12	0.10	−0.16	−0.01	0.23	0.17	0.57***

Note: FDR adjusted p-value used.

* p < 0.05.

** p < 0.01.

*** p < 0.001.

Table 6.

Pearson r correlations testing convergent validity between within-person variability on cognitive EMAs (SD) and baseline age, education, BrainCheck cognitive scores, and FACT Cog PCI (N = 51).

			BrainCheck Raw Scores
Within-person variability score	Age	Education	Trails A Time	Trails B Time	Stroop Median Reaction Time	Digit Symbol Correct Per Second	Digit Symbol Median Reaction Time (all trials)	Immediate Recall Correct	Delayed Recall Correct	Raw Combined Score	FACT-Cog PCI
N-back score	0.10	−0.08	0.22	0.25	0.15	−0.16	0.20	−0.25	0.07	−0.19	−0.03
Color Trick median reaction time	0.27	−0.01	0.38*	0.60***	0.52***	−0.49***	0.47**	−0.23	−0.13	−0.49**	−0.07
Matching Pair score	−0.24	0.10	−0.27	−0.36*	−0.33	0.43*	−0.37*	0.41*	0.17	0.45**	−0.23
Memory Matrix score	−0.26	−0.12	−0.22	−0.24	−0.36*	0.33	−0.26	0.11	0.13	0.33	0.00
Memory List 12 score	0.22	−0.01	−0.11	−0.09	0.05	0.06	−0.13	−0.22	−0.21	−0.05	0.16
Memory List 18 score	0.12	−0.19	0.24	0.22	0.11	−0.26	0.47**	−0.09	−0.19	−0.27	−0.22
CopyKat score	−0.22	−0.16	−0.32	−0.47**	−0.41*	0.49***	−0.38*	0.43*	0.43*	0.58***	0.18
Hand Swype median reaction time	−0.11	0.10	−0.05	0.11	0.30	−0.10	0.01	−0.20	−0.06	−0.14	−0.11
Hand Swype score	−0.49***	0.22	−0.49***	−0.37*	−0.41*	0.34	−0.36	−0.03	0.20	0.40*	0.15
Quick Tap median reaction time	0.32	−0.09	0.35	0.48***	0.41*	−0.43*	0.44*	−0.19	−0.29*	−0.47**	−0.24
CRCI Symptoms (EMA)	0.01	−0.09	0.17	0.25	0.09	−0.06	0.02	−0.17	−0.12	−0.16	−0.19
Cognitive Abilities (EMA)	−0.13	−0.07	0.08	0.08	−0.04	0.08	−0.05	−0.31*	−0.04	−0.05	0.32*

Note: FDR adjusted p-value used.

* p < 0.05.

** p < 0.01.

*** p < 0.001.

Discussion

We found that short-term cognitive EMA monitoring (for 28 days) is acceptable and feasible in patients with MBC as shown by high adherence to the protocol, study retention, and high satisfaction/low burden ratings. The specific cognitive and psychosocial EMAs (mobile cognitive tests and self-ratings for cognitive and psychosocial symptoms) demonstrated strong reliability and validity. High adherence of the EMA protocol (94%) aligned with previously reported adherence rates of e-Health physical activity interventions for women with MBC (94% adherence over 6 months)⁵³ and a recent EMA study in MBC patients that used a 7-day EMA protocol (80% adherence).²³ High adherence and satisfaction rates found here may also be attributed to the study procedures that encouraged participant engagement and minimized burden, including accommodating participant preferences into the timing of cognitive EMA texts each day, sending reminder texts, troubleshooting technical difficulties, and providing additional instructional support for the specific mobile cognitive tests administered. Taken together, these findings demonstrate that women with MBC are interested and engaged in e-Health modalities and ongoing symptom monitoring research via EMAs, despite the high burden of metastatic disease.

Since MBC can be a chronic condition, there is a need to help people manage their cognitive changes and live fuller and happier lives. Those with MBC often become active self-managers of their complex treatments, side effects, and inventory of their everyday QoL.⁵⁴ Methods for self-monitoring or self-evaluation could empower those with MBC to be more active participants in their treatment plans and provide important data across time to make decisions related to treatments and daily activities. Furthermore, qualitative studies report that women with MBC want to minimize their time at medical appointments and maximize their time ‘living’,¹⁰ so remote digital tools for self-evaluation, such as those employed in this study, may be especially useful to facilitate self-monitoring while living with MBC.

In the context of advanced cancer, EMAs can directly capture variation in symptoms and functional outcomes in real-life settings, provide a useful complement to more macro-level assessments that occur in longitudinal research designs or in clinic visits, and facilitate better personalized care for those impacted by MBC. One of the National Cancer Institute's (United States) research priorities is to “understand and address emerging symptom trajectories of individuals living with metastatic cancers.”⁵⁵ Additionally, newly published National Standards for Cancer Survivorship Care from the Office of Cancer Survivorship,^56,57 and newly published MASCC-ASCO Standards and Practice Recommendations for people affected by advanced or metastatic cancer^58,59 specify the need for oncology organizations to collect data on survivors’ patient-reported outcomes, including QoL, and survivors’ functional capacity. Our findings of low study burden, high utility, and high interest in using the cognitive EMA platform provide foundational evidence for incorporating EMAs for cognitive self-monitoring in clinical settings. Our findings also support the future application of cognitive EMAs for evaluating MBC-specific intervention effects, including mobile/web based supportive interventions that are in the research pipeline.^60–63

Qualitative findings in this study highlight the need to provide clearer instructions for some of mobile cognitive tests on the platform, and the provision of scheduled instead of pseudorandom assessment sessions in this population. Our team has incorporated this feedback on instruction clarity into the platform, developing instructional videos for all cognitive tests. True ‘momentary’ assessments in this population may be burdensome, and so longer periods of time for assessment completion, scheduled timing of assessments, and/or participant-initiated assessments should be considered in future EMA studies with persons who have metastatic disease.

The cognitive EMA protocol, including mobile cognitive tests and self-rated symptom assessments, administered via NeuroUX, demonstrated strong reliability across administrations, expected practice effects, and very good convergent validity overall, consistent with previous reports of strong validity and reliability of different cognitive EMAs administered across 14 days in early-stage breast cancer survivors.³¹ The lowest reliability scores were found for the Memory List tests, 12-item (0.77) and 18 item (0.67), which is expected since these tests generate new and unrelated lists of words to remember with each administration. Cognitive EMAs are an innovative solution for monitoring the cognitive health of MBC patients, but there is a need for broader validation across diverse cancer types.

The practice effects that we found are consistent with practice effects for the same mobile cognitive tests found in a community sample of healthy adults.⁴⁸ Patterns of nonlinear effects with improvements leveling off after many days were found for the three tests that are the most challenging in the protocol (N-back, Memory Matrix, Hand Swype Score). Recently, researchers have used learning curves (i.e. practice effects) across multiple days, rather than single session re-testing, as clinically relevant predictors of disease or future cognitive decline in populations with subtle cognitive deficits (i.e. preclinical Alzheimer's Disease).⁶⁴ Mobile cognitive tests with no significant practice effects in this protocol may be explained by strong alternate forms of these tests (e.g. Memory Lists). Researchers using cognitive EMA protocols in MBC populations should consider practice effects for each mobile cognitive test when designing the length of EMA protocols (i.e. number of assessments over number of days) and consider comparing learning curves of serial cognitive EMAs between those with clinical cognitive impairment and those without and/or those with MBC and matched controls.

This study did not have a matched control group. However, our findings were compared to previously published norms from a community based sample of adults living in the United States for these same mobile cognitive tests of memory and reaction time.⁴⁸ Our sample demonstrated similar scores and variability in memory tests (Memory Matrix, Memory List-12 item), but slower reaction times and greater within- person variability in reaction time on Quick Tap 1 and Hand Swype tests. Greater variability in reaction time on mobile cognitive tests is observed in cognitively symptomatic older adults compared to cognitively normal.⁶⁵ CRCI are often described as a “brain fog” or slower thinking, so it is possible that variability in reaction time may be clinically relevant in this population.^66,67 Future studies using cognitive EMAs in MBC should include a healthy, age-matched control group, and evaluate reaction time performance and variability. Furthermore, this study protocol spanned 28 days, whereby it is possible that more within- and between- person variability in symptoms would emerge in longer prospective studies with MBC and is another recommendation for future studies.

Overall, the cognitive (mobile cognitive tests and self-rated items) and psychosocial EMAs administered in this sample of women with MBC demonstrated strong convergent validity, except for N-back. Medium-to-strong correlations were evident among baseline clinical cognitive assessments, with strongest correlations identified for tests of specific cognitive domains (e.g. memory tests correlating with BrainCheck tests for immediate and delayed memory). In terms of age and education, correlations were in expected directions (i.e. higher age, lower scores, and longer reaction time; higher education, higher scores, and shorter reaction time); however, most correlations with education were not significant. These findings may be partially explained by minimal variability in education in this sample. However, NeuroUX tests are designed to be minimally influenced by socioeconomic status (SES) factors, ensuring that they provide an equitable assessment of cognitive function across diverse populations.⁶⁸ This approach enhances the reliability and reduces bias of the results, making them more applicable and actionable in diverse settings.

All the EMA self-report items showed strong relationships with all the clinical assessments; but strongest relationships emerged between EMAs tapping into the same construct, demonstrating strong convergent and ecological validity. For example, social EMAs correlated strongest with PROMIS Social, and the cognitive symptoms EMA correlated strongest with the FACT-Cog PCI. The cognitive symptom EMA not only correlated with the clinical cognitive PRO, but also several of the BrainCheck tests (i.e. Trails A and B, Stroop), suggesting that repeated sampling of cognitive PROs in this population reflects both objective and subjective CRCI, although the adjusted p values for these correlations were under the significance threshold (p's = 0.056–0.07). Significant correlations also emerged in this sample among mobile cognitive tests and CRCI symptoms, loneliness, fatigue, depressive symptoms and anxiety across time (Supplemental Figure S2). In the broader CRCI literature, objective cognitive tests rarely correlate with subjective cognitive measures or psychological symptoms.^11,69,70 Therefore, our findings provide new insights into the relationship between subjective and objective CRCI at the individual level and suggest that serial mobile cognitive assessments may be sensitive to both objective and subjective CRCI.

This study has several limitations. The relatively small sample size and low variability in race and ethnicity, decreases the generalizability. We also did not examine the within-person effects of time of day that EMAs were administered (administration times varied for each person across the study). While in is not expected that this within-person effect would change our feasibility, reliability, validity findings, future research should examine the within-person effect of time of day on cognitive EMAs. We did not collect data on metastases, limiting our interpretation of the cognitive findings related to burden of disease.^3,71 This study did not include a matched control group, limiting our interpretation of the cognitive EMA mean scores and variability in this population. Our feasibility and acceptable findings may also represent a biased sample of women who would be more willing to engage with their smartphones for monitoring purposes or those who felt well enough to participate in the study. Last, this study was conducted over 28 days. While promising, it is not clear that this would be feasible for patients who are undergoing treatments that last months to years, and a longer study is needed.

Conclusion

Despite these limitations, this study extends a growing body of literature employing EMA use for symptom monitoring in populations with metastatic disease, and provides foundational knowledge on the feasibility, reliability, and validity of employing cognitive (subjective and objective) EMAs to monitor cognitive functioning across time in women living with MBC. Future large-scale cohort and implementation studies should confirm these findings and determine how cognitive EMAs can be effectively integrated into electronic health records and clinical care.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241310474 - Supplemental material for Feasibility and psychometric quality of smartphone administered cognitive ecological momentary assessments in women with metastatic breast cancer

Supplemental material, sj-docx-1-dhj-10.1177_20552076241310474 for Feasibility and psychometric quality of smartphone administered cognitive ecological momentary assessments in women with metastatic breast cancer by Ashley M Henneghan, Emily W Paolillo, Kathleen M Van Dyk, Oscar Y Franco-Rocha, Soyeong Bang, Rebecca Tasker, Tara Kaufmann, Darren Haywood, Nicolas H Hart and Raeanne C Moore in DIGITAL HEALTH

Footnotes

Acknowledgements

This research wouldn’t be possible without the patients, survivors, thrivers, and metavivors who graciously volunteered to participate

Consent to participate

Participants provided informed written consent.

Contributorship

AMH and EWP are responsible for the manuscript's conception and design. EWP conducted the data analyses. SB and OFR conducted the qualitative content analysis. AMH was the PI on this study. AMH, KVD, RCM, DH, OFR, SB, EWP, RT, TK, NHH contributed to revisions of the manuscript. All authors approved the final version.

Data availability

De-identified data will be made available upon reasonable requests to the corresponding author and with data use agreements in place.

Declaration of conflicting interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: RCM is a co-founder of KeyWise AI, Inc. and a co-founder of NeuroUX. The terms of this arrangement have been reviewed and approved by UC San Diego in accordance with its conflict-of-interest policies. AMH was a consultant for Prodeo, Inc. The terms of this arrangement have been reviewed and approved by University of Texas at Austin in accordance with its conflict-of-interest policies.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. The University of Texas at Austin Institutional Review Board reviewed and monitored all study related procedures (STUDY00002393).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by IRG-21-135-01-IRG from the American Cancer Society (AMH). AMH is supported by a grant from the NIH: R21NR020497. NHH is supported by an NHMRC Investigator Fellowship (APP2018070). KVD is supported by grants from the NIH: K08CA241337, R21NR020497, and R35CA283926. DH is a MASCC Cognition Fellow. OYF is a MASCC Equity Fellow.

Guarantor

Ashley M Henneghan, PhD, RN, FAAN.

ORCID iD

Ashley M Henneghan

Supplemental material

Supplemental material for this article is available online.

References

Komen

SG.

Metastatic Breast Cancer, https://www.komen.org/breast-cancer/metastatic/ (2024).

O'Shaughnessy

. Extending survival with chemotherapy in metastatic breast cancer. Oncologist 2005; 10: 20–29.

Gerratana

Fanotto

Bonotto

, et al. Pattern of metastasis and outcome in patients with breast cancer. Clin Exp Metastasis 2015; 32: 125–133. 20150129.

Demirors

Goktepe

Medeck

, et al. The role of primary surgery in De Novo metastatic breast carcinoma. Eur J Breast Health 2023; 19: 110–114. 20230401.

Cardoso

Wilking

Bernardini

, et al. A multi-stakeholder approach in optimising patients’ needs in the benefit assessment process of new metastatic breast cancer treatments. Breast 2020; 52: 78–87. 20200510.

daCosta DiBonaventura

Copher

Basurto

, et al. Patient preferences and treatment adherence among women diagnosed with metastatic breast cancer. Am Health Drug Benefits 2014; 7: 386–396.

Willis

Lewis

, et al. The experience of living with metastatic breast cancer—a review of the literature. Health Care Women Int 2015; 36: 514–542. 20140422.

Cavers

Habets

Cunningham-Burley

, et al. Living with and beyond cancer with comorbid illness: a qualitative systematic review and evidence synthesis. J Cancer Surviv 2019; 13: 148–159. 20190126.

Drageset

Austrheim

Ellingsen

Quality of life of women living with metastatic breast cancer and receiving palliative care: a systematic review. Health Care Women Int 2021; 42: 1044–1065. 20210402.

10.

Lyons-Rahilly

Meskell

Carey

, et al. Exploring the experiences of women living with metastatic breast cancer [MBC]: a systematic review of qualitative evidence. PLoS One 2024; 19: e0296384. 20240105.

11.

Mayo

Lustberg

Dhillon

, et al. Cancer-related cognitive impairment in patients with non-central nervous system malignancies: an overview for oncology providers from the MASCC neurological complications study group. Support Care Cancer 2021; 29: 2821–2840. 2020/11/25.

12.

Lange

Joly

Vardy

, et al. Cancer-related cognitive impairment: an update on state of the art, detection, and management strategies in cancer survivors. Ann Oncol 2019; 30: 1925–1940. 2019/10/17.

13.

Kesler

Rao

Blayney

, et al. Predicting long-term cognitive outcome following breast cancer with pre-treatment resting state fMRI and random forest machine learning. Front Human Neurosci 2017; 11: 555.

14.

Boscher

Joly

Clarisse

, et al. Perceived cognitive impairment in breast cancer survivors and its relationships with psychological factors. Cancers (Basel) 2020; 12: 3000. 2020/10/22.

15.

Janelsins

Kesler

Ahles

, et al. Prevalence, mechanisms, and management of cancer-related cognitive impairment. Int Rev Psychiatry 2014; 26: 102–113.

16.

Wefel

Kesler

Noll

, et al. Clinical characteristics, pathophysiology, and management of noncentral nervous system cancer-related cognitive impairment in adults. CA Cancer J Clin 2015; 65: 123–138.

17.

Henneghan

Van Dyk

Haywood

, et al. Characterizing cancer-related cognitive impairments and impact on quality of life in women with metastatic breast cancer. Breast Cancer Res Treat 2024 20240913.

18.

Thong

MSY

Chan

van den Hurk

, et al. Going beyond (electronic) patient-reported outcomes: harnessing the benefits of smart technology and ecological momentary assessment in cancer survivorship research. Support Care Cancer 2021; 29: 7–10. 2020/08/28.

19.

Kida

Olver

Yennu

, et al. Optimal supportive care for patients with metastatic breast cancer according to their disease progression phase. JCO Oncol Pract 2021; 17: 177–183. 20210122.

20.

Kampshoff

Verdonck-de Leeuw

van Oijen

, et al. Ecological momentary assessments among patients with cancer: a scoping review. Eur J Cancer Care (Engl) 2019; 28: e13095. 20190514.

21.

Schuler

King

Matsveru

, et al. Wearable-triggered ecological momentary assessments are feasible in people with advanced cancer and their family caregivers: feasibility study from an outpatient palliative care clinic at a cancer center. J Palliat Med 2023; 26: 980–985. 20230503.

22.

Vehling

Gerstorf

Schulz-Kindermann

, et al. The daily dynamics of loss orientation and life engagement in advanced cancer: a pilot study to characterise patterns of adaptation at the end of life. Eur J Cancer Care (Engl) 2018; 27: e12842. 20180427.

23.

Brasky

Newton

Stephens

, et al. Testing the feasibility of mobile ecological momentary assessment for symptom burden and management among metastatic cancer patients. Digit Health 2024; 10: 20552076241261843. 20240607.

24.

Parrish

Kamarsu

Harvey

, et al. Remote ecological momentary testing of learning and memory in adults with serious mental illness. Schizophr Bull 2021; 47: 740–750. 2020/11/22.

25.

Harvey

Miller

Moore

, et al. Capturing clinical symptoms with ecological momentary assessment: convergence of momentary reports of psychotic and mood symptoms with diagnoses and standard clinical assessments. Innov Clin Neurosci 2021; 18: 24–30. 2021/06/22.

26.

Bomyea

Parrish

Paolillo

, et al. Relationships between daily mood states and real-time cognitive performance in individuals with bipolar disorder and healthy comparators: a remote ambulatory assessment study. J Clin Exp Neuropsychol 2021; 43: 813–824. 2021/09/09.

27.

Moore

Ackerman

Russell

, et al. Feasibility and validity of ecological momentary cognitive testing among older adults with mild cognitive impairment. Front Digit Health 2022; 4: 946685. 2022/08/23.

28.

Moore

Parrish

Van Patten

, et al. Initial psychometric properties of 7 NeuroUX remote ecological momentary cognitive tests among people with bipolar disorder: validation study. J Med Internet Res 2022; 24: e36665. 2022/07/30.

29.

Derbes

Hakun

Elbich

, et al. Design and methods of the mobile assessment of cognition, environment, and sleep (MACES) feasibility study in newly diagnosed breast cancer patients. Sci Rep 2024; 14: 8338. 20240409.

30.

Scott

Mogle

Sliwinski

, et al. Memory lapses in daily life among breast cancer survivors and women without cancer history. Psychooncology 2020; 29: 861–868. 20200221.

31.

Small

Jim

HSL

Eisel

, et al. Cognitive performance of breast cancer survivors in daily life: role of fatigue and depressed mood. Psychooncology 2019; 28: 2174–2180. 20190830.

32.

Brandt

Spencer

Folstein

. The telephone interview for cognitive status. Cogn Behav Neurol 1988; 1: 111–117.

33.

Castanho

Amorim

Zihl

, et al. Telephone-based screening tools for mild cognitive impairment and dementia in aging studies: a review of validated instruments. Front Aging Neurosci 2014; 6: 16. 2014/03/13.

34.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–381. Research Support, N.I.H., Extramural 2008/10/22.

35.

Harris

Taylor

Minor

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208. 2019/05/13.

36.

Henneghan

. Henneghan Lab Cognitive Health Initiative for Cancer Survivors, https://nursing.utexas.edu/researh/hl/resources (2024).

37.

Wagner

Sweet

Butt

, et al. Measuring patient self-reported cognitive function: development of the functional assessment of cancer therapy-cognitive function instrument. J Support Oncol 2009; 7: W32–W39.

38.

Groppell

Soto-Ruiz

Flores

, et al. A rapid, mobile neurocognitive screening test to aid in identifying cognitive impairment and dementia (BrainCheck): cohort study. JMIR Aging 2019; 2: e12615. 2019/09/14.

39.

Yang

Hendrix

CC.

Cancer-related cognitive impairment in breast cancer patients: influences of psychological variables. Asia Pac J Oncol Nurs 2018; 5: 296–306. 2018/07/03.

40.

West

Cavallero

Ceccherini

, et al. Impact of psychosocial, behavioral and lifestyle factors on subjective cognitive complaints and perceived quality of life in a large cohort of Italian breast cancer patients. Front Psychol 2022; 13: 1015573. 20221109.

41.

Hsu

Chen

Hsieh

, et al. Subjective and objective cognitive functioning among patients with breast cancer: effects of chemotherapy and mood symptoms. Breast Cancer 2021; 28: 236–245. 20201008.

42.

Merriman

Sereika

Brufsky

, et al. Trajectories of self-reported cognitive function in postmenopausal women during adjuvant systemic therapy for breast cancer. Psychooncology 2017; 26: 44–52. 2015/10/22.

43.

Patient-Reported Outcomes Measurement Information System 2017, http://www.healthmeasures.net/explore-measurement-systems/promis/obtain-administer-measures . (2017).

44.

Jensen

King-Kallimanis

Sexton

, et al. Measurement properties of PROMIS sleep disturbance short forms in a large, ethnically diverse cancer cohort. Psychol Test Assess Model 2016; 58: 353–370.

45.

Hughes

Waite

Hawkley

, et al. A short scale for measuring loneliness in large surveys: results from two population-based studies. Res Aging 2004; 26: 655–672.

46.

Jones

Moore

Depp

, et al.

Daily ecological momentary assessments of happy and sad moods in people with schizophrenia and bipolar disorders: what do participants who are never sad think about their activities and abilities?

Schizophr Res Cogn 2021; 26: 100202. 2021/07/01.

47.

Moore

Paolillo

Sundermann

, et al. Validation of the mobile verbal learning test: illustration of its use for age and disease-related cognitive deficits. Int J Methods Psychiatr Res 2021; 30: e1859. 2020/11/08.

48.

Paolillo

Bomyea

Depp

, et al. Characterizing performance on a suite of english-language neuroUX mobile cognitive tests in a US adult sample: ecological momentary cognitive testing study. J Med Internet Res. 2024; 26: e51978.

49.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health 2000; 23: 334–340. 2000/08/15.

50.

Sandelowski

When a cigar is not just a cigar: alternative takes on data and data analysis. Res Nurs Health 2011; 34: 342–352. 2011/04/12.

51.

Elo

Kyngas

The qualitative content analysis process. J Adv Nurs 2008; 62: 107–115. 2008/03/21.

52.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

53.

Delrieu

Pialoux

Pérol

, et al. Feasibility and health benefits of an individualized physical activity intervention in women with metastatic breast cancer: intervention study. JMIR Mhealth Uhealth 2020; 8: e12306. 20200128.

54.

van Dongen

de Nooijer

Cramm

, et al. Self-management of patients with advanced cancer: a systematic review of experiences and attitudes. Palliat Med 2020; 34: 160–178.

55.

Mollica

Smith

Tonorezos

, et al. Survivorship for individuals living with advanced and metastatic cancers. National Cancer Institute Meeting Report. J Natl Cancer Inst 2022; 114: 489–495.

56.

Institute

NC.

National Standards for Cancer Survivorship Care, https://cancercontrol.cancer.gov/ocs/special-focus-areas/national-standards-cancer-survivorship-care?cid=eb_govdel (2024).

57.

Mollica

McWhirter

Tonorezos

, et al. Developing national cancer survivorship standards to inform quality of care in the United States using a consensus approach. J Cancer Surviv 2024.

58.

Hart

Nekhlyudov

Smith

, et al. Survivorship care for people affected by advanced or metastatic cancer: MASCC-ASCO standards and practice recommendations. JCO Oncol Pract 2024: Op2300716. 20240429.

59.

Hart

Nekhlyudov

Smith

, et al. Survivorship care for people affected by advanced or metastatic cancer: MASCC-ASCO standards and practice recommendations. Support Care Cancer 2024; 32: 313. 20240429.

60.

Beatty

Kemp

Butow

, et al. Finding my way-advanced: can a web-based psychosocial intervention improve the mental quality of life for women with metastatic breast cancer vs attention-control? Study protocol of a randomised controlled trial. BMC Cancer 2022; 22: 1353. 20221224.

61.

Delrieu

Anota

Trédan

, et al. Design and methods of a national, multicenter, randomized and controlled trial to assess the efficacy of a physical activity program to improve health-related quality of life and reduce fatigue in women with metastatic breast cancer: ABLE02 trial. BMC Cancer 2020; 20: 622. 20200703.

62.

Phillips

Solk

Welch

, et al. A technology-based physical activity intervention for patients with metastatic breast cancer (Fit2ThriveMB): protocol for a randomized controlled trial. JMIR Res Protoc 2021; 10: e24254. 20210423.

63.

Gordon

Qiu

Doerksen

, et al. Addressing metastatic individuals everyday: rationale and design of the nurse AMIE for Amazon Echo show trial among metastatic breast cancer patients. Contemp Clin Trials Commun 2023; 32: 101058. 20230116.

64.

Papp

Jutten

Soberanes

, et al. Early detection of amyloid-related changes in memory among cognitively unimpaired older adults with daily digital testing. Ann Neurol 2024; 95: 507–517. 20231219.

65.

Welhaf

Wilks

Aschenbrenner

, et al. Naturalistic assessment of reaction time variability in older adults at risk for Alzheimer's disease. J Int Neuropsychol Soc 2024; 30: 428–438. 20240129.

66.

Haywood

Dauer

Baughman

, et al. “Is my brain ever going to work fully again?”: challenges and needs of cancer survivors with persistent cancer-related cognitive impairment. Cancers (Basel) 2023; 15. 20231108.

67.

Becker

Henneghan

Mikan

. When do I get my brain back? Breast cancer survivors’ experiences of cognitive problems. Clin J Oncol Nurs 2015; 19: 180–184.

68.

NeuroUX, https://www.getneuroux.com/cognitive-assessment.

69.

Bray

Dhillon

Vardy

. Systematic review of self-reported cognitive function in cancer patients following chemotherapy treatment. J Cancer Surviv 2018 2018/05/08.

70.

Costa

DSJ

Fardell

JE.

Why are objective and perceived cognitive function weakly correlated in patients with cancer?

J Clin Oncol 2019; 37: 1154–1158. 2019/03/29.

71.

Shao

Chen

, et al. Brain metastasis in de novo stage IV breast cancer. Breast 2023; 71: 54–59. 20230714.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.78 MB