Sage Journals: Discover world-class research

Abstract

Background

Increasingly, studies and reviews have highlighted the potentials of ecological momentary assessments (EMAs) and wearables in suicide research. However, to date it is only poorly understood how patients experience frequent assessment of suicidal ideation over weeks and months.

Methods

Following discharge from inpatient psychiatric care due to a suicidal crisis or suicide attempt, patients started a 21- to 24-day EMA (EMA 1) with four semi-random prompts per day. After that, participants received four prompts per day, on two randomly chosen consecutive days per week for the following 26 weeks (EMA 2). Participants were additionally given a wearable during EMA 1 and 2. Debriefings on participants’ thoughts and experiences were conducted via telephone interviews after EMA 1 (n = 68) and after EMA 2 (n = 51) using rating scales and open questions. Qualitative and quantitative methods were used to analyze the data.

Results

After EMA 1, 62% of participants stated that they had experienced a change in their behavior or mood due to the study (66% in EMA 2). Different aspects were mentioned, highlighting the helpfulness of EMA (e.g. improving insight and grounding oneself) but also the burden (e.g. feeling weighed down/exhausted) and reactivity effects (e.g. feeling worse/annoyed and increased brooding).

Discussion

The findings illustrate positive and negative effects of EMAs over longer observation intervals in individuals at high risk of suicide-related thoughts and behaviors. These findings can help in the development of study protocols, to evaluate data quality and enhance the interpretation of EMA data.

Keywords

Introduction

In suicide research, there has been an increase in studies employing real-time data collection methods such as ecological momentary assessments (EMA) and wearable sensor data. While a review of available EMA studies on suicide-related thoughts and behaviors (STBs) conducted in 2015 identified only four published studies,¹ 23 studies were included in a review published in 2022.²

Potentials of real-time data collection in suicide research

Advantages of EMAs over conventional methods like retrospective self-reports include a reduction of recall bias³ as well as high ecological validity and a fine-grained high temporal resolution matching clinically relevant time frames of hours and days.^4–6 Wearable devices have been suggested as tools to assess the physiological correlates of STBs with the aim of predicting suicidal crises through physiological markers in real-time.⁵ Wearable data is thus not affected by potential disclosure problems regarding STBs or limited insight in STBs and might be specifically relevant for individuals experiencing no suicide ideation prior to a suicide attempt.⁷ Wearables are used, for example, to measure objective sleep disturbances (i.e. changes in duration and quality) as predictors of next-day suicidal ideation or acute increases in suicidal ideation^8,9 or to assess heart rate variability, which has been shown to be negatively associated with decreases in STBs.¹⁰ In light of these advantages and potentials, EMA and wearables have been acknowledged as promising methods for evaluating and monitoring STBs and related physiological correlates in persons at risk, offering the opportunity to intervene at the moment(s) when STB risk is increased.¹¹

Measurement reactivity

Despite these promising opportunities EMAs and wearables offer for suicide research and treatment of persons at high risk of STBs, the effect of repeated measurements over days, weeks, or even months on the assessed constructs and on the participants remains somewhat unclear. Measurement reactivity is defined as changes in the underlying construct (e.g. iatrogenic effects such as exacerbation of symptoms), changes in behavior (e.g. avoidance of activities or adaption of daily routines), and response behavior (e.g. change in use of response scales during the participation period) due to intensively repeated assessments.¹² Although possible reactivity effects have been brought up frequently with regard to repeated assessments of STBs (specifically regarding iatrogenic effects such as the increase or worsening of STBs),¹³ studies systematically addressing measurement reactivity or participant experiences are rare in real-time suicide research.¹⁴ Filling this gap appears particularly relevant, because ethical and methodological concerns regarding the effects of high-frequency or EMA over longer observation intervals are often discussed.¹⁵ If measurement reactivity is evident, it could seriously limit the ecological validity of the data,¹⁶ restrict the promising opportunity to intervene in real-time¹⁷ and generally prevent researchers from implementing extensive study designs in this vulnerable population.

Previous studies addressing reactivity

A recent systematic review identified nine EMA studies addressing aspects related to feasibility and reactivity.² Most studies focused on changes in the underlying construct, targeting only one aspect of measurement reactivity. With regard to potential iatrogenic effects, these studies reported no symptom worsening (i.e. increased frequency of suicidal thoughts¹⁸) no changes in intensity¹⁹ or in the occurrence of suicidal ideation²⁰ with repeated prompts. Asked about the impact of EMA on suicidal ideation, 9% to 16% of participants reported feeling stressed or burdened by EMA or that it had triggered bad thoughts.^21,22 On the other hand, 28% of participants felt generally better after EMA²³ and 3% reported a decrease in frequency and urge of suicidal thoughts²¹ or an increased awareness of symptoms.²⁴ Interestingly, the majority of participants (approx. 60%) in a recent study¹⁴ did neither report a positive nor a negative effect on their mood. Yet, 18% of participants observed a triggering effect of EMA on their SI and 10% a worsening of SI (particularly those with higher psychopathological burden at baseline, for example, higher levels of SI, depression, anxiety or borderline personality traits). In contrast, linear multilevel models found no effects of the number of assessments per day on affective states or suicide ideation.¹⁴

Gaps in current research

We are not aware of any STB real-time studies focusing in particular on behavioral changes. Compliance and attrition rates often serve as indicators of altered response behavior (e.g. fatigue effects), but to date there have hardly been any findings on changes in everyday behavior or other changes in response behavior in EMA studies.¹⁴ Compliance ranged from 44% to 75% in studies lasting 21 to 28 days (with ≥3 prompts/day) with lower rates in clinical populations and a decline of compliance with longer study duration (i.e. suggesting a general fatigue effect and associated change of response behavior^2,25). In general, lower compliance and higher drop-out rates are associated with participant and study characteristics. For example, male and more severely burdened participants show lower compliance rates.²⁶ Moreover, interference of the assessments with everyday life (such as interrupting social contacts or travel) is associated with higher drop-out rates;²⁷ this effect is assumed to be even more pronounced in studies with a longer assessment period.²⁶ With increasing interference, the adaptation of daily routines or behaviors as well as disturbances of interactions with other persons might be more likely. To our knowledge, no study examining aspects of reactivity associated with the use of wearable devices in the field of suicide research is available to date. In other research areas, findings indicate that movement behavior is reactive in real-time studies.²⁸ Wearable use has been associated with reduced psychological distress (mediated by increased self-care and perception),²⁹ and positive affect.³⁰ In behavioral activation interventions, wearables have been perceived positively (e.g. increasing self-awareness) and negatively (e.g. being inconvenient or not interesting) by depressed individuals.³¹ In addition, most study designs entail observation periods between a few days and 4 weeks,^2,25 but data on the effects of studies with long observation intervals of weeks and months involving real-time assessments in populations at high risk of STBs is lacking. This stands in stark contrast to the fact that STB risk is heightened for several weeks or months after inpatient treatment, requiring long-term monitoring of STBs in persons at risk.³² Moreover, available studies rarely focus on participant experiences of the frequent assessment of STBs and other items addressing affect, cognitions, or context.¹⁴

To extend this limited evidence, further studies focusing on measurement reactivity of EMA in STB-related research are needed. Including the participant perspective is necessary in order to address ethical concerns regarding the burden on this vulnerable group and to understand potential limitations regarding the validity of EMA data. Hence, it is recommended to combine quantitative and qualitative methods, because quantitative indices, such as completion rate or mean level of assessed items, do not fully represent the experiences of participants.¹⁴ In addition, several aspects such as positive or unexpected reactivity effects as well as changes in daily routines cannot be sufficiently captured by quantitative methods.³³

Objective and aims

The present study therefore sought to examine the experiences of individuals at high risk of STB during their participation in a study with a long observation interval employing data collection with EMAs and wearables. Specifically, we collected quantitative and qualitative data on measurement reactivity in a study consisting of two consecutive sampling periods (EMA 1: 3-week high-frequency sampling and EMA 2: 6-month low-frequency sampling). The study was explorative and we did not have any specific hypotheses.

The following research questions were addressed:

Ratings of reactivity:

Did the participants perceive changes in daily routines, mood, and contact with other people due to EMAs?

Did the participants perceive changes in daily routines and physical activity due to the wearable use?

Association of reactivity ratings with study and participant characteristics:

Did perceived reactivity differ according to the sampling scheme (EMA 1 or EMA 2)?

Did participants with high compliance differ in their average perceived reactivity ratings compared to participants with low compliance?

Participant experiences:

How many participants experienced or observed changes in their symptoms, mood and behavior during study participation? What sort of changes in their behavior or mood did the participants describe as a result of participating in the study?

Did the experiences differ according to the sampling scheme (EMA 1 and EMA 2)?

Would the participants participate again in a similar study?

Methods

Participants and procedure

The data was collected in the context of an ongoing multicenter study on risk factors for postdischarge STBs. Recruitment started in March 2022. By February 2024, N = 184 participants were included. All participants were recruited while they received inpatient psychiatric treatment. Inclusion criteria were being admitted due to suicide attempt/suicidal crisis, being aged 18 to 75 years, fluent in German and able to give informed consent. Exclusion criteria were admission due to other reasons, insufficient German language skills, acute psychosis, and cognitive impairment. Research staff regularly attended routine meetings in seven clinics in Leipzig and the Ruhr Area around Essen to identify inpatients fulfilling these criteria. Eligible inpatients were approached and informed about the study (see Figure 1 for participant flow). In case of inclusion, a baseline assessment was conducted. After that, participants were given a wearable (Polar Unite) and instructed to download a smartphone app on their private phones (Catalyst by Metricwire Inc.). The wearable was synchronized with the app. When the patients’ discharge from the psychiatric ward was scheduled, they informed the study team via the app about the date and were briefed about the EMA assessment/wearable use via telephone. During the briefing, participants were instructed on technical aspects of data collection (i.e. wearing and regularly charging the Polar Unite; use of the app, battery status, …), guidelines for the EMA survey (i.e. responding promptly, relevance of high response rate, and incentives) as well as on safety procedures (i.e. in-app emergency contacts, drop-out in case of severe symptom increase because of the repeated assessments). EMA 1 (high frequency) started 1 to 3 days prior to discharge and ran between 21 to 24 days. Participants received a morning questionnaire at 8:00 am and four semirandom prompts (main survey) between 8:00 am and 10:00 pm per day (minimum interval of 120 minutes between prompts). The morning questionnaire included three items on sleep and the main survey 31 to 33 momentary items that could be responded to in 1 to 3 minutes. EMA 2 (low frequency) followed EMA 1 and lasted for 26 weeks. During EMA 2, participants received the same 31 to 33 items of the main survey on two randomly chosen consecutive days per week. EMA surveys were accessible for 20 minutes and expired if they had not been started. Once started, the surveys had to be completed within 10 minutes. In EMA 1 and EMA 2, participants were reminded via push-notifications to put on the wearable in the morning and to synchronize and load the device in the evening. After EMA 1 and EMA 2, the study team conducted a clinical interview via telephone (including a short debriefing). EMA compliance rates were accessible to participants in the app. In addition, compliance rates were regularly monitored by the study team during the entire sampling period. Participants received feedback on their compliance via push notifications once a week in EMA 1 and every 2 weeks in EMA 2. In case of a substantial decrease of compliance or nonresponse, the participants were contacted to check if any technical issues had evolved (e.g. app updates, smartphone loss/change). Participants received up to 150 euros for their participation (depending on the compliance rates as well as completion of the interviews during the EMA 1 and EMA 2 period).

Figure 1.

Study flow. Notes: T0 = baseline assessment following inclusion, T1 = interview via telephone 21 to 24 days after discharge, T2 = interview via telephone 6 months after discharge.

For the present analysis, data of those participants, who had either finished EMA 1 or EMA 1 and EMA 2 and took part in an extensive debriefing session held in February 2023 (see Figure 1) were analyzed. Debriefing data is available for n = 92 participants (n = 41 EMA 1, n = 24 EMA 2, and n = 27 EMA 1 & EMA 2). Of those, n = 10 (10.9%) refused to wear the Polar Unite and n = 65 (70.7%) reported a lifetime history of suicide attempt. See Table 1 for further details on the sample.

Table 1.

Sociodemographic and clinical characteristics.

	Study sampleN = 184	Subsample with debriefing datan = 92
Mean age (sd)	37.5 (14.2)	35.9 (13.8)
Gender
Male	92 (50.0%)	38 (41.3%)
Female	85 (46.2%)	53 (57.6%)
Nonbinary	4 (2.2%)	1 (1.1%)
N/A	3 (1.6%)	0 (0.0%)
Clinical diagnoses^a
F1 mental and behavioral disorders due to psychoactive substance use	63 (34.2%)	29 (31.5%)
F3 mood (affective) disorders	134 (72.8%)	73 (79.3%)
F4 neurotic, stress-related and somatoform disorders	49 (26.6%)	25 (27.2%)
F5 behavioral syndromes associated with physiological disturbances and physical factors	9 (4.9%)	5 (5.4%)
F6 disorders of adult personality and behavior	45 (24.5%)	28 (30.4%)
F9 behavioral and emotional disorders with onset usually occurring in childhood and adolescence	8 (4.3%)	3 (3.3%)
> 1 Diagnosis	96 (52.2%)	51 (55.4%)
N/A	11 (6.0%)	0 (0.0%)
Inpatient treatment
After suicide attempt	75(40.8%)	33 (35.9%)
After suicidal crisis	7 (8.2)	59 (64.1%)
N/A	2 (1.1)	0 (0.0%)

All diagnoses drawn from the medical records at the psychiatric wards.

The responsible ethics committees at the two study sites approved the study protocol (University of Leipzig/Medical Faculty no. 382/17-ek and University of Duisburg-Essen no. EA-PSY19/23/03102023). All participants gave written informed consent prior to participation. For a detailed description of the study procedure, as well as full list of EMA items see https://osf.io/axnws.

Measures

In the debriefings after EMA 1 and EMA 2, participants were invited to share their experiences and thoughts on study participation using several quantitative items and open questions. The debriefings were conducted via telephone and all questions were self-designed by the study team. The study staff recorded all answers in writing. Items and questions are available in the supplementary material. In the present analysis, we considered those items asking about the impact of the EMA survey and of the wearable on daily routines, mood, social contacts, and physical activity, respectively. Participants were asked to rate the reactivity impact (“Have the daily app questions had an impact on your daily routines, your mood or your contact with other people?”, “Did the use of the Polar watch influence your daily routines or your physical activity level?”) on a 7-point Likert scale ranging from −3 (very negative impact) to +3 (very positive impact). In addition, they were asked if they experienced any changes as a consequence of study participation. If the participants affirmed such an impact, they were asked to specify this in more detail using an open question (“What did you experience or observe?”). After EMA 2, the participants were asked, if they would participate in such a study again (rating on 7-point Likert scale from 1 “definitely” to 7 “definitely not”). Participants’ responses were documented by the research team.

EMA data was preprocessed following the recommendations and template of Revol et al.³⁴ (see https://osf.io/25utw/ for the preprocessing report). Compliance rates were separately calculated for EMA 1 and EMA 2. In EMA 1, participants received up to 96 prompts. The average compliance was 57.4% in EMA 1 (range 0.1% to 100%, SD = 25.6). In EMA 2, participants received up to 208 prompts. The average compliance was 40.5% in EMA 2 (range 0% to 100%, SD = 28.3).

Statistical analyses

Quantitative analyses: We quantitatively evaluated the responses to the debriefing items. Mean values were compared between participants with low versus high compliance using t-tests for independent samples. Participants were assigned to the low and high compliance group, respectively, based on a median split (median^EMA1 = 0.57 and median^EMA2 = 0.34). Analyses were carried out separately for EMA 1 and EMA 2. We conducted our analyses using SPSS 29. Missing data was not imputed.

Qualitative analysis: To analyze the responses to the open questions, we used the content-structuring qualitative content analysis.³⁵ Responses were documented by the research team (either writing down the statements using direct quotes or eventually using bullet points). Quotes and bullet points were summarized in excel sheets and then coded by employing consensual coding according to Hopf and Schmidt.³⁶ The main categories were derived a priori from the study items (e.g. mood, daily routines, etc.). This was followed by coding to inductively developed subcategories, which two of the authors performed independently of each other (LB and TH). The categories and codes were then collaboratively discussed and iteratively revised by LB, LS, and TH.

Results

Impact ratings of EMA surveys and wearable use on mood, daily routines, and physical activity

Participants, on average, reported no perceived change due to the high-frequency EMA assessments on their daily routines, mood, and contact in the EMA 1 phase. Mean ratings were close to zero for all items. If at all, a slightly negative impact on mood was perceived across all participants (see Table 2). As for the perceived change due to the wearable use on daily routines and physical activity, participants reported slightly positive changes with regard to average mean values.

Table 2.

Impact ratings of EMA surveys and wearable.

	Total		High compliance		Low compliance		t
	N	M (SD)	n	M (SD)	n	M (SD)	t
After EMA 1 (high frequency)
Impact of EMA survey on …
Daily routines	68	0.01 (1.06)	37	−0.03 (1.09)	31	0.06 (1.03)	−0.35
Mood	68	−0.21 (1.17)	37	−0.14 (1.16)	31	−0.29 (1.19)	0.54
Social contact	68	−0.02 (0.54)	37	−0.01 (0.71)	31	−0.03 (0.18)	0.15
Impact of wearable on …
Daily routines	58	0.40 (0.97)	34	0.38 (0.95)	24	0.42 (1.02)	−0.13
Physical activity	58	0.35 (1.00)	34	0.31 (0.97)	24	0.42 (1.06)	−0.40
After EMA 2 (low frequency)
Impact of EMA survey on …
Daily routines	49	0.06 (0.82)	32	0.16 (0.95)	17	−0.12 (0.45)	1.12
Mood	49	0.12 (1.11)	32	0.19 (1.06)	17	0.00 (1.23)	0.56
Social contact	49	0.06 (0.83)	32	0.06 (0.95)	17	0.06 (0.56)	0.15
Impact of wearable on …
Daily routines	41	0.18 (1.18)	25	0.56 (1.16)	16	−0.41 (0.99)	2.76**
Physical activity	41	0.46 (1.00)	25	0.64 (1.22)	16	0.19 (0.40)	1.71*

Notes. Participants rated the impact on a 7-point Likert scale from −3 (very negative) to 3 (very positive).

* p ≤ .05, ** p ≤ .01. EMA: ecological momentary assessment.

In the EMA 2 phase, participants reported no to slightly positive perceived changes due to the prompts on routines, mood, and contact as well as slightly positive perceived changes of the wearable use on daily routines and activity (see Table 2).

Differences between participants with high versus low compliance

There was no difference in perceived changes between participants with high versus low compliance in EMA 1. Participants with high compliance in EMA 2 rated the perceived changes due to the wearable use on routines and physical activity significantly more positively than participants with low compliance rates (see Table 2).

Participant experiences

The inductively developed categories addressed different domains (e.g. STBs, mood, cognitive processes such as rumination and introspection, behavioral changes) and directions of perceived changes (i.e. worsening or improving) (see Table 3).

Table 3.

Changes after EMA 1 and EMA 2 according to subcategories.

	After EMA 1 (n = 68)			After EMA 2 (n = 51)
	n	%		n	%
Reported changes
Increased suicidal ideation	1	1.5	Increased suicidal ideation	2	4.0
Increased suicidal behavior	1	1.5	Decreased suicidal ideation and behavior	2	4.0
Improved introspection and self-reflection	26	38.2	Improved introspection and self-reflection	22	44.0
Improved mood	1	1.5	Improved mood	1	2.0
Worsening of mood	9	13.2	Worsening of mood	9	18.0
Increased rumination	4	5.8
Changes in daily routine	2	2.9
Changes in contact with others	2	2.9

Categories were developed using consensual coding via content-structuring qualitative content analysis (Kuckartz, 2012).

EMA 1

When asked if they had experienced any changes in their behavior or mood as a result of taking part in EMA 1 (n = 68), 38% (n = 26) of participants reported that they had not noticed any changes. Responses of the remaining 62% of participants (n = 42) were assigned to subcategories (see Table 3). About one-third of participants stated that taking part in the study had improved their introspection and self-reflection. For example, one participant stated that the daily questions had made him/her more aware of his/her mood, which in turn increased mindfulness for himself/herself. Another participant commented that she/he had “dealt with [his/her] mental issues more, when I was feeling bad, I looked at it more rationally instead of freaking out straight away.” Yet, roughly 13% of participants (n = 9) reported that participation in the study had worsened their mood. One participant, for example, reported to have experienced a negative impact on mood and “things that [I] hadn’t thought about before came up: sadness, anger, humiliation.” Other participants stated that study participation led to increased rumination, specifically at the beginning of the study. As one participant put it, the days without questions after finishing EMA 1 had been more comfortable. Two individuals (2.9%) claimed that study participation resulted in changes to their daily routine (e.g. “when scheduling the day trying to take questionnaires into account, this was also constraining, but has now passed”) and changes in the contact with others (e.g. talking more openly with those around and therefore being more open with the mental disorder as a result of the daily app prompts, which was comforting because of the experience of a supportive environment). One person each stated that participation in the study had increased suicidal ideation (“rather thought about the possibility of suicide”) or suicidal behavior (“questions contributed to a suicide attempt”).

EMA 2

After EMA 2, participants were asked again whether they had experienced any changes in their behavior or mood as a result of taking part in the study (n = 51). About one-third of participant did not experience any changes (34%, n = 17). Again, almost half of the participants stated that they had experienced improvements in their ability for introspection and self-reflection (see Table 2). For example, one participant reported that the questionnaires had helped “to become more aware, but also to be able to distance myself from my own negative feelings” whereas another stated having learned to have a better perception of herself. Yet, almost one in five participants (18%) perceived a worsening of mood due to study participation (e.g. one participant explained that when he had been stressed, the app questions had burdened him even more by making him more aware of his negative feelings). Two individuals each explained having experienced an increase in suicidal ideation (e.g. after several days without prompts, the survey had been quite a downer and triggered suicidal ideation) or a decrease of STBs (e.g. “More sensitive of my behavior, being able to distract oneself from suicidal ideation and behavior”). One participant even indicated that there had been an improvement in mood (e.g. “more cheerful than usual”) as a result of participating in the study.

Future study participation

In the telephone interview following EMA 2, participants tended to indicate they would participate in a comparable study again (on a scale from 1 = definitely to 7 = not at all; M = 2.11, SD = 1.63).

Discussion

The present study sought to investigate the impact and experiences of patients at high risk of STB participating in a study with 6 months observation period incorporating EMA as well as wearables. It led to three key findings: (1) the perceived impact of the EMA surveys and the wearable on mood, daily routines, or social contacts based on participants’ reactivity ratings was small (irrespective of low vs. high sampling density and study duration). In contrast, the wearable use had a (positive) impact; (2) when asked with an open question, about two-thirds of participants reported individual changes (both positive and negative) due to study participation but would on average participate in comparable studies in the future; (3) quantitative and qualitative indicators provided diverging information and should be combined in studies examining measurement reactivity. In the following, we will address these findings in more detail and highlight strengths and limitations of the analysis.

Impact on mood, routines, and social contacts/physical activity

The reactivity ratings show that neither daily routines nor social contacts appear to be influenced by the prompts in EMA 1 and EMA 2. Only mood appears to be slightly negatively influenced by the prompts in EMA 1 (but not in EMA 2). Overall, these results suggest that measurement reactivity in terms of behavioral changes or worsening of mood seems to be negligible, if participants are asked to rate the perceived impact of the EMA survey retrospectively. This is in line with other studies looking into affective reactivity.^14,18,20 However, two aspects should be viewed critically. First, the bipolarity of the ratings might pose difficulties to the participants experiencing some aspects of mood, daily routines, etc. to be affected by the EMA and simultaneously some that are not. The observed ratings close to zero could indicate the best effort to convey the mixed experiences into a single score. Second, not all participants might be aware of subtle adaptations to everyday routines or disturbances of social contacts. Retrospective memory biases might lead to increased or decreased reporting of symptoms and other changes when participants are asked at the moment versus during the last weeks.^37,3 Yet, aligning with other studies highlighting a negative reactivity effect of EMA, the EMA surveys have a slightly negative perceived impact on participants’ mood.^14,22 This could reflect the burden/inconvenience of the repeated assessments or an increased awareness of one's negative mood. Regarding the impact of the wearable use, participants reported on average a positive impact on daily routines and physical activity at EMA 1 and EMA 2, which is in line with general findings in this field.^28,31

Associations with study and participant characteristics

The findings did not generally differ between the high- versus low-frequency sampling scheme/duration of EMA 1 and EMA 2. Out of all possible outcomes, mood appeared to be perceived the most reactive with prompts that are more frequent. The compliance (high vs. low) of the participants was not significantly associated with the reactivity ratings of EMA prompts. It might, however, play a role that participants feeling most distressed or affected by the EMA surveys had a higher chance of dropping out of the study (i.e. did not participate in the follow-up and debriefing session, suggesting a general compliance bias³⁸). It has been shown that higher psychopathological symptom burden is associated with increased reporting of iatrogenic effects (EMAs triggering or worsening SI)¹⁴ as well as with lower compliance and retention rates in general.²⁶ Since the available clinical diagnoses of the participants were reflecting the state at admission, we do not know which symptoms remained, remitted or re-occurred during the EMA phases and cannot test their association with perceived reactivity. It might be worthwhile for future studies to account for current symptom levels or additional engagement in treatment. However, participants with high compliance reported a positive impact of the wearable use on daily routines and physical activity. Most likely, getting feedback on physical activity level has led to a positive feedback loop and supported a positive reactivity effect related to the use of wearables for some participants. Getting individual feedback has been shown to increase engagement with the study³⁹ and is known to promote self-awareness.¹² The wearable might provide this opportunity (somewhat unintentionally). The functionalities of the device (e.g. indicators of activity or sleep) were accessible to the participants and might have been used by those generally showing interest in it. When participants perceive the use of the wearable as inconvenient or not interesting,³¹ they might dismiss the wearable during the study or do not consent to wear it upon inclusion in the study. Indeed, 10.9% of participants decided against the use of the wearable. Again, participants with higher psychopathological symptom burden might be overwhelmed by regularly charging, wearing, and synchronizing the device or solving technical difficulties.

Participant experiences

At both follow-ups, about two-thirds of the participants indicated that they had experienced a change in their behavior or mood as a result of study participation. And 38% (34%) of participants were not aware of any changes in their behavior or mood at debriefing following EMA 1 (EMA 2). This rate is considerably lower than in previous studies (e.g. 60% reporting neither positive nor negative effects of EMA in reference [14]) One possible explanation is provided by the composition of our sample. In comparison to other EMA studies, we specifically recruited participants at high risk of STB. Since the level of psychopathological symptoms appears to influence affective reactivity, it is conceivable that severely burdened participants may experience the EMA as more stressful and burdensome than respondents in other studies (experiencing predominately SI, with lower rates of lifetime suicide attempts).¹⁴ At EMA 1, a majority of participants described having experienced an improvement of introspection and self-reflection (63%), a worsening of mood (22%), or increased rumination (10%). At EMA 2, participants mainly reported having experienced an improvement of introspection and self-reflection (68%) and a worsening of mood (28%). The large proportion of participants who perceived an improvement in introspection and self-reflection during the study clearly represents reactivity as a result of participating in the study and has been shown before.⁴⁰ Individual quotes highlight that the increased awareness can be experienced both, as beneficial or burdensome by participants. With 22% to 28% of participants experiencing a worsening of mood, the results are in line with other studies reporting a share of 9% to 22% of participants feeling distressed or burdened by EMA.^41,14,21,22 In addition, increased self-reflection my even increase the likelihood of reporting reactivity effects. Becoming more aware of one's own mood could lead to a higher sensibility for the detection of the effects of EMA (i.e. noticing mood changes when an EMA survey is prompted).

Even though these were only a few individual cases, some people (5%–6%) stated that participation in the study had led to a worsening of suicidal thoughts and behavior. EMA studies focusing on iatrogenic effects in the assessment of suicidal ideation and behaviors concluded that repeated questions about suicidal ideation do not increase their occurrence or intensity.^19,18 However, in an EMA study across 21 days,¹⁴ as many as 18% of participants subsequently reported that the questions had triggered suicidal thoughts. Since individual participant experiences do not necessarily correspond to statistical models examining reactivity effects (i.e. by examining altered response patterns),^2,33 it seems worthwhile to address both aspects in future research. Since different experiences in relation to reactivity can exist simultaneously, questions, and items intending to asses it should be designed very carefully. Since specifically in participants at high risk of STB, researchers should carefully balance the potential burden of study length and assessment intensity with the expected gain in knowledge. Besides the absence of empirically observed measurement reactivity in terms of symptom or mood worsening and no indication of relevant adaptions of daily routines based on quantitative data, EMAs over a long observation interval do exert a reactivity effect on a relevant amount of participants if qualitative data is considered. In this regard, ethical considerations as well as possible effects on data quality should always be considered when designing a study.^12,15 We strongly recommend to inform about (rare) potential negative and positive individual effects in the study information of future studies.

Limitations and strengths

The study was not specifically designed to examine measurement reactivity and is thus explorative in nature. Reactivity was assessed using items and questions that were self-designed by the authors and have neither been pilot-tested nor validated before. The wording and the rating scale used could have affected the findings in the quantitative analysis (i.e. close to zero ratings in the items reflecting the participants’ efforts to convey mixed experiences in one score). Moreover, the relatively small sample size in the debriefings limits the power to detect effects. Most importantly, the drop-out of participants during the study represents a possible selection bias, that is, the participants experiencing the most positive or negative reactivity effects might have refrained from completing the study. For this reason we assume that reactivity might be underestimated in our study (as well as in other studies in the field) due to a compliance bias.³⁸ Accordingly, generalizability of the findings is limited. Since the reactivity ratings of the EMA prompts and the wearable as well as participant experiences were assessed retrospectively, recall biases might have confounded the findings.^3,6 Future studies would benefit from prompt-based measures of reactivity, objective assessments of behavior and the inclusion of experiences from participants that had dropped-out of the study to enhance our understanding of these processes. The experiences of the participants were further not recorded verbatim, but documented by the study team and might have been shaped by their view and interpretation.

Beyond the limitations mentioned above, the study has some major strengths, for example, the combination of quantitative and qualitative data and the nature of the sample (i.e. representing a severely burdened clinical population with high risk of STB). To our knowledge, this study is the first that particularly focusses on the effects of EMAs and wearable use over weeks and months in a sample at high risk of STB and provides unique insights into measurement reactivity and participant experiences in this context.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251339265 - Supplemental material for Promising or discouraging? Potentials and reactivity of real-time data collection in monitoring suicide-related thoughts and behaviors over weeks and months. Participants’ views on ecological momentary assessments and wearable use

Supplemental material, sj-docx-1-dhj-10.1177_20552076251339265 for Promising or discouraging? Potentials and reactivity of real-time data collection in monitoring suicide-related thoughts and behaviors over weeks and months. Participants’ views on ecological momentary assessments and wearable use by Lena Spangenberg, Luise Böhler, Tina-Marie Hoke, Jana Serebriakova, Jannik Eimen, Thomas Forkmann, Maria Strauss, Katarina Stengler and Heide Glaesmer in DIGITAL HEALTH

Footnotes

Acknowledgements

We express our gratitude to all participants and the staff of the clinics where we conducted the study. In addition, we want to thank the student assistants and research assistants (particularly Nina Hallensleben, Antje Schönfelder, and Cora Spahn) involved in data collection.

ORCID iDs

Lena Spangenberg

Tina-Marie Hoke

Jana Serebriakova

Jannik Eimen

Thomas Forkmann

Heide Glaesmer

Ethical considerations

The ethic committees approved the study protocol (University of Leipzig: no. 382/17-ek, University of Duisburg Essen no. EA-PSY19/23/03102023).

Consent to participate

All participants provided written informed consent prior study participation.

Author contributions

LS, LB, and HG involved conceptualization; LS, LB, and TH in formal analysis; LB, TH, JS, JE, MS, and KS in investigation; LS, TF, and HG in methodology; LS and LB in writing—original draft; and TH, JS, JE, TF, MS, KS, and HG in writing review and editing.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Deutsche Forschungsgemeinschaft (grant number FO 784/8-1, GL 818/8-1, SP 1556/5-1).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

Data is available upon reasonable request from the authors.

Supplemental material

Supplemental material for this article is available online.

References

Spangenberg

Forkmann

Glaesmer

. Investigating dynamics and predictors of suicidal behaviors using ambulatory assessment. Neuropsychiatrie 2015; 29: 139–143.

Kivelä

van der Does

WAJ

Riese

, et al. Don’t miss the moment: a systematic review of ecological momentary assessment in suicide research. Front Digit Health 2022; 4: 876595.

Gratch

Choo

T-H

Galfalvy

, et al. Detecting suicidal thoughts: the power of ecological momentary assessment. Depress Anxiety 2020. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2020-36808-001&site=ehost-live.

de Beurs

Kirtley

Kerkhof

, et al. The role of mobile phone technology in understanding and preventing suicidal behavior. Crisis 2015; 36: 79–82. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2015-25151-001&site=ehost-live .

Kleiman

Glenn

Liu

. The use of advanced technology and statistical methods to predict and prevent suicide. Nat Rev Psychol 2023; 2: 347–359.

Myin-Germeys

Kasanova

Vaessen

, et al. Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry 2018; 17: 123–132.

Wastler

Bryan

. Suicide attempts among adults denying active suicidal ideation: an examination of the relationship between suicidal thought content and suicidal behavior. J Clin Psychol 2022; 78: 1103–1117.

Bernert

Hom

Iwata

, et al. Objectively assessed sleep variability as an acute warning sign of suicidal ideation in a longitudinal evaluation of young adults at high suicide risk. J Clin Psychiatry 2017; 78: e678–e687.

Littlewood

Kyle

Carter

L-A

, et al. Short sleep duration and poor sleep quality predict next-day suicidal ideation: an ecological momentary assessment study. Psychol Med 2019; 49: 403–411. https://www.cambridge.org/core/product/EE235427D1F01BF5A0396D5BF30C6039 .

10.

Sheridan

Baker

Dehart

, et al. Heart rate variability and its ability to detect worsening suicidality in adolescents: a pilot trial of wearable technology. Psychiatry Investig 2021; 18: 928–935.

11.

Coppersmith

DDL

Dempsey

Kleiman

, et al. Just-in-time adaptive interventions for suicide prevention: promise, challenges, and future directions. Psychiatry 2022; 85: 317–333.

12.

Barta

Tennen

Litt

. Measurement reactivity in diary research. In: Handbook of research methods for studying daily life. New York, NY: The Guilford Press, 2012, pp.108–123.

13.

Bos

. Ecological momentary assessment as a clinical tool in psychiatry promises, pitfalls, and possibilities [Thesis fully internal (DIV), University of Groningen]. University of Groningen, 2021.

14.

Kivelä

Fiß

van der Does

, et al. Examination of acceptability, feasibility, and iatrogenic effects of ecological momentary assessment (EMA) of suicidal ideation. Assessment 2024; 31(6): 10731911231216053.

15.

Nock

Kleiman

Abraham

, et al. Consensus statement on ethical & safety practices for conducting digital monitoring studies with people at risk of suicide and related behaviors. Psychiatr Res Clin Pract 2021; 3: 57–66.

16.

Ram

Brinberg

Pincus

, et al. The questionable ecological validity of ecological momentary assessment: considerations for design and analysis. Res Hum Dev 2017; 14: 253–270.

17.

Bryan

Wastler

Allan

, et al. Just-in-time adaptive interventions (JITAIs) for suicide prevention: tempering expectations. Psychiatry 2022; 85: 341–346.

18.

Husky

Olié

Guillaume

, et al. Feasibility and validity of ecological momentary assessment in the investigation of suicide risk. Psychiatry Res 2014; 220: 564–570.

19.

Coppersmith

DDL

Fortgang

Kleiman

, et al. Effect of frequent assessment of suicidal thinking on its incidence and severity: high-resolution real-time monitoring study. Br J Psychiatry 2022; 220(1): 41–43.

20.

Law

Furr

Arnold

, et al. Does assessing suicidality frequently and repeatedly cause harm? A randomized control study. Psychol Assess 2015; 27: 1171–1181.

21.

Rogers

. Feasibility and acceptability of ecological momentary assessment in a fully online study of community-based adults at high risk for suicide. Psychol Assess 2021. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2021-61470-001&site=ehost-live.

22.

Forkmann

Spangenberg

Rath

, et al. Assessing suicidality in real time: a psychometric evaluation of self-report items for the assessment of suicidal ideation and its proximal risk factors using ecological momentary assessments. J Abnorm Psychol 2018; 127: 758–769.

23.

Czyz

Horwitz

Arango

, et al. Short-term change and prediction of suicidal ideation among adolescents: a daily diary study following psychiatric hospitalization. J Child Psychol Psychiatry 2019; 60: 732–741. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2018-47925-001&site=ehost-live .

24.

Czyz

Glenn

Arango

, et al. Short-term associations between nonsuicidal and suicidal thoughts and behaviors: a daily diary study with high-risk adolescents. J Affect Disord 2021; 292: 337–344. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2021-65105-044&site=ehost-live .

25.

Ammerman

. Using intensive time sampling methods to capture daily suicidal ideation: a systematic review. J Affect Disord 2021. https://www.sciencedirect.com/science/article/pii/S0165032721012039.

26.

Vachon

Viechtbauer

Rintala

, et al. Compliance and retention with the experience sampling method over the continuum of severe mental disorders: meta-analysis and recommendations. J Med Internet Res 2019; 21: e14475.

27.

Dejonckheere

Erbas

. Designing an experience sampling study. In: Myi-Germeys

Kuppens

(eds) The open handbook of experience sampling methodology: a step-by-step guide to designing, conducting, and analyzing ESM studies. 1st ed. Leuven: Independently published, 2021, pp.33–70.

28.

Maher

Arigo

Baga

, et al. Measurement reactivity in ecological momentary assessment studies of movement-related behaviors. J Meas Phys Behav 2024; 7: 12.

29.

Choudhury

Asan

. Impact of using wearable devices on psychological distress: analysis of the health information national trends survey. Int J Med Inf 2021: 156.

30.

Ryan

Edney

Maher

. Anxious or empowered? A cross-sectional study exploring how wearable activity trackers make their owners feel. BMC Psychol 2019; 7.

31.

Chum

Kim

Zielinski

, et al. Acceptability of the Fitbit in behavioural activation therapy for depression: a qualitative study. Evid Based Ment Health 2017; 20: 128–133.

32.

Chung

Ryan

Hadzi-Pavlovic

, et al. Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis. JAMA Psychiatry 2017; 74: 694–702.

33.

Eisele

Vachon

Lafit

, et al. The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment 2022; 29: 136–151.

34.

Revol

Carlier

Lafit

, et al. Preprocessing ESM data: a step-by-step framework, tutorialwebsite, R package, and reporting templates. Adv Methods Pract Psychol Sci 2024; 7(4).

35.

Kuckartz

. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung. Weinheim Basel: Beltz Juventa, 2012.

36.

Hopf

Schmidt

. Zum Verhältnis von innerfamilialen sozialen Erfahrungen, Persönlichkeitsentwicklung und politischen Orientierungen: Dokumentation und Erörterung des methodischen Vorgehens in einer Studie zu diesem Thema: Institut für. Heidelberg: Sozialwissenschaften der Universität Hildesheim, 1993. https://www.ssoar.info/ssoar/handle/document/45614 .

37.

Schuler

Ruggero

Mahaffey

, et al. When hindsight is not 20/20: ecological momentary assessment of PTSD symptoms versus retrospective report. Assessment 2021; 28: 238–247.

38.

van Berkel

Goncalves

Hosio

, et al. Overcoming compliance bias in self-report studies: a cross-study analysis. Int J Hum Comput Stud 2020; 134: 1–12.

39.

Folkersma

Veerman

Ornée

, et al. Patients’ experience of an ecological momentary intervention involving self-monitoring and personalized feedback for depression. Internet Interv 2021: 26.

40.

Kauer

Reid

Crooke

, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012; 14: 15–31.

41.

Jiang

Al-Dajani

King

, et al. Acceptability and feasibility of ecological momentary assessment with augmentation of passive sensor data in young adults at high risk for suicide. Psychiatry Res 2023; 326: 115347. https://www.sciencedirect.com/science/article/pii/S0165178123002974 .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB