Abstract
Researchers and clinicians working within the Diagnostic and Statistical Manual of Mental Disorders: Fifth Edition, Text Rev (DSM-5-TR) framework face a difficult question: what does it mean to have an evidence-based assessment of a nonevidence-based diagnostic construct? Alternative nosological approaches conceptualize psychopathology as (a) hierarchical, allowing researchers to move between levels of description and (b) dimensional, eliminating artificial dichotomies between disorders and the dichotomy between mental illness and mental well-being. In this article, we provide an overview of ongoing efforts to develop validated measures of transdiagnostic nosologies (i.e., the Hierarchical Taxonomy of Psychopathology; HiTOP) with applications for measurement-based care. However, descriptive models like HiTOP, which summarize patterns of covariation among psychopathology symptoms, do not address dynamic processes underlying the problems associated with psychopathology. Ambulatory assessment, well-suited to examine such dynamic processes, has also developed rapidly in recent decades. Thus, the goal of the current article is twofold. First, we provide a brief overview of developments in constructing valid measures of the HiTOP model as well as developments in ambulatory assessment practices. Second, we outline how these parallel developments can be integrated to advance measurement-based treatment. We end with a discussion of some major challenges for future research to address to integrate advances more fully in transdiagnostic and ambulatory assessment practices.
Keywords
The recent release of the Diagnostic and Statistical Manual of Mental Disorders (5th ed., text rev., DSM-5-TR; American Psychiatric Association, 2022) raises an important question for those working in mental health care: what does it mean to have evidence-based assessments of nonevidence-based diagnostic constructs? The DSM-5-TR has incorporated a variety of changes compared to Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) yet maintains its categorical representation of mental health disorders. DSM-based nosology has well-known shortcomings: it treats disorders as qualitatively different from one another and from mental health (Conway et al., 2022); diagnostic categories obscure significant within-disorder heterogeneity (e.g., Galatzer-Levy & Bryant, 2013); and comorbidity is the norm, not the exception (Kessler et al., 2005; Lenzenweger et al., 2007). These shortcomings have far-reaching consequences for research and assessment (Hyman, 2010). Alternative nosologies have recently emerged using empirically derived organization of mental health symptoms (Dalgleish et al., 2020; Kotov et al., 2017). These alternative nosological approaches have only seen partial uptake in the DSM-5 and DSM-5-TR, with the inclusion of the Alternative Model of Personality Disorders (AMPD) as an emerging model to use in future research. A parallel development has been underway with the increasing use of ambulatory assessment approaches in mental health research, with researchers emphasizing the importance of assessing mental health impairment in ways consistent with its manifestation in daily life (i.e., dynamically and across different contexts; Wright & Woods, 2020). The goal of the present review is to highlight the promise of alternative, transdiagnostic nosologies for psychopathology assessment, discuss how these nosologies can be integrated with rapidly advancing ambulatory assessment methods, and identify major obstacles on the horizon as the field continues to strengthen mental health assessment practices.
Advances in the Classification and Assessment of Mental Disorders
Motivated by the shortcomings of the dominant categorical taxonomy in the DSM, alternative approaches have emerged seeking to establish an empirically derived structure of psychopathology (Dalgleish et al., 2020). These efforts include the aforementioned AMPD, the National Institute of Mental Health’s Research Domain Criteria (RDoC; Insel et al., 2010), the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017), and various other more circumscribed transdiagnostic efforts. Approaches like HiTOP, the AMPD, and RDoC are complementary transdiagnostic efforts, and past work has emphasized the conceptual and empirical links between these models (e.g., Kotov et al., 2021; Michelini et al., 2021). So while we focus on the HiTOP framework due to its relevance for measurement-based care in applied settings, much of our discussion is also applicable to other transdiagnostic models. 1 Two features of the model are worth mention. First, HiTOP organizes psychopathology along continuous dimensions, reflecting decades of evidence showing that differences along these dimensions are a matter of degree as opposed to kind (Haslam et al., 2012, 2020). Second, HiTOP is hierarchical, allowing researchers to study features of psychopathology at varying levels of specificity. At the finest level, HiTOP provides basic descriptions of symptom components (e.g., aggression) and maladaptive traits (e.g., callousness), which differ from one another in terms of how they are expressed over time (DeYoung, Kotov, et al., 2022). One can move up the hierarchy to subfactors (e.g., antisocial behavior) or higher to the spectra level (e.g., Externalizing). At the highest level is what has been termed the p-factor, which has been argued to reflect the shared variance among all psychopathology features (Caspi et al., 2014). The dimensional, hierarchical description of psychopathology in HiTOP addresses important shortcomings of the DSM approach (namely, comorbidity and within-disorder heterogeneity). 2 Though a rapidly expanding literature has developed around HiTOP (see Kotov et al., 2021 for a recent review), a major goal of the HiTOP consortium has been to develop an empirically validated self-report instrument of the comprehensive model. Indeed, as Simms et al. (2022) note, without an adequate model-based assessment tool, HiTOP risks becoming an intellectual exercise without realizing its potential for clinical assessment and intervention.
Assessment of Transdiagnostic Dimensions of Psychopathology
Although the HiTOP model is relatively recent, it developed out of decades of research spanning multiple domains of clinical science. In fact, the philosophy underlying HiTOP is not new (Dalgleish et al., 2020); the difference between the HiTOP conceptualization of psychopathology and other comparable models is that HiTOP is simply broader in scope. For example, the Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders (Barlow et al., 2018), developed out of research showing a strong commonality between anxiety and mood disorders and the abundance of disorder-specific treatment protocols with seemingly trivial differences between them. Given the observed comorbidity among emotional disorders (e.g., major depressive disorder, panic disorder, and generalized anxiety disorder), the Unified Protocol conceptualized this set of mental health problems as phenotypic variation in the manifestation of a broader liability based on negative reactivity to emotional experiences (Wilamowska et al., 2010). The Unified Protocol has seen widespread uptake, both in terms of its conceptualization of emotional disorders, but more importantly, in treatment settings (see Carlucci et al., 2021; Cassiello-Robbins et al., 2020 for reviews). Thus, HiTOP can be seen as an extension and integration of various threads of past research focused on transdiagnostic approaches to psychopathology.
Relatedly, Simms et al. (2022) highlight a variety of existing, popular instruments that are considered “HiTOP-friendly” that, collectively, assess the major spectra of HiTOP. However, these existing measures assess HiTOP dimensions in a piecemeal fashion necessitating the development of an instrument that provides coverage of the entire model. Given the breadth of HiTOP, this is no small undertaking. Nonetheless, with the efforts of dozens of HiTOP consortium members, the HiTOP-Self-report measure (HiTOP-SR) will be the first empirically-developed self-report scale that provides coverage of the entire HiTOP model with a single instrument. 3 The initial version of the HiTOP-SR is being developed through three phases of data collection and analyses, with each phase guided by principles of valid scale development (Clark & Watson, 2019; Loevinger, 1957). These include substantive validity (conceptualizing the psychopathology dimension and developing appropriate items), structural validity (statistical and interpretative procedures that will produce homogeneous and differentiable scales), and external validity (empirical relations between the scales and relevant criteria). The HiTOP-SR has moved through these validity steps sequentially, with external validity being the next research target once the HiTOP-SR is finalized.
Aside from these steps to establish the psychometric validity of the HiTOP-SR, a unique aspect of the HiTOP measure development project is the transparency involved, given the scope and breadth of the project. In line with the movement toward greater transparency in psychological science (Nelson et al., 2018), particularly those surrounding measurement practices (Flake & Fried, 2020), the HiTOP-SR has been developed in the spirit of the broader open science movement. The accumulated data and methods will be transparently accessible to researchers and clinicians alike and available on publicly accessible platforms, which we see as a major strength of the HiTOP-SR development effort.
Translating Transdiagnostic Assessment into the Clinic
The focus of the Clinical Translation Workgroup within the HiTOP consortium has been to move the progress in assessing and conceptualizing psychopathology to the clinic (Kotov et al., 2022; Ruggero et al., 2019). Indeed, much progress has been made in translational efforts without relying on a HiTOP-specific assessment instrument. The HiTOP-Digital Assessment and Tracker (HiTOP-DAT; Jonas et al., 2022) incorporates advances in transdiagnostic assessment of psychopathology into measurement-based care and is currently in use in numerous clinics in the United States. The HiTOP-DAT comprises a selection of six HiTOP-friendly measures that were (a) free to use, (b) demonstrated empirical evidence of reliability and validity, and (c) had been administered to large, community-based samples which would allow for T-scores to be computed. The HiTOP-DAT thus provides clinicians and their patients with a profile of T-scores across a variety of relevant HiTOP dimensions, which can inform case conceptualization and treatment planning. As noted above, the hierarchical nature of HiTOP and its assessment allows clinicians to move up or down the hierarchy to best conceptualize and address a patient’s problems. It will be important to continue to examine the clinical utility of HiTOP assessments in applied settings, but recent evidence (Balling et al., 2023) from practicing clinicians had found that the HiTOP approach to case conceptualization of outpatient clinical cases was on equal footing with the DSM approach in terms of communicating with other mental health providers, but was favored over the DSM when it came to a variety of other criteria including descriptive utility and formulation of potentially effective interventions for the outpatient cases.
As with any translational endeavor, there are important practical concerns to consider when moving from the research context to real-world treatment and assessment contexts. One relevant concern is the time needed to complete a HiTOP-based assessment given their broad coverage. The estimated length of the initial HiTOP-SR is approaching 500 items—potentially too long for the practical demands of some clinics but comparable to other comprehensive self-report assessment instruments like the Personality Assessment Inventory (PAI; Morey, 2007) or the Minnesota Multiphasic Personality Inventory-3 (MMPI-3; Ben-Porath & Tellegen, 2020), which are comprised of 344 and 335 items, respectively. Nonetheless, developing a shorter version of the HiTOP-SR has been highlighted as an essential consideration for clinical translation efforts (Sauer-Zavala, 2022), and is a stated goal of the Measure Development Workgroup (Simms et al., 2022). In the meantime, the HiTOP-DAT comprises 405 items and takes approximately 40-60 minutes to complete. The HiTOP-DAT has been programmed to be administered in HIPAA-compliant Qualtrics or REDCap, and clinicians can email their patients a link to complete the assessment on patients’ own time while not in the clinic (if patients prefer). The HiTOP-DAT flags any elevations on critical items (i.e., suicide risk) and notifies the clinician immediately. In addition, after the completion of the HiTOP-DAT, clinicians are emailed the results in a tabular report which can be uploaded to electronic health record systems. Another positive feature of the HiTOP-DAT is that it can be used by any clinic free of charge.
A separate practical concern regarding HiTOP-based assessment in the clinic is that clinicians working with insurance companies for billing purposes will need to provide ICD codes. The HiTOP-DAT manual includes an appendix outlining an informal crosswalk between HiTOP-DAT profiles and ICD codes, which also vary in their specificity. Clinicians wanting to work at the broadest level of the hierarchy (i.e., the p-factor) may provide ICD code F99 (Mental disorder, not otherwise specified), for example. However, clinicians should also be aware that while a p-factor score can be provided, it does not necessarily lend itself to straightforward interpretation that is helpful in applied settings (Bonifay et al., 2017; Watts et al., 2020). Alternatively, clinicians can focus on more specific HiTOP subfactors such as antisocial behavior and provide the corresponding ICD code (F60.2 Antisocial Personality Disorder). Thus, the HiTOP-DAT provides a freely accessible and evidence-based assessment of the HiTOP approach to the conceptualization of psychopathology, while also addressing some of the practical concerns of treating patients needing diagnostic codes for insurance coverage.
Progress in the assessment and translation of the HiTOP model continues to advance in exciting ways. As its proponents recognize, however, the HiTOP model is descriptive. In treatment settings, the focus is frequently on dynamic processes that bring about the problems associated with psychopathology (Hopwood et al., 2022). Descriptive assessment tools can enhance treatment delivery by identifying those who need treatment, the appropriate level of care, and the area of dysfunction, but are not designed to capture the contextualized within-person dynamics underlying psychopathology that are commonly the target of intervention efforts. In other words, it is important that transdiagnostic assessment approaches move beyond profiles to processes. Ambulatory assessment tools are one promising avenue to develop the crosswalk between descriptive symptom profiles and within-person processes relevant to mental health problems.
Advances in Ambulatory Assessment Methods for Psychopathology
Ambulatory assessment refers to a family of data collection methods focused on capturing information from individuals’ daily lives (e.g., ecological momentary assessment, daily diaries, passive sensors; Trull & Ebner-Priemer, 2013). The rapid expansion of smart phone use has changed the landscape of ambulatory assessment and allowed these methods to proliferate in psychological research. The implications for measurement-based care are significant—the ability to assess mental health dynamics in real-time and in contexts where they are most consequential presents unique opportunities to explore how these novel assessment methods can enhance care (Fisher, 2015). For instance, asking individuals to report on their situation, thoughts, behavior, and emotions immediately before and after smoking, binge eating and purging, or cutting their skin may reveal important targets for disrupting these damaging behaviors. Nonetheless, ambulatory assessment methods also present unique challenges because they require more decisions about how the data will be collected and analyzed than typical global cross-sectional assessment instruments.
Ambulatory assessment is generally employed to assess individuals intensively and repeatedly in naturalistic settings (Moskowitz et al., 2009). Because individuals are to be assessed repeatedly over time, the assessor must decide on what has been termed the “three D’s” of an ambulatory assessment protocol: Density, Depth, and Duration (Kaurin et al., 2023; see Figure 1). Density refers to how frequently individuals will be assessed, such as daily sleep logs, several samples of emotions throughout the day, or near continuous tracking of location via GPS or heartrate via optical sensors. Depth refers to the number of questions or items administered at each assessment. The economy of assessments is at a premium due to concerns about burden. Finally, duration refers to how long a protocol lasts. In research settings, a few days to a few weeks is typical, but in clinical settings, considerably longer time periods might be employed. Consider the daily completion of diary cards in dialectical behavior therapy; these would be an example of a clinically deployed ambulatory assessment. Kaurin and colleagues (2023) found in reviewing the literature on ambulatory assessment of personality disorder that density was negatively correlated with depth and duration, whereas duration and depth were positively correlated. In other words, when investigators increase the number of times they sample individuals each day, they typically administer briefer surveys for less time. Long protocols typically employ only once-daily assessments, which are often longer. Even this briefest summary should make it obvious that applied ambulatory requires close consideration of protocol-design, perhaps more than other common assessment approaches. Additional considerations arise when interpreting the abundance of data generated by ambulatory assessment.
Considering the “Three D’s” of Applied Ambulatory Assessment.
The specific study design employed will have implications for the resulting data structure and how it can and should be analyzed. However, the resulting data is typically understood to have a nested structure or have more than one level at which it can be interrogated. For instance, sampling individuals multiple times per day provides momentary information (Level 1) that can be aggregated to get daily averages (Level 2), which can be aggregated again to get an individual’s average (Level 3). In turn, modeling approaches that can accommodate the nested structure of the data (e.g., multilevel models; Hoffman & Walters, 2022) are most common in research settings, and various advances in multilevel modeling techniques have allowed for different applied ambulatory assessment goals. Broadly, they fall into three categories: descriptive, explanatory, and predictive goals. Descriptive ambulatory assessment is most similar to cross-sectional clinical assessment and aims at outlining the kinds of problems a person is experiencing. One example would be tracking a person’s depressed mood multiple times per day for 22 weeks to examine its variability (or lack thereof) over time. Explanatory ambulatory assessment is focused on identifying the potential processes that lead to the problems a person is experiencing. For instance, examining whether the report of an interpersonal conflict coincides with increases in depressed mood. Last, predictive goals center on forecasting when a problem will occur, with less emphasis on explanation. For example, leveraging data gleaned from an ambulatory assessment protocol to predict episodes of self-harming behavior before they occur so that reliable warning signs can be identified at the sample or person level. Each of these goals presents unique challenges and opportunities for integration with transdiagnostic assessment approaches.
Integrating Transdiagnostic and Ambulatory Assessment Practices
Whether the goal of an ambulatory assessment protocol is to describe, explain, or predict a person’s problems, progress in integrating ambulatory and transdiagnostic assessment approaches can borrow from the large literature on nomothetic and idiographic approaches to measurement (Beltz et al., 2016; Molenaar, 2004). Nomothetic assessment compares a person to others (typically matched on relevant characteristics such as age and sex) in a standardized manner, while idiographic approaches are individually-tailored to maximize the data’s relevance for the specific individual. Generally, clinical assessments are nomothetic in nature and focus on establishing the relative standing of a person compared to others for clinically relevant symptoms. Despite their shift away from a categorical focus, transdiagnostic measures like the HiTOP-DAT still take a nomothetic approach to guide decision-making within treatment. Applied ambulatory assessment can be used for both nomothetic and idiographic purposes, though it tends to be person-specific in its focus. A full overview of the history and relative strengths of the idiographic and nomothetic assessment approaches is beyond the scope of the present article (see Haynes et al., 2009; Molenaar, 2004 for reviews), but we emphasize that psychopathology assessment can be enhanced by drawing on the respective strengths of the nomothetic and idiographic measurement traditions. Below, we highlight research that has demonstrated how common ambulatory assessment goals (description, explanation, and prediction) can enhance or complement cross-sectional transdiagnostic assessment. Specifically, this research has explored whether applied ambulatory assessment may help with more accurate descriptions of people’s problems, more precise explanations of why people experience their problems, and better prediction of when people’s problems will occur.
Descriptive Ambulatory Assessment
Past work has demonstrated how transdiagnostic, cross-sectional assessment can be incorporated into routine clinical procedures (e.g., Ruggero et al., 2019). There has been less focus on how to use ambulatory assessment for descriptive purposes. However, ambulatory assessment naturally fits within transdiagnostic frameworks given its focus on dynamic processes that unfold over time (e.g., affect patterns) as opposed to focusing on any particular disorder.
The overview and case-example demonstration of merging idiographic and nomothetic assessment approaches by Wright and Zimmermann (2019) is particularly informative. First, they use the recently developed Personality Dynamics Diary (PDD; Zimmermann et al., 2019) to assess behavior over the ambulatory assessment protocol. The PDD is an ambulatory assessment instrument comprising 30 items—10 items assess situational context and 20 items assess personality states and related behaviors. Importantly, the development of the PDD was directly informed by dimensional personality disorder models (i.e., the AMPD trait model) as well as taxonomies of different situational characteristics (Rauthmann et al., 2014), highlighting how ambulatory assessment can be strengthened by nomothetic data. Second, they demonstrate how person-specific statistics gleaned from ambulatory assessment can provide meaningful information about the course of psychopathology symptoms. Some of the person-specific statistics highlighted in the case example include the individual mean (iM), which represents an individual’s typical or baseline behavior over the ambulatory assessment period, and the individual’s standard deviation (iSD), which represents the individual’s general variability in behavior over the ambulatory assessment period. Though these are two simple descriptive statistics derived from ambulatory assessment data (see Table 1 in Wright & Zimmermann, 2019) for a list of other common statistics that can be informative for treatment], they provide insights that might otherwise be less accurate if asked about once a week during treatment sessions.
Summary of Three Goals for Future Research on Applied Ambulatory Assessment
Note. HiTOP-DAT = Hierarchical Taxonomy of Psychopathology-Digital Assessment and Tracker.
Other descriptive ambulatory assessment work has used structural analyses to provide insights into variability of symptom organization. For example, Fisher (2015) used individually-focused factor analysis to examine how anxiety symptoms were organized for ten patients with a principal diagnosis of generalized anxiety disorder. Fisher (2015) found important areas of overlap among patients (e.g., most patients’ factor solutions contained a factor comprised of symptoms related to worry), but also notable heterogeneity in factor loadings and overall factor structure. Moreover, only two participants exhibited factor structures consistent with the diagnostic criteria for generalized anxiety disorder (i.e., a latent factor related to worry and factors representing at least three of the six other generalized anxiety disorder criteria).
Explanatory Ambulatory Assessment
Descriptive ambulatory assessment allows for more accurate capture of relevant emotional and behavioral patterns over time and can provide the initial steps toward explaining people’s problems (Wright & Kaurin, 2020). Yet description is rarely the end goal in treatment. Researchers and practitioners are most interested in understanding why people experience their problems, in the hope of intervening. As ambulatory assessment tools have advanced, so too have the methods used to analyze the data. These methodological advancements have paved the way for more in-depth person-centered statistics derived from ambulatory assessment data. Some of the more popular approaches in explanatory ambulatory assessment include network-based methods that allow for the estimation and visualization of the various connections in a person-specific network of symptoms (e.g., Epskamp et al., 2012; Lane & Gates, 2017). In turn, person-specific networks provide meaningful information about both contemporaneous (relations between symptoms at a given assessment point) and lagged effects (relations between variables across assessments) in ambulatory assessment data. The contemporaneous and lagged relations can help identify meaningful symptom patterns relevant to treatment that might otherwise go unnoticed.
Typically, these person-specific networks serve both descriptive and explanatory purposes. For example, both Wright and Zimmermann (2019) and Fisher (2015) moved beyond individually-focused descriptive statistics to discuss explanatory links between individuals’ symptoms. Fisher (2015), for instance, used dynamic factor analysis to identify lagged effects among anxiety symptoms. The results highlighted important heterogeneity in how anxiety symptoms related to one another over time across patients (e.g., for some patients, avoidance was positively related to subsequent anxiety symptoms, while for others avoidance was negatively related to subsequent feelings of distress and fatigue).
Other recent research has sought to directly incorporate explanatory ambulatory assessment into treatment delivery. Frumkin and colleagues (2022) examined the feasibility and potential utility of incorporating person-specific models derived from ambulatory assessment data into treatment for 12 psychotherapy patients, many of whom had multiple mental health diagnoses. For one exemplary patient, their personalized symptom network identified a feedback loop between difficulty concentrating and negative affect. Specifically, negative affect predicted difficulty concentrating and difficulty concentrating predicted negative affect. In addition, the patient showed strong autoregressive paths (i.e., symptoms predicted themselves over time) for negative affect and dissatisfaction with appearance, meaning that these symptoms tended to last over three-hour time periods. These data provided treatment-relevant information for the patient that would be difficult to attain using cross-sectional or interview-based assessments. Interestingly, Frumkin and colleagues (2022) also found that patients’ own predictions about how their symptoms would be related within and across time showed little relation to their empirical symptom networks, highlighting the need for future research to better understand the lack of concordance between patient expectations about their symptoms and the empirical models.
Scholten and colleagues (2022) used ambulatory assessment to complement and enhance a functional analysis of three patients’ behavior. Specifically, Scholten and colleagues were interested in examining whether ambulatory assessment could enhance a well-established approach to case conceptualization in therapeutic contexts (i.e., functional analysis). In their approach, networks were organized into behavioral clusters using principal components analysis. For the primary case presented, these included hopelessness, procrastination, coping, and avoidance. The relations among variables in these network clusters highlighted important contemporaneous features of the patient’s symptoms (e.g., his anxiety symptoms had strong physiological components) and specific triggers of symptoms (e.g., financial restrictions).
Predictive Ambulatory Assessment
When the goal of applied ambulatory assessment is to predict a relevant outcome before it happens (e.g., onset of a depressive episode, self-harming behavior), there is less emphasis on explanation. That is, there is less concern with understanding the mechanism(s) underlying why a variable or set of variables precede the relevant outcome. Instead, what matters is whether the variables increase predictive accuracy. To construct accurate predictive models, predictive ambulatory assessment frequently employs cutting-edge methods. These methods include the use of passive sensors (e.g., heart rate monitoring, GPS tracking), which represent a more recent development in the ambulatory assessment toolbox (Cohen, 2019; Mohr et al., 2017) and are of significant interest due to their potential predictive capabilities while placing little to no burden on patients. A related development due to the need for methods that can accommodate high-dimensional and complex data provided by passive sensors and intensive repeated measurement is the rapid proliferation of machine-learning methods, which generally aim at prediction over explanation (Dwyer et al., 2018; Yarkoni & Westfall, 2017).
One focus of predictive ambulatory assessment research has been the detection of early warning signs of mood episodes. Specifically, Van de Leemput and colleagues (2014) sought to identify a critical “slowing down” process that preceded more dramatic shifts in mood, such as the shift from baseline mood to a depressive episode. Using ambulatory assessment data, Van de Leemput and colleagues (2014) found that both the strength of the relation between emotions over time (i.e., the autoregression coefficient) and variability in the fluctuation of emotions was higher in individuals with upcoming transitions. When the transition involved a worsening of depressive symptoms, the early warning signals were strongest for negative emotions. When participants reported an improvement in depressive symptoms, these signals were strongest for positive emotions. These results pointed toward the potential use of ambulatory assessment to detect early warning signals that an individual is on the verge of a major transition in mood. However, recent research investigating early warning signals of depression has found evidence of low sensitivity (i.e., no early warning signs were detected in many people who nonetheless would go on to exhibit greater depressive symptoms; Smit et al., 2022), suggesting that more research is needed before clinical applications can be pursued.
Other work has relied solely on passive sensors to predict changes in a depressed mood. Jacobson et al. (2021) examined whether passive sensors could be used to accurately predict variability in depressed mood (i.e., feelings of sadness and loneliness) in a sample of college students reporting elevated depression symptoms at the baseline assessment. Using passively sensed features that included location information, heart rate, weather, and light level, machine-learning models were trained on 24 hours of data to predict subsequent self-reported mood in a one-hour window. Jacobson et al. (2021) employed both nomothetic and idiographic machine-learning models, where idiographic predictions (i.e., person-specific predictions of depressed mood) were informed by data from the entire sample, while weighting idiographic predictions more heavily. Predicted mood levels, using only data from passive sensors, were strongly correlated with self-reported mood, demonstrating that data from passive sensors alone could reliably predict changes in depressed mood over short time periods.
Three Goals for Future Research on Applied Ambulatory Assessment
We have highlighted a small but relevant set of studies focused on each of the three goals of applied ambulatory assessment, and showed how transdiagnostic, cross-sectional assessments can and have been integrated with these efforts. However, there are various empirical questions regarding applied ambulatory assessment that require further study. Much of the existing research in this area has been to demonstrate proof-of-concept and has highlighted important questions that require further study if we are to realize a more comprehensive integration of transdiagnostic cross-sectional and ambulatory assessment tools. Some of these questions have received more empirical attention than others. While our overview is not exhaustive, we highlight three prominent issues that are relevant to all applied ambulatory assessment goals (descriptive, explanatory, and predictive). A summary of these points is also provided in Table 1.
Developing Psychometrically Valid Ambulatory Assessment Tools
A major challenge for ambulatory assessment research concerns the lack of well-validated measures. While ambulatory assessment may be at the vanguard of psychopathology assessment, the long-standing demands of psychometric validity and reliability are still essential considerations (Wright & Zimmermann, 2019). There has been recent progress in developing psychometrically sound measures for ambulatory assessment, but they are largely the exception. In their review of ambulatory assessment practices in psychopathology research, Trull and Ebner-Priemer (2020) note that few self-report measures have been subject to rigorous validation. Instead, Trull and Ebner-Priemer (2020) note that researchers often select subsets of items from larger cross-sectional measures and adapt the instructions to fit the assessment time-frame (e.g., “over the last 15 minutes”). For example, items from the Positive and Negative Affect Schedule (PANAS; Watson et al., 1988) are frequently used in ambulatory assessment studies even though validation evidence for the PANAS has largely been derived from cross-sectional studies (but see Haney et al., 2023 for a recent exception), and these results do not necessarily generalize to the within-person level (e.g., the between- and within-person factor structures may differ; Cooke et al., 2022). Divergence in between- and within-person structure has also been demonstrated for transdiagnostic psychopathology dimensions (e.g., externalizing and internalizing dimensions; Wright et al., 2015). Relatedly, in ambulatory assessment research focused on psychopathology, initial work was heavily focused on diagnostic groups according to DSM criteria (e.g., borderline personality disorder has been a frequent focus of ambulatory assessment given its inclusion of affective lability; Santangelo et al., 2014) which, as previously noted, can present challenges for the generalizability and interpretation of results given the shortcomings of diagnostic classifications.
Importantly, the lack of well-validated self-reports for use in ambulatory assessment studies is not due to a lack of guidelines. There is substantial literature outlining the steps researchers can take to establish the psychometric properties of ambulatory assessment self-report measures (e.g., Calamia, 2019; Geldhof et al., 2014; Sadikaj et al., 2021; Wilhelm & Schoebi, 2007). There is also literature outlining methodological considerations such as the utility of single-item versus multi-item scales (e.g., Cloos et al., 2023; Song et al., 2023), random versus event-contingent surveys for capturing behaviors of interest (e.g., Himmelstein et al., 2019), and more general methodological considerations to ensure that a protocol is informative for a given focus (Kaurin et al., 2023). Researchers have also outlined how ambulatory assessment research, including scale development, can be conducted transparently and in line with principles of the broader open science movement (Kirtley et al., 2021).
There are noteworthy exceptions to the more general lack of well-validated ambulatory assessment measures. As previously mentioned, the PDD (Zimmermann et al., 2019) was developed using standard principles of psychometric validity while also using well-validated dimensional models of maladaptive personality and taxonomies of situational characteristics to inform the construction of the ambulatory assessment instrument. This work has been extended to a more extensive set of scales that can be flexibly employed to study HiTOP constructs in daily life (Wright et al., 2023; Wright & Simms, 2016) and are publicly available (https://osf.io/zyh9s). Jimenez et al. (2022) recently developed an ambulatory assessment instrument for internalizing symptoms by adapting items from the Dysphoria and Well-being subscales from the Inventory of Depression and Anxiety Symptoms (IDAS; Watson et al., 2007). The IDAS was chosen as the measure to adapt for EMA purposes because it (a) had strong evidence for its psychometric validity and (b) aligned with hierarchical and dimensional conceptualizations of psychopathology. Tomko et al. (2014) used data from participants with borderline personality disorder or a depressive disorder to construct the Momentary Impulsivity Scale, a psychometrically-validated measure designed to capture various aspects of impulsivity in daily life. There is also the Aggression Experience Sampler (Aggression-ES; Borah et al., 2021) and its abbreviated version (Aggression-ES-A; Murray et al., 2022), which are designed to validly assess different types of aggressive behavior (e.g., relational aggression, verbal aggression) in day-to-day life. These measures assess a variety of transdiagnostic symptoms and behaviors and provide valuable examples for researchers interested in using well-researched psychometric principles to inform the creation of ambulatory assessment measures.
Though these measures reflect progress toward establishing validated ambulatory assessment measures, they are limited in scope. To realize the potential of ambulatory assessment in applied settings, clinicians will need to be able to draw on validated ambulatory assessment measures suited to the demands of people seeking treatment. To facilitate integration with transdiagnostic cross-sectional assessment, ambulatory assessment tools will need to be developed within a framework that enables clinicians to easily move from cross-sectional assessment to a patient-centered ambulatory assessment protocol. Since patients present with a wide variety of mental health problems, it is likely that a “menu” of validated ambulatory assessment items or scales that clinicians and patients can choose from an à la carte fashion would best meet treatment demands.
For example, if a patient completes a baseline assessment suggesting that they experience problems in functioning resulting from a wide range of internalizing symptoms, the patient and clinician may adopt a relatively broad ambulatory assessment protocol that includes items related to anxiety, sadness, hopelessness, and fear. Another patient may have more circumscribed problems, such as acute social anxiety. For this patient, daily diaries at the end of each day detailing the patient’s emotions and behaviors in response to any social situations that occurred during the day may be most informative. The diversity and nuances across presenting problems go far beyond these relatively simple examples, however. Thus, publicly available ambulatory assessment item repositories that contain validated items or scales for transdiagnostic constructs of interest would be a valuable tool to help advance applied ambulatory assessment. Item repositories are being developed for ecological momentary items used in research (Kirtley et al., 2022), and item repositories for clinical settings are also available (e.g., see supplementary material in Bos et al., 2022). However, these item repositories largely contain items used in ambulatory research as opposed to an item repository for self-report items (a) adapted from validated transdiagnostic instruments and (b) have evidence for their psychometric validity. Like the item repository being developed by Kirtley and colleagues (2022), we recommend using publicly available platforms like the Open Science Framework (osf.io) which will help facilitate a freely available set of validated ambulatory assessment items for use in both research and applied settings.
Though we have focused largely on employing ambulatory assessment to capture pathological functioning, ambulatory assessment can also be used to track adaptive responding, patient strengths, and therapeutic skill use. Like the general benefits of ambulatory assessment, assessing adaptive responding in real-time can provide important insights into coping responses to adverse events as they unfold in daily life. This may help reduce error that arises in retrospective reports, or the “parking lot effect” where patients complete homework right before a treatment session. Though considerations regarding the “three D’s” (depth, density, duration) and psychometric validity also apply to adaptive behaviors, existing approaches (e.g., Dialectical Behavioral Therapy diary cards) could be incorporated into an ambulatory assessment protocol for a given patient.
Develop Evidence Base for the Benefits of Applied Ambulatory Assessment
As previously noted, emerging research on applied ambulatory assessment has largely been to establish proof-of-concept. The next step is to demonstrate if and how applied ambulatory assessment improves treatment-related outcomes. This is admittedly an ambitious goal—ambulatory assessment may be beneficial to many aspects of treatment, with each aspect requiring different types of evidence of efficacy. However, the way ambulatory assessment is leveraged will inform where to expect its impact, and in turn, design and assess studies accordingly. For example, descriptive ambulatory assessment may be beneficial to treatment planning, symptom targeting, or increasing the therapeutic alliance through collaboration on the assessment protocol. On the contrary, ambulatory assessment used in service of predicting behavior to provide just-in-time intervention may be more circumscribed in its benefits (e.g., reducing the frequency of a specific harmful behavior). With such wide-ranging potential benefits, progress toward establishing the evidence base of applied ambulatory assessment may occur in a piecemeal fashion, organized according to the specific goals of its use. In other words, descriptive, explanatory, or predictive impacts will each need further empirical support. Due in part to its novelty, there is limited systematic research available to make strong conclusions about the potential benefits of applied ambulatory assessment, and much of the research has focused on ambulatory assessment in service of intervention delivery.
In their scoping review of ambulatory assessment-based interventions in mental health, Balaskas and colleauges (2021) identified 54 studies that used ambulatory assessment (12 of which used passive sensors) to deliver some form of mental health intervention for a wide range of mental health problems (e.g., substance use, schizophrenia, and generalized anxiety). However, regarding personally-tailored intervention strategies, only three studies used collaborative intervention strategies developed during in-person sessions with a clinician, while automated tailoring through the use of predictive modeling was used in one study with some evidence of effectiveness in reducing symptoms of depression (Wahle et al., 2016). More generally, more research is needed before making firm conclusions about the strength of evidence for ambulatory assessment-based intervention (Jacobson et al., 2022). Outside of using ambulatory assessment for intervention delivery, other work has shown tentative benefits in using ambulatory assessment for descriptive, explanatory, and predictive aims. In work that has tested the feasibility and utility of applied ambulatory assessment (e.g., Frumkin et al., 2022), patients reported that they learned novel insights from the ambulatory assessment data. However, there is still a considerable distance to cover in providing systematic empirical evidence that applied ambulatory assessment enhances treatment outcomes.
Nonetheless, demonstrating the feasibility of these techniques has been critical, and it will be important to extend these results to more diverse patient populations, though initial research has been promising (Soyster et al., 2019). This includes relevant demographic factors such as low socioeconomic status and patients with ethnic or cultural minority identities. It also includes testing the feasibility of applied ambulatory assessment for patients with more severe or complex psychopathology and also for patients considered more difficult to work with in a therapeutic context (e.g., see K. C. Lewis & Ridenour, 2020). Thus, while initial proof-of-concept results for applied ambulatory assessment are promising, more work is needed to ensure that the feasibility of applied ambulatory assessment is generalizable to diverse populations of patients, a crucial consideration in the dissemination and implementation of new treatment techniques (Weisz et al., 2014). In addition to needing evidence of ambulatory assessments treatment efficacy, evidence of effectiveness is also needed, which requires that both clinicians and patients are willing and able to use applied ambulatory assessment.
Identify Barriers to Implementation for Patients and Clinicians
An obstacle to applied ambulatory assessment, and evidence-based assessment more generally, is that the majority of treatment providers do not regularly use empirically-validated assessment tools in treatment (C. C. Lewis et al., 2019). Reasons for the lack of uptake span multiple levels (i.e., patient/clinician concerns and organizational constraints) and include response burden for patients, time and cost burdens for clinicians, and lack of financial incentives from insurance coverage providers (C. C. Lewis et al., 2019). Though these issues have been identified as barriers to using cross-sectional assessment, they are likely relevant to applied ambulatory assessment. Researchers have also begun to explore whether patients and providers perceive unique barriers related to ambulatory assessment. Though few studies are available, current results suggest that the lion’s share of skepticism about the benefits of ambulatory assessment lies with clinicians as opposed to patients. For example, Soyster and colleagues (2023) surveyed 221 practicing and in-training mental health clinicians regarding perceived benefits and barriers to using ambulatory assessment. The authors also examined a variety of potentially relevant clinician factors (e.g., theoretical orientation and demographics). The results showed generally positive interest in using applied ambulatory assessment tools in treatment. The factors identified as dampening interest included having a primarily psychodynamic or psychoanalytic orientation, more clinical experience, and greater confidence in one’s clinical skills. Perceived barriers reported by the clinicians involved a lack of understanding or training in how to use ambulatory assessment tools and properly interpret the data they provide. The results in Soyster and colleagues (2023) are consistent with other research showing that while clinicians show interest in applied ambulatory assessment they are skeptical of its added utility, are concerned about its potential burden on both patients and clinicians, and have doubts about how to implement it effectively (Frumkin et al., 2022; Wright & Zimmermann, 2019).
These are essential concerns considering the complexity of ambulatory assessment data. Recommending the use of applied ambulatory assessment without the necessary infrastructure in place places unrealistic expectations on clinicians to properly collect, manage, and interpret these data in ways helpful for treatment. Indeed, many ambulatory assessment tools are largely developed for researchers who are familiar with intensive longitudinal assessment and are not necessarily geared toward clinicians and their patients. Thus, moving ambulatory assessment from bench to bedside will require the availability of user-friendly (for both clinicians and patients) software that can provide relevant, easy-to-understand information that is amenable to patient-centered assessment (i.e., can be modified to fit the needs of a given therapeutic context involving a clinician and patient). This is a formidable obstacle for employing ambulatory assessment at scale—ambulatory assessment software that can meet the needs of both patients and clinicians will require stakeholder involvement over the course of its development as well as interdisciplinary teams that can appropriately address the needs of stakeholders. To give a few examples, this will require that software capable of providing meaningful summary statistics or other data (e.g., strength of relations between symptoms) with little to no analyses needing to be conducted by clinicians, ambulatory assessment protocols can be designed with ease and in a flexible manner (i.e., personalized), potentially be integrated with electronic health record systems, and ensure that ambulatory assessment software keeps up with routine operating system updates (Piot et al., 2022; Wasil et al., 2020). Despite these obstacles, promising evidence of how these obstacles can be addressed can be found in the web-based Personalized Treatment by Real-time Assessment application (PETRA; Bos et al., 2022). The PETRA application was developed using an iterative process over three phases that included the use of focus groups of clinicians and patients as well as testing prototypes of the software in usability sessions to improve ease-of-use. Thus, the PETRA application provides an important blueprint for future work aiming to implement ambulatory assessment in applied settings.
For patients, available evidence supports the feasibility of ambulatory assessment in treatment and that it is not perceived to place an undue burden on patients (Soyster et al., 2019). Furthermore, data suggest that patients generally have favorable attitudes toward ambulatory assessment tools being used in treatment, more so than clinicians (Frumkin et al., 2022; Zimmermann et al., 2019). Thus, while both patients and clinicians tend to have some overlapping concerns regarding applied ambulatory assessment (e.g., participant burden and privacy concerns), clinicians appear more wary of its benefits compared to its costs. With some of these identified barriers in mind, efforts to incorporate applied ambulatory assessment will be most successful when stakeholders (patients, providers, organizations such as hospitals and relevant departments) are directly involved throughout the process of deploying ambulatory assessment methods in applied settings. Encouragingly, researchers have already begun to pursue these efforts, and valuable frameworks for involving stakeholders are outlined in Bos and colleagues (2022) and Soyster and Fisher (2019).
Current Recommendations
Our focus on future directions does not preclude the present use of transdiagnostic and ambulatory assessment approaches. This is particularly true regarding transdiagnostic clinical assessment, as many transdiagnostic assessment instruments have been used in clinical assessment and treatment planning for some time (e.g., the Unified Protocol; Barlow et al., 2018). Concerning the HiTOP model, various “HiTOP-friendly” assessment instruments are widely popular (see https://hitop.unt.edu/clinical-tools/hitop-friendly-measures), and there are more explicit HiTOP-focused assessments that are freely available and currently being used in applied settings (i.e., the HiTOP-DAT; Jonas et al., 2022). Furthermore, researchers have provided guidelines for how the HiTOP model can inform assessment in clinical and psychiatric settings (Kotov et al., 2022; Ruggero et al., 2019). Though researchers are continuing to explore the clinical applications and potential treatment benefits of the HiTOP model, emerging evidence suggests that practicing clinicians view HiTOP favorably compared to the dominant approach of the DSM (Balling et al., 2023).
How to currently implement ambulatory assessment in applied settings and appropriately tailor them to transdiagnostic case conceptualization is, admittedly, more speculative. Practical demands of ambulatory assessment (e.g., accessible and affordable software) are the largest obstacles. Relatedly, some clinical settings will possess the infrastructure to more effectively merge transdiagnostic and ambulatory assessment. For example, university- or hospital-affiliated clinics with ready access to researchers with domain-expertise in ambulatory assessment can more easily overcome the barriers to implementing ambulatory assessment in the clinic and also may provide institutional access to survey platforms (e.g., Qualtrics) that can be adapted for ambulatory assessment purposes. Based on the previously described aims of ambulatory assessment (descriptive, explanatory, and predictive), descriptive aims are currently most feasible. In fact, many clinicians may already be implementing versions of descriptive ambulatory assessment, through the use of tools like DBT diary cards or Cognitive Behavioral Therapy homework sheets are designed to be completed following the use of therapeutic skills discussed in treatment.
Shifting these assessments to be closer in time to the target behaviors or symptoms by using surveys completed via smartphones and producing descriptive summaries of the assessment target would bring these existing clinical approaches to be in line with descriptive ambulatory assessment. Survey platforms like Qualtrics allow for survey links to be sent via short message/messages service (SMS), which would allow for participants to complete surveys and provide data to the clinician without needing an EMA-specific application downloaded on their phone. The depth, density, and duration of these surveys would be decided by patient and clinician and amended as necessary. For example, surveys may be designed to provide descriptive information on a limited set of symptoms or single symptom (e.g., “How sad have you felt in the last hour?”). However, we note that even basic descriptive ambulatory assessment would require that patients have smartphones capable of receiving email or text-based survey reminders and that clinicians have access to survey-building platforms like Qualtrics. Furthermore, some degree of statistical summary and data visualization would be necessary, but this is unlikely to be as prohibitive as the data analytic skills needed for predictive or explanatory ambulatory assessment aims.
Conclusion
At the outset of this review, we noted that researchers and clinicians face a difficult question in clinical assessment—what does it mean to have an evidence-based assessment of a nonevidence-based diagnostic construct? Based on current standards, evidence-based assessment is a misnomer if the diagnostic construct being assessed has not withstood theoretical or empirical scrutiny (American Psychological Association, Presidential Task Force on Evidence-Based Practice, 2006). While many clinical assessment approaches remain practically important (e.g., helping identify individuals in need of care), developments in taxonomic research can improve clinical assessment by providing an empirically-based foundation for the targets of our assessment instruments. Similarly, developments in ambulatory assessment have opened the door for clinical assessment that more closely tracks the nature of mental health problems as they are experienced in the ebb-and-flow of daily life. The progress in both transdiagnostic and ambulatory assessment provides a promising path forward toward aligning theories of psychopathology and assessment practices.
Indeed, the parallel developments in transdiagnostic and ambulatory assessment have led to exciting developments in applied settings. However, applied ambulatory assessment and its integration with established assessment approaches present as many challenges as they do opportunities. In the present article, we have outlined emerging research showing the feasibility of merging cross-sectional transdiagnostic and ambulatory assessment approaches, while also outlining challenges facing researchers and clinicians wishing to incorporate these cutting-edge approaches into applied settings. Many of the questions posed here are actively being investigated, and the results will help fill many existing knowledge gaps in applied ambulatory assessment. We are optimistic that the field will rise to the many challenges that lie ahead to realize the potential of applied ambulatory assessment.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: In the manuscript, we highlight research scales that have been developed by the second author. However, these scales are publicly available and free to use. The second author does not collect any fees or direct monetary benefits from the use of these scales.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this research was provided by the National Institute of Mental Health (K01MH130746; PI: Vize) and the National Institute on Alcohol Abuse and Alcoholism (R01AA026879; PI: Wright).
