Abstract
There are a plethora of “flexibility” constructs and measures in psychology, but the extent to which they assess the same or different constructs, and whether flexibility and inflexibility are separate constructs (vs. extremes of the same bipolar continuum), remains underexplored. We examined the distinctiveness of seven different self-report measures of psychological (in)flexibility and cognitive flexibility using an online community (N = 465) and a chronic pain sample (N = 445). We analyzed the latent structure of these questionnaires using item-level exploratory structural equation modeling that controlled for measure-specific variance, and we tested these factors in relation to a range of mental health outcomes (concurrent validity) and discriminant validity measures. Findings indicate that psychological and cognitive flexibility questionnaires can be characterized at multiple levels, including six lower-order components that span individual measures and global factors that account for their shared variance. The six factors were broadly and uniquely associated with clinically relevant variables, including symptoms and well-being. We also found support for the notion that flexibility and inflexibility exist on a single bipolar continuum, rather than being characterized as separate. Implications for clinical assessment in research and intervention settings are discussed.
Introduction
A plethora of “flexibility” constructs are studied within psychology and health more broadly, with each construct highlighting the importance of being “flexible” to promote adaptability and mental well-being. For instance, a recent review found over 23 different “flexibility” terms across research areas (e.g., clinical psychology, neuropsychology, social psychology) and treatments (e.g., acceptance and commitment therapy [ACT], cognitive behavioral therapy [CBT]; Cherry et al., 2021). This reflects a lack of consensus on what flexibility is and thus continues to pervade the scientific literature, which hinders our ability to move the flexibility field forward. A recent review found three key aspects of flexibility were consistent across flexibility terms and definitions, which consider how a person (a) approaches interference or distress (i.e., facing discomfort), (b) takes action to address interference with goals or distress, and (c) takes action to meet a situational demand so they can pursue personal goals or values (Cherry et al., 2021). However, the degree to which various measures of flexibility target similar aspects remains to be investigated.
In this paper, we focus on two of the most commonly researched forms of flexibility: psychological flexibility and cognitive flexibility. Psychological flexibility is the key therapeutic target in ACT, and it is characterized by a reorientation toward positive-valued activities in the presence of distress (McCracken & Morley, 2014). Cognitive flexibility originated in the neuropsychological literature and reflects the ability to change one’s thinking or behavior to meet the demands of a situation (Lezak, 2004; Whiting et al., 2017). Cognitive flexibility is descriptively similar to psychological flexibility in addressing the intersection of distress, action, goals, and values to deal with environmental change (Cherry et al., 2021; Whiting et al., 2017), and it is a key focus of CBT (Barlow et al., 2017, pp. 78–92). However, despite these similarities, unlike psychological flexibility, cognitive flexibility does not explicitly include acceptance-based processes (Whiting et al., 2017). This suggests that there may be important differences and similarities in the measures of these flexibility constructs, which could have further implications for their assessment in these key treatments.
When examining the association between measures of cognitive flexibility and psychological inflexibility, research has found moderate to large correlations (r = −.49; Grant & Cassidy, 2022), indicating a moderate overlap between these constructs. Another study investigated whether cognitive flexibility, as measured by the Cognitive Flexibility Scale (CFS) and psychological inflexibility, as measured by the Acceptance and Action Questionnaire (AAQ-II), improved for people with irritable bowel syndrome after receiving CBT (Lackner et al., 2022). The study found that only cognitive flexibility showed improvements following CBT, and CFS and AAQ-II change scores were not associated, indicating that they may be distinct constructs, given that they did not change in tandem. However, as highlighted below, the AAQ-II does suffer from issues with discriminant validity. Moreover, a recent review by Cherry et al. (2021) highlighted items from the CFS and Cognitive Flexibility Inventory (CFI) that appeared to overlap with the psychological (in)flexibility. For instance, the item “I am capable of overcoming the difficulties in life that I face” appeared to overlap with psychological flexibility factors such as committed action. As such, questionnaires of psychological (in)flexibility and cognitive flexibility may be assessing the same constructs, but under different labels, as some items appear to target similar constructs. However, no research has yet examined whether there are empirical distinctions between psychological (in)flexibility and cognitive flexibility questionnaires at the item level.
Given the popularity of psychological and cognitive flexibility and their theorized prominent roles in efficacious treatments, it is important to understand their similarities and differences and any unique contributions they may have to enhancing mental health. This can further promote specificity in measuring flexibility to avoid issues related to the replication crisis (e.g., reducing the vast number of flexibility scales to choose from) and enabling more accurate assessment of these processes to understand causal mechanisms in efficacious treatments (Hodson, 2021).
Such distinctions are necessary to move flexibility research forward, as cognitive flexibility and psychological (in)flexibility questionnaires may be subject to the jangle fallacy, which involves erroneously assuming two measures assess different constructs because they are labeled differently, when they actually assess the same thing (Flake & Fried, 2020; Kelley, 1927). Such fallacies hinder the development of science, as they can result in redundant measures, parallel literatures, constructs that provide no unique information in explaining psychopathology, and difficulty in consolidating findings to refine treatments (Eronen & Bringmann, 2021; Gonzalez et al., 2021). To address this issue, the current study aims to empirically examine the structure of measures of psychological (in)flexibility and cognitive flexibility, as well as their construct validity. Below, we define psychological (in)flexibility and cognitive flexibility, identify issues in their measurement and structure, and then discuss the application of flexibility constructs to chronic pain populations, which is a focus of the current study.
What Are Psychological Flexibility and Cognitive Flexibility?
A systematic review of 220 articles found that the most prominent assessment strategy of psychological flexibility was the use of self-report measures created within an ACT framework (Cherry et al., 2021). In total, 27 different psychological flexibility measures and 13 different psychological inflexibility measures were identified, highlighting the diverse ways psychological flexibility has been measured. Psychological flexibility is defined as “contacting the present moment as a conscious being, fully and without needless defence—as it is and not as what it says it is—and persisting or changing a behavior in the service of chosen values” (Hayes, Strosahl, & Wilson, 2011, p. 96).
Psychological flexibility encompasses six key interrelated processes that have also been further grouped into three overarching constructs: acceptance and cognitive defusion (grouped together as openness), present moment awareness and self-as-context (awareness), and values and committed action (engagement; Hayes, Villatte, et al., 2011; McCracken, 2023). Acceptance entails willingness and psychological acceptance, where a person is willing to maintain contact and be non-judgmental with aversive internal experiences (Hayes, Strosahl, & Wilson, 2011; McCracken, 2023). Cognitive defusion is about seeing thoughts as passing events that are not necessarily accurate, helpful, or require action (McCracken, 2023). Present-moment awareness involves intentionally focusing on present-moment experiences (i.e., “the now”) and, if needed, switching attention between different events (McCracken, 2023). Self-as-context is an unseeable process where we view oneself as transcending one’s experiences by being an observer of those experiences, rather than being defined or limited by the contents of awareness (McCracken, 2023; Rolffs et al., 2018). Values are what one chooses to hold as important and serve to guide actions, while also providing a sense of meaning (McCracken, 2023). Committed action is the extension of values and entails persisting with behaviors that serve values, even in the presence of distress (McCracken, 2023). Conversely, psychological inflexibility—“the rigid dominance of psychological reactions over chosen values and contingencies in guiding action” (Bond et al., 2011)—is essentially the inverse of psychological flexibility: lack of present moment awareness, lack of contact with values, inaction, self-as-content, fusion, and experiential avoidance (Rolffs et al., 2018).
Cognitive flexibility involves being able to change behavior (e.g., thoughts or actions) to meet situational demands (Lezak, 2004; Whiting et al., 2017). For instance, a person with high cognitive flexibility is good at thinking of solutions and taking effective action in solving a problem because they can readily switch between different mental processes and generate new responses (Dajani & Uddin, 2015). There are many ways to conceptualize cognitive flexibility, and often, operationalizations have been unclear and/or inconsistent across studies (Dajani & Uddin, 2015). One conceptualization is that cognitive flexibility is one of three core executive functioning components that build on the other two core components (i.e., inhibitory control and working memory; Diamond, 2013). Within this framework, the most common method to assess cognitive flexibility is to use set-shifting or task-switching behavioral paradigms (Diamond, 2013).
More recently, self-reported measures of cognitive flexibility have also been developed to assess a person’s perceived ability to generate alternative ways of thinking (e.g., their perceptions of flexibility), as tasks assessing behavioral responses to a stimulus may be limited in that they do not capture spontaneous thoughts as they occur in daily life outside of the lab (Dang et al., 2020). Tasks assessing cognitive flexibility are also likely more suited to detecting brain abnormalities after a traumatic brain injury than assessing individual differences in healthy populations (Dennis & Vander Wal, 2010). Consistent with these distinctions, recent research has shown that tasks that assess cognitive flexibility and self-report measures of cognitive flexibility and psychological inflexibility (measured with the AAQ-II) are unrelated (Grant & Cassidy, 2022; Howlett et al., 2021, 2022, 2024). Given that performance-based tasks of cognitive flexibility are unlikely to be related to self-report measures of either psychological flexibility or cognitive flexibility, psychological (in)flexibility is measured by self-report (Rehfeldt, 2019), and self-report may better reflect daily life experiences, we used self-report measures of cognitive flexibility in the current study. This also allowed our study’s conclusions to be directly applicable and useful to clinical settings, given that self-report measures are more commonly employed in clinical practice due to their increased reliability, ease of administration, quick completion time, and centrality of subjective experience for therapeutic purposes (Corneille & Gawronski, 2024; Sunderland et al., 2019).
Importantly, Cherry et al. (2021) identified ten self-report measures of cognitive flexibility at the time of their review, but only two are frequently used and clearly measure “cognitive flexibility.” Other measures had low psychometric quality, were used only once or twice by other studies to assess cognitive flexibility, and/or were not initially designed for this purpose. For instance, the Attributional Style Questionnaire was designed to assess internal, stable, and global attributions to good and bad events (Peterson et al., 1982) and was not selected in the present study as it was rarely used as a measure of cognitive flexibility (only one article used it from Cherry et al., 2021, review), and it did not clearly measure “cognitive flexibility” (i.e., assessed attributional styles).
Issues in Measuring Psychological Flexibility and Cognitive Flexibility
There have been numerous challenges in measuring psychological flexibility. The AAQ-II (Bond et al., 2011)—the most prominent psychological inflexibility measure and the only measure until relatively recently—has been identified as having questionable psychometric properties, including poor discriminant validity with neuroticism/negative affect (Cherry et al., 2021; Francis et al., 2016; Ong et al., 2019, 2020; Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014). Some newer psychological (in)flexibility measures have been created to address these limitations and have shown good discriminant validity with distress (Francis et al., 2016; Landi et al., 2021; Ong et al., 2020). However, there is no consistent evidence that each of the 12 facets of psychological flexibility and inflexibility are empirically distinct, making it unclear which aspects of psychological flexibility or inflexibility (or both) are most critical in understanding symptoms and well-being (Christodoulou et al., 2023; McCracken, 2023).
In addition, it is important to note that measures of psychological flexibility tend to be positively-keyed or worded (e.g., agreement reflects greater flexibility), whereas measures of psychological inflexibility tend to be negatively-keyed or worded (i.e., agreement reflects a lack of psychological flexibility). Thus, it is possible that observed differences between these constructs emerge because constructs are keyed in the same direction (e.g., inflexibility would be more related to distress, whereas flexibility measures would be more related to positive functioning; e.g., Lindwall et al., 2012). In this case, observed distinctions may be an artifact of measurement and wording (Brown, 2015), rather than substantively meaningful.
Finally, there is little research that has directly examined whether psychological flexibility and inflexibility are best modeled as a single bipolar continuum or separate constructs. Such bipolar distinctions need to be statistically accounted for to avoid misleading conclusions about the direction and magnitude (Barchard et al., 2024) of associations between important clinically relevant outcomes. Ciarrochi et al. (2014) found distinct configurations of psychological flexibility and inflexibility, and similarly, Rolffs et al. (2018) identified separate psychological flexibility and inflexibility factors, though the factors were strongly correlated (r = −.74). However, no research has examined the structure of multiple measures of inflexibility and flexibility simultaneously or accounted for the impact of item wording/direction. Thus, further work is needed to better understand the structure and measurement of psychological (in)flexibility (i.e., whether inflexibility is distinct from flexibility beyond just keying/wording effects), as well as how it relates to other similar constructs such as cognitive flexibility.
Turning to cognitive flexibility, the most prominent self-report measure is the CFI (Dennis & Vander Wal, 2010). This measure was created to assess a person’s ability to explore, evaluate, and restructure maladaptive cognitions with more adaptive ones (i.e., engage in cognitive restructuring, see D. A. Clark, 2013). The Control subscale of the CFI assesses the tendency to believe you can control difficult situations, whereas the Alternatives subscale assesses (a) being able to see different events and behaviors as having multiple possible explanations, rather than rigidly assigning a single cause; and (b) identifying alternative solutions when presented with difficult situations (Dennis & Vander Wal, 2010).
The CFI improves upon the limitations of the CFS (Martin & Rubin, 1995), which is also frequently used but focuses more narrowly on cognitive flexibility in social situations, as opposed to being able to effectively engage in cognitive restructuring broadly (Dennis & Vander Wal, 2010). Both measures include reverse-keyed items assessing cognitive inflexibility, but there has been little focus on the issue of keying in cognitive flexibility questionnaires, or whether cognitive flexibility and inflexibility are distinct and should be assessed as such. As such, there is also a need to understand whether cognitive (in)flexibility lies on a continuum or if there is a separate cognitive inflexibility domain.
Examining the structure of psychological and cognitive flexibility measures jointly would (a) address the above issues about the subcomponents of each construct and (b) determine whether psychological (in)flexibility and cognitive flexibility are empirically distinct or actually reflect the same overarching construct (i.e., the jangle fallacy). This examination would bring greater clarity to the terminology, definitions, and measurement of flexibility, allowing for better progress and synthesis of research in this area.
Applications of Flexibility to Chronic Pain Populations
Although flexibility constructs have been applied transdiagnostically, they appear to have particular relevance for populations with chronic pain (McCracken & Morley, 2014). Chronic pain refers to pain that persists for more than 3 months (Treede et al., 2019), and it is among the top five leading causes of chronic disability in almost all high-income countries (Vos et al., 2017). Substantial life interference (disability) associated with persistent pain may reflect, in part, psychological inflexibility across multiple domains, including behavior (e.g., excessive avoidance of situations or activities associated with pain cues), cognition (rigid cognitive style characterized by pain-related rumination), and emotion (difficulties regulating emotion manifested in heightened anxiety, depression)—further driving increases in pain, pain-related disability, and distress (e.g., Leeuw et al., 2007).
To help counteract this cognitive-affective and behavioral avoidance and reduce pain’s impact on a person’s daily life, psychological flexibility and cognitive flexibility may be particularly relevant. Within ACT, which is often used for chronic pain management, psychological flexibility promotes acceptance of pain and the pursuit of valued activities, even in the presence of pain (McCracken, 2023). Similarly, cognitive flexibility plays a key role in CBT, with a focus on cognitive restructuring to help correct faulty thinking patterns (catastrophizing pain beliefs) that generally maintain the problem, as a “rigid mind-set” may overinflate the degree of danger and harm of pain (Lackner et al., 2024). As such, CBT teaches people to identify irrational thoughts and generate alternative, more constructive ways of thinking so that reactions to pain are more balanced (Lackner et al., 2024).
Recent investigations of psychological flexibility and inflexibility in chronic pain show that global psychological flexibility and inflexibility have small to large correlations with pain interference, work and social adjustment, and depression, but inflexibility demonstrated greater unique predictive power than flexibility for these outcomes (Sundström et al., 2023). Moreover, a meta-analysis found that psychological flexibility and inflexibility had moderate to large correlations with pain intensity, quality of life, functional impairment, and anxiety (rs = .26 to .57; Fang & Ding, 2022). However, psychological inflexibility had stronger associations with functional impairment than flexibility, although there were few studies measuring psychological flexibility (Fang & Ding, 2022).
Much of the research that has examined cognitive flexibility within chronic pain has used neuropsychological tasks and found that people higher in cognitive flexibility had lower levels of clinically-meaningful pain at both 6 and 12 months after participants had surgery (Attal et al., 2014). Little research has examined self-reported aspects of cognitive flexibility within chronic pain, though in the general population, cognitive flexibility is robustly associated with less anxiety and depressive symptoms (Dennis & Vander Wal, 2010; Howlett et al., 2024) and better coping (e.g., more problem-focused coping; Dennis & Vander Wal, 2010). One study found that the CFI total scores were unable to differentiate between people with and without chronic pain (Howlett et al., 2024). However, another study found that compared to psychological inflexibility, only cognitive flexibility showed improvements after CBT, and psychological flexibility and cognitive flexibility change scores were not associated (Lackner et al., 2022), indicating they may be distinct constructs.
Overall, it remains unclear whether psychological and cognitive flexibility or inflexibility is most strongly related to specific outcomes in chronic pain (Sundström et al., 2023), and therefore, it is uncertain which of these constructs (or their facets) drives associations with chronic pain and its treatment.
The Current Study
Research suggests that psychological (in)flexibility and cognitive flexibility are important for improving adaptability and well-being and reducing distress, especially within the context of chronic pain. However, to move forward theoretical models and interventions targeting flexibility, there is a need to understand whether these constructs are largely redundant with one another or distinct, and to clarify whether they differ in their associations with clinical outcomes such as distress and chronic pain. Such knowledge would allow for greater clarity in treatment targets, help synthesize the literature on these constructs, and may suggest improvements in their assessment.
The first aim of the current study was to determine whether cognitive flexibility self-report measures and psychological (in)flexibility measures are structurally distinct from one another (e.g., whether psychological flexibility and psychological inflexibility are distinct), in a general community sample and a sample with chronic pain. We accounted for measure-specific variance to examine the structure of the underlying constructs with less influence from the quirks and wording of individual measures (including keying in a single direction within a measure; e.g., Brown, 2015; Morin et al., 2020). A second aim was to examine the associations of the resulting flexibility factors with measures of distress/psychological symptoms, well-being, and pain outcomes to better understand the construct validity of these different components. This allows us to determine whether some aspects of flexibility constructs are more closely associated with specific outcomes relative to other aspects of flexibility, as well as their discriminant validity viz-a-viz closely related constructs (i.e., mindfulness, neuroticism, and self-reported executive functioning) and socially desirable responding.
Method
Participants
Participants in the United Kingdom were recruited through Prolific (https://www.prolific.com) to complete an anonymous online survey distributed evenly to males and females (i.e., a gender-balanced sample). We recruited a general (non-selected) community sample of adults and a second sample of adults with chronic pain. For both samples, the inclusion criteria included that participants primarily spoke and were fluent in English and were ≥18 years old. For the chronic pain sample, participants were invited to participate only if they (a) reported having pain for more than 3 months; (b) selected “yes” to a chronic pain screening question (i.e., “Throughout our lives, most of us have had pain from time to time [such as minor headaches, sprains, and toothaches]. Have you had pain other than these everyday kinds of pain?”); and (c) reported at least mild pain (score of 3 or greater) on the Brief Pain Inventory (BPI) Pain Severity subscale (Moore et al., 2013).
The average age was 41.14 years (SD = 13.39, Min = 19, Max = 75) for the community sample and 44.21 years (SD = 12.16, Min = 18, Max = 77) for the chronic pain sample. For an overview of all assessed demographics for both samples, see Table S1 in the Online Supplemental Material. For the community and chronic pain samples, sex and gender were evenly split between males and females, with few participants identifying as either non-binary or transgender or preferring not to answer. In both samples, most participants were heterosexual, White, had finished high school or a university degree, and were either employed part-time or full-time.
For the chronic pain sample, half the participants reported a chronic pain diagnosis, with most having pain duration greater than 2 years, and few having a compensation case related to their pain or inability to work due to pain or a related condition. The sample had various triggering events and comorbid health conditions. The most problematic primary pain site was the back/spine, but there was an extensive range of primary pain sites in the sample. Table S1 in the Online Supplemental Material provides further detail regarding pain-related sample characteristics.
Procedure
All participants completed anonymous surveys through Qualtrics (www.qualtrics.com). For the chronic pain sample, potential participants were reimbursed £0.14 for their time in completing the brief screening survey (M survey time = 50 s) before completing the entire survey. Both the community and chronic pain samples were required to read the participant information form and provide consent before completing the entire survey. Both samples were reimbursed £9.00/hour, with the chronic pain sample reimbursed ￡6.00 (M survey time = 38 minutes) and the community sample reimbursed￡3.75 (M survey time = 26 minutes) for their time given that the chronic pain sample completed additional pain questionnaires.
Measures
To assess psychological (in)flexibility and cognitive flexibility, we aimed to include all commonly-used measures that had evidence of adequate psychometric quality. Thus, we included all measures that had a quality rating of 8 (out of 12) or greater in the Cherry et al. (2021) review. Although the CFS, AAQ-II, and Open and Engaged State Questionnaire (OESQ) had a slightly lower quality rating, we administered them due to their frequent use in the literature, brevity, and to have adequate coverage of each construct. However, given serious concerns about the AAQ-II’s validity and properties (e.g., Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014), we did not include it in the structural analyses of flexibility measures. Rather, we examined its association with each of the factors to help characterize them.
See Supplemental Appendix 1 for a full description of each measure’s characteristics and psychometric properties, with brief details below. To assess cognitive flexibility, we administered the CFI (Dennis & Vander Wal, 2010) and the CFS (Martin & Rubin, 1995). To assess psychological (in)flexibility, we administered the AAQ-II (Bond et al., 2011), Comprehensive Assessment of Acceptance and Commitment Therapy-15 (CompACT-15; Hsu et al., 2023), the Multidimensional Psychological Flexibility Inventory Short Form (MPFI-24; Rolffs et al., 2018), OESQ (Benoy et al., 2019), and Personalized Psychological Flexibility Index (PPFI; Kashdan et al., 2020).
To characterize the factors, we examined their associations with clinically-relevant outcomes (concurrent validity). We used the Inventory of Depression and Anxiety Symptoms-II (IDAS-II; Watson et al., 2012), Big Five Inventory (BFI; John & Srivastava, 1999) to assess personality traits, Positive and Negative Affect Scale (PANAS; Watson et al., 1988), Mental Health Continuum Short Form (MHC-SF; Lamers et al., 2011) to assess wellbeing, and Brief Pain Inventory to assess pain severity (BPI; Cleeland & Ryan, 1994). We also administered the following chronic pain-specific questionnaires to the chronic pain sample only: Chronic Pain Acceptance Questionnaire-8 (CPAQ-8; Fish et al., 2010), Psychological Inflexibility in Pain Scale (PIPS; Wicksell et al., 2010), and the Pain Interference subscale of the Brief Pain Inventory (BPI; Cleeland & Ryan, 1994). All other measures were administered in both samples.
To assess the discriminant validity of the factors from closely-related constructs, we used the following questionnaires: Web Based Executive Function Questionnaire (Webexec; Buchanan et al., 2010) for executive functioning, the Five Facet Mindfulness Questionnaire-15 (FFMQ-15; Bohlmeijer et al., 2011) for mindfulness, and the Marlowe-Crowne Social Desirability Scale—Short Form (MCSDS; Reynolds, 1982) for social desirability.
Data Analysis
Analyses were performed using R (R Core Team, 2013) and RStudio (Posit Team, 2024), except that the Exploratory Structural Equation Models (ESEM; Asparouhov & Muthén, 2009) and SEM models were estimated with Mplus version 8.11 (Muthén & Muthén, 1998/2017) using the MplusAutomation R package (Hallquist & Wiley, 2018). Although the robust maximum likelihood estimators (MLR) are advantageous in that they provide information criteria (e.g., AIC and BIC) for model selection (Preacher & Yaremych, 2023), MLR performs well only when there are at least five categories for indicators, and their distributions are symmetrical (Rhemtulla et al., 2012). Given that some scales had five response categories and that some items’ asymmetry ranged from moderate to severe, we relied on the mean and variance adjusted weighted least estimator (WLSMV), which produces less biased factor loadings irrespective of the number of categories, symmetry, sample size, and when there is a mixture of categorical and continuous items (Li, 2021).
The tidyverse, car, careless, effectsize and psych packages were used for data cleaning, graphing, and descriptive and correlational analyses (Ben-Shachar et al., 2020; Fox & Weisberg, 2019; Revelle & Revelle, 2015; Wickham et al., 2019; Yentes & Wilhelm, 2023). We also performed independent samples t-tests between the community and chronic pain sample to assess whether, on average, the chronic pain sample was experiencing more pain severity than expected. Correlation effect sizes were interpreted using Cohen (1988) guidelines, where a correlation of .|10| is considered small, .|30| medium, and .|50| large.
Data Preparation
We initially screened for duplicate responses (i.e., the same Prolific ID) and retained the most complete response out of the duplicates, as well as removing cases with extensive missing data, outliers as determined by inspection of Q-Q plots, and evidence of careless responding. We identified careless responders using Ward and Meade (2023) recommendations for what is considered an “Extensive” screening level. As part of our a priori screening, we included instructed response items (e.g., “For this question, please select number six to demonstrate your attention”; Kung et al., 2018), bogus items (e.g., “All my friends are astronauts”; Simms et al., 2011), an inconsistency check (Belliveau & Yakovenko, 2022), and page response time (Greszki et al., 2015). For the post hoc screening, we performed long string analysis, multivariate outliers’ analysis (Mahalanobis distance), psychometric synonyms, even-odd index, and person-total correlations (e.g., Curran, 2016; Goldammer et al., 2020; Meade & Craig, 2012). For each participant, we flagged a participant as careless if they failed more than one careless responding index (dichotomously coded into “pass” or ‘fail’; Ward & Meade, 2023). To identify bots, we identified and removed any Qualtrics reCAPTCHA scores below .50 (Griffin et al., 2021).
Structural Analyses
We used Exploratory Structural Equation Modeling (Asparouhov & Muthén, 2009) with geomin rotation (ε = 0.50; Morin, 2023) to examine the item-level structure of all (in)flexibility measures. ESEM combines the strengths of both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) by allowing correlated residuals (e.g., for method effects) and cross-loadings between latent factors, while freely estimating the factor structure (i.e., as in EFA, items are not a priori assigned to factors; Marsh et al., 2014). Therefore, ESEM allowed us to examine factor structure in an exploratory manner (given no similar prior studies of multiple measures) and still account for measure-specific variance via correlated residuals of all items within the same measure (Brown, 2015; Morin, 2023; Morin et al., 2020).
For both samples, we continued to extract factors in ESEM analyses until the solution was uninterpretable (e.g., a factor with one or no primary loadings; an improper solution) and then evaluated these solutions to determine the optimal one based on both fit indices and interpretability (e.g., Naragon-Gainey & DeMarree, 2017). We report standard recommended goodness of fit indices, which included the model χ2 statistic, the Steiger–Lind Root Mean Square Error of Approximation (RMSEA; Steiger, 1990) with 90% confidence intervals, the Standardized Root Mean Square Residual (SRMR), the Bentler Comparative Fit Index (CFI; Bentler, 1990), and the Tucker Lewis Index (TLI). The approximate fit guidelines used are as follows: RMSEA ≤ .06, SRMR ≤ .08, CFI and TLI ≥ .95 (Hu & Bentler, 1999; Kline, 2023). Items were retained as good markers of a factor if the primary loading was ≥0.35, secondary loadings were <0.30, and the difference between the primary and secondary loadings was <0.20 (see L. A. Clark & Watson, 2019; Morin et al., 2020).
To examine the validity associations of the flexibility factors, we tested (a) zero-order correlations of factors and validity constructs and (b) incremental validity of each flexibility factor relative to one another. For the latter, we ran regressions predicting pain interference, pain severity, anxious mood, dysphoria, and well-being using all flexibility factors as predictors. These outcomes were specified as observed variables (one outcome per model), and the flexibility factors were specified as ESEM factors within the larger SEM model. We included neuroticism as a covariate in regression models, given previous research has found a large overlap between psychological (in)flexibility and distress (e.g., Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014).
Results
Preliminary Analyses
For the community sample, 526 participants consented to the study, and two duplicate responses were removed. Similarly, 12 cases (2.29% of the full sample, completed less than 25% of the questionnaires) were removed due to missing data. We then removed 46 (9.86%) careless responders. In addition, we removed one outlier from both the community and chronic pain samples, as identified with Mahalanobis distance. This resulted in 445 and 465 participants being analyzed in the chronic pain and community samples, respectively. There were no missing data at the item level for the flexibility measures. For an overview of the cases removed, see Supplemental Figure S1.
For the chronic pain sample, 541 participants consented to the study; six duplicate responses and 33 incomplete responses (6.67% of the full sample; completed <5% of questionnaires) were removed, with 502 participants remaining. We then removed 56 (12.46%) careless responders.
Descriptive Statistics
Descriptive statistics for all measures are provided in Table S2. Most measures demonstrated good reliability (α > 0.80; L. A. Clark & Watson, 2019), except for the PPFI and the OESQ in the chronic pain sample, which were slightly below 0.80. As expected, the chronic pain sample had significantly higher pain severity on the BPI, t(880) = 22.00, p < .001, 95% CI [1.31, 1.60], than the community sample, and the difference was large (Cohen, 1988).
The correlations among the cognitive flexibility, psychological flexibility, and psychological inflexibility measures are presented in Table 1 (see Tables S3 and S4, for correlations among all measures). Overall, for both samples, correlations among psychological (in)flexibility and cognitive flexibility measures were generally large and significant (|r| = 0.31 to 0.81, M = 0.61), 1 and all were in the expected direction. Moreover, the correlations across flexibility constructs selected for inclusion in the factor analyses were generally strong in magnitude (|r| = 0.31 to 0.69, M = 0.54), suggesting substantial shared variance across constructs. Thus, the pattern of correlations suggests that the measures have some overlap (though generally not so high as to suggest complete redundancy), with little clear evidence for the distinctiveness of psychological flexibility vs. cognitive flexibility, or psychological flexibility vs. psychological inflexibility.
Correlations Between Different Flexibility Measures: Community (Bottom Diagonal) and Chronic Pain (Top Diagonal) Sample.
Note. N = 465 (community sample), N = 445 (chronic pain sample).
All correlations were p < .001.
Latent Structure of the (In)Flexibility Measures
To examine the joint item-level structure of all (in)flexibility measures, we used ESEM and accounted for measure-specific variance by correlating item residuals within a specific measure. The model fit indices are presented in Table 2. Using the interpretive guidelines for model fit described previously, for both the community and chronic pain, the one-factor model had poor fit, whereas solutions with two factors or more had good fit across indices and models. Once seven and eight factors were extracted for both samples, the residual covariance matrix was not positive definite, or the solution was uninterpretable, indicating over-extraction. 2
Goodness of Fit Indices for the Exploratory Structural Equation Models and Bifactor Models.
Note. N = 465 (community sample), N = 445 (chronic pain sample). The χ2 diff tests were Satorra-Bentler (mean-adjusted) chi-square tests.
p < .05. **p < .10. ***p < .001.
When inspecting the factor correlations for the two-to-six-factor solutions across both samples, the correlations ranged from small to large (all rs ≤ 0.51). The six-factor solution was identified as having optimal fit while also being interpretable in both samples, and was retained for subsequent analyses. However, factor loadings and correlations for the other solutions that were not examined further are presented in Supplemental Appendix 2. Of note, we observed many significant residual correlations in all solutions. Across both samples, the mean absolute residual correlation across questionnaires was small to medium in magnitude (M|r| = 0.10–0.26), albeit with considerable heterogeneity, which indicates the presence of some measure-specific variance. See Table S5 for more details regarding the residual correlations for each questionnaire.
For the six-factor model (see Table 3), we found five consistent factors in both the community sample and the chronic pain samples: “Lack of Control in Difficult Situations,” “Generating Alternatives,” “Experiential Avoidance,” “Lack of Present-Moment Awareness,” and “Committed Action,” plus a sixth factor that differed across samples. For the five consistent factors, all of Tucker’s Congruence Coefficients were above 0.85 (see Table 3), indicating at least “fair similarity” among factors (Lorenzo-Seva & Ten Berge, 2006). Factor correlations in both samples ranged from 0.02 to 0.51 and were, on average, medium (M =|0.30|–|0.31|). To help characterize which questionnaires best represented those ESEM factors, we also computed correlations between the ESEM factor scores and the scales used in the analysis (see Table S6).
Standardized Factor Loadings From the Six-Factor Exploratory Structural Equation Model.
Note. CRL = Cognitive Inflexibility—Lack of Control in Difficult Situations; ALT = Cognitive Flexibility—Generating Alternatives; DF/AC = Psychological Flexibility—Defusion/Acceptance; EXP = Psychological Inflexibility—Experiential Avoidance; LA = Psychological Inflexibility; CRL = Lack of Control in Difficult Situations; ALT = Generating Alternatives; DF/AC = Defusion/Acceptance; EXP = Experiential Avoidance; LA = Lack of Present-Moment Awareness; CA = Committed Action; AC/HNS = Acceptance and Harnessing. Coefficients shaded in grey and bolded were “good markers” of that factor.
The Defusion/Acceptance factor was compared to the Acceptance/Harnessing factor when computing the Tucker’s Congruency Coefficient.
p < .05. **p < .10. ***p < .001.
The Generating Alternatives factor contained positively keyed items largely from the CFI Alternatives subscale, as well as the CFS. Moreover, the CFI Alternatives subscale was highly correlated scale with the Generating Alternatives ESEM factor in both samples (r = .90–.92). The Lack of Present-Moment Awareness factor contained negatively keyed items from the MPFI Lack of Awareness subscale and the CompACT Behavioral Awareness subscale, which both were highly correlated with the ESEM factor in both samples (r = .88–.94). The Experiential Avoidance factor contained negatively keyed items from the CompACT Openness to Experience subscale, the MPFI Avoidant, Self-As-Content and Fusion subscales, and the OESQ. However, the MPFI Self-as-Content and CompACT Openness to Experience subscales exhibited the strongest correlations in both samples (r = .76–.83).
The Lack of Control in Difficult Situations factor contained items from the CFI Control subscale and the CFS; items were generally negatively keyed but included one CFS flexibility-worded item. The CFI Control subscale exhibited the strongest correlation with this ESEM factor (r = .89–.92). The Committed Action factor contained positively keyed items from the CompACT Valued Action, and the MPFI Values and Committed Action subscales, with all these subscales exhibiting strong correlations with the ESEM factor (r = .76–.88).
The diverging factors were “Defusion/Acceptance” for the community sample and “Acceptance and Harnessing” for the chronic pain sample, which were insufficiently similar according to Tucker’s Congruency Coefficient (TCC = 0.81; Lorenzo-Seva & Ten Berge, 2006). The Defusion/Acceptance factor contained items mainly from the MPFI Acceptance, Self-As-Context, and Defusion subscales, and one item from the PPFI Acceptance subscale. These MPFI subscales all exhibited strong correlations with the ESEM factor (r = .76–.84). The Acceptance and Harnessing factor primarily contained items from the PPFI that describe how a person uses/harnesses negative emotions to motivate valued goal pursuits, as well as an acceptance item from the PPFI and MPFI, and a self-as-context item from the MPFI. However, these subscales generally exhibited lower correlations (r = .67–.70). As such, no single subscale from either the MPFI or PPFI best represented this ESEM factor.
Overall, across both samples, factors emerged for cognitive flexibility (1 factor), cognitive inflexibility (1 factor), psychological flexibility (2 factors), and psychological inflexibility (2 factors), after accounting for the influence of measure-specific variance, and factors were only moderately correlated. Thus, we found support for the structural distinctiveness of these constructs. However, each factor reflected particular facet-level content, rather than more broadly capturing the overarching flexibility constructs that will be explored in post hoc analyses below.
Zero-Order Factor Associations With Related Constructs
See Table 4 for factor correlations with related constructs. Inflexibility subscales (Lack of Control in Difficult Situations, Experiential Avoidance, Lack of Present-Moment Awareness) across both the samples consistently demonstrated large correlations (|r| = 0.48 to 0.74, M = 0.60) with distress-related constructs (i.e., neuroticism, anxious mood, dysphoria, AAQ-II, negative affect). However, none of these correlations were excessive (i.e., |r| > 0.80), indicating reasonable distinguishability from distress for these narrower facets of (in)flexibility. For the flexibility-related facets, Generating Alternatives and Acceptance and Harnessing factors demonstrated only small to medium-sized correlations with distress (|r|s < 0.40). Conversely, all distress-related correlations for the Defusion/Acceptance factor were large (|r|s > 0.56), while the Committed Action factor’s correlations were medium to large in the community sample (|r|s = 0.44-0.50), but large in the chronic pain sample (|r|s > 0.54).
Concurrent and Discriminant Validity Correlations Between ESEM Flexibility Latent Variables and Relevant Constructs.
Note. N = 465 (community sample), N = 445 (chronic pain sample). CF = cognitive flexibility; CI = cognitive inflexibility; PI = psychological inflexibility; PF = psychological flexibility.
p < .05. **p < .10. ***p < .001.
Turning to positive-valenced constructs, for the inflexibility facets, only the Lack of Control in Difficult Situations demonstrated large correlations (rs = 0.55–0.61) with well-being and positive affect, while Experiential Avoidance and Lack of Present-Moment Awareness were small to medium in magnitude (rs = 0.18–0.47). For the flexibility facets, most associations were strong (rs > .55), with the exception that Generating Alternatives demonstrated small correlations (r = .17–.26).
For closely related and socially desirable constructs, mindfulness was strongly associated with the Defusion/Acceptance and Lack of Awareness factors (|r| = 0.65-.0.70)—consistent with their mindfulness-related content— and more moderately associated with the other factors (|r| = 0.32–0.56). Self-reported executive functioning was strongly associated with most factors (i.e., Lack of Control in Difficult Situations, Committed Action, Experiential Avoidance, Lack of Present-Moment Awareness;|r| = 0.50–0.66), but had small correlations with the Generating Alternatives, Acceptance and Harnessing, and Defusion/Acceptance factors (|r| = 0.07–0.33). Finally, most factors had a small to medium correlation with Social Desirability (|r| = 0.18–0.41). However, the Acceptance and Harnessing factor in the chronic pain sample demonstrated a non-significant correlation.
For pain-related outcomes, the (in)flexibility factors had medium correlations with pain severity in the community sample (|r| = .0.30-.46), but small correlations in the chronic pain sample (|r| = 0.06-.0.19). Other pain measures (CPAQ-8, PIPS and BPI-Pain interference scale) were administered in the chronic pain sample only, and had small to medium correlations (r = .08–.42) with latent variables denoting psychological (in)flexibility (Acceptance and Harnessing, Lack of Present-Moment Awareness, Committed Action, and Experiential Avoidance) while the strongest correlation was seen toward the Lack of Control in Difficult Situations factor (rs = .52–.53). Thus, attributes of pain experience and pain-specific psychological (in)flexibility were weakly to moderately related to psychological and cognitive (in)flexibility in the chronic pain sample.
Incremental Validity of Each Factor in Predicting Outcomes
To examine the incremental validity of the six (in)flexibility ESEM factors relative to one another (i.e., controlling for their shared variance), we ran regressions predicting pain interference, pain severity, anxious mood, dysphoria, and well-being, while also controlling for neuroticism (see Table 5 for full results). All models predicting mental health outcomes (anxious mood, dysphoria, and well-being) explained a large amount of variance (R2 = 0.61–0.78). However, the models predicting pain severity and pain interference explained considerably less variance (R2 = 0.16–0.33). Across dysphoria, anxious mood, well-being, and pain interference, all or nearly all of the flexibility factors exhibited some unique prediction, even after controlling for neuroticism. Overall, these results suggest that each flexibility factor contributed unique information in explaining some clinically relevant outcomes, and associations were not driven by one or a small number of factors.
Standardized Regression Coefficients of Each ESEM Flexibility Latent Variable Predicting Pain and Mental Health Criteria, Controlling for Neuroticism.
Note. N = 465 (community sample), N = 445 (chronic pain sample). CF = cognitive flexibility; CI = cognitive inflexibility; PI = psychological inflexibility; PF = psychological flexibility.
p < .05. **p < .10. ***p < .001.
Post Hoc Bifactor Analyses
Given that (in)flexibility constructs are likely to be hierarchical in nature (i.e., there are both superordinate and subordinate factors; Arch et al., 2023; Markon, 2019), we also tested whether broader flexibility factors underly the six facets, and to what extent the broad vs. narrow factors are reliable and distinct. Using the Bayes estimator, we estimated two bifactor models (a type of hierarchical model, e.g., Markon, 2019), relying on “good indicators” (see Table 3) from the retained six-factor ESEM solutions (after clarifying the factor structure of the questionnaires, e.g., Swami et al., 2023). 3 As a comparison for model fit, we also estimated a six-factor CFA (no general factors).
The first bifactor model estimated a single general factor representing an overarching “Flexibility” construct upon which all items loaded. The second model included three general factors, where items loaded on one of these three: cognitive flexibility, psychological flexibility, and psychological inflexibility. We again accounted for measure-specific variance by correlating residuals within a specific measure. Given that bifactor models are prone to overfitting, we do not focus on relative model fit. Instead, we used the bifactor results to determine the strength and items that characterize the general factor (e.g., if there are specific ESEM factors that have weak/no general factor loadings), and the reliability analyses to determine the usefulness of the subscale and total scores of the ESEM factors (Bornovalova et al., 2020). To determine the practical usefulness of the total and subscale scores, we relied on indices suggested by Rodriguez et al. (2016), which included explained common variance (ECV), omega (ω), omega hierarchical (OmegaH/ωH), and the omega hierarchical subscale (OmegaHS/ωHS). To determine the utility of the subscale scores using omegaHS/ωHS, we relied on guidelines provided by Dueber and Toland (2023; see Tables 2 and 4). See Supplemental Appendix 3 for further details of the bifactor model estimation, factor loadings, and fit indices.
The model fit indices are presented in Table 2, and all models showed good fit. The bifactor indices (e.g., omega) are presented in Table 6 and interpreted below. In the single-factor bifactor model, the general factors for the community sample (ωh = 0.85) and the chronic pain sample (ωh = 0.81) explained a high degree of reliable variance in the total scores, and nearly all items loaded moderately to strongly on the general factor (i.e., > .|30|). For both samples, the general factor explained only 56%–58% of the common variance in the items (ECV = 0.56–0.58). All subscores and total scores had good reliability (e.g., ω/ωS > 0.80; L. A. Clark & Watson, 2019). However, the Lack of Control in Difficult Situations factor (ωHS = 0.07–0.10) in both samples and the Defusion/Acceptance (ωHS = 0.178) factor in the community sample provided no additive value above the general factor’s total score (Dueber & Toland, 2023). However, all other specific factors provided additive value above the general factor’s total score.
Bifactor Indices for Bifactor Structural Equation Models Estimated With BAYES: Community and Chronic Pain Sample.
Note. N = 465 (community sample), N = 445 (chronic pain sample). To determine the cut scores for assessing the added value of the subscale scores beyond the total score based on ωHS and ωS, we relied on indices reported by Dueber and Toland (2023; see Tables 2 and 4). ECV = explained common variance; ECVss = explained common variance of specific factor; ω = omega; ωS = omega subscale; ωH = omega hierarchical; ωHS = omega hierarchical subscale; CF = cognitive flexibility; CI = cognitive inflexibility; PF = psychological flexibility; PI = psychological inflexibility.
For the models with three general factors (i.e., cognitive flexibility, psychological flexibility, and psychological inflexibility), all general factors were well-defined in both samples, and loadings were mostly significant and greater than .|30| (M = 0.48 to 0.58 across factors and samples). For both samples, general cognitive flexibility and general psychological inflexibility explained a relatively small amount of reliable variance in their respective total scores (ωH = 0.63–0.66) and only 54%–59% of the common variance in the items (ECV = 0.54–0.59). Conversely, in the community sample, the psychological flexibility general factor explained a higher degree of reliable variance in its total score (ωH = 0.81) and 78% of the common variance in items (ECV = 0.78). Importantly, in both samples, the general factors were all very highly correlated (rs > 0.75), indicating there may be little value in splitting the single general factor into three separate general factors. All subscores and total scores had good reliability (e.g., ω/ωS > 0.80; L. A. Clark & Watson, 2019). However, the Lack of Control in Difficult Situations subscale (ωHS =.018-.028) in both samples and the Committed Action (ωHS = .236) and Defusion/Acceptance factor (ωHS = .082) in the community sample provided no additive value above the corresponding general factor’s total score (Dueber & Toland, 2023). However, all other subscales provided additive value above the respective total scores.
Discussion
We examined the structure and construct validity of psychological flexibility, psychological inflexibility, and cognitive flexibility questionnaires within a community and chronic pain sample. While detailed findings are discussed below, there are several overarching conclusions that should be emphasized. First, measures of the three constructs demonstrated strong associations but did not appear to be empirically redundant. Second, the structure of these constructs can be characterized at multiple levels, including six lower-order components that cut across individual measures and global factors that account for their shared variance. The six factors were broadly and uniquely associated with clinically relevant variables (symptoms and well-being) after accounting for neuroticism. Third, results were mostly similar across the community and chronic pain samples, with one notable exception that will be discussed below.
Structure of Flexibility
Separability of Constructs
Correlations among cognitive flexibility (CFI and CFS), psychological flexibility (PPFI and MPFI Flexibility facet), and psychological inflexibility (CompACT, OESQ, and MPFI Inflexibility facet) indicate they are highly related but sufficiently distinct (|r| = 0.31 to 0.69, M = 0.54), consistent with limited existing past research (Grant & Cassidy, 2022; Lackner et al., 2022). Furthermore, after controlling for measure-specific effects such as similar keying and wording in ESEM models, a six-factor model emerged in both samples with one or more factors that represented each of the three constructs of interest, as well as cognitive inflexibility. That is, cognitive flexibility was captured by the Generating Alternatives factor, cognitive inflexibility by the Lack of Control in Difficult Situations factor, psychological flexibility by Committed Action and Defusion/Acceptance or Acceptance and Harnessing, and psychological inflexibility by Experiential Avoidance and Lack of Present Moment Awareness. As such, we did not find evidence for the jangle fallacy in this domain. That is, when a researcher or clinician opts to measure psychological flexibility, psychological inflexibility, or cognitive flexibility using these questionnaires, they appear to get at empirically distinct constructs. Furthermore, there was differentiation within constructs, indicating the presence of several distinguishable facets or subdomains within the flexibility constructs that cannot simply be collapsed into global processes without first considering the unique manifestation/factors of the flexibility. These factors showed differential associations with a range of related constructs, including reasonable discriminant validity and distinctiveness from measures of distress.
Given that psychological (in)flexibility and cognitive flexibility are thought to be hierarchical in nature (Arch et al., 2023), we followed up with post hoc bifactor analyses to determine whether one or more general factors underlie the six factors identified. Across both the community and chronic pain samples, there was little utility in having three general factors (corresponding to psychological flexibility, psychological inflexibility, and cognitive flexibility) compared to one overarching flexibility process. Moreover, the three general factors were all highly correlated (|r|s > 0.75), and the subscales only had some additive value above the total score in both solutions. These very strong factor correlations were comparable to the psychological flexibility and psychological inflexibility factor correlations in the MPFI (r = −.74; Rolffs et al. (2018), which were interpreted in that study as identifying two separate factors. However, we argue that these results indicate that a single general factor better represents the data, given the magnitude of correlations and value of a parsimonious solution. Moreover, recent psychometric examinations of the MPFI-24 have also found that a single general factor demonstrated good fit (Navarrete et al., 2024). It is also important to note that prior studies did not use multiple measures and conceptualisations of flexibility while also considering method effects, whereas the current study attempted to address these limitations.
Our findings provide data for some highlighted gaps in the literature described in Cherry et al. (2021), suggesting that after accounting for measure-specific variance, flexibility constructs may be best characterized structurally as operating within a single continuum, alongside lower-order facets. That is, if a researcher were to perform a mixture model/latent profile analysis on specific factors of psychological (in)flexibility and cognitive flexibility, they might find different configurations of (in)flexibility, consistent with Ciarrochi et al. (2014). At the same time, a single flexibility factor can account well for individual differences in this domain. Thus, both hierarchical levels can be useful, depending upon whether one is interested in the commonalities across flexibility components (broadband flexibility) or differences across narrow facets (six facet factors).
Combining cognitive and psychological (in)flexibility into a broader conceptualization of flexibility is not far-fetched theoretically. What is central to both cognitive and psychological flexibility is the idea of behavioral change (action or thought) to suit changes in the environment (Whiting et al., 2017). Despite cognitive flexibility items forming their own separate factors that were distinguishable from all the psychological flexibility and inflexibility domains, the cognitive flexibility factors had strong loadings onto a single general factor (> 0.30). Therefore, once we account for these very narrow manifestations of cognitive flexibility, an overarching flexibility process alongside psychological (in)flexibility factors remains (and vice versa). Considering elements of psychological and cognitive flexibility together may provide a more comprehensive and complementary conceptualization of flexibility than one in isolation, if a faceted characterization of flexibility is of interest.
Content of Narrow Factors and Replicability Across Samples
One key finding when performing the ESEM analyses in the community and chronic pain sample was that five of the six factors replicated across samples: Lack of Present Moment Awareness, Experiential Avoidance, Generating Alternatives, Lack of Control in Difficult Situations, and Committed Action. This indicates they are likely to generalize to new samples, though this requires testing with more confirmatory techniques (e.g., CFA or ESEM with Target rotation).
Conceptually, the three psychological (in)flexibility factors map onto broader groupings of the six hexaflex processes (McCracken, 2023), but not to the six individual psychological flexibility processes (nor to 12 processes, separating flexibility and inflexibility, as modeled in the MPFI). Specifically, the Experiential Avoidance factor corresponds to (low) Openness (along with acceptance/defusion, which formed separate factors), Lack of Present Moment Awareness corresponds to (low) Awareness, and Committed Action corresponds to Engagement. Note that although we named the factors for their primary loadings, there were generally also loadings from the broader processes within each of these three groups (e.g., values items also loaded on the Committed Action Factor). Consistent with these findings and theory, a network analysis found that items did not align with six distinct groupings as theorized, but rather fewer categories, with values and committed action particularly closely associated empirically (Christodoulou et al., 2023). Although the components of cognitive flexibility are less clearly hypothesized, the two factors found in the current study (Generating Alternatives and Lack of Control of Difficult Situations) generally align with the CFI subscales (as well as corresponding items from the CFS). As such, the lower-order structure suggested by the CFI was supported in our analyses.
In contrast to the above five factors, the sixth factor to emerge differed across samples. The fact that Defusion/Acceptance failed to replicate in the chronic pain sample may indicate that the Acceptance and Harnessing factor is less stable, and could be an artifact of overfitting sample-specific characteristics despite support from fit indices (Sellbom & Tellegen, 2019). On the other hand, it is also possible that Acceptance and Harnessing is a unique factor specific to chronic pain, and in particular, harnessing negative emotions items loaded weakly on factors in the community sample. It is possible that living with chronic pain may particularly call for a combination of acceptance of difficult internal experiences and harnessing negative emotions in service of their chosen values. Ecological momentary assessment (EMA) research has found that when compared to healthy controls, people with chronic pain reported more variability and higher mean levels of negative emotions (Gerhart et al., 2018; Rost et al., 2021). In such a context, relying extensively on reframing negative emotions to motivate values-driven behavior may deplete regulatory resources (Gerhart et al., 2018), but solely using acceptance of pain and associated limitations may lead to inaction. Thus, a balance of the two, depending upon the context, may be particularly important. Future research should examine the role of harnessing and acceptance in other chronic pain samples, and whether they are distinct from general or clinical populations.
Taken together, these structural results indicate that (a) psychological flexibility and cognitive flexibility may be divided into empirically-supported facets; (b) these facets do not conform to 6 separate facets hypothesized for psychological flexibility and/or psychological inflexibility, and (c) the role of acceptance, defusion, and harnessing negative emotions requires further study.
Clinical and Assessment Implications
With the exception of the Generating Alternatives factor, in both samples, all factors consistently displayed significant correlations with distress-related constructs and well-being in expected directions, ranging from small to large correlations. Furthermore, the regression analyses indicated that each of the flexibility facets predicted one or more outcomes after accounting for their shared variance; there were different patterns of predictors across different outcomes, and these results held when accounting for associated emotional styles as assessed by neuroticism. This highlights the importance of targeting specific factors of cognitive flexibility and psychological (in)flexibility in therapy, dependent upon the desired outcome. The fact that the Generating Alternatives factor was not associated with or weakly associated with relevant outcomes suggests that it may not have strong direct effects on well-being or distress. Rather, its effects may be mediated by other variables (e.g., emotion regulation or problem-focused coping), as past research has shown it has a large correlation with increased problem-focused coping (r = .51; Dennis & Vander Wal, 2010).
Although attending to specific flexibility facets is important, the bifactor analyses in the current study also suggest that enhancing any facet may have implications for improving general flexibility, and this may partially explain why some studies find that CBT and ACT lead to improvements in non-target treatment variables. Despite CBT not explicitly targeting acceptance-based processes, decentering/defusion mediates improvements in cognitive therapy (Arch et al., 2012; A-Tjak et al., 2021; Farb et al., 2018; Forman et al., 2012), and conversely, cognitive change processes show improvements in ACT (A-Tjak et al., 2021; Niles et al., 2014). One possibility that has been proposed is that the targeted process may implicitly teach or lay the groundwork for non-targeted processes (e.g., cognitive restructuring requires self-distancing, which facilitates defusion; Arch et al., 2012; Forman et al., 2012). Alternatively, the bifactor model loadings suggest that any increase in one of the narrow flexibility factors may also increase overarching flexibility. Therefore, a single general flexibility factor may explain why some treatments show improvements in variables they do not explicitly target (e.g., CBT improves defusion/decentering). Therefore, if one tries to build a person’s overarching flexibility, a good place to start may be what is most accessible to the client, for instance, people with traumatic brain injury may have issues with developing cognitive flexibility components due to their cognitive impairments, but still may benefit from psychological flexibility treatment processes (e.g., accepting negative emotions; Whiting et al., 2017).
A key implication for assessment is that psychological flexibility and inflexibility may be best assessed by the CompACT-15 and MPFI-24, whereas cognitive flexibility may be best assessed using the CFI. In the ESEM models, these questionnaires had the most “good” items and exhibited strong correlations with the ESEM factors; as such, it may not be necessary to administer multiple flexibility measures. Specifically, the MPFI-24 may be more useful for assessing a broader evaluation of psychological (in)flexibility, particularly given that it was designed to assess overarching psychological (in)flexibility (Rolffs et al., 2018), emerged consistently in the ESEM solutions, had strong general factor loadings in the bifactor analyses, and had large correlations (r = .74–.86) with all the psychological (in)flexibility specific factors.
Conversely, the CompACT-15 may be more useful for assessing specific facets of psychological (in)flexibility as it had many items loading onto specific factors in this study and was designed to comprehensively target fewer specific facets of psychological flexibility, which also supports recent research that found larger groupings of psychological flexibility (Christodoulou et al., 2023). Specifically, the CompACT Openness to Experience had very large correlations with the Experiential Avoidance factor (r = .79–.83), the CompACT Behavioral Awareness had very large correlations with the Lack of Present Moment Awareness factor (r = .94), and the CompACT Valued-Action had strong correlations with the Committed Action factor (r = .82–.88). For cognitive flexibility, the Generating Alternatives and Lack of Control in Difficult Situations were most represented by the CFI, and administering the CFS to assess these aspects appears to be redundant. Particularly, given the Alternatives and Control subscales of the CFI exhibited strong correlations with these ESEM factors (r = .89–.92). Finally, researchers could administer the PPFI to understand Harnessing, which was not captured by the other questionnaires in this study.
Limitations and Future Directions
One key limitation of the current study is the use of self-report measures, particularly given that cognitive flexibility is often assessed using task-based measures (Cherry et al., 2021). Previous research that has investigated the overlap between task-based measures and self-reported measures of cognitive flexibility has found no association in clinical and non-clinical samples (Grant & Cassidy, 2022; Howlett et al., 2021, 2022, 2024). Using only one mode of measurement may mean other aspects of flexibility are not assessed and represented. Moreover, self-report measures are vulnerable to social desirability responding and recall biases (Hohl & Dolcos, 2024), while tasks may allow for the assessment of a person’s stable level of functional capacity (Schmitter-Edgecombe et al., 2020) and capture their best performance (Dang et al., 2020). The CFI was developed to be used in clinical settings to capture a person’s perceived ability to engage in cognitive restructuring, which can be quickly and reliably assessed with self-report measures (Dennis & Vander Wal, 2010). Rather, future research should compare self-report measures of cognitive and psychological (in)flexibility (e.g., MPFI, PPFI) with cognitive flexibility tasks by using multimethod-multitrait study designs to address previous study limitations (see Eid et al., 2023). Importantly, research on related constructs has found that self-report and task measures each contribute unique and complementary information to the prediction of behaviors (Friedman & Banich, 2019; Sharma et al., 2014), illustrating the value of each and the importance of including both where possible.
Another key limitation is that this study was observational and cross-sectional, allowing only “on-average” conclusions that cannot provide causal interpretations (i.e., whether the flexibility factors caused people to have better well-being and less distress). While we controlled for neuroticism in our regression models to reduce the effects of confounding variables, future research could benefit from using ecological momentary awareness (EMA) research to assess the temporal direction of effects using repeated measurements for each person (Shiffman et al., 2008). EMA also allows assessment of dynamic individual-level processes, while cross-sectional psychometric investigations can only assess differences between people. Therefore, future research using EMA (or other longitudinal designs) could test whether the psychometric properties of these flexibility constructs are comparable at both levels (within-person and between-person) to address this study’s limitation (Schneider & Stone, 2016).
Another limitation was that this was an online sample, with only half of the chronic pain participants having a chronic pain diagnosis. In addition, pain severity and pain interference were, on average, in the moderate range for the chronic pain sample (Treede et al., 2019). Future research should determine whether the structure of flexibility generalizes to other chronic pain samples or clinical samples, as chronic overlapping pain conditions (e.g., fibromyalgia, chronic lower back pain, irritable bowel syndrome, etc.) have been associated with greater psychological symptoms (e.g., pain catastrophizing, depression, perceived stress and somatization; Fillingim et al., 2020). Moreover, people with high-impact pain compared to people with only mild chronic pain (low pain intensity and interference) showed that they differed on all ten health status indicators, but there was little difference between people with no and mild chronic pain (Von Korff et al., 2020).
Last, not all constructs/definitions of flexibility were measured, and there were more psychological flexibility measures than cognitive flexibility measures available and therefore included in the current study. Psychological flexibility is also related to other flexibility constructs, such as emotion regulation or coping flexibility (Doorley et al., 2020). Furthermore, having fewer cognitive flexibility items and fewer items for certain facets of psychological flexibility (e.g., self-as-context) could have resulted in some important undiscovered factors of cognitive flexibility and psychological flexibility. As such, future research should explore these factors’ associations with emotion regulation flexibility and other flexibility constructs (e.g., coping flexibility) and examine more cognitive flexibility questionnaires/items alongside psychological flexibility questionnaires.
Supplemental Material
sj-docx-1-asm-10.1177_10731911251399030 – Supplemental material for One Construct or Many? Clarifying the Structure and Meaning of Measures of Psychological and Cognitive Flexibility and Their Components in a Community and Chronic Pain Sample
Supplemental material, sj-docx-1-asm-10.1177_10731911251399030 for One Construct or Many? Clarifying the Structure and Meaning of Measures of Psychological and Cognitive Flexibility and Their Components in a Community and Chronic Pain Sample by Jayden Lucas, Jeffery M. Lackner, Gregory Gudleski, Andrew H. Rogers, Rodrigo Becerra and Kristin Naragon-Gainey in Assessment
Footnotes
Author Contributions
JL played a lead role in conceptualization, data collection, data curation, formal analysis, methodology, and writing of original draft; JML played a lead role in resources, funding acquisition and a supporting role in conceptualization, supervision, and reviewing and editing. AHR and GG played a lead role in resources and a supporting role in reviewing and editing. RB played a lead role in supervision, reviewing and editing. KN-G played a lead role in conceptualization, funding acquisition, project supervision, and reviewing and editing, and a supporting role in formal analysis, data collection, and data curation.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by the National Institute of Diabetes and Digestive and Kidney Diseases grant DK096606. The corresponding author (Jayden Lucas) is also supported by the Government Research Training Program Scholarship and the School of Psychological Science/Graduate Research School at the University of Western Australia. The funding bodies had no role in the study design, collection, analysis or interpretation of data, writing the manuscript, or the decision to submit the paper for publication.
Ethical Approval and Informed Consent Statements
This study was performed according to the National Statement on Ethical Conduct in Research (2018) and received ethics approval from the University of Western Australia (Approval number: 2023/ET000154) Human Research Ethics Committees. All participants viewed the information sheet and consent form prior to participation. They were required to provide informed e-consent prior to proceeding to the online survey.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
