Abstract
Mindfulness meditation (MM) training has been shown to have positive effects on working memory and focused attention tasks. Clarifying the construct of mindfulness is important so that mindfulness can be studied effectively in individual differences and cognition research. The current study tested whether trait mindfulness alone explains any of the variability on task performance. Five commonly used mindfulness scales, as well as six standardized and experimental attention and working memory tasks were administered to 164 participants with no meditation experience. Confirmatory factor analysis found that the common variance denoted by measures of trait mindfulness is unrelated to the common variance among tasks requiring focused attention. These results indicate that mindfulness scales may not be capturing the attentional aspects of mindfulness. Individuals who score high on mindfulness scales do not perform better on focused attention tasks than those who score lower on mindfulness scales. These results have implications for defining and operationalizing mindfulness.
Background
There are numerous mindfulness scales that have been developed in recent years in response to the burgeoning research interest in mindfulness and meditation. Mindfulness is an elusive construct and has been conceptualized variously as a disposition, a state, a process, and a trainable skill. Mindfulness-based interventions have recently received strong support as meditation training has been shown to alleviate distressing symptoms and enhance emotional regulation (Chiesa & Serretti, 2009; Grossman, Niemann, Schmidt, & Walach, 2004; Lutz et al., 2014). As mindfulness training has been associated with an advantageous cost to benefit ratio that is driving practitioners and individuals to promote mindfulness, it is critical to understand the constructs that lead to the benefits of mindfulness. Defining and operationalizing mindfulness so as to study individual differences and underlying constructs is methodologically challenging. Current consensus suggests that present-focused attention is an essential component of mindfulness (Bishop et al., 2004; Frewen, Lundberg, MacKinley, & Wrath, 2011; Hölzel et al., 2011; Shapiro & Carlson, 2009) and that mindfulness training may enhance focused attention (Chambers, Lo, & Allen, 2008; Jha, Krompinger, & Baime, 2007; Slagter, Lutz, Greischar, Nieuwenhuis, & Davidson, 2009; Tang et al., 2007; Zeidan, Johnson, Diamond, David, & Goolkasian, 2010)—suggesting collectively that trait mindfulness should relate to trait measures of focused attention. The goal of the present study was to examine whether current measures of mindfulness do in fact predict individual differences in focused attention.
Defining Mindfulness
Mindfulness is generally defined within the psychological literature as an intentional quality of attention with an attitude of acceptance (Frewen et al., 2011; Shapiro, Carlson, Astin, & Freedman, 2006). Kabat-Zinn’s (1994) definition of mindfulness emphasizes paying attention in the present moment, with an attitude of acceptance and nonjudgment. Relatedly, Shapiro and Carlson (2009) stress that mindfulness is “awareness that arises through intentionally attending in an open, caring, and nonjudgmental way” (p. 4). A consensus operational definition characterizes mindfulness as (a) the self-regulation of attention so that it is maintained on immediate experience, and (b) an orientation toward one’s experiences in the present moment that is characterized by openness and acceptance (Bishop et al., 2004). Hölzel et al. (2011) propose a theoretical framework of mindfulness effects based on current intervention and experimental research. They propose that the salutary effects of mindfulness meditation (MM) rest on (a) attention regulation, (b) body awareness, (c) emotion regulation, and (d) change in perspective on the self. Critically, all of the above definitions include attention as a fundamental factor of mindfulness; in fact it appears to be the only place where the multiple definitions and frameworks converge. And yet as Hölzel and colleagues (2011) note, the neuropsychology of attention in mindfulness research has been neglected, and that the relationship between attention regulation and the proposed mechanisms of mindfulness should be explored.
Measuring Mindfulness
The lack of agreement in how to define mindfulness can be easily captured by looking at the variety of measures available to researchers in this domain within the last 10 years. In 2003, Brown and Ryan developed the Mindful Attention Awareness Scale (MAAS) initially with a dual-factor perspective of individual differences in (a) awareness and (b) acceptance, but dropped the acceptance factor based on its lack of incremental validity in predicting well-being outcomes. In 2004, Baer, Smith, and Allen developed the Kentucky Inventory of Mindfulness Skills (KIMS), which 2 years later was further developed into the Five Facet Mindfulness Questionnaire (FFMQ) based on factor analytic techniques (Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006). This scale has five factors: observing, acting with awareness, describing, nonjudgment, and nonreactivity.
The Freiburg Mindfulness Inventory (FMI; Walach, Buchheld, Buttenmuller, Kleinknecht, & Schmidt, 2006) and the Toronto Mindfulness Scale (TMS; Lau et al., 2006) were both developed in 2006; the FMI is comprised of a single factor that encompasses presence, openness, and acceptance, whereas the TMS is comprised of two factors: decentering and curiosity. Feldman, Hayes, Kumar, Greeson, and Laurenceau (2007) sought to assess four facets of mindfulness (attention, awareness, present focus, and acceptance/nonjudgment) in developing the Cognitive and Affective Mindfulness Scale–Revised (CAMS-R); however, they found that the four facets loaded on a single mindfulness factor, and their scale did not capture such facets independently. In just four short years, the mindfulness community has developed scales ranging from one to five factors that are all intending to measure the same construct.
Mindfulness and Attention
One issue with the current self-report measures of mindfulness is that whether they predict objective indicators of conscious experience, namely individual differences in attention, has not been tested. Several studies have found that intensive (Chambers et al., 2008; Jha et al., 2007; Slagter et al., 2009) as well as brief (Tang et al., 2007; Zeidan et al., 2010) mindfulness training improves performance on attentional and working memory tasks. Controlled studies in our lab have found that four sessions of MM resulted in better performance on standardized and experimental tests of attention, compared with controls (Zeidan et al., 2010). If mindfulness enhances focused attention, then dispositional mindfulness should predict individual differences in attentional control. Accordingly, if self-report measures of mindfulness predict individual differences in attentional control, the validity and operational lucidity of the measures would be significantly strengthened.
Although trait mindfulness has been linked to attentional regulation in a theoretical way, there have been only a handful of studies specifically testing this relationship. Schmertz, Anderson, and Robins (2009) examined the relation between the CAMS-R, MAAS, and KIMS mindfulness scales and the Continuous Performance Test, Second Edition (CPT-II) and the Paced Auditory Serial Addition Test (PASAT). They found mixed support in that CPT-II target omissions were correlated with the MAAS and the CAMS-R, but reaction time on the CPT-II and PASAT performance were not related to any mindfulness scales. Anicha, Ode, Moeller, and Robinson (2012) showed that two factors from the FFMQ independently predicted specific types of cognitive performance that converged with their theoretical domain. Similarly, Josefsson and Broberg (2011) found that higher FFMQ–Describe scores predicted fewer errors on the Sustained Attention to response Task (SART) and the Stroop test. In a community sample, Galla, Hale, Shrestha, Loo, and Smalley (2012) found that it was the KIMS–Observe subscale that was associated with reduced attentional lapses on the CPT over prolonged exposure and better scores on the incongruent Stroop subtest. Ruocco and Direkoghu (2013) found that the Awareness subscale of the Philadelphia Mindfulness Scale was associated with CPT-II reaction time variability whereas the Acceptance subscale was associated with 2-back task efficiency. Finally, Lykins, Baer, and Gottlob (2012) did not find relationships between the FFMQ and cognitive tasks. Collectively, individual scales have been related to specific attentional and cognitive tasks, but no larger scale comparisons have been conducted that include comparing multiple trait mindfulness scales and multiple attentional and cognitive tasks. Furthermore, researchers have called for investigations in which mindfulness measures are closely related to behavioral indices of cognition (Davidson, 2010; Galla et al., 2012). The aforementioned inconsistencies and lack of a comprehensive multi-scale and multi-task study prompted the present study in which all available trait mindfulness measures were compared with a range of cognitive tasks that all require sustained attention regulation for successful performance.
There is no question that there is more to mindfulness than attentional abilities, and that an attitude of acceptance, nonjudgment, and equanimity is critical to the mechanism of mindfulness (Hölzel et al., 2011; Shapiro et al., 2006). Nonetheless, some aspects of the mindfulness construct appear easier to operationalize than others. It may be more straightforward to test whether there is a relationship between dispositional mindfulness and objective measures of sustained attention than to veridically understand a person’s intentions and attitudes at a particular moment in time. Such reports are subject to introspective limits, expectancy effects, and social desirability. Thus, the current study focuses on assessing the construct validity of the attentional aspect of mindfulness.
The tasks in the current study tap many cognitive processes in parallel (e.g., working memory, concentration, visual tracking, and word retrieval), but the timed tasks share the requirement of sustained attention and freedom from distractibility for successful performance. In the current study, standardized and experimental measures of cognitive domains that had previously been shown to be sensitive to brief MM intervention (Zeidan et al., 2010) were administered, and a confirmatory factor analysis was used to assess the relationship between the latent constructs of mindfulness and focused attention. The current study tested the validity of current mindfulness scales in predicting the attentional aspect of mindfulness. If focused attention is indeed a critical component of mindfulness, as has been suggested extensively in the literature, then the correlation between these two latent factors should be positive and at least moderate in strength.
Methods
Participants
Students from a large Southeastern university participated in exchange for course credit. The participants were recruited via the University subject pool and had no prior meditation experience. The protocol was approved by the University Institutional Review Board. A total of 197 participants entered the study, and 164 participants completed all measures of mindfulness and cognitive task performance, and thus were included in all subsequent analyses. The sample was 61% female, and ages ranged from 18 to 57 (M = 21.29, SD = 6.16) years (range = 18-57). In ethnic composition, 56.7% of the participants self-identified as White, 29.3% as African American, 4.9% as Hispanic, 3.7% as Asian, and 5.5% as biracial, Native American, or Other. Participants were excluded if they were not native speakers of English.
Measures
Mindfulness scales
All mindfulness scales used in the current study were averaged (instead of summed) to retain scales that are as consistent as possible across measures. In each case, higher scores indicate higher levels of self-reported mindfulness.
Freiburg Mindfulness Inventory, trait version, short form (Walach et al., 2006) is a 14-item assessment that measures the experience of mindfulness. Statements like “I am open to the experience of the present moment” are rated on a 5-point scale from 1 (rarely) to 5 (always). Cronbach’s alpha value in this sample was .81.
MAAS (Brown & Ryan, 2003) is a 15-item scale measuring the general tendency to be attentive to the present moment in day-to-day experiences. On a 6-point scale (1 = almost always to 6 = almost never), participants rated how often they find themselves operating on automatic pilot, and not paying attention to the present moment. Items include “I do jobs or tasks automatically, without being aware of what I’m doing” and “I tend not to notice feelings of physical tension or discomfort until they really grab my attention.” Cronbach’s alpha in this sample was .82.
CAMS-R (Feldman et al., 2007) consists of 12 items. On a 4-point scale (rarely/not at all to almost always), participants rate items such as “It’s easy for me to keep track of my thoughts and feelings,” “I am easily distracted,” or “It’s easy for me to concentrate on what I’m doing.” Cronbach’s alpha in this sample was .65.
FFMQ (Baer et al., 2006) is a 39-item scale that can be divided into the following five subscales: Observing, Describing, Acting With Awareness, Nonjudging, and Nonreactivity. The FFMQ–Total Score was used in the current study, which is the combination of all five subscales. Cronbach’s alpha in this sample was .86.
TMS (Lau et al., 2006) has 13 items and a two-factor structure (curiosity, decentering); the trait version was used in the current study (Davis, Lau, & Cairns, 2009). The items of Factor 1 (curiosity) reflect an attitude of wanting to learn more about one’s experiences. The items of Factor 2 (decentering) reflect a shift from identifying personally with thoughts and feelings to relating to one’s experience in a wider field of awareness. Cronbach’s alpha in the current sample was .83 and .67 for curiosity and decentering, respectively.
Cognition tasks
The cognitive tasks were administered to each participant individually. Controlled Oral Word Association Task (COWAT; Benton, 1989) is a measure of verbal fluency that was measured by asking participants to name words that begin with the letters F, A, and S; 1 min was given for each letter. Total score is the total number of unique words named across the three trials; higher scores indicate higher levels of verbal fluency.
Symbol Digit Modalities Test (SDMT; Smith, 1982) was used to assess working memory and attention necessary for complex scanning and visual tracking (Shum, McFarland, & Bain, 1990). SDMT requires decoding of a series of numbers listed on paper according to a corresponding template of visual symbols. The dependent measure is the number of correctly paired responses in 90 s.
The Computer Adaptive Adjustable 2-Back Task is an accuracy selectable, 2-back task that consists of 54 trials and was developed to measure information-processing speed, working memory, and attention. Participants view a sequence of letters and indicate whether or not a probe letter is the “same” or “different” as the stimulus item presented two items back. Research has provided evidence of face, concurrent, and predictive validity (Diamond, Johnson, Kaufman, & Graves, 2008; Johnson, Gur, David, & Currier, 2014; Zeidan et al., 2010). Extended hit rate (EHR) represents a run of correct responses, and was used in the current study to approximate attentional control. A measure of accuracy of .70 was required to make sure that all participants were functionally equivalent.
The forward/backward digit span subtests of the Wechsler Adult Intelligence Scale–Revised (WAIS-R; Wechsler, 1981) were used to measure immediate memory span. The dependent measure was the sum of the scores for total forward digit span and total backward digit span; subjects can repeat back a span of up to 16 and 14 digits, respectively. Higher scores are indicative of higher memory recall.
Trail Making Test, Parts A and B (Reitan, 1958), are visual tracking and attentional set-shifting tasks. In Trail A, the examinee draws a line to connect the numbers 1 through 25 in order. In Trail B, the examinee draws a line to connect in alternating sequence the numbers 1 through 13 and the letters A through L. The examinee begins with 1 and then draws a line to A, then proceeds to 2, then B, and so on until all the numbers and letters are connected. The score for both tasks is the amount of time elapsed in seconds, reverse scored, such that higher scores indicate better performance.
Procedure
After giving informed consent, participants completed the self-report measures or were administered the cognitive tasks in random order. The entire session took approximately 90 min.
Results
The descriptive statistics and bivariate correlations for the mindfulness measures and cognitive tasks are shown in Table 1. The mindfulness measures are positively correlated with each other, as are the cognitive tasks. However, bivariate correlations revealed that five commonly used scales of mindfulness were generally not associated with better performance on standardized and experimental tests of focused attention.
Descriptive Statistics and Bivariate Correlations for all Study Measures.
Note. N = 164. Reverse-scored Trails A and B were used for the correlation analyses. CAMS-R = Cognitive and Affective Mindfulness Scale–Revised; TMS = Toronto Mindfulness Scale; FMI = Friedberg Mindfulness Inventory; MAAS = Mindful Attention Awareness Scale; FFMQ = Five Facet Mindfulness Questionnaire.
p < .05. **p < .01. ***p < .001.
A confirmatory factor analysis was conducted using AMOS version 20 statistical software, with the cognitive task performance measures loading onto one factor, and the mindfulness measures loading onto a separate factor (see Figure 1). Because the Curiosity and Decentering subscales of the TMS are contained as part of the same measure, their error terms were allowed to correlate.
The factor analytic structure and standardized estimates.
Overall, the model demonstrates good or acceptable levels of fit on most indices (e.g., comparative fit index = .92, Tucker–Lewis index = .89, root mean square error of approximation = .07). Less than acceptable fit was found only on one of the commonly reported fit indices (normed fit index = .82). The model produced a χ2 = 87.43, p = .002, with a CMIN/df ratio of 1.68, which also indicates acceptable fit. 1
Recall that the hypothesis of the current study is that the latent constructs of mindfulness and focused attention will be strongly and positively associated. However, in the current sample, the correlation between the two latent factors was −.16 (p = .20), indicating that those who have higher levels of trait mindfulness tend to have lower levels of attentional control. Standardized factor loadings for the mindfulness scales range from .35 to .79, whereas these coefficients for the cognitive tasks range from .20 to .77.
Discussion
Although the self-regulation of attention described by the concept of mindfulness should be related to performance in the attentional domain of cognition, we did not find evidence to support this relationship. In terms of criterion-related validity for mindfulness scales, there is a genuine problem of introspective limits. This may be particularly salient in people unfamiliar with mindfulness concepts (Grossman, 2008). For this reason, attentional tasks as criterion may be more valid, and their relationship with mindfulness scales is an important research question. However, our results suggest that as a group, the current mindfulness scales do not appear to capture the construct of attentional focus as measured in this study.
Perhaps instead of tapping into the actual construct of mindfulness, these self-report measures only assess how mindful participants think that they are, and perhaps perceived mindfulness does not share the same theoretical relationship with attention that actual mindfulness is hypothesized to share. Concerns regarding differential item understanding and taxonomy on mindfulness scales have been raised (Grossman, 2008; Grossman & Van Dam, 2011; Sauer et al., 2013), indicating that new approaches are necessary to discover whether self-ratings of mindfulness reflect how mindful people truly are. Furthermore, although mindfulness has been approached as a construct by research psychologists, this very approach may be by its very nature problematic. Leaders in the field generally describe mindfulness as a nonconceptual experience encompassing a self-compassionate effort to be continuously present with experience (Germer, 2005; Kabat-Zinn, 2005). As such, if mindfulness is a lived experience, it may be extremely difficult to measure through self-report items on a scale.
In addition, the attention that mindfulness involves may be a specific form of attention that is not well measured by cognitive performance, which involves observing present moment experience with openness, curiosity, and acceptance. Measures of attention to emotional information have recently been developed and may provide a fruitful new avenue for investigating attentional control. For example, recovering alcohol-dependent individuals who are higher in trait mindfulness have been found to have lower attentional bias toward visual alcohol cues on dot probe tasks (Garland, 2011; Garland, Boettiger, Gaylord, Chanon, & Howard, 2012).
Finally, perhaps it is that trait mindfulness, or the amount of mindfulness that one naturally displays without any meditation experience, does not rely as strongly on the construct of attention as does state mindfulness, or mindfulness that can be cultivated through mediation practice. Previous studies show that even brief training in mindfulness led to improved performance on several attentional tasks (Tang et al., 2007; Zeidan et al., 2010), which is likely due to the specific training in MM. This supports the idea that the attentional facet of mindfulness is a trainable skill, rather than a disposition. Furthermore, Thompson and Waltz (2007) found that trait mindfulness is not necessarily predictive of state mindfulness. All of these possible explanations for the current findings suggest a need for further measurement development before continuing to move forward in studies of how mindfulness relates to other constructs that are not hypothesized to be part of the definition itself. Further measurement development may be more fruitful if it focuses on directly observable measures such as cognitive tasks requiring sustained concentration of attention, or attention to emotional information. Future researchers may develop measures that are able to capture individual differences in proficiency of present-centered awareness as a skill.
A major limitation of the current study is that due to being a follow-up of the brief mindfulness intervention study, we used the same cognitive tasks as in Zeidan et al. (2010) although arguably these tasks may not be the most appropriate to capture mindfulness of attention. Although SDMT, COWAT, and EHR were sensitive to 4 days of training in MM, they do not easily map onto mindfulness constructs. More specific measures of attention such as attentional blink, Stroop, CPT-II, Attention Network Test (ANT), and SART may be more closely related to dispositional mindfulness measures. These measures appear to be sensitive to the construct of mindfulness as sustained focus, reducing lapses of attention. However, a strength of the current study is the use of factor analysis to capture the shared variance associated with the tasks that we did use. Because focused attention is a necessary prerequisite for successful performance on each of the tasks administered, the latent factor associated with these measures should reflect (at least partially) that focused attention component. The present study adds fuel to the complex issue on the lack of convergence between self-report and laboratory tasks. This is a broad problem in psychology and not unique to mindfulness research. For example, Cyders and Coskunpinar (2011) performed a meta-analysis of comparisons between self-report and lab task measures of impulsivity and concluded that there is more variability in what is being assessed than there is overlapping content domain.
The issue of operationalizing mindfulness is still in its infancy. Some argue that Western mindfulness scales do not really capture the Buddhist nature of the phenomenon and may be capturing intervention effects in acceptance of distressing feelings, present moment awareness, and attentional training rather than mindfulness as understood in a Buddhist context (Christopher, Charoensuk, Gilbert, Neary, & Pearce, 2009; Grossman, 2008, 2011; Rosch, 2007; Sahdra, Shaver, & Brown, 2010). The current study illustrates the difficulty that current mindfulness scales show in capturing the construct of cognitive control and attention regulation. As we are still in the first wave of measuring mindfulness with the tools of Western psychology, future studies will need to explore creative measures of mindfulness using qualitative, neurobiological, and psychometric approaches.
Footnotes
Acknowledgements
The authors would like to thank Charlie Reeve and Sara Levens for helpful comments on this manuscript. The authors would like to acknowledge the help of Nishi Bryska and Elise Currier in data collection and scoring.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research and/or authorship of this article.