Abstract
Background. Persons with aphasia often present with low mood/depression, which can negatively affect their quality of life. The validity and reliability of existing depression measures for aphasia have been called into question. Eye tracking in nonstroke populations is reliable in identifying low mood/depression. Depressed persons are biased to negative emotions compared with nondepressed persons and have an absence of bias to positive emotions. However, nondepressed persons may be biased to positive emotions. Objective. To examine the feasibility of using eye tracking to measure mood in persons with aphasia. Methods. We recruited 22 persons with chronic aphasia and 12 healthy controls. Participants completed 2 self-report measures of mood. They also viewed faces that showed happy, sad, and neutral facial expressions during eye tracking. We analyzed 2 eye tracking indices: initial gaze orientation and gaze maintenance to happy, sad, and neutral faces. Results. For initial gaze orientation, participants with aphasia fixated faster on emotional faces compared with healthy controls but directed their gaze less often to happy faces compared with healthy controls. For gaze maintenance components, the duration of first fixation and total fixation duration were shorter on sad faces for participants with aphasia compared with healthy controls. Conclusion. Use of eye tracking with faces representing different mood states is feasible in persons with aphasia. Although there were some similarities, participants with aphasia had different gaze patterns to emotional faces compared with healthy controls. Further research is needed to establish whether this is a valid and reliable method of mood assessment.
Introduction
Stroke is the third leading cause of disability, with 25.7 million stroke survivors and 10.3 million new strokes worldwide in 2013. 1 Among stroke survivors, roughly 34% present with aphasia in the rehabilitation setting. 2 Aphasia is more common than conditions such as Parkinson disease and has a more negative impact on quality of life than Alzheimer disease and cancer.3,4 Aphasia affects the understanding and production of spoken language as well as reading and writing. The disorder can also increase loneliness and negatively affect the person’s ability to return to work and their social and recreational activities.5-7
Stroke survivors with aphasia are at a higher risk for low mood/depression compared with stroke survivors without aphasia.8,9 The impact of low mood/depression in aphasia has been associated with decreased quality of life, loneliness, poor outcomes from rehabilitation, and higher mortality.10-13 However, poststroke depression is difficult to identify and assess in this population because the majority of depression measures are self-report and language based. 14 Additionally, nonverbal measures of mood/depression and verbal measures that have been developed specifically for aphasia have been shown to be unreliable. 15 As a result, most studies of poststroke depression exclude individuals with aphasia. Moreover, even an intervention study 16 designed to reduce depression in aphasia excluded those with more severe aphasia. The investigators used a measure (ie, Beck Depression Inventory 17 ) that has not been validated in aphasia and indicated that most of the participants with aphasia had relatively intact language comprehension, and those with severe auditory language comprehension deficits were excluded. Therefore, low mood/depression is underdiagnosed in aphasia and our understanding of how low mood/depression presents in aphasia and how best to manage it is limited. 14
Cognitive models of depression posit that the way individuals process negative information plays a role in the onset and maintenance of depression.18-20 Nonneurologically depressed individuals may have an attentional bias to negatively valenced stimuli (eg, faces with negative expressions, negative words [eg, shame, guilt]) compared with nonneurologically nondepressed individuals.21-26 Indeed, a meta-analysis by Armstrong and Olatunji 27 and other studies25,28,29 found that nonneurologically depressed individuals show increased gaze maintenance on negatively valenced stimuli and decreased gaze toward positively valenced stimuli compared with nonneurologically nondepressed individuals. Furthermore, some eye tracking studies have found that nonneurologically nondepressed individuals direct their gaze longer toward positively valenced stimuli compared with depressed individuals.25,29 The use of eye tracking to assess depression in nonneurological populations has also shown high internal consistency as well as test-retest reliability.25,30
Many of these eye tracking studies have used free-viewing tasks with pictorial stimuli to assess depression.21,25 Free-viewing tasks better resemble the natural environment because participants are free to choose which stimuli they attend to. Free-viewing eye tracking tasks with pictorial stimuli do not rely on linguistic processing and may be advantageous to identify low mood/depression in persons with aphasia, regardless of the level of severity of their language impairment.
Eye tracking has successfully been used to assess different linguistic and cognitive functions in persons with aphasia (eg, working memory, 31 lexical-semantic processing, 32 reading, 33 and sentence production 34 ). However, eye tracking has not been used to investigate mood in persons with aphasia. The goal of this study, therefore, was to show the feasibility of using eye tracking as a viable tool for measuring mood in persons with aphasia. We hypothesized that persons with aphasia would be able to participate in a task that required free viewing of emotional and neutral facial expressions and that, depending on the presence or absence of depression, they would show bias to sad or happy faces similar to neurologically healthy individuals. Thus, those individuals with symptoms of depression/low mood would show a bias to sad facial expression and absence of bias to happy facial expressions, whereas those without symptoms of depression/low mood may show a bias to happy facial expressions.
Our study replicated the Duque and Vázquez 29 procedure in that we displayed pictorial stimuli in a free-viewing task for a longer duration (ie, 3.5 s) so that we could investigate both initial attention bias (ie, initial gaze orientation) and gaze maintenance to happy, sad, and neutral faces.
Method
Participants
We recruited 22 persons with aphasia who were participating in a variety of aphasia community groups within an urban rehabilitation facility and 12 cognitively healthy controls. Participants with aphasia had a single left-hemisphere stroke and were at least 6 months postonset without any comorbid conditions affecting communication (eg, Alzheimer dementia and Parkinson disease). They were native English speakers and had normal-to-corrected or normal vision and hearing. Informed consent was obtained by a trained researcher who had experience in communicating with persons with aphasia. While the person with aphasia silently viewed the consent form, the researcher slowly read aloud its contents. If needed, the researcher summarized each paragraph and used written keywords or pictures to augment the summary. Participants with aphasia were asked, “Do you understand?” after each paragraph. They could respond verbally or nonverbally with a “yes” or “no” head nod or by pointing to yes/no written words or images representing yes/no (eg, thumbs up/down). If participants with aphasia were not able to affirm that they understood the contents of the paragraph, the consent process was terminated after 3 tries. All participants were able to provide informed consent, and the study was approved by the Northwestern University institutional review board.
Self-report Measures of Depression
The Center for Epidemiologic Studies Depression Scale–Revised (CESD-R), a 20-item self-report measure of depression was used to assess depression. 35 The scores range from 0 to 60, with higher scores indicating greater depressive symptoms. A score ≥16 has been used to indicate the presence of major depression. The CESD has been used in a variety of populations (eg, community dwelling healthy older adults, 36 cancer, 37 and stroke 38 ). A recent meta-analysis 39 of poststroke depression instruments found the CESD to be optimal in screening persons with stroke for depression. However, the validity of CESD in persons with aphasia has not been established, although a recent study 40 did report the use of CESD as a depression screening instrument in persons with aphasia. We modified the CESD-R for persons with aphasia such that each question was presented on a single page with large font. The CESD-R questions were also read aloud by the testing researcher to facilitate the comprehension of persons with aphasia.
The Dynamic Visual Analog Mood Scales (D-VAMS), a short nonverbal scale for persons with aphasia, was also used to assess mood. 41 D-VAMS consists of 7 bipolar scales (Miserable-Satisfied, Sad-Happy, Distressed-Peaceful, Bored-Excited, Afraid-Calm, Angry-Peaceful, and Sleepy-Alert), which the participants adjust by changing the face of the actor that best reflects their own mood. The average score ranges from 0 to 100, with lower scores indicating negative mood. The D-VAMS has been validated in 11 persons with aphasia, but further investigation in a larger sample of persons with aphasia is needed to establish its validity. 41
Eye Tracking Stimuli
The free-viewing eye tracking task used faces that showed happy, sad, and neutral facial expressions that were selected from the NimStim Face stimulus set. 42 Facial expressions from the NimStim facial stimulus set have been routinely used to investigate depression and mood-related disorders.43-45 The NimStim face stimulus set consists of 672 images of facial expressions from 43 professional male and female actors (ages 21-30 years old). It contains facial expression of happy, sad, angry, fearful, surprised, disgusted, neutral, and calm expressions in color. Facial expression in the NimStim Face stimulus set have been validated with high interrater reliability. 42
We selected 20 images of happy, sad, and neutral facial expressions from 20 actors (10 male and 10 female) from the NimStim Face stimulus set. The same actors displayed the emotional and neutral facial expressions. The average validity rating of the stimuli we selected depicting the chosen emotions was 85%. Images were adjusted to have the same angle, contrast, and luminance. The size of the images was unified to 506 × 650 pixels. Based on previous studies (eg, Duque and Vázquez 29 ), we edited the noninformative aspects of faces (eg, earrings) by layering ellipses of 280 × 360 pixels. This ensured that visual attention to only facial expressions was measured. The experimental stimuli, therefore, consisted of 20 pairs of happy-neutral faces and 20 pairs of sad-neutral faces, with the same actor displaying the emotional and neutral expression in a face pair. Each face pair was presented twice to the participants, for a total of 80 trials. The emotional-neutral face pairs were equidistant from the center (480 pixels from the center of the slide and 515 pixels from the top of the slide).
Apparatus
Eye gaze was tracked using a remote infrared eye tracker with a sample rate of 300 Hz (Tobii Tx-300, Stockholm, Sweden). Participants were seated 65 cm away from the eye tracker, which resulted in a visual angle of 8.8°. Tobii Pro Lab (Tobii, Stockholm, Sweden) was used to display stimuli and record fixations. Two areas of interest (AOIs) reflecting the total area of the facial expression were drawn around the emotional and neutral faces. Only gaze inside the AOIs and to the AOIs was used in our analysis.
Procedure
Before the eye tracking experiment began, a 5-point calibration procedure was done with a maximum threshold of 1°. Each trial began with presentation of a black screen for 500 ms, a white fixation cross for 1000 ms, and the emotional-neutral face pair for 3500 ms (Figure 1). Participants were told to fixate on the white cross when it appeared and then told to look at the faces. No other instructions were provided with respect to viewing facial expressions because we did not want to bias participants’ gaze patterns to different facial expressions. Emotional and neutral faces were presented equally on the left and the right side of the screen. Three practice trials were given, so that participants were familiar with the task prior to the presentation of the test trials. After the completion of the free-viewing eye tracking task, participants completed the CESD-R and the D-VAMS. CESD-R and D-VAMS were completed at the end of the eye tracking session to ensure that participants did not receive an indication that eye tracking tasks assessed mood, which may have altered their eye gaze patterns. Participants with aphasia attended another evaluation session within 7 days of the eye tracking experiment during which their linguistic and cognitive abilities were assessed using the Western Aphasia Battery–Revised (WAB-R). 46

Example of a trial that participants saw starting with a blank screen (500 ms), followed by a fixation cross (1000 ms) and an emotional-neutral face (3500 ms).
Eye Tracking Measures of Depression
Based on previous eye tracking studies of depression,21,23,25 we included measures that reflect both initial gaze orientation and gaze maintenance. Initial gaze orientation measures were as follows:
Direction of initial gaze: location of the first fixation on either the emotional or the neutral facial expression.
Time to first fixation: the time it took for first fixation to occur on either the emotional or the neutral facial expression.
Gaze maintenance measures were as follows:
First fixation duration: the duration of the first fixation on either the emotional or the neutral facial expression.
Total fixation duration: sum of all the fixation durations on either the emotional or the neutral facial expression.
Additionally, some studies have suggested that bias scores may be better indicators of mood status than comparing values from emotional and neutral directly.21,29 Therefore, we calculated bias scores for all our eye tracking measures. Time to first fixation, first fixation duration, and total fixation time bias scores were obtained by subtracting neutral faces from emotional faces. Scores greater than 0 indicate bias toward emotional faces, and scores less than 0 indicate bias toward neutral faces. For direction of initial gaze, we calculated the percentage of trials in which a person fixated on the emotional rather than the neutral face. Scores greater than 50% indicated bias to the emotional face, whereas scores less than 50% indicated bias to the neutral face.
Data Preparation and Analysis
We used linear mixed-effects models 47 to analyze our data with random effects specified for both participants and items. The fixed effects were group (persons with aphasia vs healthy controls) and emotion (happy vs sad). Dependent variables (ie, time to first fixation, first fixation duration, and total fixation duration) were log-transformed. The package “lmerTest” 48 was used to calculate P values for the different models. Tables for mixed models were made with sjPlot package. 49
Correlational analysis was also conducted to explore the relationship between self-report scales of depression and mood (ie, CESD-R and D-VAMS) and eye tracking measures of depression and mood.
Results
Only participants with greater than 65% usable gaze data were included in the analysis. Two participants with aphasia had gaze samples less than 65% and were excluded from further analyses. Therefore, we present data from 20 participants with aphasia. Furthermore, fixations and trials where the AOIs were not fixated upon were removed from the analysis (<1% of data).
Group Characteristics
Groups did not differ in age [t(18.3) = −0.918; P = .38], education [t(20.8) = −0.224; P = .83], CESD-R score [t(23.1) = 0.945; P = .35], D-VAMS score [t(29.4) = −0.952; P = .349], and gender [χ2(1, n = 32) = 3.56; P = .06]. None of the participants had scores above the cutoff for depression on the CESD-R. Participants’ aphasia severity according to WAB-R Aphasia Quotient (AQ) ranged from severe to mild. Table 1 summarizes the participant characteristics.
Descriptive Statistics for PWAs and HCs.
Abbreviations: CESD-R, Center for Epidemiologic Depression Scale–Revised; D-VAMS, Dynamic Visual Analog Mood Scales; HC, healthy control; PWA, person with aphasia; TPO, time postonset (months); WAB-AQ, Western Aphasia Battery-Revised Aphasia Quotient.
Eye Tracking Measures
Initial Gaze Orientation
Time to first fixation
No significant difference between happy and sad emotions was found for time to first fixation, but there was a significant effect of group. Participants with aphasia had a faster time to first fixation on the emotional face compared with healthy controls. There were no significant interactions (Figure 2A). Model parameters are reported in Table 2 in the supplementary material.

Interaction effect of emotional expressions and group with their CIs of raw gaze metrics: (A) time to first fixation (TFF), (B) direction of initial gaze (DIG), (C) first fixation duration (FFD), and (D) total fixation duration (TFD).
Participants with aphasia did not show attentional biases to happy [t(19) = −0.059; P = .95] and sad faces [t(19) = −1.42; P = .17] for time to first fixation compared with neutral faces. The healthy controls also did not show bias to happy [t(11) = −1.50; P = .16] and sad faces [t(11) = −0.11; P = .92] compared with neutral faces (Figure 3A).

Bias scores of (A) time to first fixation (TFF), (B) direction of initial gaze (DIG), (C) first fixation duration (FFD), and (D) total fixation duration (TFD). Scores greater than 0 for fixation measures suggest a bias toward emotional faces, whereas scores less than 0 suggest a preference toward neutral faces. For direction of initial gaze, scores greater than 50% indicate bias toward an emotional face, and scores less than 50% indicate bias to a neutral face.
Direction of initial gaze
No significant main effect of group and emotion on direction of initial gaze was found. However, there was significant Group × Emotion interaction such that initial gaze was less directed to happy faces for participants with aphasia as compared with healthy controls (Figure 2B). Model parameters are reported in Table 3 in the supplementary material.
No attentional bias was observed toward happy faces [t(19) = −2.05; P = .06] and sad faces [t(19) = 1.24; P = .23] for participants with aphasia. Attentional biases were not reported by healthy controls toward happy faces [t(11) = 0.78; P = .45] or sad faces [t(11) = −0.72; P = .49; Figure 3B].
Gaze Maintenance
First fixation duration
No significant effect of emotion and group was found on first fixation duration. However, there was a significant Group × Emotion interaction such that first fixation duration on the sad face was shorter for participants with aphasia as compared with healthy controls (Figure 2C). Model parameters are reported in Table 4 in the supplementary material.
Participants with aphasia showed attentional bias to happy faces [t(19) = 2.44; P = .03] but not to sad faces [t(19) = −1.64; P = .12]. The healthy controls did not show a bias to either happy [t(11) = 0.065; P = .95] or sad faces [t(11) = 1.45; P = .18; Figure 3C].
Total fixation duration
A significant effect of group and emotion was found for total fixation duration. Moreover, a significant Group × Emotion interaction was found such that overall fixation duration was shorter on the sad faces for participants with aphasia as compared with healthy controls (Figure 2D). Model parameters are reported in Table 5 in the supplementary material.
Participants with aphasia showed attentional biases to happy faces [t(19) = 3.26; P < .01] and away from sad faces to neutral faces [t(19) = −3.43; P < .01]. Healthy controls also showed a bias to happy faces [t(11) = 2.50; P = .029] and not to sad faces [t(11) = −1.70; P = .27; Figure 3D].
Discussion
To our knowledge, this is the first study to show the feasibility of using eye tracking to assess mood in persons with aphasia. We were successfully able to use an eye tracking task that assesses depression/low mood in 20 persons with aphasia without excluding individuals who had severe linguistic deficits. We analyzed 2 indices for both initial gaze orientation (ie, time to first fixation and direction of initial gaze) and gaze maintenance (ie, first fixation duration and total fixation duration) to happy, sad, and neutral faces in participants with aphasia and healthy controls. For initial gaze component, time to first fixation, participants with aphasia fixated faster on emotional faces compared with healthy controls but were not biased to either happy or sad faces. For initial gaze component, direction of initial gaze, we found that participants with aphasia directed their gaze less often toward happy faces compared with healthy controls, but no attentional biases were found for either happy or sad faces. However, previous research suggests that initial gaze orientation metrics may not be reliable indices of depression.24,25 Thus, understanding metrics of gaze maintenance components may be more pertinent.
For gaze maintenance components, we found that duration of first fixation and total fixation duration were shorter on sad faces for participants with aphasia as compared with healthy controls. Participants with aphasia also showed positive biases to happy faces with regard to duration of first fixation and were biased away from sad faces to neutral faces with regard to total fixation duration. Healthy controls also showed a bias to happy faces for total fixation duration. In summary, the maintenance components of the eye tracking measures suggest that participants with aphasia had attentional biases to happy faces and an absence of bias to sad faces.
Based on previous eye tracking studies of depression, persons with aphasia in our sample had a positivity bias similar to nonneurologically nondepressed individuals.25,29 The CESD-R and the D-VAMS scores for both persons with aphasia and healthy controls indicated few depression symptoms, with no participant meeting the suggested depression cutoff score on the CESD-R. Although we did not find a correlation between the CESD-R, the D-VAMS, and the eye tracking indices of depression, the measures suggest that our sample of persons with aphasia and healthy controls had few to no depression symptoms. Our results for persons with aphasia are similar to those of Haley et al, 50 who found that the majority of persons with aphasia in their sample rated their mood as being happy on the Visual Analog Mood Scale. Indeed, the rating of happy mood in persons with aphasia in their sample was greater than sad mood by a factor of 2.
It is not clear why the participants with aphasia demonstrated a preference for positive stimuli that was greater than that of the healthy controls. One explanation may be provided by the concept of posttraumatic growth (PTG). PTG refers to positive changes that occur after a traumatic event. 51 PTG has been reported in conditions such as breast cancer (eg, Cordova et al 52 ), cardiovascular disease (eg, Bluvstein et al 53 ), and stroke (eg, McGrath and Linley 54 and Gangstad et al 55 ). For example, Gangstad et al 55 found that as time poststroke increased, PTG increased and depression decreased leading to greater psychological well-being. Participants with aphasia in our sample were all in the chronic stage (ie, post–6 months of stroke) and had taken steps to be involved in various aphasia-related community groups. Their involvement in these groups may have resulted in a positive change associated with an increased social network and learning/relearning of different communication skills. Their PTG may have been captured by their preference for viewing positively valenced stimuli. This hypothesis is consistent with research in women with breast cancer, where it has been demonstrated that attentional preference to positive stimuli reflects increased PTG. 56
Another explanation that has been associated with research in older adults is termed the positivity effect, whereby older adults move away from negative bias associated with early life to a positive bias in older age. 57 This effect has been reported in different contexts such as attention to emotional faces, facial expression recall, and short-term memory.58,59 The positivity effect has been explained through socioemotional selective theory, which posits that because older adults realize that their time is limited, they prioritize goals that improve emotional well-being. 59 Similarly, the participants with stroke-related aphasia may have had a sudden realization about their own mortality, which led them to prioritize their emotional well-being. This emotional well-being may be reflected through their gaze bias toward positively valenced stimuli.
A major limitation of the study is the selection bias of the sample. Participants with aphasia were all part of various community groups and had a strong social network. Research has shown that being part of community groups may improve the psychosocial well-being of persons with aphasia. 60 Therefore, they may not be representative of the general population of persons with aphasia. A logical next step to demonstrating feasibility of eye tracking as a tool for identifying depression in aphasia would be to recruit persons with aphasia who are not part of various community groups. Another limitation is that we did not investigate response to other emotions such as anxiety, which may be more common than depression in persons with aphasia. Thus, future studies should include different facial stimuli to permit the investigation of anxiety and other emotional problems potentially associated with aphasia. It may also be important to include different mood assessments (eg, proxy measures of depression) in future studies to more thoroughly understand depression symptoms and their relation to eye tracking metrics of depression in persons with aphasia. Finally, the small sample size of the study may have influenced our results. Future studies should include a larger sample of persons with aphasia and also investigate the test-retest reliability of eye tracking measures by repeating the free-viewing eye tracking task after a short time.
Implications
Our study has shown that eye tracking may be a feasible tool to assess mood in persons with aphasia. Results from the eye tracking metrics and self-report scales of depression are similar, suggesting that eye tracking may provide evidence of mood/depression symptoms in persons with aphasia following further research of its validity and reliability, and development of a clinical protocol with which to identify depression. Eye tracking technology has been getting cheaper over the past decade and may present a viable option to assess low mood/depression in persons with aphasia in the clinical setting. The use of nonlinguistic free-viewing tasks may be more ecologically valid than traditional language-based depression questionnaires in a population that has differing levels of language impairment.
Supplemental Material
Ashaie_Supp.Tables – Supplemental material for Eye Tracking as a Tool to Identify Mood in Aphasia: A Feasibility Study
Supplemental material, Ashaie_Supp.Tables for Eye Tracking as a Tool to Identify Mood in Aphasia: A Feasibility Study by Sameer A. Ashaie and Leora R. Cherney in Neurorehabilitation and Neural Repair
Footnotes
Acknowledgements
We would like to thank Rosalind Hurwitz, Vickie Li, and Katie McMenamin who assisted with data collection.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Coleman Foundation.
Supplementary material for this article is available on the Neurorehabilitation & Neural Repair website along with the online version of the article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
