Abstract
Background
Problems with visual attention can be an early indication for the emergence of dementia.
Objective
The current research assessed visual attention using three iPad administered, digital cancellation tests.
Methods
Letter and Symbol Cancellation Tests asked participants to circle a specific letter or symbol. On the Mixed Cancellation Test, participants alternated circling a letter, then a symbol. Five outcome variables were tallied: correct hits; mean intra-response pause or “think” time; mean drawing or “ink” time to circle correct hits; mean distance or search between correct targets; and commission errors. All but commission errors were expressed in four cumulative time epochs; 0–30 s, 0–60 s, 0–90 s, and 0–120 s. Using a protocol of paper/ pencil neuropsychological tests, Jak, Bondi criteria were used to classify 145 memory clinic patients into groups suggesting normal cognitive abilities (CL; n = 45); subtle or mild cognitive impairment (MCI; n = 62); and mild dementia (DEM; n = 38).
Results
For correct hits and mean pause/ “think” time, the three groups were dissociated from each other at 60, 90, and 120 s. For mean drawing/‘ink’ time far fewer between-group differences were found. There was no difference across the three tests for mean search. On the Symbol and Mixed Cancellation Tests, MCI and DEM patients produced more commission errors than CL participants.
Conclusions
Faster pause or “think time”, perhaps reflecting better disengagement from circling target items, may underlie better cancellation test performance. When brought to scale, The Rowan Cancellation Tests could be an effective means to screen for MCI and emergent dementia.
Keywords
Introduction
A traditional point of view has been that episodic memory is the earliest neurocognitive disability associated with Alzheimer's disease (AD). However, recent research suggests dysexecutive impairment can also be an early indication for emergent dementia.1,2 For example, in longitudinal research, Carlson and colleagues 3 found that, among some patients, dysexecutive difficulty preceded episodic memory decline by approximately three years and recommended screening for the early appearance of dysexecutive derailment. Similar results have been described by other researchers.4,5 Importantly, disproportionate dysexecutive dysfunction has been linked to declining instrumental activities of daily living (IADL). For example, Libon and colleagues 6 found that, among memory clinic patients with mild cognitive impairment (MCI), subtle to mild problems with IADL activities were highly related to dysexecutive, rather than episodic memory difficulty. Similar results have been reported by other researchers. 7
Among patients characterized with MCI and AD, working memory is likely the most extensively researched construct associated with dysexecutive difficulty.8–12 Less extensively researched are dysexecutive difficulties associated with constructs related to visual attention. Perry and colleagues 13 observed that MCI patients could be differentiated from healthy controls using experimental paradigms measuring visual search and attention. Cancellation tests 14 are commonly used in clinical practice and have been employed to operationally define behavior associated with visual attention.15,16 Cancellation tests typically consist of a complex array of stimuli where patients are asked to simultaneously attend to specific stimulus target items while ignoring foils.
Several studies have investigated deficits related to visual search and attention using cancellation tests among patients with AD. 17 For example, several studies compared patients with AD versus controls using protocols of cancellation tests with increasingly complex visual search and attentional demands.15,18–19 In this research, AD patients identified fewer targets as task demands increased. Foldi and colleagues 15 asked research participants to shift back and forth between target items (i.e., cross out a circle, then a square, etc.), and noted that AD and other patient groups struggled and continued to identify fewer target items. In follow-up research, Foldi and colleagues, 16 administered cancellation tests where there were significant similarities between targets and foils. As physical similarity in target items increased, accuracy among AD patients decreased and more commission errors were produced. With respect to patients with MCI, Wu and colleagues 20 administered the paper and pencil K-T cancellation test 21 to memory clinic patients and found that normal controls, MCI, and dementia participants differed significantly on the number of correct hits. In subsequent research Wu and colleagues 22 replicated these between-group differences using a computer administered cancellation test.
Underlying brain regions involved in cancellation test performance have recently been explored by Deng and colleagues. 23 These researchers found that within their sample of healthy, community dwelling participants, successful test performance was associated with diverse brain regions including bilateral activation involving the cerebellum, the superior temporal lobe, the precentral gyrus, the frontal gyrus, and occipital and parietal areas. Further, these researchers also noted that reduced cancellation test performance was related to reduced brain activity in supplementary motor area, middle occipital gyrus, medial and inferior frontal gyri, and putamen brain regions.
In the current research a paper and pencil neuropsychological protocol was used to statistically classify memory clinic patients as presenting with normal cognitive abilities (CN), MCI, or mild dementia (DEM). A protocol of three iPad- administered digital cancellation tests with increasing attention demands (i.e., circling a target letter, circling a target symbol, and shifting back and forth circling a letter then a symbol) was administered. In addition to tallying total hits or correct responses, a variety of novel time- based and motor- based outcome measures were evaluated. The goal of the current research was to assess how well these new cancellation test outcome measures can differentiate between these three groups.
Methods
Participants
A group of 145 patients (mean age = 73.32 ± 7.23; mean education = 15.38 ± 3.14; female = 47.6%; White = 89.6%) were recruited from the New Jersey Institute for Successful Aging (NJISA) Memory Assessment Program (MAP). All patients underwent a lengthy, standard paper and pencil neuropsychological evaluation, a CT/MRI study of the brain, serum blood tests, and were examined by a board-certified psychiatrist. Exclusion criteria included a history of head injury, substance abuse, and major psychiatric disorders including major depression; B12, folate, or thyroid deficiency; and major neurological/ medical disorders such as epilepsy or cancer requiring chemotherapy, radiation therapy, or significant surgery. This study was approved by the Rowan University institutional review board with consent obtained consistent with the Declaration of Helsinki.
Paper and pencil neuropsychological protocol
The paper and pencil protocol described below assessed three neurocognitive domains—executive control, naming/ lexical access, and verbal episodic memory. Each neurocognitive domain was assessed using three neuropsychological tests for a total of nine neuropsychological test parameters. All test scores were expressed as z-scores derived from normative data.
Executive control
This neurocognitive domain was assessed with three tests including the Boston Revision of the Wechsler Memory Scale-Mental Control subtest, 10 the letter fluency test 24 (‘FAS’); and the Trail Making Test- Part B. 25 The dependent variable for the Mental Control subtest 10 was the total non-automatized accuracy index (ACI). Raw scores from the letter fluency test and Trail Making Test-Part B were scored using demographically corrected norms provided by Heaton and colleagues. 26
Naming/lexical access
This neurocognitive domain was also assessed with three tests, including the 60-item version of the Boston Naming Test 27 ; the “animal” fluency test 24 ; and the Wechsler Adult Intelligence Scale-III Similarities subtest. 28 Raw scores from the Boston Naming Test and “animal” fluency tests were also scored using norms provided by Heaton and colleagues. 26 The dependent variable obtained from the WAIS-III Similarities subtest was the age-corrected scale score.
Verbal episodic memory
This neurocognitive domain was assessed with the 9-word California Verbal Learning Test. 29 The three 9-word CVLT variables used in the current research included total immediate free recall, delayed free recall, and delayed recognition.
Determination of patient groups
Mild cognitive impairment
Jak, Bondi criteria30,31 were used to determine MCI subtypes. Single domain memory impaired MCI was diagnosed when participants scored >1.0 standard deviations below normative expectations on at least two of the three CVLT measures (n = 7). Single domain dysexecutive MCI was diagnosed when participants scored >1.0 standard deviations below normative expectations on at least two of the three executive measures described above (n = 20). The mixed MCI subtype was diagnosed when participants scored >1.0 standard deviation below normative expectations on two scores from two or more of the three neurocognitive domains that were assessed (n = 34). All three of these groups were summed into an aggregate MCI group (n = 61).
Cognitive normal group
Among the memory clinic participants who presented for clinical evaluation, 45 did not meet Jak, Bondi criteria30,31 for MCI. All of these participants were aggregated into a single group and label as cognitively normal (CN).
Mild dementia
The classification for dementia was made clinically using Diagnostic and Statistical Manual of Mental Disorders- 5 32 (DSM-5) criteria. Patients diagnosed with dementia (n = 38) were impaired on many of the nine paper/ pencil neuropsychological tests. Also, based on information provided by the family, instrumental activities of daily living (IADL) were significantly compromised.
The Rowan Digital Cancellation Tests
The Rowan Digital Cancellation Tests consists of a suite of three cancellation tests administered using a 13-inch iPad Pro in landscape orientation. Each test was preceded by a practice trial where the examiner provided both verbal instructions and a demonstration. During all practice and test trials, a buzzer sounded when errors were made. No feedback was provided for correct hits. For all three tests, participants worked for 120 s, with 16 targets located in each quadrant (64 targets total). Target items were embedded in a random array of foils as modeled after Weintraub. 33
Digital letter cancellation test
Patients were asked to circle the letter “A”.
Digital symbol cancellation
Patients were asked to circle a specific star-like symbol.
Digital letter/symbol mixed cancellation
Patients were asked to alternate, first circling a specific letter (i.e., the letter “V”), then a specific star-like symbol, etc.
Digital cancellation outcome variables
Five digital cancellation test outcome variables were analyzed in the current research and are described below. All outcome data was tallied to express performance across four cumulative time epochs, i.e., 0–30 s, 0–60 s, 0–90 s, and 0–120 s.
Correct hits (range 0–64)
The number of correct hits circled in the allotted time.
Average intra-interval pause time/correct hit (‘think time’; range 0–1.00)
This outcome variable averaged intra-interval pause or “think time” (i.e., the average time measured from the conclusion of circling a correct target to the initiation of circling the next correct target), controlled for the number of target items available within any of the four time epochs listed above. This outcome variable was modeled, in part, on research by Geldmarker and colleagues.34,35 This outcome variable was also modeled after digital Clock Drawing Test (dCDT) intra-component latency outcome variables as described by Piers and colleagues. 36 In this research, Piers and colleagues 36 defined a number of dCDT intra-component latencies measuring the time interval in between the production of clock drawing pen strokes (i.e., the latency between drawing the clockface and the next pen stroke. 37
Average drawing time/correct hit (‘ink time’)
The average time needed to draw a circle around all correct hits was tallied and averaged based on the number of correct hits.
Visual search (range 0–1.00)
This outcome variable quantified the average distance traversed as participants circled successive correct targets yielding a score from 0 to 1.00. The score is derived by averaging the rank proximity—a measure of closeness—between the newly selected target and the targets available just before its selection. Essentially, the score shows how often the next correct target was one of the targets nearest the current selection. A low score suggests the next target was relatively far away, while a high score suggests it was relatively near.
Commission errors
The total number of commission errors (e.g., circling the incorrect letter or symbol) for all three tests was tallied.
For the first four outcome variables, all information was tallied into four time epochs: 0–30 s, 0–60 s, 0–90 s, and 0–120 s. These time epochs were designed to represent cumulative behavior and are not designed to be independent of each other such as tallying behavior from 0–30 s, 30–60 s, 60–90 s, 90–120 s. Because of the paucity of commission errors, only the total errors made during the full 120 s time epoch were analyzed.
Statistical analysis
The effect of group for correct hits, correct hit/ pause time (‘think time’), the average drawing time (‘ink time’), and average correct hit/search were assessed with a series of 3 group (CN, MCI, DEM) by 4 (0–30 s, 0–60 s, 0–90 s, 0–120 s) cumulative time epochs repeated measures analyses of variance (ANOVA). Age, education, and sex were co-varied. The dependent variables were the respective outcome variables described above for the four time epochs: 0–30 s, 0–60 s, 0–90 s, and 0–120 s. Because of non-normality, the effect of group for commission errors was assessed with non- parametric Kruskal- Wallis tests with subsequent planned comparison post- hoc analyses. The Bonferroni correction was used for all parametric analyses and significance was set at p < 0.050.
Results
Demographic characteristics
Demographic and related information can be found in Table 1. No differences were found for age. The CN group presented with more years of education versus MCI and DEM groups (p < 0.001, both tests). All three groups differed from each other on the Mini- Mental State Examination 38 (MMSE; p < 0.002, all analyses; Table 1).
Demographic and clinical information.
percent female = 47.6%.
CN: cognitive normal; MCI: mild cognitive impairment; DEM: dementia; MMSE: Mini-Mental State Examination.
Correct hits
For the letter cancellation tests, the multivariate effect for group was significant (Greenhouse-Geisser correction, F = 8.97, df = 3.63, p < 0.001, η2 = 0.114; Table 2). At 30 s CN and MCI outperformed DEM patients (p < 0.008). At 60 s, 90 s, and 120 s all three groups were dissociated from each other (p < 0.011, all analyses). For the symbol cancellation test, the multivariate effect for group was also significant (Greenhouse-Geisser correction, F = 16.81, df = 2.64, p < 0.001, η2 = 0.197). Across all four time epochs, all three groups were dissociated from each other (p < 0.014, all analysis). A similar pattern of performance was seen on the mixed letter/ symbol cancellation Test. The multivariate effect for group was significant (Greenhouse-Geisser correction, F = 19.58, df = 2.57, p < 0.001, η2 = 0.221) where, again, across all four time epochs, all three groups were dissociated from each other (p < 0.010, all analyses; Table 2).
Correct Hits: range 0–64 (mean and standard deviations).
Intra-interval pause time (‘think time’)/correct hit
For the letter cancellation test the multivariate effect for group was significant (Greenhouse-Geisser correction, F = 10.41, df = 3.71, p < 0.001, η2 = 0.130; Table 3). At 30 s, CN and MCI outperformed DEM participants (i.e., faster “think time”; p < 0.024). At 60 s, 90 s, and 120 s all three groups were dissociated from each other (p < 0.050, all analyses). There was a significant multivariate effect for group on the symbol cancellation test (Greenhouse-Geisser correction, F = 13.10, df = 3.05, p < 0.001, η2 = 0.159). Subsequent analyses found that across all four time epochs the three groups were dissociated from each other (p < 0.043), all analyses). There was also a significant multivariate effect for group on the mixed letter/ symbol cancellation test (Greenhouse-Geisser correction, F = 15.68, df = 2.60, p < 0.001, η2 = 0.184). Again, subsequent analyses found that across all four time epochs all three groups were dissociated from each other (p < 0.014, all analyses; Table 3).
Correct hit/ inter-interval pause (‘think’): range 0–1.00 (mean and standard deviations).
Average drawing time (‘ink time’)/correct hit
None of the three repeated measures analyses reached statistical significance. For descriptive purposes, subsequent analyses for all three tests can be found in Table 4. As can be seen, across the three tests some statistical differences appear to be present. However, the data presented in Table 4 should be viewed with caution as all omnibus statistical tests were not significant.
Mean correct hit drawing time (‘ink time’): means and standard deviations (seconds).
Visual search
For this outcome variable, none of the three repeated measures or subsequent analyses were statistically significant (Table 5).
Correct hit/search measure: range 0–1.00 (mean and standard deviations).
Commission errors
Total commission errors after 120 s: (means and standard deviations).
Discussion
In the current research a panel of three new digital cancellation tests was administered to a group of memory clinic patients who were statistically classified into groups suggesting cognitively normal (CN) abilities, mild cognitive impairment (MCI), and mild dementia (DEM). The three cancellation tests were developed with increasingly more neurocognitive demands and were modeled, in part, after tests and methods described in prior research.15,16,18 Correct hits has been the traditional outcome measure used with cancellation tests. 39 In the current research, differences regarding correct hits almost completely dissociated all three groups, across all four time epochs, on all three cancelation tests. Moreover, for this outcome variable, all repeated measures and subsequent analyses were accompanied by large effect sizes.
The Boston Process Approach as related to neuropsychological assessment40–42 suggests that a wide number of subordinate neurocognitive operations are likely associated with any standard summary score, such as total cancellation hits as described above. In this context, a variety of new process time- based and motor- based measures were developed. The intra-interval/ correct hit, i.e., “think time” measure was developed, in part, from prior research using the digital clock drawing test (dCDT). As noted above, Piers and colleagues 36 described a number of discrete time units, termed intra-component latencies, or pauses in between the production of pen strokes in a cohort of community dwelling participants from the Framingham Heart Study (FHS). Similariy, Dion and colleagues 37 analyzed dCDT intra-component latencies, also produced from a group of community dwelling participants, and found that slower pause time, i.e., greater “think time”, was associated with lower scores measuring working memory, processing speed, and language. In the current research, our analogous “think” time was generally able to dissociate all three groups across all three tests. For all of these analyses, effect sizes continued to be quite robust.
With respect to the average drawing or “ink” time to correctly circle target items, all multivariate tests failed to reach statistical significance. Subsequent analyses for the mixed letter/ symbol test appear to reveal some between group differences along with large effect sizes. However, additional research is needed to determine how well this digital outcome variables operates among memory clinic patients.
The visual search measure was designed to measure and operationally define the parsimony in visual search strategies. As described above, a smaller value means that, on average, after circling a correct target, participants tended to circle a near target in relation to their last correct target. None of the statistical tests that assessed for between-group differences for visual search were significant suggesting that, on average, parsimony of search did not differ between group. Finally, with respect to commissions, on average few errors were made on any of the three cancellation tests. However, some between-group differences were seen on the symbol and mixed letter/ symbol cancellation test where CN made fewer errors than other groups.
Theories designed to explain the neurocognitive constructs that underlie visual attention often center around complementary bottom-up versus top-down neurocognitive operations.43–45 Bottom-up operations are thought to reflect stimulus characteristics such as stimulus shape, novelty, or location. By contrast, top-down operations are thought to be related to executively driven, goal-directed behavior such as instructions to ignore distractor items, test contingencies involving the test conditions or interference, as well as the overall capacity to maintain the mental set associated with the task at hand, such as alternating between circling a letter, then a symbol. Prior research with AD patients suggests some relative preservation in bottom-up operations with greater impairment involving top-down operations. 43 However, it has been pointed out that such conclusions need to be viewed with caution.44,45
Posner and Petersen 46 have suggested the existence of three separate, but complementary mechanisms, each involved in a different aspect of visual attention including an alerting network responsible for arousal and maintaining the necessary mental set for the task at hand; an orienting network that operates to direct resources to specific spatial locations and task contingencies, again, consistent with the necessary mental set for the task at hand; and an executive network for the deployment of the necessary neurocognitive resources to deal with disparate or conflicting stimuli. Malhotra and colleagues 47 noted that any one or all three of these mechanisms can be impaired and contribute to visual attention deficits in MCI and dementia. Parasuraman and colleagues 48 developed a spatial cueing task, using constructs from the Posner model and were able to differentiate patients with AD from controls. Constructs involving bottom-up versus top-down neurocognitive operations are not incompatible with the model of Posner and Petersen. 46
In order to successfully navigate the cancellation tests used in the current research, “think time”, “ink time” and visual search measures would appear to involve considerable top-down as well as executive ability. Yet, the between-group differences for these outcome variables were quite different. As detailed above, there were considerable between-group differences with respect to intra-component pause or “think time”, where all three groups were generally dissociated from each other. By contrast, there were considerably fewer differences for “ink time”, and no differences with respect to the visual search outcome measures. These results suggest that differences in comparatively simple motor operations (i.e., the average time needed to circle correct hits) also appear to be modest. These results also suggest parsimony with respect to visual search among MCI and patients with mild dementia.
Difficulty with engagement then disengagement 46 or transitioning from circling one correct response to the next correct response might explain the pattern of behavior described above. In order to effectively locate objects embedded in a complex array, efficient saccadic eye movements operate to both identify requested targets as well as reject competing information. Prosaccadic tasks measure eye movement toward a novel target. By contrast, in antisaccadic tasks, participants are required to move their eyes toward the opposite side of a target. In this context, antisaccadic behavior is viewed as requiring the capacity to inhibit an automatic response and therefore thought to provide a measure of executive control. In prior research, Eraslan Boz and colleagues 49 observed diminished numbers of correct antisaccades along with greater numbers of uncorrected errors when patients with MCI were compared to healthy controls. Chehrehnegar and colleagues 50 also found that MCI patients made significantly more saccade errors, more uncorrected errors, and more target omission in antisaccade tasks, versus healthy controls. Peltsch and colleagues 51 made similar observations when they compared MCI and AD patients versus healthy controls. Therefore, it is possible that the differences for the “think time” outcome measures developed in the current research reflect derailed saccadic activity involved in rejecting foils as well as correctly selecting a valid target.
The current research is not without limitations. For example, while the outcome measures described above yield interesting between-group differences, it is not clear whether the digital outcome measures used in the current research are best understood in the context of a broad, single underlying construct, or several unique or separate underlying constructs. Also, the incremental validity of the panel of Cancellation outcome measures described above, say, for clinical decision making, as compared to other more commonly used neuropsychological tests remains an open question and is an area for further research.
Second, several methodological issues need to be acknowledged. For example, is should be understood that the results described above, reflected in cumulative test epochs, are measuring aggregated test performance rather than separate, independent time-based behavior. Also, greater research is needed to assess the effects of critical demographic variables such as age, education, and sex. Using a buzzer as a means to signal errors could have caused distraction. Also, the current research relied on a single test to assess memory (i.e., the CVLT) where several tests were used to assess other neurocognitive domains. This could have biased our results.
Third, our cognitively normal group was derived from our memory clinic sample. Future research would benefit from an analysis using a non-clinical, community dwelling sample. Fourth, future research should also assess how and/ or if dementia and MCI subtypes could be dissociated with the outcome measures described above.
Nonetheless, the current research has several strengths. The digital cancellation tests used in the current research are well tolerated, require little time for test administration, and are automatically scored. Second, the three-test cancellation portfolio, along with the data described above suggesting that patient groups can be dissociated across different time epochs, provides considerable clinical flexibility. Third, the tests used in the current research were designed with minimal linguistic content so as to maximize deployment regardless of patients’ preferred language. Fourth, MCI groups were determined using validated statistical criteria. Thus, when brought to scale, the assessment of visual attention using the tests described above could provide a much needed tool to screen for emergent MCI and dementia- related illness.
Footnotes
Acknowledgements
The authors have no acknowledgments to report.
Ethical considerations
The study was conducted in accordance with the Declaration of Helsinki and was approved by the IRB Committee of Rowan University (no. pro 2016001115) on June 11, 2025, with the need for written informed consent waived.
Consent to participate
Not applicable
Consent for publication
Not applicable
Author contribution(s)
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Libon consults to Linus Health, Inc.; Dr. Libon receives royalties from Linus Health, Inc. and Oxford University Press. Dr. Libon is an editorial board member of this journal but was not involved in the peer-review process of this article nor had access to any information regarding its peer-review.
Data availability statement
The data that support the findings of this study are available from the corresponding author with Rowan University IRB approval. Some restrictions may apply.