Cognitive Assessment with Cognivue Clarity ® : Psychometric Properties and Normative Ranges in a Diverse Population

Abstract

Background:

Detecting cognitive impairment in clinical practice is challenging as most instruments do not perform well in diverse samples of older adults. These same instruments are often used for eligibility into clinical trials making it difficult to recruit minoritized adults into Alzheimer’s disease (AD) studies. Cognivue Clarity^® is an FDA-cleared computerized 10-minute cognitive screening platform using adaptive psychophysics to detect cognitive impairment.

Objective:

Test the ability of Cognivue Clarity to measure cognitive performance in a diverse community sample compared with the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS).

Methods:

This study enrolled 452 participants across 6 US study sites and completed both Cognivue Clarity device and RBANS. Psychometric properties and exploratory factor analysis of Cognivue Clarity were explored and comparisons against RBANS across different age, sex, education, and ethnoracial groups were conducted.

Results:

Participants had a mean age of 47.9±16.1 years (range: 18–85), 63.6% were female, 45.9% had ≤12 years of education, 31.2% were African American and 9.2% were Hispanic. Cognivue Clarity had strong internal consistency, test-retest reliability and minimal practice effects. A 4-factor structure (Memory, Attention, Visuomotor, and Discrimination) had excellent goodness-of-fit. Normalizing age effects improved performance. Race and education effects were similar to those seen with RBANS. Cognivue Clarity had strong correlations with RBANS.

Conclusions:

Our study supports the use of Cognivue Clarity as an easy-to-use, brief, and valid cognitive assessment that measures cognitive performance. In the correct clinical setting, Cognivue Clarity may identify individuals with likely cognitive impairment who could be candidates for AD research studies.

INTRODUCTION

Alzheimer’s disease and related dementias (ADRD) affect over 6.7 million people in the US [1] and more than 55 million people worldwide [2]. Detection of the early stages of ADRD is a clinical challenge with many people coming to medical attention and diagnosis at the moderate stage. Mild cognitive impairment (MCI) represents a prodromal state of ADRD, and recent reports suggest that nearly 80% of people living with MCI are never diagnosed [3]. The delays in diagnosis may be particularly relevant to individuals from racial and ethnic minorities, socioeconomically disadvantaged populations, and geographic locales that lack specialty memory care settings (i.e., rural areas) and who are at higher risk for ADRD [1]. There are likely many causes contributing to low rates of diagnosis including access to care, access to specialists, under- or non-insurance, disease stigma, and health literacy [4]. One potentially addressable cause is low rates of screening for cognitive impairment with a culturally valid and sensitive measure that performs well across different sociodemographic and ethnoracial groups and does not require extensive staff or physician time.

Dementia screening could capture cases at the earliest possible stage in at-risk populations permitting early intervention and enrollment into trials [5, 6]. However, at the present time, dementia screening is not endorsed by the US Preventive Services Task Force [7]. Reasons for this lack of recommendation include both lack of clear understanding of benefits versus harms of screening, and lack of agreement about the utility of commonly used screening tests such as the Mini-Mental State Examination (MMSE) [8]. Although the Medicare Annual Wellness Visit (AWV) includes assessing for cognitive impairment as part of the required elements, there is no agreement on how this could be done and less than 25% of Medicare beneficiaries receive AWV as part of their medical care [9, 10]. The most commonly used screening instruments in the clinic and for clinical trial inclusion are the MMSE [8] and the Montreal Cognitive Assessment (MoCA) [11]. The MMSE has well recognized education and ethnoracial biases [12] including the serial subtraction task and lacks sensitivity to detecting MCI [13]. The MoCA also has educational and cultural biases including clock drawing, animal naming, and serial subtraction [14], which is only partly corrected by adding one point to those individuals with 12 years or less of education. Although a basic form of the MoCA was developed for low literacy populations [15], this format is likely not ideal for clinical trial screening.

The lack of screening and early detection, and delay in diagnosis of MCI and mild AD limits access of eligible older adults for treatment with newer disease modifying medications. Recently approved amyloid lowering therapies (i.e., Lecanemab) are only indicated for early-stage AD [16] and with most new cases diagnosed at the moderate stage [1, 4], this represents a missed window of opportunity. Further, late detection limits the opportunity for interested and otherwise eligible older adults to participate in clinical trials to test new diagnostics and therapeutics, or in other clinical research projects to further our understanding of ADRD. Lastly, even when eligible individuals are identified and enrolled into clinical trials, the research sample rarely reflects the diversity of the US population. While racial and ethnic minorities comprise 39% of the US population [17], they account for only 2–16% of clinical trial participants [5, 18].

Of 4,105 US clinical trials registered in ClinicalTrials.gov from March 2000 to March 2020 that reported race/ethnicity data, the majority of enrollees were White (median 79.7%) [18], outpacing the White population in the 2020 US Census of 71% [17]. This same report found median participation in clinical trials at 10% for African Americans, 6% for Asians, and 0% for American Indians [18]. In a separate analysis [19], participation in one company’s clinical trials by Hispanic/Latino enrollees was 15.9%, though they represent 18.7% of the US population.

Data is less available for participation in AD clinical trials. In a systematic review of 1305 articles, only 50 publications met inclusion criteria. Overall, 78.4% of patients with AD were White, 13.0% were Asian, and a cumulative 4.4% were Black or Hispanic. The median percentage of White participants in AD clinical trials was 92.5% [20]. In neuroimaging studies of AD, the median representation in 719 studies was 88.9% White or 87.4% Non-Hispanic White, 7.3% Black/African American, 3.4% Hispanic/Latino, and 0% Asian, Native Hawaiian/Pacific Islander, and American Indian/Alaska Native, Multiracial, and Other Race participants [21]. This is problematic as racial and ethnic minorities and socioeconomically disadvantaged groups have a higher lifetime risk of AD, experience a younger age of onset, present to medical attention at more advanced stages, and live more years with cognitive impairment [22]. Similar disparities are seen in between sex and educational level of the US population and that enrolled in clinical trials. Women make up 50.4% of the US population, but only 41.2% of participants in clinical trials [23]. Although 63% of US residents have less than a bachelor’s degree, 55.8% of participants in clinical trials had a college or advanced degree [24, 25].

To address these unmet needs, we conducted a study of Cognivue Clarity^®, the first FDA-cleared computerized cognitive test [26–28] to screen for cognitive impairment, in a diverse US sample across different age, education, sex, race and ethnicity strata. The Cognitive Testing in Diverse Populations to Further the Objective and Clinical Understanding of Cognivue Study (FOCUS) study was used to calculate age-normed scores, assess performance across different sociodemographic variables, measure test-retest reliability and potential practice effects, and establish validity against a Gold Standard neuropsychological battery, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), commonly used both in clinical practice and in AD clinical trials [29–31].

METHODS

Study design

FOCUS was an open-label, single visit, multi-site validity and reliability study that enrolled participants to assess the psychometric properties and performance of Cognivue Clarity across a diverse population. Inclusion criteria were (a) age ≥18 years, (b) fluency in English, (c) community dwelling, (d) overall good health with no acute medical illness, (e) full vision in at least one eye, (f) full use of at least one hand, and (g) ability to provide informed consent. Exclusion criteria were: (a) presence of an acute medical illness, (b) residing in dependent care facility or in hospice, and (c) inability to be tested in English. Participants underwent a one-time study visit that lasted up to two hours. Following informed consent, each participant completed a sociodemographic questionnaire and was then randomized for assessment groups for 90 minutes of testing on two devices: Cognivue Clarity (10-minute assessment) or an Apple iPad to complete the RBANS (30-minute assessment). Groups were randomized to either Cognivue Clarity followed by RBANS or RBANS followed by Cognivue Clarity. In order to test practice effects and test-retest reliability, for the FOCUS study each participant completed three consecutive trials of Cognivue Clarity with a 5-minute break between tests. This is distinct from the real-world administration that involves one administration lasting 10 minutes. The FOCUS study was deemed exempt by the Advarra Institutional Review Board (Pro00064617).

The primary objective of FOCUS was refinement of scoring and normative ranges with the Cognivue Clarity across different ages, sexes, race and ethnic groups, and levels of education. Secondary study objectives included determination of the level of practice effects, psychometric properties, and assessment of the overall score recommendations of Cognivue Clarity. We anticipated that most individuals in this study would perform normally as it was not selected to be an at-risk population, but it was also likely that some individuals would have subnormal performance suggesting impairment. Therefore, we compared the performance on Cognivue Clarity to a Gold Standard neuropsychological test, the RBANS, that had already conducted studies in individuals age 18+ with age-normed values.

Demographics and self-reported history

Participants provided self-reported information on age, sex, educational attainment (less than high school, high school degree or equivalent, associate degree, bachelor degrees, master or other graduate degree), racial identity (White, Black/African American, Asian, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Two or more races), ethnicity (Hispanic, Non-Hispanic), marital status, employment status, smoking history, alcohol consumption, social activity, exercise, hearing loss/use of hearing device, use of vision correction, and self-reported medical history (e.g., diabetes, hypertension, obesity, head injury, attention deficit disorder, cancer, COVID-19).

Cognivue Clarity^®

The automated Cognivue technology utilizes adaptive psychophysics assessing baseline motor skills and visual acuity to test information processing and eliminating biases that can be found in common cognitive testing mechanisms [32]. Adaptive psychophysics is the process whereby the physical characteristics of the stimuli in each trial are determined by both the stimuli and the responses to the stimuli that occurred in the previous trial [33].

Cognivue Clarity is a modified laptop computer with only a screen and flywheel device for measuring responsiveness. The patient interaction is through the CogniWheel^®, an easy-to-maneuver wheel that provides a single point of contact to select the answer. The technology collects 130,000 data points per administration, and provides a customized assessment based on each individual’s visual and motor skills. Cognivue Clarity is self-administered and takes 10 minutes to complete. An introductory video demonstrating the device and the tests can be found at https://cognivue.com/videos/.

Cognivue Clarity is comprised of 10 subtests (Fig. 1) consisting of two validity performance measures (Adaptive Motor Control, Visual Salience) that are used to set the adaptive psychophysics profile. There are four discrimination tests (Letter, Word, Shape, Motion), and four memory tests (Letter, Word, Shape, Motion). The Adaptive Motor Control subtest measures the speed at which the participant is able to manipulate the flywheel on the device. The Visual Salience subtest measures the threshold of the participant to determine visual contrast of the target stimuli. The Discrimination tests present a Letter, Word, Shape or Motion stimuli that the participant must match compared with non-target stimuli manipulating a flywheel. The Memory tests present a Letter, Word, Shape, or Motion stimuli the participant must first immediately recall and then tests delayed recall using a series of n-back paradigms.

Fig. 1

Representative Panels of the 10 Subtests in Cognivue Clarity demonstrating the (a) two tests used for the adaptive psychophysics component – Adaptive motor skill (A) and visual salience (B); (b) the 4 tests used for Discrimination – Letter discrimination (C), Word discrimination (D), shape discrimination (E), and motion discrimination (F); and (c) the 4 tests used for delayed memory – Letter memory (G), Word memory (H), shape memory (I), and motion memory (J).

After completion, the Cognivue Clarity device provides immediate results in a report (Fig. 2) and/or in a comma-separated values file. Results include an overall score, four domain specific scores, reaction times, and 10 subtest scores, as well as response tracings to assess performance validity. Score interpretations are provided as normal, borderline, or impaired performance based on prior testing consistent with the FDA-filing [26, 27]. Based on analyses conducted with the FOCUS data, we hypothesized the thresholds could change.

Fig. 2

A sample report generated by Cognivue Clarity. Panel A demonstrates the scores derived from Cognivue Clarity including a global performance, 4 domains, 10 subtests, and 2 reaction time measurements. Panel B demonstrates the calibration and subtest response tracings as well as the interpretation of those tracings.

Repeatable battery for the assessment of neuropsychological status (RBANS)

The RBANS (Pearson Assessments) Form A was administered to each participant either before or after the Cognivue Clarity using an Apple iPad^® and the standardized protocol and scoring algorithm (Q-Interactive^®) from the vendor. The RBANS is commonly used as an assessment for dementia and other neurocognitive disorders and has been used both for screening and as an outcome in ADRD clinical trials [29–31] and therefore was selected as the Gold Standard for comparison in FOCUS. Four parallel forms are available with age-normed values from 12–89 years. The RBANS provides a total score, 5 index standard scores (Immediate Memory, Delayed Memory, Visuospatial/Constructional, Language, and Attention), and can be completed in 30 minutes. An RBANS score of less than 85 is considered abnormal [29].

Statistical analyses

Statistical analyses were conducted using IBM SPSS v28 (Armonk, NY) and R (version 4.3.1). Descriptive statistics were used to summarize overall sample characteristics. Student t-tests or One-way analysis of variance (ANOVA) with Tukey-Kramer post-hoc tests were used for continuous data and Chi-square analyses were used for categorical data.

A total of 629 individuals completed Cognivue Clarity. One individual did not have age recorded and 49 individuals had an undetermined educational attainment; this left a sample size of 579 for the exploratory factor analyses, internal consistency, and group comparisons using only the first administration of Cognivue Clarity. Another 127 individuals had questionable performance validity with score swings greater than 15 points between the three Cognivue Clarity trials. The demographic characteristics of these 127 individuals were not different from the larger sample although their mean scores were lower. As general observation, the two most common issues were (a) a first testing score lower than the 10% quartile of the overall scores suggesting that these individuals randomly chose answers before having two consistent scores the 2nd and 3rd trial, or (b) the second administration that had the widest swings in scores suggesting invalid efforts in taking the repeat test. There were no instances of a high score on the first try, followed by two low scores. These fluctuations detrimentally affected reliability estimates. This fluctuation in scores was approximately 10% higher than the acceptable base rate of noncredible performance in clinical non-forensic settings [34–36]. Removal of the 127 left a sample size of 452 participants for measurement of practice effects and test-retest reliability. Of these 452 participants, 110 had missing RBANS data, so the sample size for direct comparisons between Cognivue Clarity and RBANS was reduced to 342. Finally, a sensitivity analysis was performed removing 92 individuals who self-reported a neurological (e.g., head injury, transient ischemic attack, multiple sclerosis) or psychiatric (e.g., attention deficit disorder, depression, bipolar disorder) diagnosis. This was to ensure that Cognivue Clarity scores were not unduly influenced by neuropsychiatric conditions and to ensure results reflected a normative sample.

Internal consistency was examined as the proportion of the variability in the responses that is the result of differences in the respondents, reported as the Cronbach’s alpha reliability coefficient. Coefficients greater than 0.7 are good measures of internal consistency. Test-retest reliability was determined using the Spearman-Brown prediction formula.

A scree plot and exploratory factor analysis (EFA) with Oblique rotation was used to determine the individual factor loading of the 10 Cognivue subtests. Oblique rotation allows factors to be correlated with each other, frequently resulting in more interpretable factor loadings. We fit three possible models and examined the fit indices including Root Mean Square Error of Approximation with 90% confidence intervals (RMSEA), Chi-Square fit index, and the Tucker-Lewis Index (TLI). For RMSEA, values less than 0.05 are considered good. Ideally the Chi-Square test should be non-significant (with more than 0 degrees of freedom); however, this is commonly not achieved in real data as sample sizes sufficient for factor analysis are also frequently large enough to detect even minor deviations between the estimated factor structure and the data. A good TLI is above 0.95. The factors derived from the EFA were then evaluated with a principal components analysis biplot which revealed 4 clear clusters replicating the EFA. A correlation heatmap was created to visualize associations between Cognivue Clarity average and subtest scores and RBANS total and index scores using Pearson product-moment correlation coefficients.

Group validity was assessed by examining Cognivue Clarity and RBANS by sample characteristics (sociodemographic and lifestyle characteristics, medical history). Linear regression was used to test the association between Cognivue Clarity and sociodemographic characteristics (e.g., age, race, education). To further study race effects, a stratified analysis was performed comparing performance on Cognivue Clarity and RBANS between White and Black participants. Unadjusted analyses demonstrated differences in age and education so adjusted analyses were performed. Cognivue Clarity was adjusted for age and education while RBANS, which is already age-normed, was adjusted for education only.

Cognivue Clarity scores were residualized on age to provide optimized age-norming for comparisons to RBANS scores which were already age-normed. Cognivue Clarity total and subtest scores were compared by cognitive status based on RBANS score cutoffs (<85) for cognitive impairment. Receiver operator characteristics (ROC) curves and area under the curve (AUC) were used to test discriminative properties of Cognivue Clarity and determine sensitivity, specificity, and optimal cut-points (by using closest top-left criteria). Multiple comparisons were addressed using the Bonferroni correction.

RESULTS

Sample characteristics

FOCUS was conducted from September 8, 2022, to December 16, 2022, and included 452 subjects from 6 sites covering all 4 regions of the US (Northeast, Midwest, South, West) with valid Cognivue Clarity data. Participants had a mean age of 47.9±16.1 years (range: 18–85), 63.6% were female, 45.9% had 12 or less years of education, 35.9% were married, 41.9% were employed full time, 17.1% were current smokers, 20.0% consumed 2 or more alcohol drinks per week, 21.9% exercised daily, and 21.5% participated in daily social activities. The racial identity of the sample was 63.4% White, 31.2% Black/African American, while 5.4% reported other races. Hispanic ethnicity was reported in 9.2% of the sample. The sample self-reported common health problems including obesity (41.5%), diabetes (1.9%), hypertension (43.7%), cancer (4.8%), prior history of head injury (7.2%), vision correction (54.9%), hearing loss (3.7%), mood disturbance (6.3%), and COVID-19 (60.3%). The mean Cognivue Clarity score for the sample was 80.2±11.4 (range: 33–99) and the mean RBANS total score was 93.0±15.4 (range: 48–146).

Psychometric properties of Cognivue Clarity

Cognivue Clarity exhibited strong internal consistency and test-retest reliability. The Cronbach’s alpha was 0.812 (95% CI: 0.785–0.837); p < 0.001 and the test-retest reliability was 0.85. After eliminating individuals with exceptionally large swings between measurement occasions (subjects with between measurement differences of 15 or higher) the practice effect for Cognivue Clarity was 2.99 points per measurement, or approximately a 0.20 SD increase in score.

An exploratory factor analysis was conducted to establish the underlying data structure of Cognivue Clarity. The scree plot of the 10 subtests suggested a 1-, 3-, or 4-factor structure with fit indices shown in Table 1. The results suggest that not only is the 4-factor model the best fit, but it is also a good fit to the data, with all three fit indices in the acceptable range. The 4-factor model with factor loadings, sum of squares, eigenvalues, and variance are shown in Table 2. A delayed memory factor was the strongest, with high loadings from all memory scores. An executive attention factor (Letter and Word Discrimination), a visuomotor factor (Adaptive Motor Control and Visual Salience) and a discrimination factor (Shape and Motion Discrimination) were also defined. The factor structure had a cumulative variance of 0.52.

Table 1

Goodness of fit indices

Scree plot options	RMSEA (90% CI)	Chi-square test	TLI
One factor	0.142 (0.131, 0.154)	p < 0.0001	0.75
Three factors	0.052 (0.035, 0.07)	p < 0.0001	0.97
Four factors	0.029 (0, 0.055)	p < 0.11	0.99
RMSEA, Root Mean Square Error of Approximation; TLI, Tucker-Lewis Index

Table 2

Factor analysis of cognivue clarity

Cognivue subtests	Factor 1 memory	Factor 2 attention	Factor 3 visuomotor	Factor 4 discrimination
Adaptive motor control			0.81
Visual salience			0.61
Letter discrimination		0.37
Word discrimination		0.86
Shape discrimination				0.39
Motion discrimination				0.59
Letter memory	0.67
Word memory	0.85
Shape memory	0.71
Motion memory	0.48
Eigenvalue	3.89	1.37	0.99	0.75
Sum of squares loading	2.03	1.34	0.99	0.85
Proportion variance	0.20	0.13	0.10	0.08
Cumulative variance	0.20	0.34	0.44	0.52

Cognivue Clarity performance across sociodemographic groups

Table 3 shows the performance on Cognivue Clarity and RBANS by sociodemographic categories of age, education, sex, ethnicity, and race. There was no difference in Cognivue Clarity performance between men and women, or between individuals by ethnicity. There was a significant age effect (p < 0.001) with 18–39-year-olds performing best and individuals over age 70 performing worst. There were significant differences in performance by highest educational attainment (p < 0.001) with individuals with less than a Bachelor’s degree performing similarly and individuals with Bachelor’s degrees and graduate degrees performing similarly.

Table 3

Group comparisons of cognivue and RBANS scores

	Sex			Race			Ethnicity
	Men	Women	p	White	Black	p	Non-Hispanic	Hispanic	p
Cognivue	79.6 (12.0)	80.6 (11.1)	0.447	81.6 (10.7)	77.0 (12.6)	<0.001	80.2 (11.6)	82.4 (9.4)	0.143
RBANS	93.5 (16.6)	92.8 (14.7)	0.692	96.9 (14.3)	85.2 (14.9)	<0.001	93.4 (15.5)	89.5 (14.3)	0.208
	Age Categories
	18–29	30–39	40–49	50–59	60–69	70+	p
Cognivue	86.3 (8.8)	85.3 (8.9)	80.5 (11.6)	79.9 (9.7)	75.4 (11.8)	70.8 (17.5)	<0.001 a
RBANS	89.5 (15.6)	86.2 (14.2)	98.3 (13.6)	94.5 (14.2)	94.6 (15.9)	93.1 (17.3)	<0.001 b
	Highest Educational Attainment
	<High School	High School	Associate	Bachelor	Graduate	p
Cognivue	73.4 (14.1)	79.2 (12.0)	79.5 (11.6)	83.6 (9.6)	82.9 (8.1)	<0.001 c
RBANS	83.8 (9.8)	87.4 (15.3)	94.0 (12.1)	97.1 (12.7)	106.2 (15.2)	<0.001 d

Mean (SD), Bold p-values are significant after correction for multiple comparisons (adjusted p = 0.01). Post-Hoc Analyses. ^aAge Groups (Cognivue): 18–29 and 30–39 not different; 40–49 and 50–59 not different, all other groups different from each other. ^bAge Groups (RBANS): 18–29 and 30–39 not different, 40–49, 50–59, 60–69, and 70 + not different from each other, all other groups different from each other. ^cEducation Groups (Cognivue): High School not different from < High School or Associate, Bachelor not different from Graduate, all other groups different from each other. ^dEducation Groups (RBANS): High School not different from < High School, Associate not different from Bachelor, all other groups different from each other. RBANS, Repeatable Battery Assessing Neuropsychological Status.

For race, only comparisons between White and Black participants were considered since there were too few participants in other racial groups to provide meaningful conclusions. There were significant differences between White and Black participants on Cognivue Clarity (p < 0.001). There was no interaction between race and education supporting that Cognivue Clarity performance differences in Black participants are not due to differences in education. Differences in scores were further analyzed by conducting stratified analyses comparing White and Black participants by Cognivue Clarity overall score and subtests, and RBANS total score and index scores (Table 4). In unadjusted analyses for Cognivue Clarity, African American participants performed worse than White participants in the average score and the motion discrimination, shape memory, and motion memory subtests. After adjusting for age and education, Adaptive Motor Control scores were also different. For RBANS, the total score and all index scores were different in unadjusted analyses with only delayed memory index losing significance after adjusting for education.

Table 4

Race stratified analyses for cognivue clarity and RBANS

Variable	White	Black	Unadjusted p	Adjusted pa
Age	50.9 (16.3)	45.7 (15.6)	0.009	–
Education, % 12y or less	37.0	64.0	<0.001	–
Cognivue average score	81.9 (10.7)	76.4 (12.7)	<0.001	<0.001
Cognivue adaptive motor control	55.3 (15.6)	49.6 (17.3)	0.008	<0.001
Cognivue visual salience	80.7 (13.1)	79.8 (14.6)	0.612	0.089
Cognivue letter discrimination	75.7 (14.0)	72.0 (15.2)	0.055	0.020
Cognivue word discrimination	78.3 (16.2)	78.2 (15.4)	0.967	0.330
Cognivue shape discrimination	86.9 (14.5)	81.7 (16.8)	0.012	0.004
Cognivue motion discrimination	83.8 (18.6)	73.9 (21.7)	<0.001	<0.001
Cognivue letter memory	82.6 (16.6)	81.1 (16.6)	0.460	0.407
Cognivue word memory	88.9 (15.8)	83.7 (19.3)	0.026	0.008
Cognivue shape memory	80.6 (21.4)	71.4 (24.9)	0.002	0.001
Cognivue motion memory	82.3 (22.5)	71.8 (29.8)	0.003	0.002
RBANS total score	97.3 (14.4)	85.7 (15.1)	<0.001	<0.001
RBANS immediate memory index	91.6 (16.5)	81.5 (16.8)	<0.001	<0.001
RBANS delayed memory index	97.8 (16.3)	90.6 (18.0)	0.001	0.016
RBANS attention index	102.3 (17.9)	93.9 (19.3)	<0.001	0.005
RBANS visuospatial index	103.4 (15.5)	91.6 (18.0)	<.001	<0.001
RBANS language index	95.1 (13.0)	87.6 (16.0)	<.001	0.003

Mean (SD) or %, Bold p-values are significant after correction for multiple comparisons (adjusted p = 0.0045 for Cognivue; adjusted p = 0.0083 for RBANS), ^aCognivue adjusted for age and education, RBANS adjusted for education only. RBANS, Repeatable Battery Assessing Neuropsychological Status.

Age adjustment of Cognivue Clarity

To explore the age effect, a simple linear regression was fitted and demonstrated a strong association between age and Cognivue Clarity performance. Increasing age decreases the average Cognivue Clarity score (94.38 – 0.297×age, R² = 17.2%). We also fit a multiple linear regression model with race and education and found that White participants scored on average 5.47 points higher than Black participants matched for age and educational attainment (R² = 27.8%). Education did not provide additional fit to the regression model.

Comparison of Cognivue Clarity performance to RBANS

Since no formal clinical evaluation was included in FOCUS, the RBANS was treated as a gold standard for cognitive performance with a cut-point of 85 to represent possible cognitive impairment. A total of 342 participants had complete and valid Cognivue Clarity and RBANS data available. Table 3 shows the RBANS performance across the different sociodemographic groups with significant differences by education and race, similar to what is seen with Cognivue Clarity. The Pearson’s correlation between Cognivue Clarity and RBANS score was 0.385 with an AUC 0.666 (95% CI: 0.601–0.731) for RBANS impaired versus RBANS not impaired. Since RBANS scores are provided as age-normed scores, residualizing the Cognivue Clarity score on age improved the correlation to 0.495, and the AUC to 0.730 AUC. To achieve the optimal cut point age-normed Cognivue Clarity score (residual) based on the post-hoc analyses from Table 1, we categorized age into three groups (18–39, 40–59, and 60+), then de-age normed back the original Cognivue Clarity score. Table 5 shows the AUC, sensitivity, specificity, and optimal cut for Cognivue Clarity score for each age group.

Table 5

Age-strata performance of cognivue compared with RBANS

	Age 18–39			Age 40–59			Age 60+
	RBANS not impaired	RBANS impaired	p	RBANS not impaired	RBANS impaired	p	RBANS not impaired	RBANS impaired	p
Cognivue average score	88.8 (5.8)	82.4 (9.2)	<0.001	82.5 (9.2)	74.1 (10.8)	<0.001	76.9 (9.7)	64.9 (13.8)	<0.001
RBANS total score	98.1 (8.7)	74.1 (9.7)	<0.001	101.3 (9.6)	75.3 (9.5)	<0.001	101.1 (13.3)	76.3 (6.4)	<0.001
Impaired by cognivue, %	19.4	53.3	<0.001	34.9	76.0	<0.001	39.3	60.7	<0.001
Area under the curve (95% CI)	0.695 (0.593–0.797)			0.716 (0.613–0.819)			0.750 (0.651–0.849)
Best cut point	86			81			68
Sensitivity	65.1			65.8			88.4
Specificity	65.1			72.4			57.1

Mean (SD) or %, Bold p-values are significant after correction for multiple comparisons (adjusted p = 0.017). RBANS, Repeatable Battery Assessing Neuropsychological Status.

We conducted a principal component analysis (PCA) bi-plot of age-normed scores from the 10 Cognivue Clarity subtests with RBANS scores and categorized participants into four clear subgroups consistent with domains derived from the factor analysis (data not shown). We then calculated the correlation of the 10 age-normed subtests scores with the five RBANS indexes (Immediate Memory, Visuospatial/Constructional, Language, Attention, and Delayed Memory). A correlation heatmap was created to visualize associations between Cognivue Clarity average and subtest scores and RBANS total and index scores using Pearson product-moment correlation coefficients. Figure 3 shows that the Motion Memory, Shape Memory and Word Memory scores are highly correlated with all RBANS indexes except language index, while the letter Memory score is more weakly correlated with these same RBANS indexes. The Motor, Visual, Motion Discrimination and Shape Discrimination scores are correlated with RBANS Visuospatial/Constructional index and attention domain index, while the Letter Discrimination and Word Discrimination scores are correlated with RBANS attention domain index only.

Fig. 3

Correlation Map for Cognivue Clarity and RBANS. A correlation heatmap was created to visualize associations between Cognivue Clarity average and subtest scores and RBANS total and index scores using Pearson product-moment correlation coefficients. Motion memory, shape memory and word memory scores are highly correlated with all RBANS indexes except language index, while correlation is weaker between letter memory score and these same RBANS indexes. The motor, visual, motion discrimination and shape discrimination scores are correlated with RBANS visuospatial/constructional index and attention domain index, while the letter discrimination and word discrimination scores are correlated with RBANS attention domain index only.

Sensitivity analysis

While only self-reported medical history was available, 92 individuals reported neurologic (e.g., head injury, stroke) or psychiatric (e.g., attention deficit disorder, bipolar disorder, depression) conditions. These individuals had a mean age 46.9±15.7 (range 18–73), 23.9% were African American, 11.9% were Hispanic, and 41.3% had 12 years or less of education. To ensure that our results were not influenced by the performance of these 92 individuals, we repeated the analyses comparing Cognivue Clarity scores by RBANS status after removing the 92 individuals. As shown in Table 6, there was no difference in Cognivue Clarity performance on average scores or 10 subtest scores between the original and sensitivity samples. The RBANS total score and age were also similar between the two samples.

Table 6

Sensitivity analysis of cognivue clarity

Variable	Original sample (n = 452)			Sensitivity sample (n = 360)
	RBANS not impaired	RBANS impaired	p	RBANS not impaired	RBANS impaired	p
Age	49.5 (15.9)	45.6 (17.3)	0.044	50.1 (15.9)	46.1 (17.6)	0.08
Cognivue average score	82.4 (9.7)	75.2 (13.3)	<0.001	82.3 (9.9)	74.9 (13.6)	<0.001
Cognivue adaptive motor control	56.3 (16.1)	49.2 (16.7)	<0.001	55.5 (15.9)	48.7 (16.3)	0.001
Cognivue visual salience	82.2 (12.6)	77.8 (14.1)	0.005	81.7 (13.0)	77.8 (14.6)	0.030
Cognivue letter discrimination	75.8 (13.3)	72.8 (16.3)	0.083	75.6 (13.4)	72.1 (16.9)	0.091
Cognivue word discrimination	78.0 (15.1)	78.8 (16.7)	0.653	78.4 (15.4)	78.4 (17.3)	0.980
Cognivue shape discrimination	87.8 (13.1)	79.1 (18.2)	<0.001	87.7 (13.3)	79.2 (18.5)	<0.001
Cognivue motion discrimination	83.2 (19.2)	76.1 (21.1)	0.003	82.9 (19.3)	75.1 (21.4)	0.004
Cognivue letter memory	83.4 (15.8)	79.6 (17.5)	0.079	83.1 (15.8)	79.5 (17.4)	0.108
Cognivue word memory	89.4 (14.5)	80.3 (20.5)	<0.001	89.4 (14.8)	81.0 (20.3)	<0.001
Cognivue shape memory	81.0 (19.8)	69.7 (27.2)	<0.001	80.9 (19.9)	69.3 (27.7)	<0.001
Cognivue motion memory	84.5 (21.3)	68.4 (29.2)	<0.001	83.9 (21.7)	67.6 (30.0)	<0.001
RBANS total score	100.4 (10.7)	75.0 (8.8)	<0.001	100.8 (10.6)	75.1 (8.6)	<0.001

Mean (SD), Bold p-values are significant after correction for multiple comparisons (adjusted p = 0.0036). RBANS, Repeatable Battery Assessing Neuropsychological Status.

DISCUSSION

Cognivue Clarity offers a quick, valid, and reliable measure of cognitive function with limited practice effects in a study of community-dwelling individuals from age 18–85. It is self-administered, takes 10 minutes, and is automatically scored immediately for ease of use. This suggests that Cognivue Clarity can be used to evaluate cognitive performance and screen individuals for cognitive impairment. Cognivue Clarity has a 4-factor structure that provides not only a global performance score but also information on delayed memory, executive-attention, visuomotor, and discrimination-perceptual abilities. Cognivue Clarity performed equally well between men and women but had age effects with improved discriminability following age-norming. African Americans scored 5.5 points lower than White participants matched for age and education. At least some of these differences can be explained by differential performance in motion discrimination, motion memory, and shape memory subtests but further research is warranted.

Cognivue Clarity overall score and subtests showed good correlation with RBANS total score and 4 of 5 index scores consistent with the 4-factor structure. RBANS Language index scores showed weak to no correlation with Cognivue Clarity overall score or subtests scores. Using a cut-off of 85 on the RBANS total score, we were able to explore sensitivity and specificity of Cognivue Clarity to detect probable cognitive impairment in those individuals who also had sub-optimal performance on RBANS. However, we do not have sufficient clinical information about the sample to explain reasons for impaired performance on the Cognivue Clarity or RBANS in individuals aged 18–59. Further studies of individuals with established diagnoses will be required to address this. However, Cognivue Clarity performed similarly to RBANS across different age, education, sex, race, and ethnicity groups.

The detection of MCI and AD is limited in community settings [37–39] due in part to the use of brief screening tests of objective cognitive performance that lack psychometric properties similar to those of Gold Standard tests [4, 39]. However, Gold Standard tests, such as the RBANS, are lengthy and require specialized staff to administer, score and interpret, making them impractical in the clinic. Cognivue Clarity offers a global assessment of cognitive functioning for easy screening while also providing domain and subtest scores and reaction times for more detailed characterization of patients. Cognivue Clarity adaptive psychophysics also provides the potential for use in clinical research with good test-retest reliability, internal consistency, and small practice effects for repeated testing in the context of a clinical trial [40, 41]. In tests relying on adaptive psychophysics, a threshold value is measured and other characteristics of the psychometric function underlying perceptual performance, such as slope and pattern of the response tracing, is developed [32, 33]. Adaptive procedures allow accurate and fast determination of psychophysical thresholds by reducing the number of stimulus presentations when the subject is far from threshold. The adaptive motor control evaluates the participants ability to control rotary movement responses and corrects further testing for their best motor performance. The visual salience test evaluates the participants ability to locate a stimuli and maintain the ability to follow it as the stimuli degrades. This adjusts further testing for their best visual performance. In doing so, Cognivue Clarity sets each participant as their own control for best responses and eliminates biases that can be seen with interindividual differences in motor reaction and visual perception.

The FOCUS study, which recruited a patient population that was diverse in age, race, and level of educational attainment, enhances our understanding regarding the normative ranges in cognitive assessment. This is important as the proportion of US adults who are diverse is increasing [42]. Historically underserved and underrepresented groups constitute 39% of the US population and FOCUS was able to closely match these demographic characteristics as well as educational attainment in which 63% of the US population has less than a Bachelor’s degree [43]. Studies that include diverse cohorts will enable the development of more diverse normative ranges and more precise interpretations of cognitive tests. This can further inform the use of dementia screening tools such as the Cognivue Clarity in clinical practice for early detection and in research for clinical trial eligibility.

Study limitations

FOCUS was a cross-sectional study so that longitudinal changes in Cognivue Clarity were not able to be discerned. Health history was self-reported, so no independent verification was available. A greater than expected number of individuals under age 60 had difficulty with both the Cognivue Clarity and RBANS, but in the absence of a comprehensive clinical evaluation the reasons remain unclear. We removed 127 individuals from the analyses who had greater than a 15-point difference between 2 trials. These extreme outliers had 1 score out of three that was lower than the 10% quartile of the overall scores. Although in clinical settings Cognivue Clarity is only administered one time, it is possible that some individuals tire during the exam and may randomly select answers rather than giving full effort. Future research should include formal performance validity measures to account for this.

While the sample size was sufficient to test the psychometric properties of Cognivue Clarity, higher numbers of individuals in each age strata would provide more confident estimates of normative data. Future research projects should test Cognivue Clarity properties in diagnosed individuals and provide greater numbers of cognitively normal individuals to refine normative data. One such study is the Bio-Hermes study conducted by the Global Alzheimer’s Platform [28] that includes characterized individuals with imaging, plasma and digital AD biomarkers, including Cognivue Clarity. While FOCUS had good representation of Non-Hispanic White, African American and Hispanic individuals from age 18–85, no conclusions about Cognivue Clarity performance can be determined regarding other races and ethnicities. Future research in these understudied groups is needed with several studies already underway. This may inform whether adjustment for ethnoracial identity or other sociodemographic characteristics may be needed.

Conclusions

FOCUS was able to recruit a diverse sample that reflects many of the demographic characteristics of the US population. This study provides new information on the psychometric properties and normative values of Cognivue Clarity. The psychometric properties of Cognivue Clarity suggests that it could play a significant role in early detection and screening for cognitive impairment across diverse populations capturing both global performance as well as domain- and subtest-specific attributes. This study provides supportive evidence for the current application of Cognivue Clarity as an easy-to-use, brief, and valid assessment to measure cognitive performance and screen individuals for likely cognitive impairments. These individuals could be further evaluated in clinical settings or be recruited as candidates for clinical research studies.

AUTHOR CONTRIBUTIONS

James E. Galvin (Conceptualization; Formal analysis; Investigation; Methodology; Writing – original draft; Writing – review & editing); Lun-Ching Chang (Formal analysis; Writing – review & editing); Paul Estes (Conceptualization; Data curation; Writing – review & editing); Heather M. Harris (Conceptualization; Data curation; Writing – review & editing); Ernest Fung (Methodology; Visualization; Writing – review & editing).