Validation of Noninvasive Body Sensor Network Technology in the Detection of Agitation in Dementia

Abstract

Objective:

Agitated behaviors are one of the most frequent reasons that patients with dementia are placed in long-term care settings. This study aims to validate the ability of a custom Body Sensor Network (BSN) to capture the presence of agitation against currently accepted subjective measures, the Cohen-Mansfield Agitation Inventory (CMAI) and the Aggressive Behavior Scale (ABS) and to discriminate between agitation and cognitive decline.

Methods:

Six patients identified as being at high risk for agitated behaviors were enrolled in this study. The devices were applied at three sites for three hours while behaviors were annotated simultaneously and subsequently repeated twice for each enrolled subject.

Results:

We found that the BSN was a valid measure of agitation based on construct validity testing and secondary validation using non-parametric ANOVAs.

Discussion:

The BSN shows promise from these pilot results. Further testing with a larger sample is needed to replicate these results.

Introduction

Agitated behaviors are one of the most frequent reasons that patients with dementia are placed in long-term care (LTC) settings. ^1,2 These behaviors are indicators of distress in patients and are associated with increased risk of injury to patients and caregivers. Most patients with dementia experience agitation during the course of their disease ^{3 –5} with increasing symptomatology reported as the illness progresses ⁶ resulting in increasing difficulty in expressing their feelings and thus making subjective measurements inefficient. Researchers have quantified behavioral manifestations and factored them into subsyndromes. ⁷ The complexity of agitation has been attributed to the impact of cognitive, medical, environmental, and psychological factors. ⁸ Subsequent research has identified intervention techniques, ^9,10 a temporal pattern of agitation, ¹¹ and nonpharmacologic treatment options with individualized interventions. ^12,13 Other researchers have utilized the Cohen-Mansfield Agitation Inventory (CMAI) ⁷ for validation of agitation in the development of other methods of measuring agitation. ¹⁴ Pragmatic interventions are difficult to define due to the complexity of the syndrome and the inconsistent identification of escalating agitation before behavioral dysregulation becomes disruptive and uncontrollable.

While agitation can be expressed in more than just a physical context, most instruments used in the measurement of agitation including the CMAI incorporate a range of physical actions as signifying agitation in dementia.

Technology-Enabled Medical Precision Observation

Actometry was used by Janno et al ¹⁵ to measure subtle body movements. Movement disorders such as Parkinsonian tremors have been discriminated with technology-enabled medical precision observation (TEMPO) technology. ^{16 –18} Actometry provides an objective assessment of symptom severity if other hyperactive movement disorders are controlled for. Studies using actometry to measure agitation in dementia are few in number. Nagels et al ¹⁹ concluded that actometry was a useful tool for examining agitated behaviors in dementia. Bharucha et al ²⁰ identified 58 technologies in an extensive review and only 3 were applicable to patients with dementia and 2 of these utilized constant videotaping of patients with dementia. None of these previous studies utilized the quality assessments of the TEMPO, which captures real-time data without compromising the participant’s privacy, and is portable, wireless, and lightweight. The new TEMPO 3.1 (Figure 1), a network of custom miniature wireless inertial body sensors developed by University of Virginia engineers, gathers data from patients while they perform their normal activities of living. The motion capture capabilities provided by the TEMPO are essential and cannot be provided by existing off the shelf general activity monitors, such as the actigraph. This study proposes to differentiate normal movement from agitated movement. An actigraph only measures total movement without such differentiation.

Figure 1.

The third generation of the technology-enabled medical precision observation (TEMPO) technology.

Preliminary Studies

In previous observational work in the outpatient setting, the research team from Carilion Clinic and University of Virginia found the body sensor network (BSN) to be acceptable to elderly patients with dementia in the outpatient setting. Preliminary results were promising in the identification of agitation in this setting. Data were collected on 5 patients during regular clinic visits. Patients were asked to perform routine tests or were observed in a naturalistic setting over several sessions. The data were correlated with annotations of observations taken during the same period to determine how well agitation could be detected and quantified. The most promising technique used in this study was the Teager energy function. ²¹ This technique provides a measure of aggregate energy of movement, separating agitation from the raw motion data. Teager energy and other measures were plotted over time, and the waveforms were overlaid with the agitation annotations. As shown in Figure 2, annotations noting increased agitation correspond to increases in Teager energy.

Figure 2.

A time-domain plot of the Teager energy (figure) and the real-time annotations inserted during the data collections (table).

Methods

The interdisciplinary team combined function (electrical engineering), applicability (medical research and health care), and accessibility (technology design) of the technology as part of the collaboration between the University of Virginia, the Carilion Clinic Center for Healthy Aging, and Virginia Tech. We used the Multitrait-Multimethod (MMTM) approach ²² to test the construct validity of the BSN. Three clinical instruments with strong validity and reliability in the clinical setting ^{23 –25} —the CMAI, ⁷ Aggressive Behavior Scale (ABS), ¹⁴ and Mini-Mental State Examination (MMSE) ²⁶ —were used as the benchmarks to test convergent and discriminant validity of the wrist, waist, and ankle readings from the BSN. We also compared the observed and annotated levels of agitation with the Teager energy function.

We present the following hypotheses:

Hypothesis 1: There will be an association between the BSN variables and the CMAI ⁷ dimensions.

Hypothesis 2: There will be an association between the BSN variables and the ABS ¹⁴ dimensions.

Hypothesis 3: There will be no association between the BSN variables and the MMSE ²⁶ dimensions.

We recruited 6 participants residing on an LTC dementia unit; they were selected based on the underlying Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) diagnoses of dementia and the attending physician having placed a geriatric psychiatry consultation order. All participants were female and mostly Caucasian (4 of 6). The mean age was 81.8 years (range 67-93 years). Exclusion criteria included the presence of other psychiatric diagnoses such as schizophrenia, impaired mobility such as being wheelchair-bound, or the presence of movement disorders such as Parkinson’s disease. As this was not an interventional study, medications were managed separately from the study.

This study was approved by the Institutional Review Board at Carilion Clinic and informed consent was obtained from each participant. Because of the nature of this study, most potential participants did not have the cognitive capacity to make an informed decision to consent. We, therefore, used a legally authorized representative (LAR) process whereby 2 physicians or a physician and a clinical psychologist signed a form attesting to the incapacity of the potential participant and informed consent was then obtained from the participant’s LAR.

Each patient was studied for 3 hours with 3 nodes (dominant wrist, waist, and opposite leg) on 3 separate occasions and at different times of the day. Thus, a total of 9 hours of data were collected on each participant. All behaviors were observed and annotated electronically in real time by our research assistant.

The critical incidents relevant to the usability ²⁷ of the BSN were also recorded during the observation period. Smith-Jackson and Williges ²⁸ used a similar method for individuals with spinal cord injury to collect usability-related data and were able to demonstrate high reliability of critical incident reporting among a diverse group of participants (Cronbach's a coefficients ranged from .88 to .96). With each application of the sensor, the research team sought a verbal assent from each participant. Each BSN includes 3 axes of gyroscopes (for rotational motion) in addition to the 3 axes of linear accelerometers, providing 6 degrees of freedom motion capture and providing it with the capability of performing full rotational and translational sensing at the same site, virtually eliminating errors due to mechanical offsets.

The data were wirelessly transmitted from the BSN to the “supernode” which enabled fine synchronization of the data. The CMAI ⁷ and the ABS ¹⁴ were administered by a member of the research team near the time of and/or on the day of the readings. The MMSE ²⁶ was administered to the participant at each reading.

Instruments

Cohen-Mansfield Agitation Inventory ⁷ —a 29-item interview asking clinicians for observed behaviors of agitation over the previous 2 weeks. Each behavior is rated on a 1-7 scale of frequency:

1—never;

2—< once a week but still occurring;

3—once or twice a week;

4—several times a week;

5—once or twice a day;

6—several times a day;

7—several times an hour.

Interrater reliability with 3 sets of raters was r = .88 (n = 31), .92 (n = 23), and .92 (n = 16). Test–retest reliabilities at 1 month ranged from 0.74 to 0.92. The global scores range from 0 to 203, including 4 subscale scores: physically aggressive (PAs) behaviors—9 items; physically nonaggressive (PNA) behavior—13 items; verbally aggressive (VA) behavior—3 items; and verbally nonaggressive behavior (VNA)—4 items.

Aggressive Behavior Scale ¹⁴ —a tool based on the minimum data set (MDS). It is a summary scale of 4 MDS items: verbal abuse, physical abuse, socially inappropriate or disruptive, and resisting care. The frequency over the previous 7 days is reported as:

behavior occurred in 1 to 3 days of the past 7 days;

behavior occurred in 4 to 6 days of the past 7 days but less than daily;

behavior occurred daily.

Scores range from 0 to 12 with higher scores indicating more behaviors and increased frequency of behaviors. Average interrater reliability with the MDS 2.0 was κ of .72 for all behavioral items.

The Mini-Mental State Examination ²⁶ is an assessment of cognitive function. Categories include memory, recall, orientation, attention, and language use. Scores range from 1 to 30. The typical cutoff is 24 of 30 with 30 of 30 being a perfect score. Scores under 24 of 30 indicate increasing degrees of cognitive impairment with decreasing scores.

Data Analysis

Signal processing algorithms were developed to extract the clinically relevant information from the raw motion data. In a previous study, we selected the techniques that showed the most promise for statistical validation. These signal processing assessments were then correlated with the data annotations to determine how well agitation could be detected and quantified. The mechanical energy of the system is proportional to the product of the amplitude squared and the frequency squared, that is, the Teager energy function.

$E\cong A^2{\text{ω}}^2$

This provides a measure of aggregate energy of movement.

Our goal was to discriminate the degree of agitation in nursing home patients with dementia using the TEMPO system BSN and Teager energy analysis. Annotations by the research observer were studied alongside the Teager energy scores. Data captured from the supernode were downloaded and transmitted along with the annotations to researchers at University of Virginia for analysis. Transmission was blinded so that University of Virginia researchers did not know patients’ assessment instrument scores. Vector magnitude and time series accelerations data were analyzed by employing various feature extraction techniques in conjunction with adaptive classifiers.

The Teager energy scores along with the annotations and scores from the various instruments were then transmitted to researchers at Virginia Tech. The raw energy values for each participant for the wrist, waist, and ankle measures and the CMAI, ABS, and MMSE were tested for normality. All distributions were non-Gaussian with Shapiro-Wilk values for the BSN data ranging from 0.27 to 0.89.

Because the distributions were non-Gaussian, we used the Spearman-rho equation for the correlation coefficient. Due to the number of observations per participant, we established a conservative α criterion of α = .01 as the standard for significance testing (2-tailed tests) to reduce the influence of experiment-wise error because of the number of degrees of freedom based on repeated measures on each patient (degrees of freedom ranged from 197 to 301).

Construct validation (convergent) was conducted using 2 expert clinical observer rating scales: the CMAI, which measures agitation, and the ABS, which measures aggression as a component of agitation. Discriminant validity was explored using the MMSE.

Results

Over a 6-week period, the 3 MMSE results of each 6 participants ranged from 0 to 11 with an average range of 5.17 to 6.17. The MMSE results at this level validate the severity of the dementia in the participants.

Two sets of CMAI data were collected for each participant. Findings revealed a great spread of behaviors in our 6 participants. Two thirds scored 7 (indicating highest levels) for general restlessness and half scored 7 for pacing and wandering through all measures. Using the CMAI’s definition for various agitated behaviors, ²⁹ only 1 participant was found not to display agitation. Of the 6 participants, 4 exhibited PA behavior and PNA behavior while 2 of 6 displayed VA behavior.

Like the CMAI, the ABS is also a retrospective measure. The ABS scores have a range of 0 to 12. The range in this study was 1 to 12 in aggressive behavior with average of 4.00 to 5.83 across 3 measures at least 7 days apart. There were 3 missing data points as a result of having periods of less than 7-day intervals between readings. Nonstructured interviews with staff and family members indicated strong resistance with waist BSN based on the tendency of dementia patients to fiddle with things. ²⁹

Tables 1 to 3 show the Multitrait-Multimethod (MMTM) convergent validity data for morning, afternoon, and evening intervals, respectively.

Hypothesis 1: There will be an association between the BSN variables and the CMAI dimensions.

Morning: confirmed for all sensor data collected at this time. Correlations between the CMAI and the BSN wrist, waist, and ankle sensor data were r _s = .27, P < .001; r _s = .36, P < .001; and r _s = .34, P < .001, respectively. The direction of the correlation was as expected; that is, the higher the CMAI score, the higher the BSN values.

Afternoon: not confirmed as the correlations were nonsignificant but the direction of the correlations were as expected.

Evening: not confirmed.

Hypothesis 2: There will be an association between the BSN variables and the ABS dimensions.

Morning: confirmed for all sensor data for the morning interval. Correlations between the ABS and the BSN wrist, waist, and ankle sensor data were r _s = .35, P< .001; r _s = .22, P< .001; and r _s = .27, P < .001, respectively. The direction of the correlation was as expected; that is, the higher the ABS score, the higher the BSN values.

Table 1.

The MMTM Convergent Validity Data From the “Morning” Interval ^a

Morning (Spearman r)
	(M) Wrist	(M) Waist	(M) Ankle	(M) CMAI	(M) ABS	(M) MMSE
(M) Wrist	1
(M) Waist	.510	1
	<.001
(M) Ankle	.587	.701	1
	<.001	<.001
(M) CMAI	.269	.355	.338	1
	<.001	<.001	<.001
(M) ABS	.354	.224	.268	.903	1
	<.001	<.001	<.001	<.001
(M) MMSE	.042	–.314	–.345	–.062	.198	1
	.465	<.001	<.001	.281	<.001

Abbreviations: ABS, Aggressive Behavior Scale; CMAI, Cohen-Mansfield Agitation Inventory; MMSE, Mini-Mental State Examination; MMTM, Multitrait-Multimethod.

^a Indicates statistical significance at alpha criterion of 0.001 or 0.01.

Table 2.

MMTM Convergent Validity Data From the “Afternoon” Interval ^a

	(M) Wrist	(M) Waist	(M) Ankle	(M) CMAI	(M) ABS	(M) MMSE
(M) Wrist	1
(M) Waist	.621	1
	<.001
(M) Ankle	.595	.686	1
	<.001	<.001
(M) CMAI	.014	.152	.125	1
	.817	.031	.039
(M) ABS	–.092	.011	.060	.912	1
	0.132	.881	.328	<.001
(M) MMSE	–.0002	–.161	–.053	.124	–.077	1
	.998	.023	.382	.041	.201

Abbreviations: ABS, Aggressive Behavior Scale; CMAI, Cohen-Mansfield Agitation Inventory; MMSE, Mini-Mental State Examination; MMTM, Multitrait-Multimethod.

^a Indicates statistical significance at alpha criterion of 0.001 or 0.01.

Table 3.

MMTM Convergent Validity Data From the “Evening” Interval ^a

	(M) Wrist	(M) Waist	(M) Ankle	(M) CMAI	(M) ABS	(M) MMSE
(M) Wrist	1
(M) Waist	.601	1
	<.001
(M) Ankle	.790	.676	1
	<.001	<.001
(M) CMAI	–.188	.025	–.049	1
	.007	.722	.472
(M) ABS	–.287	.011	–.185	.878	1
	<.001	.873	<.001	<.001
(M) MMSE	–.014	–.156	–.021	–.149	–.467	1
	.841	.028	.762	.022	<.001

Abbreviations: ABS, Aggressive Behavior Scale; CMAI, Cohen-Mansfield Agitation Inventory; MMSE, Mini-Mental State Examination; MMTM, Multitrait-Multimethod.

^a Indicates statistical significance at alpha criterion of 0.001 or 0.01.

Afternoon: not confirmed.

Evening: confirmed for the wrist and ankle sensor data with significant correlations of r _s = −.29, P < .001 and r _s = −.19, P < .001. However, the directions of the correlations were negative, with higher ABS scores associated with lower BSN measures.

Hypothesis 3: There will be no association between BSN variables and the MMSE dimensions.

As a test of discriminant validity, the MMSE was not expected to correlate with the BSN. We found no associations between the BSN and the MMSE for 7 of the 9 correlations. The morning waist and ankle measures, r _s = .31 and −.34, respectively, and P values of <.001 were significant.

Secondary validation of the BSN

A preliminary test was conducted to determine the sensitivity of the BSN measures to detect agitation. The working hypothesis was that Teager scores during an agitation event would be significantly different from Teager scores during the preagitation and postagitation events (2-tailed test). We also expected Teager scores to be higher during agitation events compared to pre- and postagitation events. Using the full data set consisting of all measures on each participant across sessions (morning, afternoon, and evening), we set 3 epochs using the annotations provided by the observer. The agitation periods in the BSN data were synchronized with the observer’s time stamps during the observation sessions. Each agitation event varied in length, but a 2-minute sample could be derived for each event. We sampled 2 minutes of signals from each agitation event, followed by selecting 2 additional epochs referred to as preagitation and postagitation epochs. The preagitation epoch was selected by sampling 2 minutes of signal data immediately before the onset of each agitation event. The postagitation epoch was selected by sampling 2 minutes of signal data occurring immediately after the end of an agitation event. Thus, we derived 3 epochs for each agitation event (preagitation, agitation, and postagitation) across all sessions (morning, afternoon, and evening) and across all body measures (wrist, waist, and ankle). This data set yielded 11 569 observations.

Friedman analyses of variance (ANOVAs) for all Teager scores were tested for normality using a Kolmogorov-Smirnov Test, and all values were significant, indicating non-Gaussian distributions (D _range = .33-.45, P < .01). As a consequence, nonparametric analyses of variance (Friedman ANOVAs) were conducted using epoch as the predictor with 3 levels (preagitation, agitation, and postagitation). This model was run separately on the wrist, waist, and ankle measures. Wilcoxon signed rank tests were used to calculate the post hoc paired comparisons. Due to the number of comparisons, we applied a Bonferroni adjustment to minimize experiment-wise error; thus, the α criterion for significance was α = .0167.

All Teager values in the morning sessions regardless of location on the body (wrist, waist, or ankle) showed significant differences between epochs; F _r2 = 63.63, P < .0001; F _r2 = 190.37, P < .0001; and F _r2 = 49.57, P < .0001 for the Teager values derived from the wrist, waist, and ankle measures, respectively. Post hoc tests of pairwise comparisons for the wrist measures showed significant differences between preagitation and agitation Teager scores (P < .001) and agitation and postagitation Teager scores (P < .01). There were no significant differences between the preagitation and postagitation scores (P = .12).

The waist and ankle measures in the morning session only partially confirmed the hypothesis. For the waist measures, all 3 epochs showed significant differences from each other. For the ankle measures, the preagitation and agitation values were significantly different from each other, but the preagitation and postagitation values were also significantly different from each other. Figure 3 illustrates the median Teager values for the wrist, waist, and ankle locations in the morning sessions.

Figure 3.

Median Teager scores for each epoch in the morning sessions.

Friedman ANOVAs were significant for all body sites in the afternoon session. F _r values ranged from 169.90 to 441.44, P < .0001. Post hoc comparisons using Wilcoxon sign rank tests revealed significant differences between all pairwise comparisons of epochs at P < .0001. The medians are illustrated in Figure 4.

Figure 4.

Median Teager scores for each epoch in the afternoon sessions.

Friedman ANOVAs were significant for all body sites in the evening sessions. F _r values ranged from 45.32 to 204.72, P < .0001. All pairwise comparisons were significantly different from each other (P < .0001) with the exception of agitation and postagitation for the waist measures (P = .28). The medians are illustrated in Figure 5.

Figure 5.

Median Teager scores for each epoch in the evening sessions.

Critical incidents result

The critical incident technique was used to conduct a formative usability assessment of the BSN. Critical incidents were any events indicating a problem with the design or operation of the BSN. The total number of critical incidents was 76 across all observations for all participants. This is a relatively small number of critical incidents, given the length of time each participant wore the bands.

Discussion

Even with a relatively conservative criterion, we achieved a reasonable level of validation by way of construct validity, although the support was not 100%. We found acceptable convergent validity for the morning measures of the BSN data, the CMAI, and the ABS but less convergent validity with the afternoon and evening measures. A possible explanation for this may be the use of medications in the management of the neuropsychiatric symptoms of dementia causing a reduction in manifested agitation.

Strong support was found in the secondary validation comparing Teager scores of agitation events to pre- and postagitation events. Teager energy scores differed significantly between epochs. Although there were 2 instances where Teager values for agitation scores were lower rather than higher compared to the pre- and postagitation periods, this exception was likely due to extraneous noise in the data.

We demonstrated strong discriminant validity between the BSN and the MMSE. Overall, the BSN seems to be a useful and effective measure of agitation. The instrument will require additional refinement, which is currently underway. Interestingly, there were few usability problems with the BSN, and participants seem to tolerate the device while proceeding through their usual activities. Connecting and removing the BSN from patients also seems to be relatively easy for the researchers and participants. From this study, we found the wrist sensor to be the most sensitive in detecting agitation. This increases the population that could benefit from the BSN to include those who are wheelchair bound.

The aging of the population promises to bring increases in the prevalence of dementia. ³⁰ Detecting agitation in a timely manner, before the patient becomes more overtly distressed, would help caregivers offer nonpharmacologic interventions while they might still be effective. Previous studies on agitation reveal the considerable amount of time and detail necessary to capture all the elements of agitation and enable clinical discrimination and intervention. ^12,29 Direct observations of behaviors that are aggressive (cursing, hitting, etc), nonaggressive (pacing and repetitive behaviors), and verbal (asking for attention, critical comments, etc) have been identified as subsyndromes of agitation. ⁷ We applied a socio-technical systems approach that accounts for the constraints on personnel and the need for effective and efficient assessment tools.

Studies have shown the efficacy of behavioral interventions in the treatment of agitation in patients with dementia. ^{12,31 –33} These studies had small sample sizes, no control group, no randomization or blinding, or poorly defined outcome criteria. ¹⁴ Consequently, generalizations cannot be made as a result of these qualitative deficits. ¹³ A consensus report by Salzman et al ³⁴ has called for further research that is scientifically sound and adequately powered. Use of the BSN would provide consistency in data collection and in turn provide consistent higher quality research outcomes from which generalization would be possible. This would help further in the development of evidence-based treatments and interventions for agitation in dementia.

Currently, staff guidance on action for the behavioral management of patients with dementia with agitation uses a shotgun approach. The BSN would allow for focused and consistent staff guidance. Pragmatic interventions for direct caregivers could be tailored to the expertise level of the care provider. This would eliminate the use of a broad dimension of interventions on all patients. The workload imposed by existing subjective assessments can also be reduced with usable, objective assessments with the BSN. Agitated behavior tends to build up over time resulting in increasing episodes in response to decreasing intensity of stimuli. Being able to identify triggers for agitated behavior earlier could help prevent this behavioral cascade. Further refinement of the BSN could allow for early identification of specific triggers to distress.

Timely, precise, and defined treatment interventions and recommendations could help increase employee satisfaction and potentially reduce caregiver burnout, institutionalization, and staff turnover. Improvements through use of the BSN would impact overall consistency of care, one of the initiatives for the Advancing Excellence in America’s Nursing Homes’ national campaign to improve nursing home quality of care for the 1.5 million currently in LTC. ³⁵

All patients with dementia who experience some level of agitation will potentially benefit, whether it is to allow for evidence-based nonpharmacologic interventions, to prevent increases or additions of neuroleptic medications, or to gauge early effects when medication management is implemented. In the future, when a patient requires medication, the BSN could potentially provide information on the positive or negative impact on the patient based on the frequency and intensity of agitation as compared to the patient’s baseline. One of the objectives for dementia in Healthy People 2020 ³⁶ calls for reducing preventable hospitalizations. Discrimination of agitation to allow health care providers to discern patterns and identify causes of behavioral changes through the BSN could allow caregivers to appropriately address agitation and discern its cause, whether physiological or psychological, in persons unable to describe their symptoms.

According to the recent Alzheimer’s Association report, ³⁷ in 2011 unpaid caregivers provided over $210 billion worth of care to patients with dementia and it is estimated that there will be almost a million new cases of dementia every year by 2050. This research could allow for more care in the home and improved care in community and institutional settings.

There are several strengths in this study, including a collaborative effort between the different specialties of electrical engineering, medical and nursing research and health care, and technology design. Focusing on the human factors approach that emphasizes user-centered and accessible technology design is also a strength. The limitations of this study include the small sample size of the study, nonrandomization, and the lack of a control group.

Nevertheless, this study demonstrates that the TEMPO system and Teager energy analysis can be used to discriminate the degree of agitation in nursing home patients with dementia experiencing agitation. The hypotheses that were not confirmed by the construct validity testing will be used to explore further design iterations to improve the validity of the BSN. The BSN shows promise and as such we will continue to refine the technology and conduct further testing with larger samples.