Abstract
Background
Handwriting and speech are served as reliable signatures for detecting cognitive decline, playing a pivotal role in the early diagnosing Alzheimer's disease (AD) and mild cognitive impairment (MCI). However, current unimodal approaches for diagnosing AD and MCI have demonstrated constraints in classification accuracy, potentially overlooking the synergistic value of combining handwriting and speech data.
Objective
Presenting an innovative multi-modal screening classification model, that harnesses handwriting and speech analysis to enhance MCI detection, aiming to overcome the constraints of single-modality approaches by integrating data from both modalities, thereby improving diagnostic accuracy.
Methods
Proposing a multimodal classification model based on gated recurrent unit (GRU) and attention mechanism, treating handwriting and speech data as sequence inputs. The model was constructed and tested on a dataset of 41 participants, including 20 MCI patients and 21 cognitively normal (CN) individuals. To mitigate the risk of overfitting due to the small sample size, we employed a 10-fold cross-validation strategy to ensure the robustness of the results.
Results
Our multimodal classification model achieved an accuracy of 95.2% for MCI versus CN individuals, which shows a significant improvement compared to the results of single-modality. This result indicates the effectiveness of the cross-fusion model in enhancing classification performance, offering a promising approach for the early diagnosis of neurodegenerative diseases.
Conclusions
The proposed GRU_CA effectively improves early MCI detection by fusing handwriting and speech data, outperforming a single modality. It shows strong potential for deployment in primary healthcare settings and establishes a foundation for future research on more complex diagnostic tasks, including CN, MCI, and AD classification, as well as longitudinal studies.
Keywords
Get full access to this article
View all access options for this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
