MANSHIP: Mobile-based assistive notification service for hearing-impaired people using a hybrid deep learning model

Abstract

Hearing impairment, often caused by noise-induced trauma, significantly affects sound perception, communication, and cognitive abilities while increasing the risk of secondary accidents—individuals with hearing impairment are twice as likely to experience accidents as those with normal hearing. According to a 2023 WHO report, approximately 432 million adults and 34 million children globally are affected by hearing loss. While Assistive Listening Devices are commonly recommended, they are inadequate for individuals with total hearing loss. Therefore, alternatives are necessary to enhance safety and reduce accident risks. The present study introduces a hybrid deep learning model combining Very Deep Convolutional Networks (VGG16) and Residual Networks (ResNet-50) for efficient sound wave analysis and classification. Trained and validated on a comprehensive urban sound dataset, the model achieved a remarkable accuracy of 97.14%, surpassing existing state-of-the-art solutions. Furthermore, a mobile-based assistive notification system, MANSHIP, was developed to detect environmental sounds and alert individuals with profound or total hearing loss to potential hazards. MANSHIP addresses critical safety challenges and demonstrates the potential to improve the quality of life for those with severe hearing impairments by fostering safer environments and reducing caregiver dependency.