Hearing impairment, often caused by noise-induced trauma, significantly affects sound perception, communication, and cognitive abilities while increasing the risk of secondary accidents—individuals with hearing impairment are twice as likely to experience accidents as those with normal hearing. According to a 2023 WHO report, approximately 432 million adults and 34 million children globally are affected by hearing loss. While Assistive Listening Devices are commonly recommended, they are inadequate for individuals with total hearing loss. Therefore, alternatives are necessary to enhance safety and reduce accident risks. The present study introduces a hybrid deep learning model combining Very Deep Convolutional Networks (VGG16) and Residual Networks (ResNet-50) for efficient sound wave analysis and classification. Trained and validated on a comprehensive urban sound dataset, the model achieved a remarkable accuracy of 97.14%, surpassing existing state-of-the-art solutions. Furthermore, a mobile-based assistive notification system, MANSHIP, was developed to detect environmental sounds and alert individuals with profound or total hearing loss to potential hazards. MANSHIP addresses critical safety challenges and demonstrates the potential to improve the quality of life for those with severe hearing impairments by fostering safer environments and reducing caregiver dependency.
YangHJLayYLLinCS,et al.The radar-graphic speech learning system for hearing impaired. Expert Syst Appl2009; 36: 4804–4809.
2.
HendersonDSubramaniamMBoettcherFA. Individual susceptibility to noise-induced hearing loss: an old topic revisited. Ear Hear1993; 14: 152–168.
3.
MizoueTMiyamotoTShimizuT. Combined effect of smoking and occupational exposure to noise on hearing loss in steel factory workers. Occup Environ Med2003; 60: 56–59.
4.
NomuraKNakaoMYanoE. Hearing loss associated with smoking and occupational noise exposure in a Japanese metal working company. Int Arch Occup Environ Health2005; 78: 178–184.
SimonovićMKovandžićMĆirićI,et al.Acoustic recognition of noise-like environmental sounds by using artificial neural network. Expert Syst Appl2021; 184: 115484.
7.
LemaitreG. Everyday sound. A journey through perception, cognition, interaction, and imitation of sounds that surround us. Cognitive science. Université Pierre et Marie Curie (Paris 6), 2016. <tel-01448929>.
8.
AlettaFDe CoenselBLindborgP. Human perception of environmental sounds. Front Psychol2021; 12: 714591.
GórrizJMÁlvarez-IllánIÁlvarez-MarquinaA, et al.Computational approaches to explainable artificial intelligence: advances in theory, applications and trends. Inf Fusion2023; 100: 101945.
12.
KalchbrennerNGrefenstetteEBlunsomP. A convolutional neural network for modelling sentences. 52nd Annual Meeting of the Association for Computational Linguistics. 2014 - Proc Conf 2014, vol. 1, pp.655–665.
13.
SeoSKimCKimJH. Convolutional neural networks using log mel-spectrogram separation for audio event classification with unknown devices. J Web Eng2021; 21: 497–522.
14.
VermaHMandalSGuptaA. Temporal deep learning architecture for prediction of COVID-19 cases in India. Expert Syst Appl2022; 195: 116611.
15.
DielemanSSchrauwenB. End-to-end learning for music audio. In 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), 2014 May 4, pp. 6964-6968. IEEE.
16.
HersheySChaudhuriSEllisDP, et al.CNN architectures for large-scale audio classification. In 2017 ieee international conference on acoustics, speech and signal processing (icassp), 2017 Mar 5, pp. 131-135. IEEE.
17.
CostaYMOliveiraLSSilla JrCN. An evaluation of convolutional neural networks for music classification using spectrograms. Appl Soft Comput2017; 52: 28–38.
18.
RavanelliMBengioY. Speaker recognition from raw waveform with sincnet. In 2018 IEEE spoken language technology workshop (SLT), 2018 Dec 18, pp. 1021-1028. IEEE.
19.
PiczakKJ. Environmental sound classification with convolutional neural networks. In 2015 IEEE 25th international workshop on machine learning for signal processing (MLSP), 2015 Sep 17, pp. 1-6. IEEE.
20.
OrtacGOzcanG. Comparative study of hyperspectral image classification by multidimensional Convolutional Neural Network approaches to improve accuracy. Expert Syst Appl2021; 182: 115280.
21.
ChoiSKimKKamyodC,et al.Ethical use of web-based welfare technology for caring elderly people who live alone in Korea: a case study. J Web Eng2022; 21: 1239–1264.
22.
SahingozOKCekmezUBulduA. Internet of Things (IoTs) security: intrusion detection using deep learning. J Web Eng2021; 20: 1721–1760.
23.
ElbasaniEKimJD. AMR-CNN: abstract meaning representation with convolution neural network for toxic content detection. J Web Eng2022; 21: 677–692.
24.
LockeyKJenningsMBShawL. Exploring hearing aid use in older women through narratives. Int J Audiol2010; 49: 542–549.
25.
RekkedalAM. Assistive hearing technologies among students with hearing impairment: factors that promote satisfaction. J Deaf Stud Deaf Educ2012; 17: 499–517.
TuckerBP. The feel of silence. Philadelphia: Temple University Press, 1995.
28.
Payzan-LeNestourEPradierLDoranJ, et al.Impact of ambient sound on risk perception in humans: neuroeconomic investigations. Sci Rep2021; 11: 5392.
29.
DejaJADela TorreALeeHJ, et al.Vitune: a visualizer tool to allow the deaf and hard of hearing to see music with their eyes. In Extended Abstracts of the 2020 CHI conference on human factors in computing systems, 2020 Apr 25, pp. 1-8.
30.
YangLZhaoH. Sound classification based on multihead attention and support vector machine. Math Probl Eng2021; 2021: 9937383.
31.
LiSYaoYHuJ, et al.An ensemble stacked convolutional neural network model for environmental event sound recognition. Appl Sci2018; 8: 1152.
32.
AbdoliSCardinalPKoerichAL. End-to-end environmental sound classification using a 1D convolutional neural network. Expert Syst Appl2019; 136: 252–263.
33.
ChenYGuoQLiangX, et al.Environmental sound classification with dilated convolutions. Appl Acoust2019; 148: 123–132.
MuWYinBHuangX, et al.Environmental sound classification using temporal-frequency attention based convolutional neural network. Sci Rep2021; 11: 21552.
36.
VidalEGZarricuetaEFCheeinFA. Human-inspired sound environment recognition system for assistive vehicles. J Neural Eng2015; 12: 016012.
37.
BeritelliFGrassoR. A pattern recognition system for environmental sound classification based on MFCCs and neural networks. In 2008 2nd international conference on signal processing and communication systems, 2008 Dec 15, pp. 1-4. IEEE.
38.
SalamonJJacobyCBelloJP. A dataset and taxonomy for urban sound research. In Proceedings of the 22nd ACM international conference on Multimedia, 2014 Nov 3, pp. 1041-1044.
SimonyanKZissermanA. Very deep convolutional networks for large-scale image recognition. In 3rd international conference on learning representations (ICLR 2015), 2015 Apr 10. Computational and Biological Learning Society.
41.
HeKZhangXRenS,et al.Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
42.
ZhangWXuLYanM, et al.A probability distribution and location-aware ResNet approach for QoS prediction. J Web Eng2021; 20: 1251–1290.
43.
DyzelVOosterom-CaloRWormM,et al.Assistive technology to promote communication and social interaction for people with deafblindness: a systematic review. In Frontiers in Education, 2020 Sep 17, Vol. 5, p. 578389. Frontiers Media SA.
44.
PiczakKJ. ESC: Dataset for environmental sound classification. In Proceedings of the 23rd ACM international conference on Multimedia, 2015 Oct 13, pp. 1015-1018.
