Real time anomalies detection in crowd using convolutional long short-term memory network

Abstract

Violence is a critical social problem and demands to evaluate through computer vision approaches. At present, the incidences of violent actions get grown in the community, particularly in public places due to several economic and social causes. Moreover, our society’s populations are increasing day by day and it is challenging to keep citizens within limits as well as monitoring human activities in crowd is too hard. Thus, government organizations including local bodies, require examining such occurrences through smart surveillance. In this research, a lightweight computational architecture has been presented to classify non-violent and violent activities. A model has been proposed to extract time-based features using smart devices, high-speed wireless networks and cloud servers to classify real-time human activities. For this purpose, a deep learning-based model is employed to detect violent activities and assist the stakeholders in exposing such activities in real-time. Convolutional long short-term memory (Conv-LSTM) is employed to extend fully connected LSTM (FC-LSTM) to capture the frame and detect violent actions. The proposed model accomplished 95.16% validation accuracy using a standard crowd anomaly dataset.

Keywords

Anomaly detection convolutional long short-term memory (Conv-LSTM) network human action recognition smart surveillance social security

Get full access to this article

View all access options for this article.

References

Saba

Haseeb

Ahmed

et al. Secure and energy-efficient framework using Internet of Medical Things for e-healthcare. J Infect Pub Health 2020; 13(10): 1567–1575.

Saba

Rehman

AlGhamdi

. Weather forecasting based on hybrid neural model. App Water Sci 2017; 7(7): 3869–3874.

Meethongjan

Dzulkifli

Rehman

et al. An intelligent fused approach for face recognition. J Intell Syst 2013; 22(2): 197–212.

Halder

Chatterjee

. CNN-BiLSTM model for violence detection in smart surveillance. SN Comput Sci 2020; 1(4): 1–9.

Sharif

Mehmood

Mahmood

et al. Scene analysis and search using local features and support vector machine for effective content-based image retrieval. Artif Intell Rev 2019; 52(2): 901–925.

Saba

Rehman

Jamail

NSM

et al. Categorizing the students’ activities for automated exam proctoring using proposed deep L2-GraftNet CNN network and ASO based feature selection approach. IEEE Access 2021; 9: 47639–47656.

Khan

Jabeen

Khan

MUG

et al. A realistic image generation of face from text description using the fully trained generative adversarial networks. IEEE Access 2020; 9: 1250–1260.

Khan

Zhang

Khan

et al. A resource conscious human action recognition framework using 26-layered deep convolutional neural network. Multimed Tools Appl. Epub ahead of print 1 August 2020. DOI: 10.1007/s11042-020-09408-1.

Chaudhry

Rahim

MSM

Saba

et al. Crowd detection and counting using a static and dynamic platform: state of the art. Int J Computat Vis Robot 2019; 9(3): 228–259.

10.

Mahadevan

Bhalodia

et al. Anomaly detection in crowded scenes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Francisco, CA, 13–18 June 2010, https://doi.org/10.1109/CVPR.2010.5539872

11.

Aurangzeb

Haider

Khan

et al. Human behavior analysis based on multi-types features fusion and Von Nauman entropy based features reduction. J M Imag Health Inform 2019; 9: 662–669.

12.

Sharif

Khan

Javed

et al. Intelligent human action recognition: a framework of optimal features selection based on Euclidean distance and strong correlation. J Control Eng Appl Inform 2019; 21: 3–11.

13.

Khan

Javed

Khan

et al. Human action recognition using fusion of multiview and deep features: an application to video surveillance. Multimed Tools Appl. Epub ahead of print 14 March 2020. DOI: 10.1007/s11042-020-08806-9.

14.

Khan

Sharif

Akram

et al. Hand-crafted and deep convolutional neural network features fusion and selection strategy: an application to intelligent human action recognition. Appl Soft Comput 2020; 87: 105986.

15.

Raza

Sharif

Yasmin

et al. Appearance based pedestrians’ gender recognition by employing stacked auto encoders in deep learning. Future Gener Comput Syst 2018; 88: 28–39.

16.

Ullah

Muhammad

Haq

et al. Action recognition using optimized deep autoencoder and CNN for surveillance data streams of non-stationary environments. Future Gener Comput Syst 2019; 96: 386–397.

17.

Wang

Liu

Shen

et al. Order-aware convolutional pooling for video based action recognition. Pattern Recognit 2019; 91: 357–365.

18.

Sharif

Khan

Akram

et al. A framework of human detection and action recognition based on uniform segmentation and combination of Euclidean distance and joint entropy-based features selection. EURASIP J Image Video Process 2017; 2017(1): 1–18.

19.

Shi

Chen

Wang

et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Adv Neural Inf Process Syst 2015; 2015: 802–810.

20.

Vinayakumar

Alazab

Soman

et al. Deep learning approach for intelligent intrusion detection system. IEEE Access 2019; 7: 41525–41550.

21.

Adnan

Rahim

MSM

Rehman

et al. Automatic image annotation based on deep learning models: a systematic review and future challenges. IEEE Access 2021; 9: 50253–50264.

22.

Chaudhry

Rahim

MSM

Saba

et al. Crowd region detection in outdoor scenes using color spaces. Int J Model Simul Sci Comput 2018; 9(2): 1850012.

23.

Sharif

Attique

Tahir

et al. A machine learning method with threshold based parallel feature fusion and feature selection for automated gait recognition. J Organ End User Comput 2020; 32: 67–92.

24.

Afza

Khan

Sharif

et al. A framework of human action recognition using length control features fusion and weighted entropy-variances based feature selection. Image Vis Comput 2021; 106: 104090.