Analysis and comparison of various deep learning models to implement suspicious activity recognition in CCTV surveillance

Abstract

The paper aims to analyze and compare various deep learning (DL) algorithms in order to develop a Suspicious Activity Recognition (SAR) system for closed-circuit television (CCTV) surveillance. Automated systems for detecting and classifying suspicious activities are crucial as technology’s role in safety and security expands. This paper addresses these challenges by creating a robust SAR system using machine learning techniques. It analyzes and compares evaluation metrics such as Precision, Recall, F1 Score, and Accuracy using various deep learning methods (convolutional neural network (CNN), Long short-term memory (LSTM) – Visual Geometry Group 16 (VGG16), LSTM – ResNet50, LSTM – EfficientNetB0, LSTM – InceptionNetV3, LSTM – DenseNet121, and Long-term Recurrent Convolutional Network (LRCN)). The proposed system improves threat identification, vandalism deterrence, fight prevention, and video surveillance. It aids emergency response by accurately classifying suspicious activities from CCTV footage, reducing reliance on human security personnel and addressing limitations in manual monitoring. The objectives of the paper include analyzing existing works, extracting features from CCTV videos, training robust deep learning models, evaluating algorithms, and improving accuracy. The conclusion highlights the superior performance of the LSTM-DenseNet121 algorithm, achieving an overall accuracy of 91.17% in detecting suspicious activities. This enhances security monitoring capabilities and reduces response time. Limitations of the system include subjectivity, contextual understanding, occlusion, false alarms, and privacy concerns. Future improvements involve real-time object tracking, collaboration with law enforcement agencies, and performance optimization. Ongoing research is necessary to overcome limitations and enhance the effectiveness of CCTV surveillance.

Keywords

Deep learning machine learning DenseNet121 activity recognition convolutional neural network video processing

Get full access to this article

View all access options for this article.

References

Popoola

Wang

. Video-Based Abnormal Human Behavior Recognition – A Review. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 2012; 42(6): 865-878.

Kyrkou

Theocharides

. EmergencyNet: Efficient Aerial Image Classification for Drone-Based Emergency Monitoring Using Atrous Convolutional Feature Fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2020; 13: 1687-1699.

Hou

Chen

Shah

. Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos. 2017; IEEE International Conference on Computer Vision (ICCV). 2017; p. 5823-5832. Available from: https//api.semanticscholar.org/CorpusID:206771624.

De Geest

Tuytelaars

. Modeling Temporal Structure with LSTM for Online Action Detection. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV); 2018. pp. 1549-1557.

Gkountakos

Touska

Ioannidis

Tsikrika

Vrochidis

Kompatsiaris

. Spatio-Temporal Activity Detection and Recognition in Untrimmed Surveillance Videos. In: Proceedings of the 2021 International Conference on Multimedia Retrieval. ICMR ’21. New York, NY, USA: Association for Computing Machinery; 2021. pp. 451-455. Available from: doi: 10.1145/3460426.3463591.

Yeung

Russakovsky

Mori

Fei-Fei

. End-to-End Learning of Action Detection from Frame Glimpses in Videos. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. pp. 2678-2687.

Scariaa

Abahai

Isaacc

. Suspicious Activity Detection in Surveillance Video using Discriminative Deep Belief Network; 2017. Available from: https//api.semanticscholar.org/CorpusID:43258079.

Adam

Rivlin

Shimshoni

Reinitz

. Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2008; 30(3): 555-560.

Boudihir

. Intelligent video surveillance system architecture for abnormal activity detection. The International Conference on Informatics and Applications (ICIA2012); 2012; 1: 102-111.

10.

Tripathi

Jalal

Agrawal

. Suspicious Human Activity Recognition: A Review. Artif Intell Rev. 2018 Aug; 50(2): 283-339. Available from: doi: 10.1007/s10462-017-9545-7.

11.

Yang

Yuan

Xing

. SCNN: Sequential convolutional neural network for human action recognition in videos. In: 2017 IEEE International Conference on Image Processing (ICIP); 2017. pp. 355-359.

12.

Wang

Gao

Song

Shen

. Beyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action Recognition. IEEE Signal Processing Letters. 2017; 24(4): 510-514.

13.

Yao

Lei

Zhong

. A review of Convolutional-Neural-Network-based action recognition. Pattern Recognition Letters. 2019; 118: 14-22. Cooperative and Social Robots: Understanding Human Activities and Intentions. Available from: https//www.sciencedirect.com/science/article/pii/S0167865518302058.

14.

Chéron

Laptev

Schmid

. P-CNN: Pose-Based CNN Features for Action Recognition. In: 2015 IEEE International Conference on Computer Vision (ICCV); 2015. pp. 3218-3226.

15.

Ullah

Ahmad

Muhammad

Sajjad

Baik

. Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features. IEEE Access. 2018; 6: 1155-1166.

16.

Zhao

Huang

. An Investigation on Sparsity of CapsNets for Adversarial Robustness. In: Proceedings of the 1st International Workshop on Adversarial Learning for Multimedia. ADVM ’21. New York, NY, USA: Association for Computing Machinery; 2021. pp. 55-61. Available from: doi: 10.1145/3475724.3483609.

17.

Schuldt

Laptev

Caputo

. Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. vol. 3; 2004. pp. 32-36.