Group Activity Recognition (GAR) is the task of recognizing an overall activity in a multi-individual scene. Most of the existing methods have achieved significant progress by incorporating the attributes and relations between individuals. However, these methods still suffer from the ability to automatically detect, recognize, and infer potential connections in group behavior. To address the issue, inspired by the role of latent spatial position present in video frames, we propose a novel method for learning graph structures by incorporating the distances between individuals. Specifically, we design a graph reasoning module based on Graph Convolutional Networks (GCNs) to learn the hierarchical relationship between individual behaviors and group intentions. To evaluate the feasibility and effectiveness of our proposed model, we conduct experiments on publicly available datasets. Through the experimental results, we validate the effectiveness of our approach, demonstrating its ability to accurately analyze and interpret group behavior.
QingJXiantaiGWeidongJ, et al.Intention recognition of aerial targets based on Bayesian optimization algorithm. In: 2017 2nd IEEE international conference on intelligent transportation engineering (ICITE), Singapore, 01–03 September 2017, pp. 356–359. IEEE.
2.
GuangleiMRunnanZBiaoW, et al.Target tactical intention recognition in multiaircraft cooperative air combat. International Journal of Aerospace Engineering2021; 2021: 1–18.
3.
LiCTTsaiYCChenCY, et al.Graph neural networks for tabular data learning: a survey with taxonomy and directions. arXiv preprint arXiv:2401.02143, 2024.
4.
GuoJHuangKYiX, et al.Rethinking spectral graph neural networks with spatially adaptive filtering. arXiv preprint arXiv:2401.09071, 2024.
5.
DefferrardMBressonXVandergheynstP. Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst2016; 3844–3852.
6.
HamiltonWYingZLeskovecJ. Inductive representation learning on large graphs. Adv Neural Inf Process Syst2017; 1025–1035.
7.
HeRFuZLiuQ, et al.Sparse relation graph for group activity recognition. In: 2022 IEEE 24th international workshop on multimedia signal processing (MMSP), Shanghai, China, 26–28 September 2022, pp. 1–5. IEEE.
8.
WatanabeTItoSYokoiK. Image feature descriptor using co‐occurrence histograms of oriented gradients for human detection. J Inst Image Inf Televis Eng2017; 71(1): J28–J34.
9.
KrähenbühlPKoltunV. Efficient inference in fully connected crfs with Gaussian edge potentials. Adv Neural Inf Process Syst2011, pp. 109–117.
10.
KongLQinJHuangD, et al.Hierarchical attention and context modeling for group activity recognition. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), Calgary, AB, Canada, 15–20 April 2018, pp. 1328–1332. IEEE.
11.
LuJXiongCParikhD, et al.Knowing when to look: adaptive attention via a visual sentinel for image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017.
12.
PramonoRRChenYTFangWH. Empowering relational network by self-attention augmented conditional random fields for group activity recognition. ECCV2020; 2020: 71–90.
13.
LiSCaoQLiuL, et al.Groupformer: group activity recognition with clustered spatial-temporal transformer. In: Proceedings of the IEEE/CVF international conference on computer vision, Montreal, QC, Canada, 10–17 October 2021, pp. 13668–13677.
14.
Kim D, Lee J, Cho M, et al. Detector-free weakly supervised group activity recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, New Orleans, LA, USA, 2022, pp. 20083–20093.
15.
ZhangHLuGZhanM, et al.Semi-supervised classification of graph convolutional networks with Laplacian rank constraints. Neural Process Lett2022; 54: 1–12.
16.
WuJWangLWangL, et al.Learning actor relation graphs for group activity recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 15–20 June 2019, pp. 9964–9974.
17.
LuPBaiWRueckertD, et al.Dynamic spatio-temporal graph convolutional networks for cardiac motion analysis. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI), Nice, France, 13–16 April 2021, pp. 122–125. IEEE.
18.
SunQLiJPengH, et al.Graph structure learning with variational information bottleneck. Proc AAAI Conf Artif Intell2022; 36(4): 4165–4174.
19.
ZhuYXuWZhangJ, et al.Deep graph structure learning for robust representations: a survey. arXiv preprint arXiv:2103.03036, 2021, 14.
20.
BagautdinovTAlahiAFleuretF, et al.Social scene understanding: end-to-end multi-person action localization and collective activity recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017, pp. 4315–4324.
21.
ButerezDJanetJPOglicD, et al.Masked attention is all you need for graphs. arXiv preprint arXiv:2402.10793, 2024.
22.
QiMWangY, et al.stagNet: An Attentive Semantic RNN for Group Activity and Individual Action Recognition,” In:IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 2, pp. 549–565, Feb. 2020.
23.
YuanHNiDWangM. Spatio-temporal dynamic inference network for group activity recognition. In: IEEE/CVF International Conference on Computer Vision (ICCV),Montreal, QC, Canada, 2021, pp. 7456–7465.
24.
LiQHanZWuXM. Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence,32(1), 2018.
25.
LiuMGaoHJiS. Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining,20(8), 2020, pp. 338–348.
26.
SzegedyCVanhouckeVIoffeS, et al.Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016, pp. 2818–2826.
27.
WangTZhengXZhangL, et al.Graph structure learning-based compression method for convolutional neural networks. In: International conference on algorithms and architectures for parallel processing. Singapore: Springer Nature Singapore, 2023, pp. 130–146.
28.
SuGWangHZhangY, et al.Simple and deep graph attention networks. Knowl Base Syst2024; 293: 111649.
29.
AzarSMGhadimi AtighMNickabadiA, et al.Convolutional relational machine for group activity recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, 15–20 June 2019, pp. 7892–7901.
