Most people primarily rely on subjective opinions and action assessment from others in the process of learning badminton, which leads to biases and unreliable assessment of player performance. This paper presents a method of deep learning-based recognition and quality assessment of badminton actions to accurately identify player actions and assess player performance. In this research, we construct a video dataset of standard badminton actions for training networks and design a human pose estimation and tracking network to detect keypoints of individual players in the video dataset and track their trajectories. Furthermore, badminton action recognition is carried out based on the SlowFast network framework, and a Siamese network with it as the backbone network is proposed for automating the quality assessment of badminton actions. Experimental results demonstrate that the mean average precision (mAP) of human body pose estimation reaches 83.2%, the Multiple Object Tracking Accuracy (MOTA) of pose detection and recognition in badminton games reaches 81.4%, and the Multiple Object Tracking Precision (MOTP) reaches 90.7%. The accuracy of professional players in identifying badminton strokes is 83.08% for Top-1 and 96.89% for Top-3. Therefore, the proposed method can be effectively applied to badminton action recognition and quality assessment.
LiBXuX. Application of artificial intelligence in basketball sport. Journal of Education, Health and Sport(2021); 11: 54–67.
2.
RaviDWongCLoB,et al.Deep learning for human activity recognition: a resource efficient implementation on low-power devices. In: 2016 IEEE 13th international conference on wearable and implantable body sensor networks (BSN). Piscataway, NJ: IEEE, 2016, pp. 71–76.
3.
ReyazNAhamadGKhanNJ,et al.Machine learning in sports talent identification: a systematic review. In: 2022 2nd international conference on emerging frontiers in electrical and electronic technologies (ICEFEET). Piscataway, NJ: IEEE, 2022, pp. 1–6.
4.
WangYFangWMaJ, et al.Automatic badminton action recognition using cnn with adaptive feature extraction on sensor data. In: Intelligent computing theories and application: 15th international conference, ICIC, Nanchang, China, August 3–6. Berlin, Germany: Springer, 2019, pp. 131–143.
5.
QinLWangDGuoM, et al.Optimizing badminton action recognition with deep learning and sensor fusion: a study of sensor numbers and combinations. In: 2023 IEEE 13th international conference on CYBER technology in automation, control, and intelligent systems (CYBER). Piscataway, NJ: IEEE, 2023, pp. 273–278.
6.
YanSXiongYLinD. Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI conference on artificial intelligence. Palo Alto, CA: AAAI Press, 2018, pp. 1-9.
7.
FeichtenhoferCFanHMalikJ,et al.Slowfast networks for video recognition. In: Proceedings of the IEEE/CVF international conference on computer vision. Palo Alto, CA: IEEE, Piscataway, 2019, pp. 6202–6211.
8.
ParmarPMorrisB. Action quality assessment across multiple actions. In: 2019 IEEE winter conference on applications of computer vision (WACV). Piscataway, NJ: IEEE, pp. 1468–1476.
9.
ShaoDZhaoYDaiB,et al.Finegym: a hierarchical video dataset for fine-grained action understanding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2020, pp. 2616–2625.
10.
XuJRaoYYuX, et al.Finediving: a fine-grained dataset for procedure-aware action quality assessment. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2022, pp. 2949–2958.
11.
BanK-WSeeJAbdullahJ,et al.Badmintondb: a badminton dataset for player-specific match analysis and prediction. In: Proceedings of the 5th international ACM workshop on multimedia content analysis in sports. New York, NY: ACM, 2022, pp. 47–54.
12.
LiQChiuT-CHuangH-W, et al.Videobadminton: a video dataset for badminton action recognition, in: 2024 IEEE International Conference on Big Data (BigData). Piscataway, NJ: IEEE, 2024, pp. 1387–1392.
13.
SunKXiaoBLiuD,et al.Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2019, pp. 5693–5703.
14.
WangJLongXGaoY, et al.Graph-pcnn: two stage human pose estimation with graph pose refinement. In: Computer vision–ECCV 2020: 16th European conference, proceedings, part XI 16, August 23–28, Glasgow, UK. Berlin, Germany: Springer, 2020, pp. 492–508.
15.
NingGPeiJHuangH. Lighttrack: a generic framework for online top-down human pose tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. Piscataway, NJ: IEEE, 2020, pp. 1034–1035.
16.
SnowerMKadavALaiF,et al.15 Keypoints is all you need. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2020, pp. 6738–6748.
17.
XiaoBWuHWeiY. Simple baselines for human pose estimation and tracking. In: Proceedings of the European conference on computer vision (ECCV). Berlin, Germany: Springer, 2018, pp. 466–481.
18.
KreissSBertoniLAlahiA. Pifpaf: composite fields for human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2019, pp.11977–11986.
19.
NewellAHuangZDengJ. Associative embedding: End-to-end learning for joint detection and grouping. Adv Neural Inf Process Syst2017, 1-1130.
20.
ChengBXiaoBWangJ, et al.Higherhrnet: scale-aware representation learning for bottom-up human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, IEEE2020, pp. 5386–5395.
21.
RaajYIdreesHHidalgoG,et al.Efficient online multi-person 2d pose tracking with recurrent spatio-temporal affinity fields. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2019, pp. 4620–4628.
22.
JinSLiuWOuyangW,et al.Multi-person articulated tracking with spatial and temporal embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2019, pp. 5664–5673.
23.
VaswaniAShazeerNParmarN, et al.Attention is all you need. Adv Neural Inf Process Syst201730, 1-11.
24.
GhoshPYaoYDavisL,et al.Stacked spatio-temporal graph convolutional networks for action segmentation. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. Piscataway, NJ: IEEE, 2020, pp. 576–585.
25.
ShiLZhangYChengJ,et al.Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2019, pp. 12026–12035.
26.
DuanHZhaoYChenK, et al.Revisiting skeleton-based action recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2022, pp. 2969–2978.
27.
RajAConsulPPalSK. Fast neural accumulator (NAC) based badminton video action classification. In: Intelligent systems and applications: proceedings of the 2020 intelligent systems conference (IntelliSys) volume 1. Berlin, Germany: Springer, 2021, pp. 452–467.
28.
RahmadNAAs’ ariMAIbrahimMF, et al.Vision based automated badminton action recognition using the new local convolutional neural network extractor. In: Enhancing health and sports performance by design: proceedings of the 2019 movement, health & exercise (MoHE) and international sports science conference (ISSC). Berlin, Germany: Springer, 2020, pp. 290–298.
29.
IsaWHMAbdullahMARazmanMAM, et al.Deep learning algorithms for recognition of badminton strokes: a study using SDNN, RNN, and RNN-GRU models with off-court video capture. In: International conference on mechatronics and intelligent robotics. Berlin, Germany: Springer, 2023, pp. 53–60.
30.
BertasiusGWangHTorresaniL. Is space-time attention all you need for video understanding? In: ICML. ACM, New York, NY, 2021, p. 4.
31.
LiuZNingJCaoY, et al.Video swin transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2022, pp. 3202–3211.
32.
GaoJZhengW-SPanJ-H, et al.An asymmetric modeling for action assessment. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part XXX 16. Berlin, Germany: Springer, 2020, pp. 222–238.
33.
PanJ-HGaoJZhengW-S. Action assessment by joint relation graphs. In: Proceedings of the IEEE/CVF international conference on computer vision. Piscataway, NJ: IEEE, 2019, pp. 6331–6340.
34.
BaiYZhouDZhangS, et al.Action quality assessment with temporal parsing transformer. In: European conference on computer vision. Berlin, Germany: Springer, 2022, pp. 422–438.
LinT-YMaireMBelongieS, et al.Microsoft coco: common objects in context. In: Computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, September 6–12, 2014, proceedings, part V 13. Berlin, Germany: Springer, 2014, pp.740–755.
37.
GamraMBAkhloufiMA. A review of deep learning techniques for 2D and 3D human pose estimation. Image Vis Comput2021114,104282.
38.
RedmonJFarhadiA. Yolov3: An incremental improvement, arXiv preprint arXiv:1804.02767. 2018.
39.
SongY-FZhangZ and ShanC,et al.Richly activated graph convolutional network for robust skeleton-based action recognition. IEEE Trans Circuits Syst Video Technol2020; 31: 1915–1925.
40.
HeKZhangXRenS,et al.Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Piscataway, NJ: IEEE, 2016, pp. 770–778.
