With the development of human-computer interaction, gesture recognition has gradually become one of the research hotspots. The cost reduction and the richer information of RGB-D images make the research of gesture recognition based on RGB-D images more and more. However, the current gesture processing methods for RGB-D images still can not fully utilize the information contained. Aiming at the above problems, this paper studies the feature extraction method of RGB-D image, and proposes a multimodal and multilevel feature extraction method. By extracting multimodal and multilevel image features for mapping and splicing, the utilization of RGB-D image information and the accuracy in recognition are improved effectively. Finally, the experiments verified the effectiveness and robustness of the proposed method based on the self-built gesture database. Compared and analyzed with several other RGB-D processing methods, the processing method of this paper is more advanced and effective, and can achieve better results in gesture recognition.
ChengW.T., SunY., LiG.F., JiangG.Z. and LiuH.H., Jointly network: A network based on CNN and RBM for gesture recognition, Neural Computing and Applications31(Suppl 1) (2019), 309–323.
2.
BelgacemS., ChatelainC. and PaqueT., Gesture sequence recognition with one shot learned CRF/HMM hybrid model, Image and Vision Computing61 (2017), 12–21.
3.
SunY., LiC.Q., LiG.F., JiangG.Z., JiangD., LiuH.H., ZhengZ.G. and ShuW.N., Gesture recognition based on Kinect and sEMG signal fusion, Mobile Networks and Applications23(4) (2018), 797–805.
4.
ChakrabortyB.K., SarmaD., BhuyanM.K. and MacDormanK.F., Review of constraints on vision-based gesture recognition for human-computer interaction, IET COMPUTER VISION12(1) (2018), 3–15.
5.
LiJ., WangJ.X. and JuZ.J., A novel hand gesture recognition based on high-level features, International Journal of Humanoid Robotics15(2) (2018). DOI: 10.1142/S0219843617500220
6.
LiJ., MiY., LiG.F. and JuZ.J., CNN-Based facial expression recognition from annotated RGB-D images for Human–Robot interaction, Humanoid Robotics (2019). DOI: 10.1142/S0219843619410020
7.
TanC., SunY., LiG.F., JiangG.Z., ChenD.S. and LiuH.H., Research on gesture recognition of smart data fusion features in the IoT, Computing and Applications (2019). DOI: 10.1007/s00521-019-04023-0
8.
PigouL., Van Den OordA. and
DielemanS., Beyond temporal pooling: Recurrence and temporal convolutions for gesture recognition in video, International Journal of Computer Vision126(2–4) (2018), 430–439.
9.
HeY., LiG.F., LiaoY.J., SunY., KongJ.Y., JiangG.Z., JiangD., TaoB., XuS. and LiuH.H., Gesture recognition based on an improved local sparse representation classification algorithm, Cluster Computer (2017). DOI: 10.1007/s10586-017-1237-1
10.
QiJ.X., JiangG.Z., LiG.F. and SunY., Intelligent human-computer interaction based on surface EMG gesture recognition, IEEE Access7 (2019), 61378–61387.
11.
ZhouY., JiangG. and LinY., A novel finger and hand pose estimation technique for real-time hand gesture recognition, Pattern Rrcognition49 (2016), 102–114.
12.
BlumM.K., KumarD.A. and MacDormanK.F., A novel set of features for continuous hand gesture recognition, Journal on Multimodal User Interfaces8(4) (2014), 333–343.
13.
SharpT., KeskinC. and RobertsonD., Accurate, robust, and flexible real-time hand tracking, Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing System (2015), 3633–3642.
14.
BarrosP., Maciel-JuniorN.T., FernandesB.J.T., BezerraB.L.D. and FernandesS.M.M., A dynamic gesture recognition and prediction system using the convexity approach, Computer Vision and Image Understanding155 (2017), 139–149.
15.
EscalanteH.J., GuyonI., AthitsosV., JangyodsukP. and WanJ., Principal motion components for one-shot gesture recognition, Pattern Analysis and Applications20(1) (2017), 167–182.
16.
LenzI., LeeH. and SaxenaA., Deep learning for detecting robotic grasps, The International Journal of Robotics Research34 (2015), 705–724.
17.
FangY.F., ZhouD.L., LiK. and LiuH.H., Interface prostheses with classifier-feedback-based user training, IEEE Transactions on Neural Systems and Rehabilitation Engineering64(11) (2017), 2575–2583.
18.
SagayamK.M. and HemanthD.J., Application of Pseudo 2-D hidden Markov model for hand gesture recognition, Advances in Intelligent Systems and Computing507 (2017), 179–188.
19.
MarinG., DominioF. and ZanuttighP., Hand gesture recognition with jointly calibrated leap motion and depth sensor, Multimedia Tools and Application75(22) (2016), 14991–15015.
20.
ZhaoW. and DuS., Spectral-spatial feature extraction for hypersperctral image classification: A dimension reduction and deep learning approach, IEEE Transactions on Geoscience and Remote Sensing54(8) (2016), 4544–4554.
21.
PigouL., VanDenOordA. and DielemanS., Beyond temporal pooling: Recurrence and temporal convolutions for gesture recognition in video, International Journal of Computer Vision126(2–4) (2018), 430–439.
22.
WangP., LiW. and OgunbonaP., RGB-D-based human motion recognition with deep learning: A survey, Computer Vision and Image Understanding171 (2018), 118–139.
23.
HeidariS., NaseriM., GheibiR., BaghfalakiM., PourarianM.R. and FaroukA., A new quantum watermarking based on quantum wavelet transforms, Communications in theoretical Physics67(6) (2017), 732–742.
24.
NaseriM., AbdolmalekyM., ParandinF., FatahiN., FaroukA. and NazariR., A new quantum gray-scale image encoding scheme, Communications in Theoretical Physics69(2) (2018), 215–226.
25.
NagataK., NakamuraT. and FaroukA., Quantum cryptography based on the Deutsch-Jozsa algorithm, International Journal of Theoretical Physics56(9) (2017), 2887–2897.
26.
BatleJ., OoiC.R., FaroukA., AbutalibM. and AbdallaS., Do multipartite correlations speed up adiabatic quantum computation or quantum annealing?Quantum Information Processing15(8) (2016), 3081–3099.
27.
BatleJ., FaroukA., TarawnehO. and AbdallaS., Multipartite quantum correlations among atoms in QED cavities, Frontiers of Physics13(1) (2018), 130305.
28.
BatleJ., OoiC.R., FaroukA., AlkhambashiM.S. and AbdallaS., Global versus local quantum correlations in the Grover search algorithm, Quantum Information Processing15(2) (2016), 833–849.
29.
NagataK., NakamuraT., GeurdesH., BatleJ., AbdallaS. and FaroukA., Creating very true quantum algorithms for quantum energy based computing, International Journal of Theoretical Physics57(4) (2018), 973–980.
30.
JiangD., LiG.F., SunY., KongJ.Y. and TaoB., Gesture recognition based on skeletonization algorithm and CNN with ASL database, Multimedia Tools and Applications (2018). DOI: 10.1007/s11042-018-6748-0
31.
LiG.F., JiangD., ZhouY.L., JiangG.Z., KongJ.Y. and ManogaranG., Human Lesion Detection Method Based on Image Information and Brain Signal, IEEE Access7 (2019), 11533–11542. DOI: 10.1109/ACCESS.2019.2891749
32.
LeeD.L. and YouW.S., Recognition of complex static hand gestures by using the wristband-based contour features, IET Image Processing12(1) (2018), 80–87.
33.
SunY., HuJ.B., LiG.F., JiangG.Z., XiongH.G., TaoB., ZhengZ.J. and JiangD., Gear Reducer Optimal Design based on Computer Multimedia Simulation, The Journal of Supercomputing (2018). DOI: 10.1007/s11227-018-2255-3
34.
NyirarugiraC., ChoiH.R. and KimT., Hand gesture recognition using particle swarm movement, Mathematical Problems in Engineering1 (2016), 1–8.
35.
JiangD., ZhengZ.J., LiG.F.,
SunY,
KongJ.Y.,
JiangG.Z.,
XiongH.G.,
TaoB.,
XuS.,
LiuH.H. and
JuZ.J., Gesture recognition based on binocular vision, Cluster Computing (2018). DOI: 10.1007/s10586-018-1844-5
36.
LiuK. and KehtarnavazWangN., A, Real-time robust vision-based hand gesture recognition using stereo images, Journal of Real-time Image Processing11(1) (2016), 201–209.
37.
ChenD.S., LiG.F., SunY., KongJ.Y., JiangG.Z., TangH., JuZ.J., YuH. and LiuH.H., An interactive image segmentation method in hand gesture recognition, Sensors17(2) (2017), 253. DOI: 10.3390/s17020253
38.
D’OrazioT., MaraniR., RenoV. and CicirelliG., Recent trends in gesture recognition: How depth data has improved classical approaches, Image and Vision Computing52 (2016), 56–72.
39.
LiB., SunY., LiG.F., KongJ.Y., JiangG.Z., JiangD., TaoB., XuS. and LiuH.H., Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm, Cluster Computing22(Suppl 1) (2019), 503–512.
40.
OboT., LooCK., SeeraM. and KubotaN., Hybrid evolutionary neuro-fuzzy approach based on mutual adaptation for human gesture recognition, Applied Soft Computong42 (2016), 377–389.
41.
LiC.C., LiG.F., JiangG.Z., ChenD.S. and LiuH.H., Surface EMG data aggregation processing for intelligent prosthetic action recognition, Neural Computing and Applications (2018). DOI: 10.1007/s00521-018-3909-z
42.
HuangY.J., YangX.C., LiY.F., ZhouD.L. and LiuH.H., Ultrasound-based sensing models for finger motion classification, IEEE Journal of Biomedical and Health Informatics22(5) (2018), 1395–1405.
43.
LiG.F., KongJ.Y., YangJ.T., HuangX.C. and HouY., Genetic algorithm and its application research, prospect in mechanical optimization design, Dynamics of Continuous Discrete and Impulsive Systems-Series A-Mathematical Analysis13 (2006), 1446–1453.
44.
DinhD.L., LeeS. and KimT.S., Hand number gesture recognition using recognized hand parts in depth images, Multimedia Tools and Applications75(2) (2016), 1333–1348.
45.
HuJ.B., SunY., LiG.F., JiangG.Z. and TaoB., Probability analysis for grasp planning facing the field of medical robotics, Measurement141 (2019), 227–234.
46.
NuzziC., PasinettiS., LanciniM., DocchioF. and SansoniG., Deep learning-based hand gesture recognition for collaborative robots, IEEE Instrumentation & Measurement Magazine22(2) (2019), 44–51.
47.
HuaJ., LiG.F., JiangD., ZhaoH.Y. and QiJ.X., An optimized selection method of channel numbers and electrode layouts for hand motions recognition, International Journal of Humanoid Robotics (2019). DOI: 10.1142/S0219843619410068
48.
PanellaM. and AltilioR., A smartphone based application using machine Learning for gesture recognition, IEEE Instrumentation & Measurement Magazine8(1) (2019), 25–29.
49.
ZhangC. and ZhangZ., Calibration between depth and color sensors for commodity depth cameras, Computer Vision and Machine Learning with RGB-D Sensors (2014). DOI: 10.1007/978-3-319-08651-4_3
50.
LiG.F., WuH., JiangG.Z., XuS. and LiuH.H., Dynamic gesture recognition in the internet of things, IEEE Access7 (2019), 23713–23724.
51.
SkariaS., Al-HouraniA., LechM. and EvansR.J., Hand-gesture recognition using Two-Antenna doppler radar with deep convolutional neural networks, IEEE Sensors Journal19(8) (2019), 3041–3048.
52.
LiG.F., ZhangL.L., SunY. and
KongJ.Y, Towards the sEMG hand: Internet of things sensors and haptic feedback application, Multimedia Tools and Applications (2018). DOI: 10.1007/s11042-018-6293-x
53.
DengJ., YangY., TaoJ., DengL.G., LiuD.Q., GuanZ.Q., LiG.F., LiZ.L., YuS.H., ZhengG.X., LiZ.Y. and ZhangS., Spatial Frequency Multiplexed Meta-Holography and Meta-Nanoprinting, ACS Nano13 (2019), 9237–9246.
54.
WuH., JiangZ.G., ZhangH., WangY., YangY.H. and LiY., An integrated MCDM approach considering demands-matching for reverse logistics, Journal of Cleaner Production208 (2018), 199–210.
55.
LiG.F., TangH., SunY., KongJ.Y., JiangG.Z., JiangD., TaoB., XuS. and LiuH.H., Hand gesture recognition based on convolution neural network, Cluster Computing (2017). DOI: 10.1007/s10586-017-1435-x
56.
SagayamK.M. and HemanthD.J., A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition, Computational Intelligence35(1) (2019), 59–81.
57.
LuoB.W., SunY., LiG.F., ChenD.S. and JuZ.J., Decomposition algorithm for depth image of human health posture based on brain health, Neural Computing and Applications (2019). DOI: 10.1007/s00521-019-04141-9
58.
ByunS.W. and LeeS.P., Hand gesture recognition suitable for wearable devices using flexible epidermal tactile sensor array, Journal of Elecrical Engineering & Technology13(4) (2018), 1731–1738.
59.
JuZ.J., JiX.F., LiJ. and LiuH.H., An integrative framework of human hand gesture segmentation for human-robot interaction, IEEE Systems Journal11(3) (2017), 1326–1336.
60.
YuM.C., LiG.F., JiangD., TaoB. and ChenD.S., Hand medical monitoring system based on machine learning and optimal EMG feature set, Personal and Ubiquitous Computing (2019). DOI: 10.1007/s00779-019-01285-2
61.
LiK., FangY.F., ZhouY., JuZ.J. and LiuH.H., Haptics model for human fingertips based on gaussian distribution, Journal of Intelligent & Fuzzy Systems36(5) (2019), 3945–3955.
62.
LiG.F., LiJ.H., JuZ.J., SunY. and KongJ.Y., A novel feature extraction method for machine learning based on surface electromyography from healthy brain, Neural Computing and Applications (2019). DOI: 10.1007/s00521-019-04147-3
63.
ChoiH.R. and KimT., Modified dynamic time warping based on direction similarity for fast gesture recognition, Mathematical Problems in Engineering (2018), 1–9.
64.
QiJ.X., JiangG.Z., LiG.F., SunY. and TaoB., Surface EMG hand gesture recognition system based on PCA and GRNN, Neural Computing and Applications (2019). DOI: 10.1007/s00521-019-04142-8
65.
JiangD., LiG.F., SunY., KongJ.Y., TaoB. and ChenD.S., Grip Strength Forecast and Rehabilitative Guidance Based on Adaptive Neural Fuzzy Inference System Using sEMG, Personal and Ubiquitous Computing (2019). DOI: 10.1007/s00779-019-01268-3
