Hand gesture intention-based identity recognition using various recognition strategies incorporated with VGG convolution neural network-extracted deep learning features

Abstract

With fast developments of artificial intelligence, human behaviors can be further acknowledged by means of the biometric information of hand gesture actions made by the person. Such hand gesture information revealing the specific intention of the person will be undoubtedly a critical clue to cognize human behaviors. Furthermore, identity recognition of the hand gesture-making person is one of the most important technique issues in hand gesture recognition applications. This work explores hand gesture intention-based identity recognition where various deep learning recognition strategies are presented. The well-know image sensor of Leap Motion Controller (LMC) is employed in this work for acquisitions of active hand gesture data. This paper presents four different deep learning strategies for hand gesture intention-based identity recognition, all of which are based on the deep learning model of the visual geometry group (VGG)-type convolution neural network (CNN). The presented deep learning strategies to perform hand gesture intention-based identity recognition are typical VGG-16 CNN deep learning, dynamic time warping (DTW) classifications with VGG-16 CNN extracted deep learning features, DTW classifications by VGG-16 CNN extracted deep learning features with principal component analysis (PCA) data reduction, and PCA centroid classifications using VGG-16 CNN extracted deep learning features with PCA. Compared with traditional hand gesture recognition by classifications of only the geometrical space feature of LMC 3D-(x, y, z) data without any deep learning, most of presented VGG-CNN based deep learning approaches have more outstanding performances on recognition accuracy. In the situation of real-time recognition that considers both of recognition accuracy and computation time, PCA centroid classifications by VGG-16 CNN extracted deep learning features with PCA reduction, FC1-PCA and FC2-PCA features that are estimated from the first and the second fully connected (FC) layer of VGG-CNN respectively (i.e. FC1 and FC2 layers) and then significantly reduced the data dimension by PCA, apparently performs best among all presented deep learning strategies.