Computer Science and Information Systems 2022 Volume 19, Issue 3, Pages: 1371-1388
https://doi.org/10.2298/CSIS220322037L
Full text (
397 KB)
Cited by
A novel art gesture recognition model based on two channel region-based convolution neural network for explainable human-computer interaction understanding
Li Pingping (School of Fine Arts, Zhengzhou Normal University Zhengzhou, China), 910675024@qq.com
Zhao Lu (School of Fine Arts, Yulin Normal University Yulin, China), zhaoluvip@163.com
The application development of hot technology is both an opportunity and a challenge. The vision-based gesture recognition rate is low and real-time performance is poor, so various algorithms need to be studied to improve the accuracy and speed of recognition. In this paper, we propose a novel gesture recognition based on two channel region-based convolution neural network for explainable human-computer interaction understanding. The input gesture image is extracted through two mutually independent channels. The two channels have convolution kernel with different scales, which can extract the features of different scales in the input image, and then carry out feature fusion at the fully connection layer. Finally, it is classified by the softmax classifier. The two-channel convolutional neural network model is proposed to solve the problem of insufficient feature extraction by the convolution kernel. Experimental results of gesture recognition on public data sets NTU and VIVA show that the proposed algorithm can effectively avoid the over-fitting problem of training models, and has higher recognition accuracy and stronger robustness than traditional algorithms.
Keywords: explainable human-computer interaction understanding, two channel region-based convolution neural network, gesture recognition, softmax classifier, feature fusion
Show references
Nguyen K A. ”Utilizing a Human-Computer Interaction Approach to Evaluate the Design of Current Pharmacogenomics Clinical Decision Support,” Journal of Personalized Medicine, vol. 11, 2021.
Zhong Q, Yang Q. ”Analyzing the Mental States of the Sports Student Based on Augmentative Communication with Human-Computer Interaction,” Journal of Interconnection Networks, 2021.
Jing Yu, Hang Li, Shoulin Yin. ”Dynamic Gesture Recognition Based on Deep Learning in Human-to-Computer Interfaces,” Journal of Applied Science and Engineering, vol. 23, no. 1, pp. 31-38, 2020.
Chaaba Ne S, Etien Ne A M, Schyns M, et al. ”The Impact of Virtual Reality Exposure on Stress Level and Sense of Competence in Ambulance Workers,” Journal of Traumatic Stress, 2021.
X. Zhang, F. Zhang and C. Xu. ”Joint Expression Synthesis and Representation Learning for Facial Expression Recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1681-1695, 2022.
Feng T. ”Mask RCNN-based Single Shot Multibox Detector For Gesture Recognition In Physical Education,” Journal of Applied Science and Engineering, vol. 26, no. 3, pp. 377-385, 2022.
Gasteiger N, Hellou M, Ahn H S. ”Factors for Personalization and Localization to Optimize Human-Robot Interaction: A Literature Review,” International Journal of Social Robotics, 2021:1-13.
Eskofier B M. ”A Smart Capacitive Sensor Skin with Embedded Data Quality Indication for Enhanced Safety in Human-Robot Interaction,” Sensors, vol. 21, 2021.
Hamuda E, Ginley B M, Glavin M, et al. ”Automatic crop detection under field conditions using the HSV colour space and morphological operations,” Computers & Electronics in Agriculture, vol. 133(Complete), pp. 97-107, 2017.
Udoh N, Ekpenyong M. ”A Knowledge-Based Framework for Cost Implication Modeling of Mechanically Repairable Systems with Imperfect Preventive Maintenance and Replacement Schedule,” Journal of Applied Science and Engineering, vol. 26, no. 2, pp. 221-234, 2022.
Bhattacharjee H, Anesiadis N, Vlachos D G. ”Regularized machine learning on molecular graph model explains systematic error in DFT enthalpies,” Scientific Reports, vol. 11, no. 1, 2021.
J. Wan, Q. Ruan, G. An and W. Li, ”Gesture recognition based on Hidden Markov Model from sparse representative observations,” 2012 IEEE 11th International Conference on Signal Processing, 2012, pp. 1180-1183, doi: 10.1109/ICoSP.2012.6491787.
Q. Chen, N. D. Georganas and E. M. Petriu, ”Real-time Vision-based Hand Gesture Recognition Using Haar-like Features,” 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007, 2007, pp. 1-6, doi: 10.1109/IMTC.2007.379068.
Lin S, Yuan W, Jing L, et al. Blurred palm-print recognition based on fusion of Laplacian smoothing transform and geometric features of hand,” Chinese Journal of Scientific Instrument, vol. 34, no. 2, pp. 415-422, 2013.
Asaari M S M, Suandi S A, Rosdi B A. Fusion of Band Limited Phase Only Correlation and Width Centroid Contour Distance for finger based biometrics,” Expert Systems with Applications, vol. 41, no. 7, pp. 3367-3382, 2014.
Liu F, Liu H Y, Gao L, et al. Hand shape recognition based on fusion features of fingers and particle swarm optimization,” Optics & Precision Engineering, vol. 23, no. 6, pp. 1774-1782, 2016.
X. Zhu, W. Liu, X. Jia and K. -Y. K. Wong, ”A two-stage detector for hand detection in egocentric videos,” 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1-8, 2016, doi: 10.1109/WACV.2016.7477665.
Qingwu Shi, Shoulin Yin, Kun Wang, Lin Teng and Hang Li. Multichannel convolutional neural network-based fuzzy active contour model for medical image segmentation,” Evolving Systems, 2021. https://doi.org/10.1007/s12530-021-09392-3
Wu S. ”Simulation of classroom student behavior recognition based on PSO-kNN algorithm and emotional image processing,” Journal of Intelligent & Fuzzy Systems, vol. 40, no. 4, pp. 7273-7283, 2021.
Nguyen-Trong K, Vu H N, Trung N N, et al. ”Gesture Recognition Using Wearable Sensors With Bi-Long Short-Term Memory Convolutional Neural Networks,” IEEE Sensors Journal, vol. 21, no. 13, pp. 15065-15079, , 2021.
Y Peng,Wang J, Pang K, et al. ”A Physiology-Based Flexible Strap Sensor for Gesture Recognition by Sensing Tendon Deformation,” IEEE Sensors Journal, vol. 21, no. 7, pp. 9449-9456, 2021.
Fang Y, Zhang X, Zhou D, et al. ”Improve Inter-day Hand Gesture Recognition Via Convolutional Neural Network-based Feature Fusion,” International Journal of Humanoid Robotics, 2021.
Singh D K. ”3D-CNN based Dynamic Gesture Recognition for Indian Sign Language Modeling,” Procedia Computer Science, vol. 189, pp. 76-83, 2021.
Rahman M A. ”Recognition of Static Hand Gestures of Alphabet in Bangla Sign Language,” IOSR Journal of Computer Engineering, vol. 8, no. 1, pp. 07-13, 2012.
M. Panwar, ”Hand gesture recognition based on shape parameters,” 2012 International Conference on Computing, Communication and Applications, pp. 1-6, 2012, doi: 10.1109/ICCCA. 2012.6179213.
Dominio F, Donadeo M, Zanuttigh P. ”Combining multiple depth-based descriptors for hand gesture recognition,” Pattern Recognition Letters, vol. 50, pp. 101-111, 2014.
Yang X, Feng Z, Huang Z, et al. ”Gesture Recognition Based on Combining Main Direction of Gesture and Hausdorff-like Distance,” Journal of Computer-Aided Design & Computer Graphics, 2016.
X. Zhang, F. Zhang and C. Xu, ”Joint Expression Synthesis and Representation Learning for Facial Expression Recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1681-1695, March 2022, doi: 10.1109/TCSVT.2021.3056098.
Y. Xia, W. Zheng, Y. Wang, H. Yu, J. Dong and F. -Y. Wang, ”Local and Global Perception Generative Adversarial Network for Facial Expression Synthesis,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1443-1452, March 2022, doi: 10.1109/TCSVT.2021.3074032.
H. Zhang,W. Su, J. Yu and Z.Wang, ”Identity-Expression Dual Branch Network for Facial Expression Recognition,” IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 4, pp. 898-911, Dec. 2021, doi: 10.1109/TCDS.2020.3034807.
Chen C., Liu MY., Tuzel O., Xiao J. ”R-CNN for Small Object Detection,” Computer Vision - ACCV 2016. ACCV 2016. Lecture Notes in Computer Science, vol. 10115, 2017. Springer, Cham.
T. H. N. Le, Y. Zheng, C. Zhu, K. Luu and M. Savvides, ”Multiple Scale Faster-RCNN Approach to Driver‘s Cell-Phone Usage and Hands on Steering Wheel Detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 46-53, 2016, doi: 10.1109/CVPRW.2016.13.
D. Poux, B. Allaert, N. Ihaddadene, I. M. Bilasco, C. Djeraba and M. Bennamoun, ”Dynamic Facial Expression Recognition Under Partial Occlusion With Optical Flow Reconstruction,” IEEE Transactions on Image Processing, vol. 31, pp. 446-457, 2022, doi: 10.1109/TIP.2021.3129120.
F. Zhang, T. Zhang, Q. Mao and C. Xu, ”Geometry Guided Pose-Invariant Facial Expression Recognition,” IEEE Transactions on Image Processing, vol. 29, pp. 4445-4460, 2020, doi: 10.1109/TIP.2020.2972114.
Ahmad M, Ahmed I, Jeon G. ”An IoT-enabled real-time overhead view person detection system based on Cascade-RCNN and transfer learning,” Journal of Real-Time Image Processing, vol. 6, 2021.
Sukhbaatar S, Bruna J, Paluri M, et al. Training convolutional networks with noisy labels[OL]. [2017-06-01]. https://arxiv.org/abs/1406.2080
Wang J, Lv P, Wang H, et al. ”SAR-U-Net: squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver CT segmentation,” Computer Methods and Programs in Biomedicine, vol. 208, 2021.
Jin Y, Bhatia A, Wanvarie D. Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation. 2021. https://doi.org/10.48550/arXiv.2104.09765
M Trusca, Spanakis G. ”Hybrid Tiled Convolutional Neural Networks (HTCNN) Text Sentiment Classification.” 2020. https://doi.org/10.48550/arXiv.2001.11857
Zhang C., He D., Li Z., Wang Z. ”Parallel Connecting Deep and Shallow CNNs for Simultaneous Detection of Big and Small Objects,” Pattern Recognition and Computer Vision. PRCV 2018. Lecture Notes in Computer Science, vol. 11259, 2018. Springer, Cham.