Pose Invariant Hand Gesture Recognition using Two Stream Transfer Learning Architecture
Anjali R. Patil1, S. Subbaraman2
1Mrs. Anjali R. Patil*, Assist. Prof., Electronics Engineering, DKTE‟s Textile and Engineering Institute, Ichalkaranji, Maharashtra, India.
2Dr. S. Subbaraman Professor, Electronics Engineering Department, Walchand College of Engineering, Sangli, Maharashtra, India.
Manuscript received on September 22, 2019. | Revised Manuscript received on October 20, 2019. | Manuscript published on October 30, 2019. | PP: 1771-1777 | Volume-9 Issue-1, October 2019 | Retrieval Number: F9058088619/2019©BEIESP | DOI: 10.35940/ijeat.F9058.109119
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: The hand gesture detection problem is one of the most prominent problems in machine learning and computer vision applications. Many machine learning techniques have been employed to solve the hand gesture recognition. These techniques find applications in sign language recognition, virtual reality, human machine interaction, autonomous vehicles, driver assistive systems etc. In this paper, the goal is to design a system to correctly identify hand gestures from a dataset of hundreds of hand gesture images. In order to incorporate this, decision fusion based system using the transfer learning architectures is proposed to achieve the said task. Two pretrained models namely ‘Mobile Net’ and ‘Inception V3’ are used for this purpose. To find the region of interest (ROI) in the image, YOLO (You Only Look Once) architecture is used which also decides the type of model. Edge map images and the spatial images are trained using two separate versions of the Mobile Net based transfer learning architecture and then the final probabilities are combined to decide upon the hand sign of the image. The simulation results using classification accuracy indicate the superiority of the approach of this paper against the already researched approaches using different quantitative techniques such as classification accuracy.
Keywords: Convolutional Neural Networks; edge map images; hand gestures; ROI, Transfer learning; YOLO.