Abstract
WiFi-based gesture recognition emerges in recent years and attracts extensive attention from researchers. Recognizing gestures via WiFi signal is feasible because a human gesture introduces a time series of variations to the received raw signal. The major challenge for building a ubiquitous gesture recognition system is that the mapping between each gesture and the series of signal variations is not unique, exact the same gesture but performed at different locations or with different orientations towards the transceivers generates entirely different gesture signals (variations). To remove the location dependency, prior work proposes to use gesture-level location-independent features to characterize the gesture instead of directly matching the signal variation pattern. We observe that gesture-level features cannot fully remove the location dependency since the signal qualities inside each gesture are different and also depends on the location. Therefore, we divide the signal time series of each gesture into segments according to their qualities and propose customized signal processing techniques to handle them separately. To realize this goal, we characterize signal's sensing quality by building a mathematical model that links the gesture signal with the ambient noise, from which we further derive a unique metric i.e., error of dynamic phase index (EDP-index) to quantitatively describe the sensing quality of signal segments of each gesture. We then propose a quality-oriented signal processing framework that maximizes the contribution of the high-quality signal segments and minimizes the impact of low-quality signal segments to improve the performance of gesture recognition applications. We develop a prototype on COTS WiFi devices. The extensive experimental results demonstrate that our system can recognize gestures with an accuracy of more than 94% on average, and significant improvements compared with state-of-arts.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Towards Robust Gesture Recognition by Characterizing the Sensing Quality of WiFi Signals
- Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. WiGest: A ubiquitous wifi-based gesture recognition system. In 2015 IEEE Conference on Computer Communications (INEOCOM). IEEE, 1472--1480.Google ScholarCross Ref
- Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 90--102.Google ScholarDigital Library
- Xiaoxiao Cao, Bing Chen, and Yanchao Zhao. 2016. Wi-Wri: Fine-grained writing recognition using Wi-Fi signals. In 2016 IEEE Trustcom/BigDataSE/ISPA. IEEE, 1366--1373.Google Scholar
- R. H. Clarke. 1968. A statistical theory of mobile-radio reception. The Bell System Technical Journal 47, 6 (1968), 957--1000. https://doi.org/10.1002/j.1538--7305.1968.tb00069.xGoogle ScholarCross Ref
- Emily A Cooper and Hany Farid. 2020. A Toolbox for the Radial and Angular Marginalization of Bivariate Normal Distributions. arXiv preprint arXiv:2005.09696 (2020).Google Scholar
- Biyi Fang, Nicholas D Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. 2016. BodyScan: A Wearable Device for Contact-less Radio-based Sensing of Body-related Activities. In 14th ACM Conference on Mobile Systems, Applications, and Services (MobiSys' 16). ACM.Google ScholarDigital Library
- Makiko Funasaka, Yu Ishikawa, Masami Takata, and Kazuki Joe. 2015. Sign language recognition using leap motion controller. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA). The Steering Committee of The World Congress in Computer Science, 263.Google Scholar
- Ruiyang Gao, Mi Zhang, Jie Zhang, Yang Li, Enze Yi, Dan Wu, Leye Wang, and Daqing Zhang. 2021. Towards Position-Independent Sensing for Gesture Recognition with Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--28.Google ScholarDigital Library
- Google. 2015. Project Soli. Retrieved Jan 15, 2020 from https://www.youtube.com/watch?v=0QNiZfSsPc0Google Scholar
- Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool release: gathering 802.11 n traces with channel state information. ACM SIGCOMM Computer Communication Review 41, 1 (2011), 53--53.Google ScholarDigital Library
- Wenfeng He, Kaishun Wu, Yongpan Zou, and Zhong Ming. 2015. Wig: Wifi-based gesture recognition system. In Computer Communication and Networks (ICCCN), 2015 24th International Conference on. IEEE, 1--7.Google ScholarCross Ref
- Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, 269--282.Google ScholarDigital Library
- Hong Li, Wei Yang, Jianxin Wang, Yang Xu, and Liusheng Huang. 2016. WiFinger: talk to your smart devices with finger-grained gesture. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 250--261.Google ScholarDigital Library
- Shengjie Li, Zhaopeng Liu, Yue Zhang, Qin Lv, Xiaopeng Niu, Leye Wang, and Daqing Zhang. 2020. WiBorder: Precise Wi-Fi based Boundary Sensing via Through-wall Discrimination. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--30.Google ScholarDigital Library
- Xiang Li, Daqing Zhang, Qin Lv, Jie Xiong, Shengjie Li, Yue Zhang, and Hong Mei. 2017. IndoTrack: Device-free indoor human tracking with commodity Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 72.Google ScholarDigital Library
- Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1515--1525.Google ScholarDigital Library
- Kai Niu, Fusang Zhang, Yuhang Jiang, Jie Xiong, Qin Lv, Youwei Zeng, and Daqing Zhang. 2019. WiMorse: A Contactless Morse Code Text Input System Using Ambient WiFi Signals. IEEE Internet of Things Journal 6, 6 (2019), 9993--10008.Google ScholarCross Ref
- Kai Niu, Fusang Zhang, Xuanzhi Wang, Qin Lv, Haitong Luo, and Daqing Zhang. 2021. Understanding WiFi Signal Frequency Features for Position-Independent Gesture Sensing. IEEE Transactions on Mobile Computing (2021), 1--1. https://doi.org/10.1109/TMC.2021.3063135Google ScholarCross Ref
- Sameera Palipana, David Rojas, Piyush Agrawal, and Dirk Pesch. 2018. FallDeFi: Ubiquitous Fall Detection using Commodity Wi-Fi Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 155.Google ScholarDigital Library
- Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 27--38.Google ScholarDigital Library
- Muhammad Shahzad and Shaohu Zhang. 2018. Augmenting user identification with WiFi based gesture recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--27.Google ScholarDigital Library
- Chao Sun, Tianzhu Zhang, and Changsheng Xu. 2015. Latent support vector machine modeling for sign language recognition with Kinect. ACM Transactions on Intelligent Systems and Technology (TIST) 6, 2 (2015), 20.Google Scholar
- D. Tse and P. Viswanath. 2005. Fundamentals of Wireless Communication. Cambridge University Press. https://books.google.com.tr/books?id=66XBb5tZX6ECGoogle Scholar
- Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-Level Localization with a Single WiFi Access Point.. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Vol. 16. 165--178.Google Scholar
- Raghav H Venkatnarayan, Griffin Page, and Muhammad Shahzad. 2018. Multi-User Gesture Recognition Using WiFi. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 401--413.Google ScholarDigital Library
- Aditya Virmani and Muhammad Shahzad. 2017. Position and orientation agnostic gesture recognition using wifi. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 252--264.Google ScholarDigital Library
- Chuyu Wang, Jian Liu, Yingying Chen, Hongbo Liu, Lei Xie, Wei Wang, Bingbing He, and Sanglu Lu. 2018. Multi-Touch in the Air: Device-Free Finger Tracking and Gesture Recognition via COTS RFID. In Proceedings of the Conference on Computer Communications (INFOCOM). IEEE, 1691--1699.Google ScholarDigital Library
- Hao Wang, Daqing Zhang, Junyi Ma, Yasha Wang, Yuxiang Wang, Dan Wu, Tao Gu, and Bing Xie. 2016. Human respiration detection with commodity wifi devices: do user location and body orientation matter?. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 25--36.Google ScholarDigital Library
- Hao Wang, Daqing Zhang, Kai Niu, Qin Lv, Yuanhuai Liu, Dan Wu, Ruiyang Gao, and Bing Xie. 2017. MFDL: A Multicarrier Fresnel Penetration Model based Device-Free Localization System leveraging Commodity Wi-Fi Cards. arXiv preprint arXiv:1707.07514 (2017).Google Scholar
- Jue Wang, Deepak Vasisht, and Dina Katabi. 2014. RF-IDraw: virtual touch screen in the air using RF signals. In ACM SIGCOMM Computer Communication Review, Vol. 44. ACM, 235--246.Google ScholarDigital Library
- Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st annual international conference on mobile computing and networking. ACM, 65--76.Google ScholarDigital Library
- Wei Wang, Alex X Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 82--94.Google Scholar
- Xuyu Wang, Chao Yang, and Shiwen Mao. 2017. PhaseBeat: Exploiting CSI phase data for vital sign monitoring with commodity WiFi devices. In Distributed Computing Systems (ICDCS), 2017 IEEE 37th International Conference on. IEEE, 1230--1239.Google ScholarCross Ref
- Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proceedings of the 20th annual international conference on Mobile computing and networking. 617--628.Google ScholarDigital Library
- Wikipedia. 2022. Signal-to-noise ratio. Retrieved Feb 15, 2022 from https://en.wikipedia.org/wiki/Signal-to-noise_ratioGoogle Scholar
- Dan Wu, RuiYang Gao, Youwei Zeng, Jinyi Liu, Leye Wang, Tao Gu, and Daqing Zhang. 2020. FingerDraw: Sub-wavelength Level Finger Motion Tracking with WiFi Signals. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol 1, 1 (2020).Google ScholarDigital Library
- Dan Wu, Daqing Zhang, Chenren Xu, Yasha Wang, and Hao Wang. 2016. WiDir: walking direction estimation using wireless signals. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, 351--362.Google ScholarDigital Library
- Nan Yu, Wei Wang, Alex X Liu, and Lingtao Kong. 2018. QGesture: Quantifying Gesture Distance and Direction with WiFi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 51.Google ScholarDigital Library
- Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 15--28.Google ScholarDigital Library
- Youwei Zeng, Dan Wu, Ruiyang Gao, Tao Gu, and Daqing Zhang. 2018. FullBreathe: Full Human Respiration Detection Exploiting Complementarity of CSI Phase and Amplitude of WiFi Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 148.Google ScholarDigital Library
- Youwei Zeng, Dan Wu, Jie Xiong, Enze Yi, Ruiyang Gao, and Daqing Zhang. 2019. Farsense: Pushing the range limit of wifi-based respiration sensing with csi ratio of two antennas. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--26.Google ScholarDigital Library
- Daqing Zhang, Hao Wang, and Dan Wu. 2017. Toward centimeter-scale human activity sensing with Wi-Fi signals. Computer 50, 1 (2017), 48--57.Google ScholarDigital Library
- Fusang Zhang, Kai Niu, Jie Xiong, Beihong Jin, Tao Gu, Yuhang Jiang, and Daqing Zhang. 2019. Towards a diffraction-based sensing approach on human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (2019), 1--25.Google ScholarDigital Library
- Feng Zhang, Chenshu Wu, Beibei Wang, Hung-Quoc Lai, Yi Han, and KJ Ray Liu. 2019. WiDetect: Robust motion detection with a statistical electromagnetic model. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.Google ScholarDigital Library
- Fusang Zhang, Daqing Zhang, Jie Xiong, Hao Wang, Kai Niu, Beihong Jin, and Yuxiang Wang. 2018. From Fresnel Diffraction Model to Fine-grained Human Respiration Sensing with Commodity Wi-Fi Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 53.Google ScholarDigital Library
- Ouyang Zhang and Kannan Srinivasan. 2016. Mudra: User-friendly Fine-grained Gesture Recognition using WiFi Signals. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies. ACM, 83--96.Google ScholarDigital Library
- Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. 2018. Through-wall human pose estimation using radio signals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7356--7365.Google ScholarCross Ref
- Mingmin Zhao, Yonglong Tian, Hang Zhao, Mohammad Abu Alsheikh, Tianhong Li, Rumen Hristov, Zachary Kabelac, Dina Katabi, and Antonio Torralba. 2018. RF-based 3D skeletons. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. 267--281.Google ScholarDigital Library
- Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 313--325.Google ScholarDigital Library
Index Terms
- Towards Robust Gesture Recognition by Characterizing the Sensing Quality of WiFi Signals
Recommendations
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Whole-home gesture recognition using wireless signals
MobiCom '13: Proceedings of the 19th annual international conference on Mobile computing & networkingThis paper presents WiSee, a novel gesture recognition system that leverages wireless signals (e.g., Wi-Fi) to enable whole-home sensing and recognition of human gestures. Since wireless signals do not require line-of-sight and can traverse through ...
Augmenting User Identification with WiFi Based Gesture Recognition
Over the last few years, researchers have proposed several WiFi based gesture recognition systems that can recognize predefined gestures performed by users at runtime. As most environments are inhabited by multiple users, the true potential of WiFi based ...
Comments