Raghav H. Venkatnarayan
ABSTRACT
WiFi based gesture recognition has received significant attention over the past few years. However, the key limitation of prior WiFi based gesture recognition systems is that they cannot recognize the gestures of multiple users performing them simultaneously. In this paper, we address this limitation and propose WiMU, a WiFi based Multi-User gesture recognition system. The key idea behind WiMU is that when it detects that some users have performed some gestures simultaneously, it first automatically determines the number of simultaneously performed gestures (Na) and then, using the training samples collected from a single user, generates virtual samples for various plausible combinations of Na gestures. The key property of these virtual samples is that the virtual samples for any given combination of gestures are identical to the real samples that would result from real users performing that combination of gestures. WiMU compares the detected sample against these virtual samples and recognizes the simultaneously performed gestures. We implemented and extensively evaluated WiMU using commodity WiFi devices. Our results show that WiMU recognizes 2, 3, 4, 5, and 6 simultaneously performed gestures with accuracies of 95.0, 94.6, 93.6, 92.6, and 90.9%, respectively.
Supplemental Material
- https://www.ettus.com/product/category/USRP-Networked-Series.Google Scholar
- Amazon Echo. http://www.businessinsider.com/the-inside-story-of-how-amazon-created-echo-2016-4.Google Scholar
- Independent Component Analysis. https://en.wikipedia.org/wiki/Independent_component_analysis.Google Scholar
- Jaccard Index. https://en.wikipedia.org/wiki/Jaccard_index.Google Scholar
- USRP N210. https://www.ettus.com/product/details/UN210-KIT.Google Scholar
- Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. WiGest: A ubiquitous WiFi-based gesture recognition system. In Proceedings of IEEE INFOCOM.Google ScholarCross Ref
- Fadel Adib, Zachary Kabelac, and Dina Katabi. 2015. Multi-person Localization via RF Body Reflections. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation (NSDI'15). USENIX Association, Berkeley, CA, USA, 279--292. http://portal.acm.org/citation.cfm?id=2789790 Google ScholarDigital Library
- Fadel Adib and Dina Katabi. 2013. See Through Walls with WiFi!. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM '13). ACM, New York, NY, USA, 75--86. Google ScholarDigital Library
- Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C Miller. 2015. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, 837--846. Google ScholarDigital Library
- J-F Cardoso. 1998. Blind signal separation: statistical principles. Proc. IEEE 86, 10 (1998), 2009--2025.Google ScholarCross Ref
- Lisa Y Chen, Benjamin C-K Tee, Alex L Chortos, Gregor Schwartz, Victor Tse, Darren J Lipomi, H-S Philip Wong, Michael V McConnell, and Zhenan Bao. 2014. Continuous wireless pressure monitoring and mapping with ultra-small passive sensors for health monitoring and critical care. Nature communications 5 (2014), 5028.Google Scholar
- Christopher Dondzila and Dena Garner. 2016. Comparative accuracy of fitness tracking modalities in quantifying energy expenditure. Journal of medical engineering & technology 40, 6 (2016), 325--329.Google ScholarCross Ref
- Biyi Fang, Nicholas D Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. 2016. BodyScan: Enabling radio-based sensing on wearable devices for contactless activity and vital sign monitoring. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 97--110. Google ScholarDigital Library
- Biyi Fang, Nicholas D Lane, Mi Zhang, and Fahim Kawsar. 2016. Headscan: A wearable system for radio-based sensing of head and mouth-related activities. In Proceedings of the 15th International Conference on Information Processing in Sensor Networks. IEEE Press, 21. Google ScholarDigital Library
- Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool Release: Gathering 802.11n Traces with Channel State Information. ACM SIGCOMM CCR 41, 1 (Jan. 2011), 53. Google ScholarDigital Library
- Lieve Hamers, Yves Hemeryck, Guido Herweyers, Marc Janssen, Hans Keters, Ronald Rousseau, and André Vanhoutte. 1989. Similarity measures in scientometric research: the Jaccard index versus Salton's cosine formula. Info. Processing & Management 25, 3 (1989), 315--318. Google ScholarDigital Library
- Chunmei Han, Kaishun Wu, Yuxi Wang, and Lionel M Ni. 2014. WiFall: Device-free fall detection by wireless networks. In Proceedings of IEEE INFOCOM. 271--279.Google ScholarCross Ref
- Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: algorithms and applications. Neural networks 13, 4 (2000), 411--430. Google ScholarDigital Library
- Kanitthika Kaewkannate and Soochan Kim. 2016. A comparison of wearable fitness devices. BMC public health 16, 1 (2016), 433.Google Scholar
- Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing Gesture Recognition to All Devices. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI'14). USENIX Association, Berkeley, CA, USA, 303--316. http://dl.acm.org/citation.cfm?id=2616448.2616477 Google ScholarDigital Library
- Taesu Kim, Hagai Thomas Attias, Soo-Young Lee, and Te-Won Lee. 2007. Blind source separation exploiting higher-order frequency dependencies. IEEE Trans. on Audio, Speech, and Language Processing 15, 1 (2007), 70--79. Google ScholarDigital Library
- Dana E King, Eric Matheson, Svetlana Chirina, Anoop Shankar, and Jordan Broman-Fulks. 2013. The status of baby boomers' health in the United States: the healthiest generation? JAMA internal medicine 173, 5 (2013), 385--386.Google Scholar
- Changzhi Li, Victor M Lubecke, Olga Boric-Lubecke, and Jenshan Lin. 2013. A review on recent advances in Doppler radar sensors for noncontact healthcare monitoring. IEEE Transactions on microwave theory and techniques 61, 5 (2013), 2046--2060.Google ScholarCross Ref
- Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015. Contactless sleep apnea detection on smartphones. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 45--57. Google ScholarDigital Library
- Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Wholehome Gesture Recognition Using Wireless Signals. In Proceedings of the 19th Annual International Conference on Mobile Computing & Networking (MobiCom '13). ACM, New York, NY, USA, 27--38. Google ScholarDigital Library
- Souvik Sen, Božidar Radunovic, Romit Roy Choudhury, and Tom Minka. 2012. You are facing the mona lisa: spot localization using phy layer information. In Proceedings of ACM MobiSys. 183--196. Google ScholarDigital Library
- Li Sun, Souvik Sen, Dimitrios Koutsonikolas, and Kyu-Han Kim. 2015. Widraw: Enabling hands-free drawing in the air on commodity win devices. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 77--89. Google ScholarDigital Library
- Sheng Tan and Jie Yang. 2016. WiFinger: leveraging commodity WiFi for finegrained finger gesture recognition. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 201--210. Google ScholarDigital Library
- David Tse and Pramod Viswanath. 2005. Fundamentals of wireless communication. Cambridge university press. Google ScholarDigital Library
- Aditya Virmani and Muhammad Shahzad. 2017. Position and Orientation Agnostic Gesture Recognition Using WiFi. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 252--264. Google ScholarDigital Library
- Guanhua Wang, Yongpan Zou, Zimu Zhou, Kaishun Wu, and Lionel M. Ni. 2014. We Can Hear You with Wi-Fi!. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (MobiCom 14). ACM, New York, NY, USA, 593--604. Google ScholarDigital Library
- Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and Modeling of WiFi Signal Based Human Activity Recognition. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 65--76. Google ScholarDigital Library
- Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2017. Device-Free Human Activity Recognition Using Commercial WiFi Devices. IEEE Journal on Selected Areas in Communications 35, 5 (2017), 1118--1131.Google ScholarDigital Library
- Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: In-home Device-free Activity Identification Using Fine-grained WiFi Signatures. In Proceedings of ACM MobiCom. Google ScholarDigital Library
- Susan Krauss Whitbourne and Sherry L Willis. 2014. The baby boomers grow up: Contemporary perspectives on midlife. Psychology Press.Google Scholar
- Joey Wilson and Neal Patwari. 2011. See-through walls: Motion tracking using variance-based radio tomography networks. IEEE Transactions on Mobile Computing 10, 5 (2011), 612--621. Google ScholarDigital Library
- Wei Xi, Jizhong Zhao, Xiang-Yang Li, Kun Zhao, Shaojie Tang, Xue Liu, and Zhiping Jiang. 2014. Electronic Frog Eye: Counting Crowd Using WiFi. In Proceedings of IEEE INFOCOM.Google ScholarCross Ref
Index Terms
- Multi-User Gesture Recognition Using WiFi
Recommendations
Position and Orientation Agnostic Gesture Recognition Using WiFi
MobiSys '17: Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and ServicesWiFi based gesture recognition systems have recently proliferated due to the ubiquitous availability of WiFi in almost every modern building. The key limitation of existing WiFi based gesture recognition systems is that they require the user to be in ...
Augmenting User Identification with WiFi Based Gesture Recognition
Over the last few years, researchers have proposed several WiFi based gesture recognition systems that can recognize predefined gestures performed by users at runtime. As most environments are inhabited by multiple users, the true potential of WiFi ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Comments