Tianxing Li
Xia Zhou
ABSTRACT
We present LiSense, the first-of-its-kind system that enables both data communication and fine-grained, real-time human skeleton reconstruction using Visible Light Communication (VLC). LiSense uses shadows created by the human body from blocked light and reconstructs 3D human skeleton postures in real time. We overcome two key challenges to realize shadow-based human sensing. First, multiple lights on the ceiling lead to diminished and complex shadow patterns on the floor. We design light beacons enabled by VLC to separate light rays from different light sources and recover the shadow pattern cast by each individual light. Second, we design an efficient inference algorithm to reconstruct user postures using 2D shadow information with a limited resolution collected by photodiodes embedded in the floor. We build a 3 m x 3 m LiSense testbed using off-the-shelf LEDs and photodiodes. Experiments show that LiSense reconstructs the 3D user skeleton at 60 Hz in real time with 10 degrees mean angular error for five body joints.
- Google Project Jacquard. https://www.google.com/atap/project-jacquard/.Google Scholar
- Adib, F., Kabelac, Z., and Katabi, D. Multi-Person Motion Tracking via RF Body Reflections. In Proc. of NSDI (2015). Google ScholarDigital Library
- Adib, F., Kabelac, Z., Katabi, D., and Miller, R. C. 3D Tracking via Body Radio Reflections. In Proc. of NSDI (2014). Google ScholarDigital Library
- Adib, F., and Katabi, D. See through walls with WiFi! In Proc. of SIGCOMM (2013). Google ScholarDigital Library
- Afgani, M. Z., Haas, H., Elgala, H., and Knipp, D. Visible light communication using OFDM. In Proc. of TRIDENTCOM (2006).Google ScholarCross Ref
- Arai, S., et al. Experimental on hierarchical transmission scheme for visible light communication using LED traffic light and high-speed camera. In Proc. of VTC (2007).Google ScholarCross Ref
- Ashok, A., et al. Challenge: Mobile optical networks through visual MIMO. In Proc. of MobiCom (2010). Google ScholarDigital Library
- Association, I. S., et al. IEEE Std. for Local and metropolitan area networks-Part 15.7: Short-Rang Wireless Optical Communication Using Visible Light. IEEE Computer Society (2011).Google Scholar
- Beaudoin, P., Poulin, P., and van de Panne, M. Adapting wavelet compression to human motion capture clips. In Proc. of GI (2007). Google ScholarDigital Library
- Chen, V. C., Li, F., Ho, S.-S., and Wechsler, H. Analysis of micro-doppler signatures. IEE Proceedings-Radar, Sonar and Navigation 150, 4 (2003), 271--276.Google ScholarCross Ref
- Cheung, G. K., Kanade, T., Bouguet, J.-Y., and Holler, M. A real time system for robust 3D voxel reconstruction of human motions. In Proc. of CVPR (2000).Google ScholarCross Ref
- Dietz, P., Yerazunis, W., and Leigh, D. Very low-cost sensing and communication using bidirectional LEDs. In Proc. of UbiComp (2003).Google ScholarCross Ref
- Duhamel, P., and Hollmann, H. Split radix FFT algorithm. Electronics letters 20, 1 (1984), 14--16.Google Scholar
- Elgala, H., Mesleh, R., Haas, H., and Pricope, B. OFDM visible light wireless communication based on white LEDs. In Proc. of VTC (2007).Google ScholarCross Ref
- Fernandez-Baena, A., Susın, A., and Lligadas, X. Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In Proc. of INCoS (2012). Google ScholarDigital Library
- Ghaddar, M., Talbi, L., and Denidni, T. Human body modelling for prediction of effect of people on indoor propagation channel. Electronics Letters 40, 25 (2004), 1592--1594.Google ScholarCross Ref
- Gupta, S., Morris, D., Patel, S., and Tan, D. SoundWave: Using the Doppler effect to sense gestures. In Proc. of CHI (2012). Google ScholarDigital Library
- Hao, T., Zhou, R., and Xing, G. COBRA: Color barcode streaming for smartphone systems. In Proc. of MobiSys (2012). Google ScholarDigital Library
- Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proc. of ISER (2010).Google Scholar
- Herda, L., Fua, P., Plankers, R., Boulic, R., and Thalmann, D. Skeleton-based motion capture for robust reconstruction of human motion. In Proc. of International Conference on Computer Animation (2000). Google ScholarDigital Library
- Howe, N. R., Leventon, M. E., and Freeman, W. T. Bayesian Reconstruction of 3D Human Motion from Single-Camera Video. In Proc. of NIPS (1999).Google Scholar
- Hu, W., Gu, H., and Pu, Q. LightSync: Unsynchronized visual communication over screen-camera links. In Proc. of MobiCom (2013). Google ScholarDigital Library
- Hu, W., Mao, J., Huang, Z., Xue, Y., She, J., Bian, K., and Shen, G. Strata: Layered Coding for Scalable Visual Communication. In Proc. of MobiCom (2014). Google ScholarDigital Library
- Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. of UIST (2011). Google ScholarDigital Library
- Jerri, A. J. The Shannon sampling theorem--Its various extensions and applications: A tutorial review. Proc. of the IEEE (1977).Google ScholarCross Ref
- Jhuang, H., Gall, J., Zuffi, S., Schmid, C., and Black, M. J. Towards understanding action recognition. In Proc. of ICCV (2013). Google ScholarDigital Library
- Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P. Digits: Freehand 3D Interactions Anywhere Using a Wrist-worn Gloveless Sensor. In Proc. of UIST (2012). Google ScholarDigital Library
- Kim, H.-S., et al. An indoor visible light communication positioning system using a RF carrier allocation technique. Journal of Lightwave Technology 31, 1 (2013), 134--144.Google ScholarCross Ref
- Komine, T., and Nakagawa, M. Integrated system of white LED visible-light communication and power-line communication. IEEE Transactions on Consumer Electronics 49, 1 (2003), 71--79. Google ScholarDigital Library
- Komine, T., and Nakagawa, M. Fundamental analysis for visible-light communication system using LED lights. IEEE Transactions on Consumer Electronics 50, 1 (2004), 100--107. Google ScholarDigital Library
- Krosshaug, T., and Bahr, R. A model-based image-matching technique for three-dimensional reconstruction of human motion from uncalibrated video sequences. Journal of biomechanics 38, 4 (2005), 919--929.Google Scholar
- Kuo, Y.-S., Pannuto, P., Hsiao, K.-J., and Dutta, P. Luxapose: Indoor positioning with mobile phones and visible light. In Proc. of MobiCom (2014). Google ScholarDigital Library
- Le-Minh, H., et al. 100-Mb/s NRZ visible light communications using a postequalized white LED. Photonics Technology Letters, IEEE 21, 15 (2009), 1063--1065.Google ScholarCross Ref
- Le Minh, H., O'Brien, D., Faulkner, G., Zeng, L., Lee, K., Jung, D., Oh, Y., and Won, E. T. 100-Mb/s NRZ visible light communications using a postequalized white LED. Photonics Technology Letters, IEEE 21, 15 (2009), 1063--1065.Google Scholar
- Lee, K., and Park, H. Modulations for visible light communications with dimming control. Photonics Technology Letters, IEEE 23, 16 (2011), 1136--1138.Google ScholarCross Ref
- Li, L., Hu, P., Peng, C., Shen, G., and Zhao, F. Epsilon: A visible light based positioning system. In Proc. of NSDI (2014). Google ScholarDigital Library
- Li, T., An, C., Xiao, X., Campbell, A. T., and Zhou, X. Real-Time Screen-Camera Communication Behind Any Scene. In Proc. of MobiSys (2015). Google ScholarDigital Library
- Little, T. D. C., et al. Using LED lighting for ubiquitous indoor wireless networking. In Proc. of WiMob (2008). Google ScholarDigital Library
- Liu, C. B., Sadeghi, B., and Knightly, E. W. Enabling vehicular visible light communication (V2LC) networks. In Proc. of VANET (2011). Google ScholarDigital Library
- McKinney, J. C., and Hopkins, C. D. R. ATSC digital television standard. Advanced Television System Committee (1995).Google Scholar
- Moeslund, T. B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Computer vision and image understanding 104, 2 (2006), 90--126. Google ScholarDigital Library
- Mohr, R., Quan, L., and Veillon, F. Relative 3D reconstruction using multiple uncalibrated images. The International Journal of Robotics Research 14, 6 (1995), 619--632. Google ScholarDigital Library
- Moschini, D., and Fusiello, A. Tracking human motion with multiple cameras using an articulated model. In Computer Vision/Computer Graphics Collaboration Techniques. Springer, 2009, pp. 1--12. Google ScholarDigital Library
- Perli, S. D., Ahmed, N., and Katabi, D. PixNet: Interference-free wireless links using LCD-camera pairs. In Proc. of MobiCom (2010). Google ScholarDigital Library
- Post, E. R., Orth, M., Russo, P. R., and Gershenfeld, N. E-broidery: Design and fabrication of textile-based computing. IBM Systems Journal 39, 3.4 (2000), 840--860. Google ScholarDigital Library
- Pu, Q., Gupta, S., Gollakota, S., and Patel, S. Whole-home gesture recognition using wireless signals. In Proc. of MobiCom (2013). Google ScholarDigital Library
- Quintana, C., et al. Reading lamp-based visible light communication system for in-flight entertainment. Consumer Electronics, IEEE Transactions on 59, 1 (2013), 31--37.Google Scholar
- Rajagopal, N., Lazik, P., and Rowe, A. Visual light landmarks for mobile devices. In Proc. of IPSN (2014). Google ScholarDigital Library
- Rajagopal, S., Roberts, R., and Lim, S.-K. IEEE 802.15.7 visible light communication: modulation schemes and dimming support. Communications Magazine, IEEE 50, 3 (2012), 72--82.Google Scholar
- Ryckaert, J., De Doncker, P., Meys, R., de Le Hoye, A., and Donnay, S. Channel model for wireless communication around human body. Electronics Letters 40, 9 (2004), 543--544.Google ScholarCross Ref
- Saxena, A., Chung, S. H., and Ng, A. Y. 3-D depth reconstruction from a single still image. International journal of computer vision 76, 1 (2008), 53--69. Google ScholarDigital Library
- Sb\^ırlea, D., Burke, M. G., Guarnieri, S., Pistoia, M., and Sarkar, V. Automatic detection of inter-application permission leaks in android applications. IBM Journal of Research and Development 57, 6 (2013), 10--1. Google ScholarDigital Library
- Schill, F., Zimmer, U. R., and Trumpf, J. Visible spectrum optical communication and distance sensing for underwater applications. In In Proc. of Australasian Conference on Robotics and Automation (2004).Google Scholar
- Schmid, S., Corbellini, G., Mangold, S., and Gross, T. R. LED-to-LED visible light communication networks. In Proc. of MobiHoc (2013). Google ScholarDigital Library
- Segen, J., and Kumar, S. Shadow gestures: 3D hand pose estimation using a single camera. In Proc. of CVPR (1999).Google ScholarCross Ref
- Shoemaker, G., Tang, A., and Booth, K. S. Shadow reaching: a new perspective on interaction for large displays. In Proc. of UIST (2007). Google ScholarDigital Library
- Shotton, J., et al. Real-time human pose recognition in parts from single depth images. In Proc. of CVPR (2011). Google ScholarDigital Library
- Silaghi, M.-C., Plankers, R., Boulic, R., Fua, P., and Thalmann, D. Local and global skeleton fitting techniques for optical motion capture. In Modelling and Motion Capture Techniques for Virtual Environments. Springer, 1998, pp. 26--40. Google ScholarCross Ref
- Tsonev, D., et al. A 3-Gb/s Single-LED OFDM-based Wireless VLC Link Using a Gallium Nitride μLED. Photonics Technology Letters, IEEE PP, 99 (2014), 1--1.Google Scholar
- Wall, M. 'Li-fi' via LED light bulb data speed breakthrough. BBC News, 2013.Google Scholar
- Wang, A., Ma, S., Hu, C., Huai, J., Peng, C., and Shen, G. Enhancing Reliability to Boost the Throughput over Screen-camera Links. In Proc. of MobiCom (2014). Google ScholarDigital Library
- Wang, J., Vasisht, D., and Katabi, D. RF-IDraw: Virtual Touch Screen in the Air Using RF Signals. In Proc. of SIGCOMM (2014). Google ScholarDigital Library
- Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., and Liu, H. E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures. In Proc. of MobiCom (2014). Google ScholarDigital Library
- Weinberg, Z., Chen, E. Y., Jayaraman, P. R., and Jackson, C. I still know what you visited last summer: Leaking browsing history via user interaction and side channel attacks. In Proc. of IEEE Symposium on Security and Privacy (2011). Google ScholarDigital Library
- Welch, T., Musselman, R., Emessiene, B., Gift, P., Choudhury, D., Cassadine, D., and Yano, S. The effects of the human body on UWB signal propagation in an indoor environment. IEEE Journal on Selected Areas in Communications 20, 9 (2002), 1778--1782. Google ScholarDigital Library
- Whitaker, R. T. A level-set approach to 3D reconstruction from range data. International journal of computer vision 29, 3 (1998), 203--231. Google ScholarDigital Library
- Yeung, K.-Y., Kwok, T.-H., and Wang, C. C. Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications. Journal of Computing and Information Science in Engineering 13, 4 (2013), 041007.Google ScholarCross Ref
- Zeng, L., et al. High data rate multiple input multiple output (MIMO) optical wireless communications using white LED lighting. IEEE Journal on Selected Areas in Communications 27, 9 (2009), 1654--1662. Google ScholarDigital Library
- Zhang, W., and Kavehrad, M. Comparison of VLC-based indoor positioning techniques. In Proc. of SPIE (2013).Google ScholarCross Ref
- Zhang, Z., et al. I Am the Antenna: Accurate Outdoor AP Location Using Smartphones. In Proc. of MobiCom (2011). Google ScholarDigital Library
- Zhou, X., and Campbell, A. Visible Light Networking and Sensing. In Proc. of HotWireless (2014). Google ScholarDigital Library
Index Terms
- Human Sensing Using Visible Light Communication
Recommendations
Practical Human Sensing in the Light
MobiSys '16: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and ServicesWe present StarLight, an infrastructure-based sensing system that reuses light emitted from ceiling LED panels to reconstruct fine-grained user skeleton postures continuously in real time. It relies on only a few (e.g., 20) photodiodes placed at ...
LED-to-LED visible light communication networks
MobiHoc '13: Proceedings of the fourteenth ACM international symposium on Mobile ad hoc networking and computingVisible Light Communication (VLC) with Light Emitting Diodes (LEDs) as transmitters and receivers enables low bitrate wireless adhoc networking. LED-to-LED VLC adhoc networks with VLC devices communicating with each other over free-space optical links ...
Using consumer LED light bulbs for low-cost visible light communication systems
VLCS '14: Proceedings of the 1st ACM MobiCom workshop on Visible light communication systemsLED-to-LED Visible Light Communication (VLC) based on Light Emitting Diodes (LEDs) and microcontrollers provide a foundation for networking using visible light as communication medium. We describe a low-complexity smart LED light bulb prototype that is ...
Comments