Chulhong Min
Akhil Mathur
ABSTRACT
In this paper, we explore audio and kinetic sensing on earable devices with the commercial on-the-shelf form factor. For the study, we prototyped earbud devices with a 6-axis inertial measurement unit and a microphone. We systematically investigate the differential characteristics of the audio and inertial signals to assess their feasibility in human activity recognition. Our results demonstrate that earable devices have a superior signal-to-noise ratio under the influence of motion artefacts and are less susceptible to acoustic environment noise. We then present a set of activity primitives and corresponding signal processing pipelines to showcase the capabilities of earbud devices in converting accelerometer, gyroscope, and audio signals into the targeted human activities with a mean accuracy reaching up to 88% in varying environmental conditions.
- Oliver Amft, Mathias Stäger, Paul Lukowicz, and Gerhard Tröster. 2005. Analysis of chewing sounds for dietary monitoring. In International Conference on Ubiquitous Computing. Springer, 56--72. Google Scholar
Digital Library
- Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 37. Google ScholarDigital Library
- Yujie Dong, Jenna Scisco, Mike Wilson, Eric Muth, and Adam Hoover. 2014. Detecting periods of eating during free-living by tracking wrist motion. IEEE journal of biomedical and health informatics 18, 4 (2014), 1253--1260.Google Scholar
- Davide Figo, Pedro C Diniz, Diogo R Ferreira, and Joao MP Cardoso. 2010. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing 14, 7 (2010), 645--662. Google ScholarDigital Library
- Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 (2016). Google ScholarDigital Library
- Inseok Hwang, Chungkuk Yoo, Chanyou Hwang, Dongsun Yim, Youngki Lee, Chulhong Min, John Kim, and Junehwa Song. 2014. TalkBetter: family-driven mobile intervention care for children with language delay. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 1283--1296. Google ScholarDigital Library
- Eric C Larson, TienJui Lee, Sean Liu, Margaret Rosenfeld, and Shwetak N Patel. 2011. Accurate and privacy preserving cough sensing using a low-cost microphone. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 375--384. Google ScholarDigital Library
- Steven F LeBoeuf, Michael E Aumer, William E Kraus, Johanna L Johnson, and Brian Duscha. 2014. Earbud-based sensor for the assessment of energy expenditure, heart rate, and vo2max. Medicine and science in sports and exercise 46, 5 (2014), 1046.Google Scholar
- Youngki Lee, Chulhong Min, Chanyou Hwang, Jaeung Lee, Inseok Hwang, Younghyun Ju, Chungkuk Yoo, Miri Moon, Uichin Lee, and Junehwa Song. 2013. Sociophone: Everyday face-to-face interaction monitoring platform using multi-phone sensor fusion. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 375--388. Google ScholarDigital Library
- Chengwen Luo and Mun Choon Chan. 2013. Socialweaver: Collaborative inference of human conversation networks using smartphones. In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems. ACM, 20. Google ScholarDigital Library
- Afra Mashhadi, Akhil Mathur, Marc Van den Broeck, Geert Vanderhulst, and Fahim Kawsar. 2016. Understanding the impact of personal feedback on face-to-face interactions in the workplace. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. ACM, 362--369. Google ScholarDigital Library
- Akhil Mathur, Marc Van den Broeck, Geert Vanderhulst, Afra Mashhadi, and Fahim Kawsar. 2015. Tiny habits in the giant enterprise: understanding the dynamics of a quantified workplace. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 577--588. Google ScholarDigital Library
- Akhil Mathur, Tianlin Zhang, Sourav Bhattacharya, Petar Veličković, Leonid Joffe, Nicholas D. Lane, Fahim Kawsar, and Pietro Lió. 2018. Using Deep Data Augmentation Training to Address Software and Hardware Heterogeneities in Wearable and Smartphone Sensing Devices. In Proceedings of the 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN '18). IEEE Press, Piscataway, NJ, USA, 200--211. Google ScholarDigital Library
- Anh Nguyen, Raghda Alqurashi, Zohreh Raghebi, Farnoush Banaei-Kashani, Ann C Halbower, and Tam Vu. 2016. A lightweight and inexpensive in-ear sensing system for automatic whole-night sleep stage monitoring. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM. ACM, 230--244. Google ScholarDigital Library
- Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. ACM, 127--140. Google ScholarDigital Library
- Wai-Tian Tan, Mary Baker, Bowon Lee, and Ramin Samadani. 2013. The sound of silence. In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems. ACM, 19.Google ScholarDigital Library
- Edison Thomaz, Irfan Essa, and Gregory D Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1029--1040. Google ScholarDigital Library
Index Terms
- Exploring audio and kinetic sensing on earable devices
Recommendations
Excerpt of ToothSonic: Earable Authentication via Acoustic Toothprint
UbiComp/ISWC '22 Adjunct: Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable ComputersEarables (ear wearable) are rapidly emerging as a new platform to enable a variety of personal applications. The traditional authentication methods thus become less applicable and inconvenient for earables due to their limited input interface. Earables, ...
Earable Authentication via Acoustic Toothprint
CCS '21: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications SecurityEarables (ear wearable) are rapidly emerging as a new platform to enable a variety of personal applications. The traditional authentication methods thus become less applicable and inconvenient for earables due to their limited input interface. Earables, ...
EarComp 2023: Fourth International Workshop on Earable Computing
UbiComp/ISWC '23 Adjunct: Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable ComputingThe objective of the 4th ACM International Workshop on Earable Computing (EarComp 2023) is to provide an academic forum and bring together researchers, practitioners, and design experts to discuss how sensory earables technologies have and can ...
Comments