Skip to main content
SpringerLink
Log in

Sensor-Based Behavior Recognition

  • Chapter
  • First Online:
Human Behavior Analysis: Sensing and Understanding

Abstract

In this monograph, sensor-based behavior recognition mainly refers to the use of emerging sensor network technologies for behavior monitoring and understanding. The generated sensor data from sensor-based monitoring are mainly time series of state changes and/or various parameter values that are usually processed through data fusion, probabilistic, or statistical analysis methods and formal knowledge technologies for behavior recognition. Specifically, sensors can be attached to an actor under observation, namely wearable sensors or smartphones, or objects that constitute the environment, namely dense sensing. Wearable sensors often use inertial measurement units and radio frequency identification (RFID) tags to gather a user’s behavioral information. This approach is effective to recognize physical movements such as physical exercises. In contrast, dense sensing infers behaviors by monitoring human–object interactions through the usage of multiple multimodal miniaturized sensors. In this chapter, we first give a brief introduction to the historical evolution of sensor-based behavior recognition. Afterwards, we present the mobile device-enabled behavior recognition approach, which is a typical type of sensor-based behavior recognition, followed by a discussion on the key issues of developing behavior recognition systems using mobile devices.

Part of this chapter is based on a previous work: L. Chen, J. Hoey, C. D. Nugent, D. J. Cook and Z. Yu, “Sensor-Based Activity Recognition,” in IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 790–808, Nov. 2012. DOI: https://doi.org/10.1109/TSMCC.2012.2198883

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. M. C. Mozer, “The neural network house: An environment that adapts to its inhabitants,” in Proc. AAAI Spring Symp. Intell. Environ., 1998, pp. 110–114.

    Google Scholar 

  2. U. Leonhardt and J. Magee, “Multi-sensor location tracking,” in Proc. 4th ACM/IEEE Int. Conf. Mobile Comput. Netw., 1998, pp. 203–214

    Google Scholar 

  3. A. R. Golding and N. Lesh, “Indoor navigation using a diverse set of cheap, wearable sensors,” in Proc. 3rd Int. Symp. Wearable Comput., Oct. 1999, pp. 29–36.

    Google Scholar 

  4. A. Ward and A. Hopper, “A new location technique for the active office,” IEEE Personal Commun., vol. 4, no. 5, pp. 42–47, Oct. 1997.

    Google Scholar 

  5. A. Schmidt, M. Beigl, and H. Gellersen, “There is more to context than location,” Comput. Graph., vol. 23, no. 6, pp. 893–901, 1999.

    Article  Google Scholar 

  6. C. Randell and H. L. Muller, “Context awareness by analyzing accelerometer data,” in Proc. 4th Int. Symp. Wearable Comput., 2000, pp. 175–176.

    Google Scholar 

  7. H. W. Gellersen, A. Schmidt, and M. Beigl, “Multi-sensor context awareness in mobile devices and smart artifacts,” Mobile Netw. Appl., vol. 7, no. 5, pp. 341–351, Oct. 2002.

    Google Scholar 

  8. A. Schmidt and K. Van Laerhoven, “How to build smart appliances,” IEEE Pers. Commun., vol. 8, no. 4, pp. 66–71, Aug. 2001.

    Google Scholar 

  9. K. Van Laerhoven and K. A. Aidoo, “Teaching context to applications,” J. Pers. Ubiquitous Comput., vol. 5, no. 1, pp. 46–49, 2001.

    Article  Google Scholar 

  10. K. Van Laerhoven, K. Aidoo, and S. Lowette, “Real-time analysis of data from many sensors with neural networks,” in Proc. 5th Int. Symp. Wearable Comput., 2001, pp. 115–123.

    Google Scholar 

  11. K. Van Laerhoven and O. Cakmakci, “What shall we teach our pants?” in Proc. 4th Int. Symp. Wearable Comput., 2000, pp. 77–84.

    Google Scholar 

  12. F. Foerster and J. Fahrenberg, “Motion pattern and posture: Correctly assessed by calibrated accelerometers,” Behav. Res. Methods Instrum. Comput., vol. 32, no. 3, pp. 450–457, 2000.

    Article  Google Scholar 

  13. K. Van Laerhoven and H. W. Gellersen, “Spine versus Porcupine: A study in distributed wearable activity recognition,” in Proc. 8th Int. Symp. Wearable Comput., 2004, pp. 142–150.

    Google Scholar 

  14. S. W. Lee and K. Mase, “Activity and location recognition using wearable sensors,” IEEE Pervasive Comput., vol. 1, no. 3, pp. 24–32, Jul.–Sep. 2002.

    Article  Google Scholar 

  15. L. Bao and S. Intille, “Activity recognition from user-annotated acceleration data,” in Proc. Pervasive, 2004, vol. 3001, pp. 1–17.

    Google Scholar 

  16. D. J. Patterson, L. Liao, D. Fox, and H. Kautz, “Inferring high-level behavior from low-level sensors,” in Proc. 5th Conf. Ubiquitous Comput., 2003, pp. 73–89.

    Google Scholar 

  17. M. Chan, D. Esteve, C. Escriba, and E. Campo, “A review of smart homes—present state and future challenges,” Comput. Methods Programs Biomed., vol. 91, no. 1, pp. 55–81, 2008.

    Article  Google Scholar 

  18. C. D. Nugent, “Experiences in the development of a smart lab,” Int. J. Biomed. Eng. Technol., vol. 2, no. 4, pp. 319–331, 2010.

    Article  Google Scholar 

  19. The ambient assisted living joint programme [Online]. Available: www.aal-europe.eu

  20. The house of the future [Online]. Available: http://architecture.mit.edu/house_n

  21. P. Rashidi and D. Cook, “Activity knowledge transfer in smart environments,” J. Pervasive Mobile Comput., vol. 7, no. 3, pp. 331–343, 2011.

    Article  Google Scholar 

  22. The gator-tech smart house [Online]. Available: http://www.icta.ufl.edu/gt.htm

  23. The inHaus project in Germany [Online]. Available: http://www.inhaus.fraunhofer.de

  24. The aware home [Online]. Available: http://awarehome.imtc.gatech.edu

  25. The DOMUS laboratory [Online]. Available: http://domus.usherbrooke.ca

  26. The iDorm project [Online]. Available: http://cswww.essex.ac.uk/iieg/idorm.htm

  27. N. Kern, B. Schiele, H. Junker, P. Lukowicz, and G. Troster, “Wearable sensing to annotate meeting recordings,” Pers. Ubiquitous Comput., vol. 7, no. 5, pp. 263–274, Oct. 2003.

    Google Scholar 

  28. P. Lukowicz, J. A. Ward, H. Junker, and T. Starner, “Recognizing workshop activity using body worn microphones and accelerometers,” in Proc. Pervasive Comput., Apr. 2004, pp. 18–23.

    Google Scholar 

  29. J. Mantyjarvi, J. Himberg, and T. Seppanen, “Recognizing human motion with multiple acceleration sensors,” in Proc. IEEE Int. Conf. Syst., Man, Cybern., 2001, vol. 2, pp. 747–752.

    Google Scholar 

  30. D. Ashbrook and T. Starner, “Using GPS to learn significant locations and predict movement across multiple users,” Pers. Ubiquitous Comput., vol. 7, no. 5, pp. 275–286, Oct. 2003.

    Google Scholar 

  31. M. Philipose, K. P. Fishkin, M. Perkowitz, D. J. Patterson, D. Fox, H. Kautz, and D. Hahnel, “Inferring activities from interactions with objects,” IEEE Pervasive Comput., vol. 3, no. 4, pp. 50–57, Oct./Dec. 2004.

    Google Scholar 

  32. K. P. Fishkin, M. Philipose, and A. Rea, “Hands-on RFID: Wireless wearables for detecting use of objects,” in Proc. 9th IEEE Int. Symp. Wearable Comput., 2005, pp. 38–43.

    Google Scholar 

  33. D. J. Patterson, D. Fox, H. Kautz, and M. Philipose, “Fine-grained activity recognition by aggregating abstract object usage,” in Proc. 9th IEEE Int. Symp. Wearable Comput., 2005, pp. 44–51.

    Google Scholar 

  34. M. R. Hodges and M. E. Pollack, “An object-use fingerprint: The use of electronic sensors for human identification,” in Proc. 9th Int. Conf. Ubiquitous Comput., 2007, pp. 289–303.

    Google Scholar 

  35. M. Buettner, R. Prasad, M. Philipose, and D. Wetherall, “Recognizing daily activities with RFID-based sensors,” in Proc. 11th Int. Conf. Ubiquitous Comput., 2009, pp. 51–60.

    Google Scholar 

  36. D. Wilson and C. Atkeson, “Simultaneous tracking and activity recognition (STAR) using many anonymous, binary sensor,” in Proc. 3rd Int. Conf. Pervasive Comput., 2005, pp. 62–79.

    Google Scholar 

  37. C. R. Wren and E. M. Tapia, “Toward scalable activity recognition for sensor networks,” in Proc. 2nd Int. Workshop Location Context Awareness, 2006, pp. 168–185.

    Google Scholar 

  38. R. Aipperspach, E. Cohen, and J. Canny, “Modeling human behavior from simple sensors in the home,” in Proc. Pervasive, 2006, vol. 3968, pp. 337–348.

    Google Scholar 

  39. T. Gu, Z. Wu, X. Tao, H. K. Pung, and J. Lu, “epSICAR: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition,” in Proc. 7th IEEE Int. Conf. Pervasive Comput. Commun., 2009, pp. 1–9.

    Google Scholar 

  40. L. Liao, D. J. Patterson, D. Fox, and H. Kautz, “Learning and inferring transportation routines,” Artif. Intell., vol. 171, no. 5–6, pp. 311–331, 2007.

    Article  MathSciNet  Google Scholar 

  41. A. Montanari, C. Mascolo, K. Sailer, and S. Nawaz, “Detecting Emerging Activity-Based Working Traits through Wearable Technology,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 3, pp. 86, 2017.

    Article  Google Scholar 

  42. Y. Zhang, M. Haghdan, and K. S. Xu, “Unsupervised motion artifact detection in wrist-measured electrodermal activity data,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2017.

    Google Scholar 

  43. J. M. Echterhoff, J. Haladjian, and B. Brügge, “Gait and jump classification in modern equestrian sports,” In Proceedings of the 2018 ACM International Symposium on Wearable Computers, pp. 88–91, 2018.

    Google Scholar 

  44. S. A. Elkader, M. Barlow, and E. Lakshika. “Wearable sensors for recognizing individuals undertaking daily activities.” Proceedings of the 2018 ACM International Symposium on Wearable Computers. ACM, 2018.

    Google Scholar 

  45. V. Becker, P. Oldrati, L. Barrios, and G. Sörös, “Touchsense: classifying finger touches and measuring their force with an electromyography armband,” In Proceedings of the 2018 ACM International Symposium on Wearable Computers, pp. 1–8, 2018.

    Google Scholar 

  46. P. Siirtola and J. Röning, “Recognizing human activities user independently on smartphones based on accelerometer data,” Int. J. Interact. Multimedia Artif. Intell., vol. 1, no. 5, pp. 38–45, 2012.

    Google Scholar 

  47. A. M. Khan, A. Tufail, A. M. Khattak, and T. H. Laine, “Activity recognition on smartphones via sensor-fusion and KDA-based SVMS,” Int. J. Distrib. Sensor Netw., vol. 10, no. 5, pp. 1–14, 2014.

    Article  Google Scholar 

  48. M. Shoaib, S. Bosch, Ö. D. Incel, H. Scholten, and P. J. M. Havinga, “Fusion of smartphone motion sensors for physical activity recognition,” Sensors, vol. 14, no. 6, pp. 10146–10176, 2014.

    Article  Google Scholar 

  49. H. Lu, W. Pan, N. D. Lane, T. Choudhury, and A. T. Campbell, “Soundsense: Scalable sound sensing for people-centric applications on mobile phones,” in Proc. MobiSys, 2009, pp. 165–178.

    Google Scholar 

  50. H. Du, Z. Yu, F. Yi, Z. Wang et al., “Recognition of group mobility level and group structure with mobile devices,” IEEE Transactions on Mobile Computing, vol. 17, no. 4, pp. 884–897, 2018.

    Article  Google Scholar 

  51. T. Hao, G. Xing, and G. Zhou, “RunBuddy: A smartphone system for running rhythm monitoring,” in Proc. UBICOMP 2015, pp. 133–144.

    Google Scholar 

  52. X. Sun, Z. Lu, W. Hu, and G. Cao, “Symdetector: Detecting sound-related respiratory symptoms using smartphones,” in Proc. UbiComp 2015, pp. 97–108.

    Google Scholar 

  53. A. Mottelson and K. Hornbæk, “An affect detection technique using mobile commodity sensors in the wild,” In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 781–792, 2016.

    Google Scholar 

  54. B. Cao, L. Zheng, C. Zhang, P. S. Yu, A. Piscitello, et al. “Deepmood: modeling mobile phone typing dynamics for mood detection,” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,” pp. 747–755, 2017.

    Google Scholar 

  55. X. Zhang, W. Li, X. Chen, and S. Lu. MoodExplorer: Towards Compound Emotion Detection via Smartphone Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 4, pp. 176, 2018.

    Google Scholar 

  56. Lane N D, Miluzzo E, Lu H, et al. A survey of mobile phone sensing[J]. IEEE Communications Magazine, 2010, 48(9):140–150.

    Article  Google Scholar 

  57. B. Guo, Z. Wang, Z. Yu, Y. Wang, N.Y. Yen, R. Huang, and X. Zhou. Mobile Crowd Sensing and Computing: The Review of an Emerging Human-Powered Sensing Paradigm. ACM Comput. Surv. 48, 1, Article 7 (August 2015), 31 pages.

    Article  Google Scholar 

  58. K. Ylli, D. Hoffmann, A. Willmann, P. Becker, B. Folkmer, and Y. Manoli. Energy harvesting from human motion: exploiting swing and shock excitations. Smart Materials and Structures, vol. 24, no. 2, pp. 025–029, 2015.

    Article  Google Scholar 

  59. K. Li, C. Yuen, B. Kusy, R. Jurdak, A. Ignatovic, S. Kanhere, and S. K. Jha. Fair scheduling for data collection in mobile sensor networks with energy harvesting. IEEE Transactions on Mobile Computing, 2018.

    Google Scholar 

  60. D. Liaqat, S. Jingoi, E. de Lara, A. Goel, W. To, et al. Sidewinder: An energy efficient and developer friendly heterogeneous architecture for continuous mobile sensing. ACM SIGARCH Computer Architecture News, vol. 44, no. 2, pp. 205–215, 2016.

    Article  Google Scholar 

  61. Wang Y, Lin J, Annavaram M, Quinn JA, Jason H, Bhaskar K, Sadeh N (2009) A framework of energy efficient mobile sensing for automatic user state recognition. In: Proceedings of the 7th ACM international conference on mobile systems, applications, and services, New York, pp 179–192.

    Google Scholar 

  62. Zappi P, Lombriser C, Stiefmeier T, Farella E, Roggen D, Benini L, Troster G (2008) Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. Wirel Sens Netw Lect Notes Comput Sci 943: 17–33.

    Article  Google Scholar 

  63. Li X, Cao H, Chen E, Tian J (2012) Learning to infer the status of heavy-duty sensors for energy efficient context-sensing. ACM Trans Intell Syst Technol 3(2):1–23.

    Google Scholar 

  64. Bouten C, Koekkoek K, Verduin M, Kodde R, Janssen JD (1997) A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity. IEEE Trans Biomed Eng 44:136–147.

    Article  Google Scholar 

  65. Khan AM, Lee Y, Lee SY, Kim T (2010) A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf Technol Biomed 14:1166–1172.

    Article  Google Scholar 

  66. Kwapisz JR, Weiss GM, Moore SA (2010) Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter 12.

    Google Scholar 

  67. K. Kahatapitiya, C. Weerasinghe, J. Jayawardhana, H. Kuruppu, K. Thilakarathna, and D. Días, Low-power step counting paired with electromagnetic energy harvesting for wearables. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. pp. 218–219, 2018.

    Google Scholar 

  68. Maurer U, Smailagic A, Siewiorek DP, Deisher M (2006) Activity recognition and monitoring using multiple sensors on different body positions. In: Proceedings of international workshop on wearable and implantable body sensor networks, Cambridge, pp 113–116.

    Google Scholar 

  69. Liang Y, Zhou X, Yu Z, et al. Energy-Efficient Motion Related Activity Recognition on Mobile Devices for Pervasive Healthcare. Mobile Networks & Applications, 2014, 19(3):303–317.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Northwestern Polytechnical University, Xi’an, China

    Zhiwen Yu & Zhu Wang

Authors
  1. Zhiwen Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yu, Z., Wang, Z. (2020). Sensor-Based Behavior Recognition. In: Human Behavior Analysis: Sensing and Understanding. Springer, Singapore. https://doi.org/10.1007/978-981-15-2109-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-2109-6_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-2108-9

  • Online ISBN: 978-981-15-2109-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation