Allan Stisen
Henrik Blunck
Mikkel Baun Kjærgaard
Mads Møller Jensen
ABSTRACT
The widespread presence of motion sensors on users' personal mobile devices has spawned a growing research interest in human activity recognition (HAR). However, when deployed at a large-scale, e.g., on multiple devices, the performance of a HAR system is often significantly lower than in reported research results. This is due to variations in training and test device hardware and their operating system characteristics among others. In this paper, we systematically investigate sensor-, device- and workload-specific heterogeneities using 36 smartphones and smartwatches, consisting of 13 different device models from four manufacturers. Furthermore, we conduct experiments with nine users and investigate popular feature representation and classification techniques in HAR research. Our results indicate that on-device sensor and sensor handling heterogeneities impair HAR performances significantly. Moreover, the impairments vary significantly across devices and depends on the type of recognition technique used. We systematically evaluate the effect of mobile sensing heterogeneities on HAR and propose a novel clustering-based mitigation technique suitable for large-scale deployment of HAR, where heterogeneity of devices and their usage scenarios are intrinsic.
- M. V. Albert, S. Toledo, M. Shapiro, and K. Kording. Using mobile phones for activity recognition in parkinson's patients. Frontiers in neurology, 3, 2012.Google Scholar
- O. Amft. On the need for quality standards in activity recognition using ubiquitous sensors. In How To Do Good Research In Activity Recognition. Workshop in conjunction with Pervasive, 2010.Google Scholar
- S. A. Antos, M. V. Albert, and K. P. Kording. Hand, belt, pocket or bag: Practical activity tracking with mobile phones. Journal of Neuroscience Methods, 231:22--30, 2014.Google ScholarCross Ref
- O. Banos, A. Calatroni, M. Damas, H. Pomares, I. Rojas, H. Sagha, J. del R Millán, G. Troster, R. Chavarriaga, and D. Roggen. Kinect= imu? learning mimo signal mappings to automatically translate activity recognition systems across sensor modalities. In IEEE Int. Symp. Wearable Computers (ISWC), 2012. Google ScholarDigital Library
- L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. In 2nd Intl. Conf. Pervasive Computing (Pervasive), pages 1--17, 2004.Google ScholarCross Ref
- C. Barthold, K. Subbu, and R. Dantu. Evaluation of gyroscope-embedded mobile phones. In IEEE Intl. Conf. Systems, Man, and Cybernetics (SMC), Oct 2011.Google ScholarCross Ref
- P. Batista, C. Silvestre, P. Oliveira, and B. Cardeira. Accelerometer calibration and dynamic bias and gravity estimation: Analysis, design, and experimental evaluation. IEEE Trans. Control Systems Technology, 19(1):1128--1137, 2011.Google ScholarCross Ref
- S. Bhattacharya, H. Blunck, M. Kjærgaard, and P. Nurmi. Robust and energy-efficient trajectory tracking for mobile devices. IEEE Trans. Mobile Computing (TMC), 14(2):430--443, 2015.Google ScholarCross Ref
- S. Bhattacharya, P. Nurmi, N. Hammerla, and T. Plötz. Using unlabeled data in a sparse-coding framework for human activity recognition. Pervasive and Mobile Computing, 15(0):242--262, 2014.Google ScholarDigital Library
- G. Bieber, P. Koldrack, C. Sablowski, C. Peter, and B. Urban. Mobile physical activity recognition of stand-up and sit-down transitions for user behavior analysis. In Intl. Conf. Pervasive Technologies Related to Assistive Environments (PETRA). ACM, 2010. Google ScholarDigital Library
- G. Bieber, J. Voskamp, and B. Urban. Activity recognition for everyday life on mobile phones. In Universal Access in Human-Computer Interaction (UAHCI). Springer, 2009. Google ScholarDigital Library
- C. M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006. Google ScholarDigital Library
- H. Blunck, N. O. Bouvin, T. Franke, K. Grønbæk, M. B. Kjærgaard, P. Lukowicz, and M. Wüstenberg. On heterogeneity in mobile sensing applications aiming at representative data collection. In UbiComp '13 Adjunct, pages 1087--1098. ACM, 2013. Google ScholarDigital Library
- T. Brezmes, J.-L. Gorricho, and J. Cotrina. Activity recognition from accelerometer data on a mobile phone. In Intl. Work-Conf. Artificial Neural Networks (IWANN). Springer, 2009. Google ScholarDigital Library
- A. Bulling, U. Blanke, and B. Schiele. A tutorial on human activity recognition using body-worn inertial sensors. Computing Surveys (CSUR, 46(3), 2014. Google ScholarDigital Library
- F. Buttussi and L. Chittaro. Mopet: A context-aware and user-adaptive wearable system for fitness training. Artificial Intelligence in Medicine, 42(2):153--163, 2008. Google ScholarDigital Library
- Y. Chen, Z. Zhao, S. Wang, and Z. Chen. Extreme learning machine-based device displacement free activity recognition model. Soft Computing, 16(9), 2012. Google ScholarDigital Library
- J. Dai, X. Bai, Z. Yang, Z. Shen, and D. Xuan. Mobile phone-based pervasive fall detection. Personal Ubiquitous Comput., 14(7), 2010. Google ScholarDigital Library
- S. Dey, N. Roy, W. Xu, R. R. Choudhury, and S. Nelakuditi. Accelprint: Imperfections of accelerometers make smartphones trackable. Network and Distributed System Security Symp. (NDSS), 2014.Google ScholarCross Ref
- D. Figo, P. C. Diniz, D. R. Ferreira, and J. M. P. Cardoso. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing, 14(7):645--662, 2010. Google ScholarDigital Library
- B. J. Frey and D. Dueck. Clustering by passing messages between data points. science, 315(5814):972--976, 2007.Google Scholar
- F. Gulmammadov. Analysis, modeling and compensation of bias drift in mems inertial sensors. IEEE 4th Intl. Conf. Recent Advances in Space Technologies (RAST), June 2009.Google ScholarCross Ref
- N. Y. Hammerla, R. Kirkham, P. Andras, and T. Ploetz. On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution. In ISWC. ACM, 2013. Google ScholarDigital Library
- S. Hemminki, P. Nurmi, and S. Tarkoma. Accelerometer-based transportation mode detection on smartphones. In 11th ACM Conf. Embedded Networked Sensor Systems (SenSys), 2013. Google ScholarDigital Library
- A. Henpraserttae, S. Thiemjarus, and S. Marukatat. Accurate activity recognition using a mobile phone regardless of device orientation and location. In IEEE Body Sensor Networks Conference (BSN), 2011. Google ScholarDigital Library
- J. D. Hol. Sensor fusion and calibration of inertial sensors, vision, Ultra-Wideband and GPS. PhD thesis, Linköping University, Sweden, 2011.Google Scholar
- Y. Kawahara, H. Kurasawa, and H. Morikawa. Recognizing User Context Using Mobile Handsets with Acceleration Sensors. In IEEE Intl. Conf. Portable Information Devices (PORTABLE), pages 1--5, 2007.Google Scholar
- K. Kunze and P. Lukowicz. Dealing with sensor displacement in motion-based onbody activity recognition systems. In ACM Intl. Joint Conf. on Pervasive and Ubiquitous Computing (UbiComp), pages 20--29. ACM, 2008. Google ScholarDigital Library
- J. R. Kwapisz, G. M. Weiss, and S. A. Moore. Activity recognition using cell phone accelerometers. In SIGKDD Explorations Newsletter. ACM, 2011. Google ScholarDigital Library
- N. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. Campbell. A survey of mobile phone sensing. Communications Magazine, IEEE, 48(9), 2010. Google ScholarDigital Library
- J. Lester, T. Choudhury, and G. Borriello. A practical approach to recognizing physical activities. In 4th Intl. Conf. Pervasive Computing (Pervasive), 2006. Google ScholarDigital Library
- B. Logan, J. Healey, M. Philipose, E. M. Tapia, and S. Intille. A long-term evaluation of sensing modalities for activity recognition. In Proceedings of the 10th international conference on Ubiquitous computing (UbiComp), 2007. Google ScholarDigital Library
- J. C. Lötters, J. Schipper, P. H. Veltink, W. Olthuis, and P. Bergveld. Procedure for in-use calibration of triaxial accelerometers in medical applications. Sensors and Actuators A: Physical, 68(1--3):221--228, 1998.Google Scholar
- S. Mazilu, M. Hardegger, Z. Zhu, D. Roggen, G. Troster, M. Plotnik, and J. M. Hausdorff. Online detection of freezing of gait with smartphones and machine learning techniques. In IEEE Intl. Conf. Pervasive Computing Technologies for Healthcare (PervasiveHealth), pages 123--130. IEEE, 2012.Google ScholarCross Ref
- Q. McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153--157, 1947.Google ScholarCross Ref
- OpenSignal. Android Fragmentation Visualized. http://opensignal.com/reports/2014/ android-fragmentation/, 2014. Accessed 17-Mar-2015. {37} J. Pärkkä, M. Ermes, P. Korpipää, J. Mäntyjärvi, J. Peltola, and I. Korhonen. Activity classification using realistic data from wearable sensors. Biomedicine, 10(1):119--128, 2006. Google ScholarDigital Library
- C. Pham and P. Olivier. Slice&dice: Recognizing food preparation activities using embedded accelerometers. In Intl. Conf. Ambient Intelligent (AmI), 2009. Google ScholarDigital Library
- T. Plötz, N. Y. Hammerla, and P. Olivier. Feature learning for activity recognition in ubiquitous computing. In Intl. Joint Conf. Artificial Intelligence (IJCAI), volume 22, page 1729, 2011. {40} T. Plötz, P. Moynihan, C. Pham, and P. Olivier. Activity recognition and healthier food preparation. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, 2011. Google ScholarDigital Library
- S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, and D. Howard. A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Trans. Biomedical Engineering, 56(3):871--879, 2009.Google ScholarCross Ref
- S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, and M. Srivastava. Using mobile phones to determine transportation modes. ACM Trans. Sen. Netw., 6(2):13:1--13:27, 2010. Google ScholarDigital Library
- H. Sagha, S. Digumarti, J. del R Millan, R. Chavarriaga, A. Calatroni, D. Roggen, and G. Tröster. Benchmarking classification techniques using the Opportunity human activity dataset. In IEEE Intl. Conf. Systems, Man, and Cybernetics (SMC), 2011.Google ScholarCross Ref
- P. Siirtola and J. Röning. Recognizing human activities user-independently on smartphones based on accelerometer data. Intl. Journ. Interactive Multimedia and Artificial Intelligence, 1(5), 2012.Google Scholar
- X. Su, H. Tong, and P. Ji. Activity recognition with smartphone sensors. Tsinghua Science and Technology, 19(3):235--249, 2014.Google ScholarCross Ref
- L. Sun, D. Zhang, B. Li, B. Guo, and S. Li. Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations. In Ubiquitous intelligence and computing (UIC). Springer, 2010. Google ScholarDigital Library
- J. Yang. Toward physical activity diary: motion recognition using simple acceleration features with mobile phones. In 1st Intl. Workshop Interactive Multimedia for Consumer Electronics, pages 1--10. ACM, 2009. Google ScholarDigital Library
Index Terms
- Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition
Recommendations
Energy-Efficient Motion Related Activity Recognition on Mobile Devices for Pervasive Healthcare
Activity recognition plays an important role for pervasive healthcare such as health monitoring, assisted living and pro-active services. Despite of the continuous and transparent sensing with various built-in sensors in mobile devices, activity ...
Recognizing the use of portable electrical devices with hand-worn magnetic sensors
Pervasive'11: Proceedings of the 9th international conference on Pervasive computingThe new method proposed here recognizes the use of portable electrical devices such as digital cameras, cellphones, electric shavers, and video game players with hand-worn magnetic sensors by sensing the magnetic fields emitted by these devices. Because ...
Capturing Daily Student Life by Recognizing Complex Activities Using Smartphones
MobiQuitous 2017: Proceedings of the 14th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and ServicesIn-depth understanding of student life is essential to proactively support students in their academic educations. However, there is no work that identifies and recognizes a sufficient set of activities to capture a daily student life since complex ...
Comments