Stanislaw Saganowski
ABSTRACT
Wearables equipped with pervasive sensors enable us to monitor physiological and behavioral signals in our everyday life. We propose the WellAff system able to recognize affective states for wellbeing support. It also includes health care scenarios, in particular patients with chronic kidney disease suffering from bipolar disorders. For the need of a large-scale field study, we revised over 50 off-the-shelf devices in terms of usefulness for emotion, stress, meditation, sleep, and physical activity recognition and analysis. Their usability directly comes from the types of sensors they possess as well as the quality and availability of raw signals. We found there is no versatile device suitable for all purposes. Using Empatica E4 and Samsung Galaxy Watch, we have recorded physiological signals from 11 participants over many weeks. The gathered data enabled us to train a classifier that accurately recognizes strong affective states.
- H Alawieh, Z Dawy, E Yaacoub, and et al.2019. A Real-Time ECG Feature Extraction Algorithm for Detecting Meditation Levels within a General Measurement Setup. In 41st Annual Int. Conf. of the IEEE Eng. in Med. and Biol. Soc. IEEE, 99–103.Google Scholar
- Amani Albraikan, Basim Hafidh, and Abdulmotaleb El Saddik. 2018. iAware: A Real-Time Emotional Biofeedback System Based on Physiological Signals. IEEE Access 6(2018), 78780–78789.Google ScholarCross Ref
- H Allahbakhshi, L Conrow, B Naimi, and R Weibel. 2020. Using Accelerometer and GPS Data for Real-Life Physical Activity Type Detection. Sensors 20, 3 (2020), 588.Google ScholarCross Ref
- G Aroganam, N Manivannan, and D Harrison. 2019. Review on wearable technology sensors used in consumer sport applications. Sensors 19, 9 (2019), 1983.Google ScholarCross Ref
- KR Arunkumar and M Bhaskar. 2020. Heart rate estimation from wrist-type photoplethysmography signals during physical exercise. Biomedical Signal Processing and Control 57 (2020), 101790.Google ScholarCross Ref
- Mahmoud Assaf, Aïcha Rizzotti-Kaddouri, and Magdalena Punceva. 2018. Sleep detection using physiological signals from a wearable device. In EAI International Conference on IoT Technologies for HealthCare. Springer, 23–37.Google Scholar
- MI Cajita, CE Kline, and et al.2020. Feasible but Not Yet Efficacious: a Scoping Review of Wearable Activity Monitors in Interventions Targeting Physical Activity, Sedentary Behavior, and Sleep. Current Epidem. Rep. 7, 1 (2020), 25–38.Google ScholarCross Ref
- Burçin Camcı, Cem Ersoy, and Hakan Kaynak. 2019. Abnormal respiratory event detection in sleep: a prescreening system with smart wearables. Journal of biomedical informatics 95 (2019), 103218.Google ScholarDigital Library
- Yekta Said Can, Bert Arnrich, and Cem Ersoy. 2019. Stress detection in daily life scenarios using smart phones and wearable sensors: A survey. Journal of biomedical informatics(2019), 103139.Google Scholar
- Yekta Said Can, Niaz Chalabianloo, Deniz Ekiz, and Cem Ersoy. 2019. Continuous stress detection using wearable sensors in real life: Algorithmic programming contest case study. Sensors 19, 8 (2019), 1849.Google ScholarCross Ref
- Rossana Castaldo, Marta Prati, Luis Montesinos, Vishwesh Kulkarni, Micheal Chappell, Helen Byrne, Pasquale Innominato, Stephen Hughes, and Leandro Pecchia. 2019. Investigating the Use of Wearables for Monitoring Circadian Rhythms: A Feasibility Study. In International Conference on Biomedical and Health Informatics. Springer, 275–280.Google Scholar
- Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321–357.Google ScholarCross Ref
- Gloria Cosoli, Susanna Spinsante, and Lorenzo Scalise. 2020. Wrist-worn and chest-strap wearable devices: systematic review on accuracy and metrological characteristics. Measurement (2020), 107789.Google Scholar
- Massimiliano de Zambotti, Leonardo Rosas, Ian M Colrain, and Fiona C Baker. 2019. The sleep of the ring: comparison of the ŌURA sleep tracker against polysomnography. Behavioral sleep medicine 17, 2 (2019), 124–136.Google Scholar
- Urtnasan Erdenebayar, Yoon Ji Kim, Jong-Uk Park, Eun Yeon Joo, and Kyoung-Joung Lee. 2019. Deep learning approaches for automatic detection of sleep apnea events from an electrocardiogram. Computer methods and programs in biomedicine 180 (2019), 105001.Google Scholar
- Luz Fernández-Aguilar, Arturo Martínez-Rodrigo, José Moncho-Bogani, Antonio Fernández-Caballero, and José Miguel Latorre. 2019. Emotion Detection in Aging Adults Through Continuous Monitoring of Electro-Dermal Activity and Heart-Rate Variability. In Int. Work-Conference on the Interplay Between Natural and Artificial Computation. Springer, 252–261.Google Scholar
- Arindam Ghosh, Morena Danieli, and Giuseppe Riccardi. 2015. Annotation and prediction of stress and workload from physiological and inertial signals. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 1621–1624.Google ScholarCross Ref
- G. Giannakakis, D. Grigoriadis, K. Giannakaki, O. Simantiraki, A. Roniotis, and M. Tsiknakis. 2019. Review on psychological stress detection using biosignals. IEEE Transactions on Affective Computing(2019).Google Scholar
- Alireza Golgouneh and Bahram Tarvirdizadeh. 2019. Fabrication of a portable device for stress monitoring using wearable sensors and soft computing algorithms. Neural Computing and Applications(2019), 1–23.Google Scholar
- Rosenberger M Grandner M. 2019. Actigraphic sleep tracking and wearables: Historical context, scientific applications and guidelines, limitations, and considerations for commercial sleep devices. In Sleep and Health. Elsevier, 147–157.Google Scholar
- Juan Lorenzo Hagad, Kenichi Fukui, and Masayuki Numao. 2019. Deep visual models for EEG of mindfulness meditation in a workplace setting. In International Workshop on Health Intelligence. Springer, 129–137.Google Scholar
- BP Harne, Y Bobade, RS Dhekekar, and A Hiwale. 2019. SVM classification of EEG signal to analyze the effect of OM Mantra meditation on the brain. In 2019 IEEE 16th India Council International Conference (INDICON). IEEE, 1–4.Google ScholarCross Ref
- Cheng He, Yun-jin Yao, and Xue-song Ye. 2017. An emotion recognition system based on physiological signals obtained by wearable sensors. In Wearable sensors and robots. Springer, 15–25.Google Scholar
- RT Hurlburt and CL Heavy. 2002. Interobserver reliability of descriptive experience sampling. Cognitive Therapy and Research 26, 1 (2002), 135–142.Google ScholarCross Ref
- Eiman Kanjo, Eman MG Younis, and Chee Siang Ang. 2019. Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection. Information Fusion 49(2019), 46–56.Google ScholarDigital Library
- A Léonard, S Clément, C-D Kuo, and MU Manto. 2019. Changes in Heart Rate Variability During Heartfulness Meditation: A Power Spectral Analysis Including the Residual Spectrum. Frontiers in cardiovascular medicine 6 (2019), 62.Google Scholar
- M Maier, C Marouane, and D Elsner. 2019. DeepFlow: Detecting Optimal User Experience From Physiological Data Using Deep Neural Networks. In Proc. of the 18th Int. Conf. on Autonom. Agents and MultiAgent Sys. Int. Found. for Autonom. Agents and Multiagent Sys., 2108–2110.Google ScholarCross Ref
- F Mendonca, SS Mostafa, AG Ravelo-García, F Morgado-Dias, and T Penzel. 2018. A review of obstructive sleep apnea detection approaches. IEEE journal of biomedical and health informatics 23, 2(2018), 825–837.Google Scholar
- Andre Matthias Müller, Nan Xin Wang, Jiali Yao, Chuen Seng Tan, Ivan Cherh Chiet Low, Nicole Lim, Jeremy Tan, Agnes Tan, and Falk Müller-Riemenschneider. 2019. Heart rate measures from wrist-worn activity trackers in a laboratory and free-living setting: validation study. JMIR mHealth and uHealth 7, 10 (2019), e14120.Google Scholar
- H Murakami, R Kawakami, S Nakae, Y Yamada, Y Nakata, and et al.2019. Accuracy of 12 Wearable Devices for Estimating Physical Activity Energy Expenditure Using a Metabolic Chamber and the Doubly Labeled Water Method: Validation Study. In JMIR mHealth and uHealth.Google Scholar
- B. Nakisa, M. N. Rastgoo, A. Rakotonirainy, F. Maire, and V. Chandran. 2018. Long short term memory hyperparameter optimization for a neural network based emotion recognition framework. IEEE Access 6(2018), 49325–49338.Google ScholarCross Ref
- Grzegorz J Nalepa, Krzysztof Kutt, Barbara Giżycka, Paweł Jemioło, and Szymon Bobek. 2019. Analysis and Use of the Emotional Context with Wearable Devices for Games and Intelligent Assistants. Sensors 19, 11 (2019), 2509.Google ScholarCross Ref
- Sami Nikkonen, Isaac O Afara, Timo Leppänen, and Juha Töyräs. 2019. Artificial neural network analysis of the oxygen saturation signal enables accurate diagnostics of sleep apnea. Scientific reports 9, 1 (2019), 1–9.Google Scholar
- Selena R. Pasadyn, Mohamad Soudan, Marc Gillinov, Penny Houghtaling, Dermot Phelan, Nicole Gillinov, Barbara Bittel, and Milind Y. Desai. 2019. Accuracy of commercially available heart rate monitors in athletes: a prospective study. Cardiovascular Diagnosis and Therapy 9, 4 (2019).Google Scholar
- Reddipogu Pavani and Akshay Berad. 2019. Study of Effect of Meditation on Galvanic Skin Response in Healthy Individuals. International Journal of Medical and Biomedical Studies 3, 4(2019).Google ScholarCross Ref
- JM Peake, G Kerr, and JP Sullivan. 2018. A critical review of consumer wearables, mobile applications, and equipment for providing biofeedback, monitoring stress, and sleep in physically active populations. Frontiers in physiology 9 (2018), 743.Google ScholarCross Ref
- S Pham, D Yeap, and et al.2020. Wearable Sensor System to Monitor Physical Activity and the Physiological Effects of Heat Exposure. Sensors 20, 3 (2020).Google Scholar
- S. Saganowski, A. Dutkowiak, A. Dziadek, M. Dziezyc, J. Komoszynska, W. Michalska, A. G. Polak, M. Ujma, and P. Kazienko. 2020. Emotion Recognition Using Wearables: A Systematic Literature Review - Work-in-progress. In 2020 Int. Conf. on Perv. Comp. and Comm. Workshops. IEEE, 1–6.Google Scholar
- P. Schmidt, A. Reiss, R. Duerichen, C. Marberger, and K. Van Laerhoven. 2018. Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection. In Proc. of the 2018 on Int. Conf. on Multimodal Interaction. ACM, 400–408.Google Scholar
- P. Schmidt, A. Reiss, R. Dürichen, and K. Van Laerhoven. 2018. Labelling Affective States in the Wild: Practical Guidelines and Lessons Learned. In The 2018 ACM Int. Joint Conf. and 2018 Int. Symp. on Pervasive and Ubiquitous Comp. and Wearable Comp. ACM, 654–659.Google Scholar
- F Setiawan, SA Khowaja, AG Prabono, and et al.2018. A Framework for Real Time Emotion Recognition Based on Human ANS Using Pervasive Device. In 2018 IEEE 42nd Annual Comp. Software and App. Conf., Vol. 1. IEEE, 805–806.Google Scholar
- H. Sharma, R. Raj, and M. Juneja. 2019. EEG signal based classification before and after combined Yoga and Sudarshan Kriya. Neurosc. letters 707(2019), 134300.Google Scholar
- Saul Shiffman, Arthur A Stone, and Michael R Hufford. 2008. Ecological Momentary Assessment. Annu. Rev. Clin. Psychol. 4 (2008), 1–32.Google ScholarCross Ref
- M Shoaib, S Bosch, OD Incel, and et al.2014. Fusion of Smartphone Motion Sensors for Physical Activity Recognition. Sensors 14, 6 (2014), 10146–10176.Google ScholarCross Ref
- MZ Soroush, K Maghooli, SK Setarehdan, and AM Nasrabadi. 2017. A Review on EEG signals based emotion recognition. Int. Clinic. Neurosc. Jour. 4, 4 (2017), 118.Google ScholarCross Ref
- J Wei and J Boger. 2019. You Are How You Sleep: Personalized Sleep Monitoring Based on Wrist Temperature and Accelerometer Data. In 13th Int. Conf. on Perv. Comp. Tech. for Healthcare-Demos and Posters. European Alliance for Innovation.Google Scholar
- Bin Yu, Mathias Funk, Jun Hu, and Loe Feijs. 2018. Unwind: a musical biofeedback for relaxation assistance. Behaviour & Inform. Technology 37, 8 (2018), 800–814.Google ScholarCross Ref
- Roberto Zangróniz, Arturo Martínez-Rodrigo, María T López, José Manuel Pastor, and Antonio Fernández-Caballero. 2018. Estimation of mental distress from photoplethysmography. Applied Sciences 8, 1 (2018), 69.Google ScholarCross Ref
- R Zangróniz, A Martínez-Rodrigo, J Pastor, and et al.2017. Electrodermal activity sensor for classification of calm/distress condition. Sensors 17, 10 (2017), 2324.Google ScholarCross Ref
Recommendations
Usability of a Low-Cost Wearable Health Device for Physical Activity and Sleep Duration in Healthy Adults
MobileHealth '15: Proceedings of the 2015 Workshop on Pervasive Wireless HealthcareFor public health and epidemiology, an understanding of physical activity and sleep duration is important. People found decreased physical activity and short sleep duration as they aged. The development of healthy behaviors in the early life will ensure ...
Affective Wearables
ISWC '97: Proceedings of the 1st IEEE International Symposium on Wearable ComputersAn "affective wearable" is a wearable system equipped with sensors and tools which enables recognition of its wearer's affective patterns. Affective patterns include expressions of emotion such as a joyful smile, an angry gesture, a strained voice or a ...
Workplace Indicators of Mood: Behavioral and Cognitive Correlates of Mood Among Information Workers
DH '16: Proceedings of the 6th International Conference on Digital Health ConferencePositive wellbeing in the workplace is tied to better health. However, lack of wellbeing in the workplace is a serious problem in the U.S, is rising continually, and can lead to poor health conditions. In this study we investigate factors that might be ...
Comments