Michalis Papakostas
ABSTRACT
Distracted driving is a leading cause of accidents worldwide. The tasks of distraction detection and recognition have been traditionally addressed as computer vision problems. However, distracted behaviors are not always expressed in a visually observable way. In this work, we introduce a novel multimodal dataset of distracted driver behaviors, consisting of data collected using twelve information channels coming from visual, acoustic, near-infrared, thermal, physiological and linguistic modalities. The data were collected from 45 subjects while being exposed to four different distractions (three cognitive and one physical). For the purposes of this paper, we experiment with visual and physiological information and explore the potential of multimodal modeling for distraction recognition. In addition, we analyze the value of different modalities by identifying specific visual and physiological groups of features that contribute the most to distraction characterization. Our results highlight the advantage of multimodal representations and reveal valuable insights for the role played by the two modalities on identifying different types of driving distractions.
- Yehya Abouelnaga, Hesham M Eraqi, and Mohamed N Moustafa. 2017. Real-time distracted driver posture classification. arXiv preprint arXiv:1706.09498(2017).Google Scholar
- Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: an open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1–10.Google ScholarCross Ref
- Daniela Cardone, David Perpetuini, Chiara Filippini, Edoardo Spadolini, Lorenza Mancini, Antonio Maria Chiarelli, and Arcangelo Merla. 2020. Driver Stress State Evaluation by Means of Thermal Imaging: A Supervised Machine Learning Approach Based on ECG Signal. Applied Sciences 10, 16 (Aug 2020), 5673. https://doi.org/10.3390/app10165673Google ScholarCross Ref
- Hyun-Seung Cho, Jin-Hee Yang, Sang-Min Kim, Jeong-Whan Lee, Hwi-Kuen Kwak, Je-Wook Chae, and Joo-Hyeon Lee. 2020. Development of a Chest-Belt-Type Biosignal-Monitoring Wearable Platform System. Journal of Electrical Engineering & Technology 15, 4(2020), 1847–1855.Google ScholarCross Ref
- Curtis Florence, Thomas Simon, Tamara Haegerich, Feijun Luo, and Chao Zhou. 2015. Estimated lifetime medical and work-loss costs of fatal injuries—United States, 2013. Morbidity and Mortality Weekly Report 64, 38 (2015), 1074–1077.Google ScholarCross Ref
- Jerome H Friedman. 2002. Stochastic gradient boosting. Computational statistics & data analysis 38, 4 (2002), 367–378.Google Scholar
- Guido Grassi, Sabrina Vailati, Giovanni Bertinieri, Gino Seravalle, Maria Luisa Stella, Raffaella Dell’Oro, and Giuseppe Mancia. 1998. Heart rate as marker of sympathetic activity. Journal of hypertension 16, 11 (1998), 1635–1639.Google ScholarCross Ref
- David Michael Herman. 2020. Monitoring of steering wheel engagement for autonomous vehicles. US Patent App. 16/294,541.Google Scholar
- Michael J Kane, Andrew RA Conway, Timothy K Miura, and Gregory JH Colflesh. 2007. Working memory, attention control, and the N-back task: a question of construct validity.Journal of Experimental Psychology: Learning, Memory, and Cognition 33, 3(2007), 615.Google ScholarCross Ref
- Neslihan Kose, Okan Kopuklu, Alexander Unnervik, and Gerhard Rigoll. 2019. Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC). IEEE, 3236–3242.Google ScholarDigital Library
- Matthew N Levy. 1971. Brief reviews: sympathetic-parasympathetic interactions in the heart. Circulation research 29, 5 (1971), 437–445.Google ScholarCross Ref
- Andy Liaw, Matthew Wiener, 2002. Classification and regression by randomForest. R news 2, 3 (2002), 18–22.Google Scholar
- Tianchi Liu, Yan Yang, Guang-Bin Huang, Yong Kiang Yeo, and Zhiping Lin. 2015. Driver distraction detection using semi-supervised machine learning. IEEE transactions on intelligent transportation systems 17, 4(2015), 1108–1120.Google ScholarCross Ref
- Ralph Oyini Mbouna, Seong G Kong, and Myung-Geun Chun. 2013. Visual analysis of eye state and head pose for driver alertness monitoring. IEEE transactions on intelligent transportation systems 14, 3(2013), 1462–1469.Google ScholarDigital Library
- Anthony D. McDonald, Thomas K. Ferris, and Tyler A. Wiener. 2020. Classification of Driver Distraction: A Comprehensive Analysis of Feature Generation, Machine Learning, and Input Measures. Human Factors 62, 6 (2020), 1019–1035. https://doi.org/10.1177/0018720819856454 arXiv:https://doi.org/10.1177/0018720819856454PMID: 31237788.Google ScholarCross Ref
- H Meyers. 2010. ProComp Infiniti/BioGraph Infiniti biofeedback system (version 5.1. 2). Montreal, QB: Thought Technology Ltd(2010).Google Scholar
- Centers for Disease Control & Prevention (CDC) National Center for Injury Prevention & Control. 2019. Distracted Driving. https://www.cdc.gov/motorvehiclesafety/distracted_driving/index.html. [Online; accessed 13-April-2020].Google Scholar
- Centers for Disease Control & Prevention (CDC) National Center for Injury Prevention & Control. 2019. Road Traffic Injuries and Deaths—A Global Problem. https://www.cdc.gov/injury/features/global-road-safety/index.html. [Online; accessed 13-April-2020].Google Scholar
- Centers for Disease Control & Prevention (CDC) National Center for Injury Prevention & Control. 2020. Cost of Injury Data. https://www.cdc.gov/injury/wisqars/cost/index.html. [Online; accessed 13-April-2020].Google Scholar
- US Department of Transportation National Highway Traffic Safety Administration (NHTSA). 2019. Distracted Driving. https://www.nhtsa.gov/risky-driving/distracted-driving. [Online; accessed 13-April-2020].Google Scholar
- Shotaro Odate, Naohiro Sakamoto, and Yukinori Midorikawa. 2020. Development of Electrostatic Capacity Type Steering Sensor Using Conductive Leather. Technical Report. SAE Technical Paper.Google Scholar
- Michalis Papakostas, Kapotaksha Das, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. 2021. Distracted and Drowsy Driving Modeling Using Deep Physiological Representations and Multitask Learning. Applied Sciences 11, 1 (2021), 88.Google ScholarCross Ref
- Michalis Papakostas, Akilesh Rajavenkatanarayanan, and Fillia Makedon. 2019. CogBeacon: A Multi-Modal Dataset and Data-Collection Platform for Modeling Cognitive Fatigue. Technologies 7, 2 (2019), 46.Google ScholarCross Ref
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825–2830.Google Scholar
- Xuli Rao, Feng Lin, Zhide Chen, and Jiaxu Zhao. 2019. Distracted driving recognition method based on deep convolutional neural network. Journal of Ambient Intelligence and Humanized Computing (2019), 1–8.Google Scholar
- Kais Riani, Michalis Papakostas, Hussein Kokash, Mohamed Abouelenien, Mihai Burzo, and Rada Mihalcea. 2020. Towards Detecting Levels of Alertness in Drivers Using Multiple Modalities. In Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Greece) (PETRA ’20). Association for Computing Machinery, New York, NY, USA, Article 12, 9 pages. https://doi.org/10.1145/3389189.3389192Google ScholarDigital Library
- Fabien Ringeval, Björn Schuller, Michel Valstar, Shashank Jaiswal, Erik Marchi, Denis Lalanne, Roddy Cowie, and Maja Pantic. 2015. Av+ ec 2015: The first affect recognition challenge bridging across audio, video, and physiological data. In Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge. 3–8.Google ScholarDigital Library
- Sol M Rodríguez-Colón, Fan He, Edward O Bixler, Julio Fernandez-Mendoza, Alexandros N Vgontzas, Susan Calhoun, Zhi-Jie Zheng, and Duanping Liao. 2015. Sleep variability and cardiac autonomic modulation in adolescents–Penn State Child Cohort (PSCC) study. Sleep medicine 16, 1 (2015), 67–72.Google Scholar
- Jason M Saragih, Simon Lucey, and Jeffrey F Cohn. 2011. Deformable model fitting by regularized landmark mean-shift. International journal of computer vision 91, 2 (2011), 200–215.Google ScholarDigital Library
- Pragya Sharma, Xiaonan Hui, Jianlin Zhou, Thomas B Conroy, and Edwin C Kan. 2020. Wearable radio-frequency sensing of respiratory rate, respiratory volume, and heart rate. NPJ digital medicine 3, 1 (2020), 1–10.Google Scholar
- GiriBabu Sinnapolu and Shadi Alawneh. 2020. Intelligent wearable heart rate sensor implementation for in-vehicle infotainment and assistance. Internet of Things 12(2020), 100277.Google ScholarCross Ref
- Erin T Solovey, Marin Zec, Enrique Abdon Garcia Perez, Bryan Reimer, and Bruce Mehler. 2014. Classifying driver workload using physiological and driving performance data: two field studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 4057–4066.Google ScholarDigital Library
- Y-I Tian, Takeo Kanade, and Jeffrey F Cohn. 2001. Recognizing action units for facial expression analysis. IEEE Transactions on pattern analysis and machine intelligence 23, 2(2001), 97–115.Google ScholarDigital Library
- R. Verma, B. Mitra, and Sandip Chakraborty. 2019. Avoiding Stress Driving: Online Trip Recommendation from Driving Behavior Prediction. 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom (2019), 1–10.Google ScholarCross Ref
- K. Wang, Y. L. Murphey, Y. Zhou, X. Hu, and X. Zhang. 2019. Detection of driver stress in real-world driving environment using physiological signals. In 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Vol. 1. 1807–1814.Google Scholar
- Yongquan Xie, Yi L Murphey, and Dev Kochhar. 2019. Personalized Driver Workload Estimation Using Deep Neural Network Learning from Physiological and Vehicle Signals. IEEE Transactions on Intelligent Vehicles(2019).Google Scholar
- Sebastian Zepf, Neska El Haouij, Jinmo Lee, Asma Ghandeharioun, Javier Hernandez, and Rosalind W. Picard. 2020. Studying Personalized Just-in-Time Auditory Breathing Guides and Potential Safety Implications during Simulated Driving(UMAP ’20). Association for Computing Machinery, New York, NY, USA, 275–283. https://doi.org/10.1145/3340631.3394854Google ScholarDigital Library
Index Terms
- Understanding Driving Distractions: A Multimodal Analysis on Distraction Characterization
Index terms have been assigned to the content through auto-classification.
Recommendations
Detection and Recognition of Driver Distraction Using Multimodal Signals
Distracted driving is a leading cause of accidents worldwide. The tasks of distraction detection and recognition have been traditionally addressed as computer vision problems. However, distracted behaviors are not always expressed in a visually observable ...
Towards detecting levels of alertness in drivers using multiple modalities
PETRA '20: Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsDistracted and drowsy driving are two very common causes of car accidents as they contribute to 2.3% of all the fatalities caused on the US roads. Therefore, in the era of smart driving there is an increased need of technologies able to monitor driver's ...
Numerical Analysis of Tractor Accidents using Driving Simulator for Autonomous Driving Tractor
ICMRE'19: Proceedings of the 5th International Conference on Mechatronics and Robotics EngineeringAutonomous driving of automobiles is a hot research topic in recent years. The autonomous driving tractor also has been studied in the agricultural field as well as an autonomous driving automobile. On the other hand, tractor accidents frequently occur ...
Comments