Wentao Bao
ABSTRACT
Traffic accident anticipation aims to predict accidents from dashcam videos as early as possible, which is critical to safety-guaranteed self-driving systems. With cluttered traffic scenes and limited visual cues, it is of great challenge to predict how long there will be an accident from early observed frames. Most existing approaches are developed to learn features of accident-relevant agents for accident anticipation, while ignoring the features of their spatial and temporal relations. Besides, current deterministic deep neural networks could be overconfident in false predictions, leading to high risk of traffic accidents caused by self-driving systems. In this paper, we propose an uncertainty-based accident anticipation model with spatio-temporal relational learning. It sequentially predicts the probability of traffic accident occurrence with dashcam videos. Specifically, we propose to take advantage of graph convolution and recurrent networks for relational feature learning, and leverage Bayesian neural networks to address the intrinsic variability of latent relational representations. The derived uncertainty-based ranking loss is found to significantly boost model performance by improving the quality of relational features. In addition, we collect a new Car Crash Dataset (CCD) for traffic accident anticipation which contains environmental attributes and accident reasons annotations. Experimental results on both public and the newly-compiled datasets show state-of-the-art performance of our model. Our code and CCD dataset are available at https://github.com/Cogito2012/UString.
Supplemental Material
- Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight Uncertainty in Neural Networks. In International Conference on Machine Learning.Google Scholar
- James Bradbury, Stephen Merity, Caiming Xiong, and Richard Socher. 2017. Quasi-Recurrent Neural Networks. In International Conference on Learning Representations.Google Scholar
- Zhaowei Cai and Nuno Vasconcelos. 2018. Cascade R-CNN: Delving into High Quality Object Detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Fu-Hsiang Chan, Yu-Ting Chen, Yu Xiang, and Min Sun. 2016. Anticipating Accidents in Dashcam Videos. In Asian Conference on Computer Vision.Google Scholar
- Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations Using RNN Encoder--Decoder for Statistical Machine Translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.Google ScholarCross Ref
- Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A Recurrent Latent Variable Model for Sequential Data. In Proceedings of Neural Information Processing Systems.Google Scholar
- G. Corcoran and J. Clark. 2019. Traffic Risk Assessment: A Two-Stream Approach Using Dynamic Attention. In Conference on Computer and Robot Vision.Google Scholar
- Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Proceedings of Neural Information Processing Systems.Google Scholar
- John S. Denker and Yann LeCun. 1990. Transforming Neural-Net Output Levels to Probability Distributions. In Proceedings of Neural Information Processing Systems.Google Scholar
- J. Fang, D. Yan, J. Qiao, J. Xue, H. Wang, and S. Li. 2019. DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark. In IEEE Intelligent Transportation Systems Conference.Google Scholar
- Yarin Gal and Zoubin Ghahramani. 2016. Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. In International Conference on Learning Representations (Workshop).Google Scholar
- Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarDigital Library
- Alex Graves. 2011. Practical Variational Inference for Neural Networks. In Proceedings of Neural Information Processing Systems.Google Scholar
- Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. 2019. Variational Graph Recurrent Neural Networks. In Proceedings of Neural Information Processing Systems.Google Scholar
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation, Vol. 9, 8 (1997), 1735--1780.Google ScholarDigital Library
- Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. In Proceedings of Neural Information Processing Systems.Google Scholar
- Diederik P Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. In International Conference on Learning Representations.Google Scholar
- Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. In Proceedings of Neural Information Processing Systems (Workshop).Google Scholar
- Thomas N Kipf and Max Welling. 2017. Semi-supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations.Google Scholar
- Yongchan Kwon, Joong-Ho Won, Beom Joon Kim, and Myunghee Cho Paik. 2018. Uncertainty Quantification Using Bayesian Neural Networks in Classification: Application to Ischemic Stroke Lesion Segmentation. In Medical Imaging with Deep Learning.Google Scholar
- Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature Pyramid Networks for Object Detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Zachary C Lipton, John Berkowitz, and Charles Elkan. 2015. A Critical Review of Recurrent Neural Networks for Sequence Learning. arXiv:1506.00019 (2015).Google Scholar
- Shugao Ma, Leonid Sigal, and Stan Sclaroff. 2016. Learning Activity Progression in LSTMs for Activity Detection and Early Detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Radford M Neal. 2012. Bayesian learning for neural networks. Vol. 118. Springer Science & Business Media.Google Scholar
- Lukas Neumann, Andrew Zisserman, and Andrea Vedaldi. 2019. Future Event Prediction: If and When. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Workshop).Google ScholarCross Ref
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of Neural Information Processing Systems.Google Scholar
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Proceedings of Neural Information Processing Systems.Google Scholar
- Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In International Conference on Machine Learning.Google Scholar
- Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. 2018. Structured Sequence Modeling with Graph Convolutional Recurrent Networks. In Proceedings of Neural Information Processing Systems.Google ScholarDigital Library
- Ankit Shah, Jean Baptiste Lamare, Tuan Nguyen Anh, and Alexander Hauptmann. 2018. CADP: A Novel Dataset for CCTV Traffic Camera based Accident Analysis. In International Workshop on Traffic and Street Surveillance for Safety and Security.Google ScholarCross Ref
- Kumar Shridhar, Felix Laumann, and Marcus Liwicki. 2018. Uncertainty Estimations by Softplus Normalization in Bayesian Convolutional Neural Networks with Variational Inference. arXiv:1806.05978 (2018).Google Scholar
- Tomoyuki Suzuki, Hirokatsu Kataoka, Yoshimitsu Aoki, and Yutaka Satoh. 2018. Anticipating Traffic Accidents with Adaptive Loss and Large-scale Incident DB. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Yoshiaki Takimoto, Yusuke Tanaka, Takeshi Kurashima, Shuhei Yamamoto, Maya Okawa, and Hiroyuki Toda. 2019. Predicting Traffic Accidents with Event Recorder Data. In Proceedings of ACM SIGSPATIAL International Workshop on Prediction of Human Mobility.Google ScholarDigital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of Neural Information Processing Systems.Google Scholar
- Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated Residual Transformations for Deep Neural Networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Yu Yao, Mingze Xu, Yuchen Wang, David J Crandall, and Ella M Atkins. 2019. Unsupervised Traffic Accident Detection in First-person Videos. In International Conference on Intelligent Robots and Systems.Google Scholar
- Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. 2020. BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Kuo-Hao Zeng, Shih-Han Chou, Fu-Hsiang Chan, Juan Carlos Niebles, and Min Sun. 2017. Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Rui Zhao, Kang Wang, Hui Su, and Qiang Ji. 2019. Bayesian Graph Convolution LSTM for Skeleton Based Action Recognition. In Proceedings of the IEEE International Conference on Computer Vision.Google ScholarCross Ref
Index Terms
- Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learning
Recommendations
Dynamic spatio-temporal integration of traffic accident data
SIGSPATIAL '18: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information SystemsUp to 50% of delay in traffic is due to non-reoccurring events such as traffic accidents. Accidents lead to delays, which can be costly for transport companies. Road authorities are also very interested in warning drivers about accidents, e.g., to ...
Vision-based highway traffic accident detection
AIIPCC '19: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud ComputingThe highway traffic scene is relatively simple, the traffic flow is small but the speed of traffic is fast. The traffic accidents on highways are sudden and harmful. Based on highway traffic monitoring video, this paper uses machine learning to ...
V2V Communication-based AEB Validation in Traffic Accident Simulation Scenario
ICFEICT 2021: International Conference on Frontiers of Electronics, Information and Computation TechnologiesAutonomous Emergency Braking (AEB) system can effectively avoid traffic accidents and reduce the degree of casualties. However, AEB systems based on traditional sensors will have blind spots and are greatly affected by environmental factors. This paper ...
Comments