This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Driver Distraction Detection with a Two-stream Convolutional Neural Network
Technical Paper
2020-01-1039
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Driver distraction detection is crucial to driving safety when autonomous vehicles are co-piloted. Recognizing drivers’ behaviors that are highly related with distraction from real-time video stream is widely acknowledged as an effective approach mainly due to its non-intrusiveness. In recently years, deep learning neural networks have been adopted to by-pass the procedure of designing features artificially, which used to be the major downside of computer-vision based approaches. However, the detection accuracy and generalization ability is still not satisfying since most deep learning models extracts only spatial information contained in images. This research develops a driver distraction model based on a two-stream, spatial and temporal, convolutional neural network (CNN). The CNN in both stream is improved with Batch Normalization-Inception (BN-Inception) modules which increase the sparsity in the inception modules in GoogLeNet, so that the network is further speeded up and also more adapted to features at various-scales. The original RGB image is fed into the spatial stream CNN to extract static information, and the feature map of optical flow field extracted from adjacent image frames is fed into the temporal stream CNN to extract motion information. Based on a weighted fusion of two streams, the model outputs the classification result. Experimental results show that the proposed model recognize, with high accuracy, the driver’s hand and head movements and hence can predict the driver’s level of distraction with a good level of confidence.
Recommended Content
Authors
Citation
Ma, Y., Yin, Z., and Nie, L., "Driver Distraction Detection with a Two-stream Convolutional Neural Network," SAE Technical Paper 2020-01-1039, 2020, https://doi.org/10.4271/2020-01-1039.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 | ||
Unnamed Dataset 2 | ||
Unnamed Dataset 3 | ||
Unnamed Dataset 4 | ||
Unnamed Dataset 5 | ||
Unnamed Dataset 6 |
Also In
References
- Gasser , T. , Westhoff , D. , and BASt BASt-study: Definitions of automation and legal issues in Germany Bergisch Gladbach Federal Highway Research Institute 2012 10.1016/S0386-1112(14)60126-6
- SAE International Surface Vehicle Information Report: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems New York SAE International 2014 10.4271/j3016_201401
- National Highway Traffic Safety Administration Preliminary Statement of Policy Concerning Automated Vehicles Washington DC National Highway Traffic Safety Administration 2013 10.4135/9781483346526.n340
- Lu , Z. , Happee , R. , Cabrall , C. et al. Human Factors of Transitions in Automated Driving: A General Framework and Literature Survey Transportation Research Part F 43 183 198 2016 10.1016/j.trf.2016.10.007
- Akamatsu , M. , Green , P. , and Bengler , K. Automotive Technology and Human Factors Research: Past, Present, and Future International Journal of Vehicular Technology 1 27 2013 10.1155/2013/526180
- Yan , C. , Coenen , F. , and Zhang , B. Driving Posture Recognition by Convolutional Neural Networks IET Computer Vision 10 2 103 114 2016 10.1049/iet-cvi.2015.0175
- Koesdwiady , A. , Bedawi , S. , Ou , C. et al. Transfer Learning Using Convolutional neural networks for Face Anti-Spoofing 2017 10.1007/978-3-319-59876-5_2
- Xing , Y. and Lv, C., Zhang, Z., Et Al. Identification and Analysis of Driver Postures for In-vehicle Driving Activities and Secondary Tasks Recognition IEEE Transactions on Computational Social Systems 5 1 95 108 2018 10.1109/tcss.2017.2766884
- Xing , Y. , Tang , J. , Liu , H. , et al. End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning IEEE Intelligent Vehicles Symposium (IV) IEEE 2018 1626 1631 10.1109/ivs.2018.8500548
- Zhang , Y. , Gao , X. , Zhu , J. , et al. A Novel Approach to Driving Fatigue Detection Using Forehead EOG International Ieee/embs Conference on Neural Engineering Hawaii 2015 707 710 10.1109/ner.2015.7146721
- Lee , D. , Chong , T. , and Lee , B. Stress Events Detection of Driver by Wearable Glove System IEEE Sensors Journal 17 1 194 204 2017 10.1109/jsen.2016.2625323
- Choi , M. , Koo , G. , Seo , M. et al. Wearable Device-Based System to Monitor a Drivers Stress, Fatigue, and Drowsiness IEEE Transactions on Instrumentation and Measurement 67 3 634 645 2018 10.1109/tim.2017.2779329
- Zhao , C. , Zhang , B. , and He, J., Et Al. Recognition of Driving Postures by Contourlet Transform and Random Forests IET Intelligent Transport Systems 6 6 161 168 2012 10.1049/iet-its.2011.0116
- Martin , S. , Tawari , A. , Murphy-Chutorian , E. , et al. On the Design and Evaluation of Robust Head Pose for Visual User Interfaces: Algorithms, Databases, and Comparisons Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications New York: ACM 2012 149 154 10.1145/2390256.2390281
- Hoang , T. , Zheng , Y. , Zhu , C. et al. Multiple scale faster-rcnn approach to driver's cell-phone usage and hands on steering wheel detection Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 46 53 2016 10.1109/cvprw.2016.13
- Yan , C. , Coenen , F. , and Zhang , L. Driving Posture Recognition by Joint Application of Motion History Image and Pyramid Histogram of Oriented Gradients Advanced Materials Research 846-847 1102 1105 2014 10.4028/www.scientific.net/amr.846-847.1102
- Yan , C. , Zhang , B. , Coenen , F. Driving Posture Recognition by a Hierarchal Classification System with Multiple Features IEEE Proceedings of the International Congress on Image and Signal Processing New York: IEEE 2014 83 88 10.1109/cisp.2014.7003754
- Gupta , R. , Mangalraj , P. , Agrawal , A. , et al. Posture Recognition for Safe Driving IEEE. Proceedings of the Third International Conference on Image Information Processing(ICIIP) New York: IEEE 2015 141 146 10.1109/iciip.2015.7414755
- Baheti , B. , Gajre , S. , Talbar , S. Detection of Distracted Driver Using Convolutional Neural Network IEEE. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops New York: IEEE 2018 1032 1038 10.1109/cvprw.2018.00150
- Tran , C. , Trivedim , M. Introducing “XMOB”: Extremity Movement Observation Framework for Upper Body Pose Tracking in 3D IEEE. Proceedings of the IEEE International Symposium on Multimedia New York: IEEE 2009 446 447 10.1109/ism.2009.89
- Zhao , M. , Beurier , G. , Wang , H. et al. In Vehicle Diver Postural Monitoring Using a Depth Camera Kinect SAE Technical Paper 2018-01-0505 2018 https://doi.org/10.4271/2018-01-0505
- Park , D. , Jones , M. , Miller , C. et al. In-vehicle Occupant Head Tracking Using a Low-cost Depth Camera SAE Technical Paper 2018-01-1172 2018 https://doi.org/10.4271/2018-01-1172
- Simonyan , K. and Zisserman , A. Two-Stream Convolutional Networks for Action Recognition in Videos Advances in Neural Information Processing Systems 1 4 568 576 2014
- Tran , D. , Bourdev , L. , Fergus , R. , et al. Learning Spatiotemporal Features with 3D Convolutional Networks IEEE International Conference on Computer Vision (ICCV) 2015 4489 4497 10.1109/iccv.2015.510
- Karpathy , A. , Toderici , G. , Shetty , S. , et al. Large-Scale Video Classification with Convolutional Neural Networks Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Wachingtm DC: IEEE 2014 1725 1732 10.1109/cvpr.2014.223
- Abu-El-Haija , S. , Kothari , N. , Lee , J. , et al. 2016
- Fernando , B. , Gavves , E. , Oramas , J. et al. Rank Pooling for Action Recognition IEEE Transactions on Pattem Analysis & Machine Intelligence 39 4 773 787 2015 10.1109/tpami.2016.2558148
- Jia , Y.S. et al. 2014 675 678 10.1145/2647868.2654889
- Wang , L. , Xiong , Y. , Wang , Z. et al. Temporal Segment Networks: Towards Good Practices for Deep Action Recognition ACM Transactions on Information Systems 22 1 20 36 2016 10.1007/978-3-319-46484-8_2
- Wang , L. , Xiong , Y. , Lin , D. , et al. 2017 6402 6411 10.1109/cvpr.2017.678
- Arnheim , R. The Perception of the Visual Wbrid Houghton Mimin 1950 10.2307/426044
- Wang , H. , Klaser , A. , Schmid , C. , et al. Action Recognition by Dense Trajectories Computer Vision and Pattern Recognition (CVPR) IEEE Conference on 2011 3169 3176 10.1109/cvpr.2011.5995407
- Ioffe , S. , Szegedy , C. 2015 448 456
- Zach , C. , Pock , T. , and Bischof , H. A Duality Based Approach for Realtime tv-L1 Optical Flow 29th DAGM Symposium on Pattern Recognition 2007 214 223 10.1007/978-3-540-74936-3_22
- Soomro , K. , Zamir , A.R. , and Shah , M. Ucflol: ADataset of 101 Human Actions Classes from Videos in the Wild Computer Science 2012