This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety
Technical Paper
2020-01-1211
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Driving inattention caused by drowsiness has been a significant reason for vehicle crash accidents, and there is a critical need to augment driving safety by monitoring driver drowsiness behaviors. For real-time drowsy driving awareness, we propose a vision-based driver drowsiness monitoring system (DDMS) for driver drowsiness behavior recognition and analysis. First, an infrared camera is deployed in-vehicle to capture the driver’s facial and head information in naturalistic driving scenarios, in which the driver may or may not wear glasses or sunglasses. Second, we propose and design a multi-modal features representation approach based on facial landmarks, and head pose which is retrieved in a convolutional neural network (CNN) regression model. Finally, an extreme learning machine (ELM) model is proposed to fuse the facial landmark, recognition model and pose orientation for drowsiness detection. The DDMS gives promptly warning to the driver once a drowsiness event is detected. The proposed CNN and ELM models are trained in a drowsy driving dataset and are validated on public datasets and field tests. Comparing to the end-to-end CNN recognition model, the proposed multi-modal fusion with the ELM detection model allows faster and more accurate detection with minimal intervention. The experimental result demonstrates that DDMS is able to provide real-time and effective drowsy driving alerts under various light conditions to augment driving safety.
Recommended Content
Authors
Topic
Citation
Li, R., Brand, H., Gopinath, A., Kamarajugadda, S. et al., "Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety," SAE Technical Paper 2020-01-1211, 2020, https://doi.org/10.4271/2020-01-1211.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 |
Also In
References
- A. F. for Traffic Safety 2019
- NHTSA 2017
- Yong , L.C. , Li , J. , and Calvert , G.M. Sleep-Related Problems in the US Working Population: Prevalence and Association with Shiftwork Status Occup Environ Med 74 2 93 104 2017
- Klauer , S.G. , Dingus , T.A. , Neale , V.L. , Sudweeks , J.D. , Ramsey , D.J. et al. 2006
- https://www.nsc.org/road-safety/safety-topics/fatigued-driving Oct. 21, 2019
- A. F. for Traffic Safety 2016
- Li , B. , Munoz , J.P. , Rong , X. , Chen , Q. et al. Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People IEEE Transactions on Mobile Computing 18 3 702 714 2018
- Li , B. , Munoz , J.P. , Rong , X. , Xiao , J. , Tian , Y. , and Arditi , A. Isana: Wearable Context-Aware Indoor Assistive Navigation with Obstacle Avoidance for the Blind European Conference on Computer Vision 2016 448 462
- Li , R. , Wang , W. , Chen , Y. , Srinivasan , S. , and Krovi , V.N. An End-To-End Fully Automatic Bay Parking Approach for Autonomous Vehicles ASME 2018 Dynamic Systems and Control Conference 2018
- Lin , C. , Chang , C. , Lin , B. , Hung , S. et al. A Real-Time Wireless Brain-Computer Interface System for Drowsiness Detection IEEE Transactions on Biomedical Circuits and Systems 4 4 214 222 2010
- Hu , S. and Zheng , G. Driver Drowsiness Detection With Eyelid Related Parameters by Support Vector Machine Expert Systems with Applications 36 4 7651 7658 2009
- Li , G. , Lee , B. , and Chung , W. Smartwatch-Based Wearable EEG System for Driver Drowsiness Detection IEEE Sensors Journal 15 12 7169 7180 2015
- Vicente , J. , Laguna , P. , Bartra , A. , and Bailón , R. Drowsiness Detection Using Heart Rate Variability Medical & Biological Engineering & Computing 54 6 927 937 2016
- Garcia , I. , Bronte , S. , Bergasa , L.M. , Almazán , J. , and Yebes , J. Vision-Based Drowsiness Detector for Real Driving Conditions 2012 IEEE Intelligent Vehicles Symposium 2012 618 623
- Danisman , T. , Bilasco , I.M. , Djeraba , C. , and Ihaddadene , N. Drowsy driver detection system using eye blink patterns 2010 International Conference on Machine and Web Intelligence 2010 230 233
- Omidyeganeh , M. , Javadtalab , A. , and Shirmohammadi , S. Intelligent Driver Drowsiness Detection Through Fusion of Yawning and Eye Closure 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings 2011 1 6
- Choi , I.-H. and Kim , Y.-G. Head Pose and Gaze Direction Tracking for Detecting a Drowsy Driver 2014 international conference on big data and smart computing (BIGCOMP) 2014 241 244
- Huang , G. , Zhu , Q. , and Siew , C. Extreme Learning Machine: Theory and Applications Neurocomputing 70 1-3 489 501 2006
- Huang , G. , Zhou , H. , Ding , X. , and Zhang , R. Extreme Learning Machine for Regression and Multiclass Classification IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42 2 513 529 2011
- Wang , W. , Li , R. , Chen , Y. , and Jia , Y. Human Intention Prediction in Human-Robot Collaborative Tasks Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction 2018 279 280
- Dwivedi , K. , Biswaranjan , K. , and Sethi , A. Drowsy Driver Detection Using Representation Learning 2014 IEEE International Advance Computing Conference (IACC) 2014 995 999
- Park , S. , Pan , F. , Kang , S. , and Yoo , C.D. Driver Drowsiness Detection System Based on Feature Representation Learning Using Various Deep Networks Asian Conference on Computer Vision 2016 154 164
- Milborrow , S. and Nicolls , F. Locating Facial Features with an Extended Active Shape Model European Conference on Computer Vision 2008 504 513
- Abtahi , S. , Hariri , B. , and Shirmohammadi , S. Driver drowsiness monitoring based on yawning detection 2011 IEEE International Instrumentation and Measurement Technology Conference 2011 1 4
- Kazemi , V. and Sullivan , J. One Millisecond Face Alignment with an Ensemble of Regression Trees Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2014 1867 1874
- Patacchiola , M. and Cangelosi , A. Head Pose Estimation in the Wild Using Convolutional Neural Networks and Adaptive Gradient Methods Pattern Recognition 71 132 143 2017
- Weng , C. , Lai , Y. , and Lai , S. Driver Drowsiness Detection Via a Hierarchical Temporal Deep Belief Network Asian Conference on Computer Vision 2016 117 133
- Hastie , T. , Tibshirani , R. , Friedman , J. , and Franklin , J. The Elements of Statistical Learning: Data Mining, Inference and Prediction The Mathematical Intelligencer 27 2 83 85 2005
- Huang , G. and Chen , L. Enhanced Random Search Based Incremental Extreme Learning Machine Neurocomputing 71 16-18 3460 3468 2008
- Huang , G. and Chen , L. Convex Incremental Extreme Learning Machine Neurocomputing 70 16-18 3056 3062 2007