This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
A Robust Failure Proof Driver Drowsiness Detection System Estimating Blink and Yawn
Technical Paper
2020-01-1030
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
The fatal automobile accidents can be attributed to fatigued and distracted driving by drivers. Driver Monitoring Systems alert the distracted drivers by raising alarms. Most of the image based driver drowsiness detection systems face the challenge of failure proof performance in real time applications. Failure in face detection and other important part (eyes, nose and mouth) detections in real time cause the system to skip detections of blinking and yawning in few frames. In this paper, a real time robust and failure proof driver drowsiness detection system is proposed. The proposed system deploys a set of detection systems to detect face, blinking and yawning sequentially. A robust Multi-Task Convolutional Neural Network (MTCNN) with the capability of face alignment is used for face detection. This system attained 97% recall in the real time driving dataset collected. The detected face is passed on to ensemble of regression trees to detect the 68 facial landmarks. The eye and mouth landmarks are isolated to detect the blink and yawning by identifying the open/closed state by calculating their aspect ratios. The proposed system combines the two detectors with failure proof strategies. A Minimization of Sum Squared Error based object tracker is used to track the face in the video when the face detector fails. A new binary classification neural network is deployed to reinforce the detection of eye and mouth state when the face landmark detection fails. A combination of the face, landmark detector and the failure proof systems such as the object tracker and classification neural network provides a driver drowsiness detection system with nearly 99.9% accuracy in detecting blinking and yawning.
Citation
Jesudoss, Y. and Park, J., "A Robust Failure Proof Driver Drowsiness Detection System Estimating Blink and Yawn," SAE Technical Paper 2020-01-1030, 2020, https://doi.org/10.4271/2020-01-1030.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 | ||
| Unnamed Dataset 3 | ||
| Unnamed Dataset 4 | ||
| Unnamed Dataset 5 | ||
| Unnamed Dataset 6 |
Also In
References
- Arnold , L.S. and Tefft , B.C. Prevalence of Self - Reported Drowsy Driving , United States : 2015 AAA Foundation for Traffic Safety 2017
- National Center for Statistics and Analysis 2017
- Sahayadhas , A. , Sundaraj , K. , and Murugappan , M. Detecting Driver Drowsiness Based on Sensors: A Review Sensors 12 12 16937 16953 2012 10.3390/s121216937
- Vicente , J. , Laguna , P. , Bartra , A. , and Bailon , R. Drowsiness Detection Using Heart Rate Variability Med Biol Eng Comput. 54 927 937 2016 10.1007/s11517-015-1448-7
- Lin , F.-C. , Ko , L.-W. , Chuang , C.-H. , Tung-Ping , S. , and Lin , C.-T. Generalized EEG-Based Drowsiness Prediction System by Using a Self-Organizing Neural Fuzzy System IEEE Transactions on Circuits and Systems I: Regular Papers 59 9 2044 2055 2012 10.1109/tcsi.2012.2185290
- Morris , D.M. , Pilcher , J.J. , and Switzer , F.S. Lane Heading Difference: An Innovative Model for Drowsy Driving Detection Using Retrospective Analysis around Curves Accident Analysis & Prevention 80 2015 117 24 10.1016/j.aap.2015.04.007
- Li , K. , Wang , S. , Chang , D. , Huang , Y. et al. Accurate Fatigue Detection Based on Multiple Facial Morphological Features Journal of Sensors 2019 1 10 2019 10.1155/2019/7934516
- Knapik , M. and Cyganek , B. Driver’s Fatigue Recognition Based on Yawn Detection in Thermal Images Neurocomputing 338 274 292 2019 10.1016/j.neucom.2019.02.014
- Zhang , Z. and Zhang , J. J. Control Theory Appl. 8 181 2010 https://doi.org/10.1007/s11768-010-8043-0
- Soukupova , T. and Cech , J. 2016
- Alioua , N. , Amine , A. , and Rziza , M. Driver’s Fatigue Detection Based on Yawning Extraction International Journal of Vehicular Technology 2014 2014 https://doi.org/10.1155/2014/678786
- Sun , X. , Wu , P. , and Hoi, S. C.h. Face Detection Using Deep Learning: An Improved Faster RCNN Approach Neurocomputing 299 2018 42 50 10.1016/j.neucom.2018.03.030
- Schroff , F. , Kalenichenko , D. , and Philbin , J. FaceNet: A Unified Embedding for Face Recognition and Clustering 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015 10.1109/cvpr.2015.7298682
- Zhang , K. and Zhang , Z. and Li , Z. and Qiao , Y. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks 2016 IEEE Signal Processing Letters 23 10.1109/LSP.2016.2603342
- Simonyan , K. and Zisserman , A. Very Deep Convolutional Networks for Large-Scale Image Recognition CVPR 2014
- Yang , S. , Luo , P. , Loy , C.C. , Tang , X. WIDER FACE: A Face Detection Benchmark IEEE conference on Computer Vision and Pattern Recognition (CVPR) 2016
- Do , N.-T. , Kim , S.-H. , Yang , H.-J. , Lee , G.-S. , and Na , I.-S. Face Tracking with Convolutional Neural Network Heat-map Proceedings of the 2nd International Conference on Machine Learning and Soft Computing - ICMLSC 18 February 2018 10.1145/3184066.3184081
- Jain , V. and Learned-Miller , E. 2010
- Ågren , S. 2017
- Lukežič , A. , Vojíř , T. , Zajc , L.Č. , Matas , J. , and Kristan , M. Discriminative Correlation Filter Tracker with Channel and Spatial Reliability International Journal of Computer Vision 126 7 671 688 August 2018 https://doi.org/10.1007/s11263-017-1061-3
- Uzkent , B. , and Seo , Y. EnKCF: Ensemble of Kernelized Correlation Filters for High-Speed Object Tracking 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) 2018
- Bolme , D. , Beveridge , J.R. , Draper , B.A. , Lui , Y.M. Visual Object Tracking using Adaptive Correlation Filters 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition 10.1109/cvpr.2010.5539960
- Kazemi , V. , and Sullivan , J. One Millisecond Face Alignment with an Ensemble of Regression Trees 2014 IEEE Conference on Computer Vision and Pattern Recognition 2014 https://doi.org/10.1109/cvpr.2014.241
- Xiong , X. , and De La Torre , F. Supervised Descent Method and Its Applications to Face Alignment 2013 IEEE Conference on Computer Vision and Pattern Recognition 2013 https://doi.org/10.1109/cvpr.2013.75
- Faithfull , W.J. 2015 https://will.faithfull.me/blinks-dataset/
- Yang , S. , Luo , P. , Loy , C.-C. , and Tang , X. From Facial Parts Responses to Face Detection: A Deep Learning Approach 2015 IEEE International Conference on Computer Vision (ICCV) 2015 https://doi.org/10.1109/iccv.2015.419
- Diederik , P. , Jimmy , B.A. December 2014
- Chen , D. , Ren , S. , Wei , Y. , Cao , X. , and Sun , J. Joint Cascade Face Detection and Alignment Computer Vision - ECCV 2014 Lecture Notes in Computer Science 109 122 2014 https://doi.org/10.1007/978-3-319-10599-4_8
- Asthana , A. , Zafeiriou , S. , Cheng , S. , and Pantic , M. Incremental Face Alignment in the Wild 2014 IEEE Conference on Computer Vision and Pattern Recognition 2014 https://doi.org/10.1109/cvpr.2014.240
- Abtahi , S. , Omidyeganeh , M. , Shirmohammadi , S. , and Hariri , B. YawDD: A Yawning Detection Dataset Singapore Proc. ACM Multimedia Systems 24 28 March 19-21 2014 10.1145/2557642.2563678