This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Driving inattention caused by drowsiness has been a significant reason for vehicle crash accidents, and there is a critical need to augment driving safety by monitoring driver drowsiness behaviors. For real-time drowsy driving awareness, we propose a vision-based driver drowsiness monitoring system (DDMS) for driver drowsiness behavior recognition and analysis. First, an infrared camera is deployed in-vehicle to capture the driver’s facial and head information in naturalistic driving scenarios, in which the driver may or may not wear glasses or sunglasses. Second, we propose and design a multi-modal features representation approach based on facial landmarks, and head pose which is retrieved in a convolutional neural network (CNN) regression model. Finally, an extreme learning machine (ELM) model is proposed to fuse the facial landmark, recognition model and pose orientation for drowsiness detection. The DDMS gives promptly warning to the driver once a drowsiness event is detected. The proposed CNN and ELM models are trained in a drowsy driving dataset and are validated on public datasets and field tests. Comparing to the end-to-end CNN recognition model, the proposed multi-modal fusion with the ELM detection model allows faster and more accurate detection with minimal intervention. The experimental result demonstrates that DDMS is able to provide real-time and effective drowsy driving alerts under various light conditions to augment driving safety.
CitationLi, R., Brand, H., Gopinath, A., Kamarajugadda, S. et al., "Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety," SAE Technical Paper 2020-01-1211, 2020, https://doi.org/10.4271/2020-01-1211.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- A. F. for Traffic Safety , “2018 Traffic Safety Culture Index,” Traffic Safety Culture Index, 2019.
- NHTSA , “Drowsy Driving 2015,” Traffic Safety Facts, 2017.
- Yong, L.C., Li, J., and Calvert, G.M. , “Sleep-Related Problems in the US Working Population: Prevalence and Association with Shiftwork Status,” Occup Environ Med 74(2):93-104, 2017.
- Klauer, S.G., Dingus, T.A., Neale, V.L., Sudweeks, J.D., Ramsey, D.J. et al. , “The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data,” 2006.
- “Drowsy Driving Is Impaired Driving,” https://www.nsc.org/road-safety/safety-topics/fatigued-driving, accessed Oct. 21, 2019.
- A. F. for Traffic Safety , “2015 Traffic Safety Culture Index,” 2016.
- Li, B., Munoz, J.P., Rong, X., Chen, Q. et al. , “Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People,” IEEE Transactions on Mobile Computing 18(3):702-714, 2018.
- Li, B., Munoz, J.P., Rong, X., Xiao, J., Tian, Y., and Arditi, A. , “Isana: Wearable Context-Aware Indoor Assistive Navigation with Obstacle Avoidance for the Blind,” in European Conference on Computer Vision, 2016, 448-462, Springer.
- Li, R., Wang, W., Chen, Y., Srinivasan, S., and Krovi, V.N. , “An End-To-End Fully Automatic Bay Parking Approach for Autonomous Vehicles,” in ASME 2018 Dynamic Systems and Control Conference, American Society of Mechanical Engineers Digital Collection, 2018.
- Lin, C., Chang, C., Lin, B., Hung, S. et al. , “A Real-Time Wireless Brain-Computer Interface System for Drowsiness Detection,” IEEE Transactions on Biomedical Circuits and Systems 4(4):214-222, 2010.
- Hu, S. and Zheng, G. , “Driver Drowsiness Detection With Eyelid Related Parameters by Support Vector Machine,” Expert Systems with Applications 36(4):7651-7658, 2009.
- Li, G., Lee, B., and Chung, W. , “Smartwatch-Based Wearable EEG System for Driver Drowsiness Detection,” IEEE Sensors Journal 15(12):7169-7180, 2015.
- Vicente, J., Laguna, P., Bartra, A., and Bailón, R. , “Drowsiness Detection Using Heart Rate Variability,” Medical & Biological Engineering & Computing 54(6):927-937, 2016.
- Garcia, I., Bronte, S., Bergasa, L.M., Almazán, J., and Yebes, J. , “Vision-Based Drowsiness Detector for Real Driving Conditions,” in 2012 IEEE Intelligent Vehicles Symposium, 2012, 618-623, IEEE.
- Danisman, T., Bilasco, I.M., Djeraba, C., and Ihaddadene, N. , “Drowsy driver detection system using eye blink patterns,” in 2010 International Conference on Machine and Web Intelligence, 2010, 230-233, IEEE.
- Omidyeganeh, M., Javadtalab, A., and Shirmohammadi, S. , “Intelligent Driver Drowsiness Detection Through Fusion of Yawning and Eye Closure,” in 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings, 2011, 1-6, IEEE.
- Choi, I.-H. and Kim, Y.-G. , “Head Pose and Gaze Direction Tracking for Detecting a Drowsy Driver,” in 2014 international conference on big data and smart computing (BIGCOMP), 2014, 241-244, IEEE.
- Huang, G., Zhu, Q., and Siew, C. , “Extreme Learning Machine: Theory and Applications,” Neurocomputing 70(1-3):489-501, 2006.
- Huang, G., Zhou, H., Ding, X., and Zhang, R. , “Extreme Learning Machine for Regression and Multiclass Classification,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42(2):513-529, 2011.
- Wang, W., Li, R., Chen, Y., and Jia, Y. , “Human Intention Prediction in Human-Robot Collaborative Tasks,” in Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, 2018, 279-280.
- Dwivedi, K., Biswaranjan, K., and Sethi, A. , “Drowsy Driver Detection Using Representation Learning,” in 2014 IEEE International Advance Computing Conference (IACC), 2014, 995-999, IEEE.
- Park, S., Pan, F., Kang, S., and Yoo, C.D. , “Driver Drowsiness Detection System Based on Feature Representation Learning Using Various Deep Networks,” in Asian Conference on Computer Vision, 2016, 154-164, Springer.
- Milborrow, S. and Nicolls, F. , “Locating Facial Features with an Extended Active Shape Model,” European Conference on Computer Vision, 2008, 504-513, Springer.
- Abtahi, S., Hariri, B., and Shirmohammadi, S. , “Driver drowsiness monitoring based on yawning detection,” in 2011 IEEE International Instrumentation and Measurement Technology Conference, 2011, 1-4, IEEE.
- Kazemi, V. and Sullivan, J. , “One Millisecond Face Alignment with an Ensemble of Regression Trees,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 1867-1874.
- Patacchiola, M. and Cangelosi, A. , “Head Pose Estimation in the Wild Using Convolutional Neural Networks and Adaptive Gradient Methods,” Pattern Recognition 71:132-143, 2017.
- Weng, C., Lai, Y., and Lai, S. , “Driver Drowsiness Detection Via a Hierarchical Temporal Deep Belief Network,” in Asian Conference on Computer Vision, 2016, 117-133, Springer.
- Hastie, T., Tibshirani, R., Friedman, J., and Franklin, J. , “The Elements of Statistical Learning: Data Mining, Inference and Prediction,” The Mathematical Intelligencer 27(2):83-85, 2005.
- Huang, G. and Chen, L. , “Enhanced Random Search Based Incremental Extreme Learning Machine,” Neurocomputing 71(16-18):3460-3468, 2008.
- Huang, G. and Chen, L. , “Convex Incremental Extreme Learning Machine,” Neurocomputing 70(16-18):3056-3062, 2007.