This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Development of a Camera-Based Driver State Monitoring System for Cost-Effective Embedded Solution
Technical Paper
2020-01-1210
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
To prevent the severe consequences of unsafe driving behaviors, it is crucial to monitor and analyze the state of the driver. Developing an effective driver state monitoring (DSM) systems is particularly challenging due to limited computation capabilities of embedded systems in automobiles and the need for finishing processing in real-time. However, most of the existing research work was conducted in a lab environment with expensive equipment while lacking in-car benchmarking and validation. In this paper, a DSM system that estimates driver's alertness and drowsiness level as well as performs emotion detection built with a cost-effective embedded system is presented. The proposed system consists of a mono camera that captures driver's facial image in real-time and a machine learning based detection algorithm that detects facial landmark points and use that information to infer driver's state. In the detection module, driver's distraction level is evaluated by estimating head-pose through solving a perspective-n-point problem, drowsiness level is estimated by processing eyelid related parameters extracted from facial keypoints data, and machine learning approach was used for emotion state monitoring. It is discovered for emotion recognition that using multilayer perceptron (MLP) network reached the accuracy of 93% on DISFA (Denver Intensity of Spontaneous Facial Action) dataset when combined with Action Units (AU) analysis. The performance of the developed DSM system has been verified in both laboratory and in-vehicle condition, and the experimental results showed its effectiveness in both normal and low lighting conditions. Moreover, the performance of the developed algorithm using commercially available low-cost cameras as well as memory and processing speed analysis considering the required embedded system design has also been examined.
Recommended Content
Journal Article | Advances in Experimental Vehicle Soiling Tests |
Technical Paper | Robust Validation Platform of Autonomous Capability for Commercial Vehicles |
Journal Article | Lateral Control for Automated Vehicle Following System in Urban Environments |
Authors
Topic
Citation
Zhou, X., Wingert, T., Sauer, M., and Kundu, S., "Development of a Camera-Based Driver State Monitoring System for Cost-Effective Embedded Solution," SAE Technical Paper 2020-01-1210, 2020, https://doi.org/10.4271/2020-01-1210.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 | ||
Unnamed Dataset 2 | ||
Unnamed Dataset 3 | ||
Unnamed Dataset 4 |
Also In
References
- Young , K. , Regan , M. , and Hammer , M.J.D.d. Driver Distraction: A Review of the Literature 2007 379 405 2007
- Horne , J.A. and Reyner , L.A.J.B. Sleep Related Vehicle Accidents 310 6979 565 567 1995
- Regan , M.A. , Lee , J.D. , and Young , K. Driver Distraction: Theory, Effects, and Mitigation CRC Press 2008
- Brown , I.D.J.H.f. Driver Fatigue 36 2 298 314 1994
- Cai , H. , Lin , Y. and Mourant , R.R. Study on Driver Emotion in Driver-Vehicle-Environment Systems Using Multiple Networked Driving Simulators Proceedings of the Driving Simulation Conference North America (DSC’07) 2007
- Kleinberg , R. 2000
- TOYOTA 2018
- Solutions , B.M.
- Sahayadhas , A. , Sundaraj , K. , and Murugappan , M.J.S. Detecting Driver Drowsiness Based on Sensors: A Review 12 12 16937 16953 2012
- Hu , S. and Zheng , G.J.E.S.w.A. Driver Drowsiness Detection with Eyelid Related Parameters by Support Vector Machine 36 4 7651 7658 2009
- Ahmad , R. , Borole , J.J.I.-I.J.o.C.S. , and Technologies , I. Drowsy Driver Identification Using Eye Blink Detection 6 1 270 274 2015
- Topham , G. The End of Road Rage? A Car which Detects Emotions 2018
- Tango , F. and Botta , M.J.I.T.o.I.T.S. Real-Time Detection System of Driver Distraction Using Machine Learning 14 2 894 905 2013
- Liu , T. , Yang , Y. , Huang , G.-B. , Yeo , Y.K. , and Lin , Z.J.I.t.o.i.t.s. Driver Distraction Detection Using Semi-Supervised Machine Learning 17 4 1108 1120 2016
- Kumar , A. and Patra , R. Driver Drowsiness Monitoring System Using Visual Behaviour and Machine Learning 2018 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE) 2018 339 344
- Chen , L.-l. , Zhao , Y. , Zhang , J. , and Z.,.J.-z.J.E.S.w.A. Automatic Detection of Alertness/Drowsiness from Physiological Signals Using Wavelet-Based Nonlinear Features and Machine Learning 42 21 7344 7355 2015
- Viola , P. and Jones , M. Rapid Object Detection Using a Boosted Cascade of Simple Features Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on 2001 1 I I
- King , D.E.J.J.o.M.L.R Dlib-Ml: A Machine Learning Toolkit 10 1755 1758 2009
- Bradski , G. and Kaehler , A.J.D.D.s.j.o.s.t. OpenCV 3 2000
- Moré , J.J. The Levenberg-Marquardt Algorithm: Implementation and Theory Numerical Analysis Springer 1978 105 116
- Dinges , D.F. and Grace , R.J.U.D.o.T 1998
- Alvaro , P.K. , Jackson , M.L. , Berlowitz , D.J. , Swann , P. , and Howard , M.E.J.J.o.c.s.m. Prolonged Eyelid Closure Episodes during Sleep Deprivation in Professional Drivers 12 08 1099 1103 2016
- Mavadati , S.M. , Mahoor , M.H. , Bartlett , K. , Trinh , P. , and C.,.J.F.J.I.T.o.A.C. Disfa: A Spontaneous Facial Action Intensity Database 4 2 151 160 2013
- Mandal , M. , Verma , M. , Mathur , S. , Vipparthi , S.K. et al. Regional Adaptive Affinitive Patterns (RADAP) with Logical Operators for Facial Expression Recognitio N 13 5 850 861 2019
- Shan , C. , Gong , S. , McOwan , P.W.J.I. , and Computing , V. Facial Expression Recognition Based on Local Binary Patterns: A Comprehensive Study 27 6 803 816 2009
- Lai , C.-C. , Ko , C.-H.J.O.-I.J.f.L. , and Optics , E. Facial Expression Recognition Based on Two-Stage Features Extraction 125 22 6678 6680 2014
- Jabid , T. , Kabir , M.H. , and Chae , O.J.E.j. Robust Facial Expression Recognition Based on Local Directional Pattern 32 5 784 794 2010
- Rivera , A.R. , Castillo , J.R. , and Chae , O.O.J.I.t.o.i.p. Local Directional Number Pattern for Face Analysis: Face and Expression Recognition 22 5 1740 1752 2012
- Rivera , A.R. , Castillo , J.R. , and Chae , O.J.P.R.L. Local Directional Texture Pattern Image Descriptor 51 94 100 2015
- Simonyan , K. and Zisserman , A.J.a.p.a. Very Deep Convolutional Networks for Large-Scale Image Recognition 2014
- He , K. , Zhang , X. , Ren , S. , and Sun , J. Deep Residual Learning for Image Recognition Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016 770 778
- Lucas , B.D. and Kanade , T. An Iterative Image Registration Technique with an Application to Stereo Vision 1981