This content is not included in your SAE MOBILUS subscription, or you are not logged in.
In Vehicle Diver Postural Monitoring using a Depth Camera Kinect
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 03, 2018 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Monitoring driver’s postures has extensive applications. The postural information could be used for the development of smart airbags, for detecting possible fatigue in long travel and for recognizing activities which may determine if the driver has enough time to take over the control in an intelligent vehicle when encountering hazardous situations. Microsoft Kinect is one of the best candidates for monitoring driver’s posture thanks to its innovative feature of real time motion capture without use of markers and its low cost. However, when body parts are partially occluded, the accuracy of Kinect data will drop markedly. Inspired by previous researches, the present work focused on testing a data driven approach for improving driver’s upper body movement reconstruction with a Kinect camera. Firstly, we organize a database of accurately captured driver poses from different motion clips with filtered structures, including a global motion graph and multiple local motion graphs. The reliability of individual skeletal joint from Kinect posture is objectively evaluated to work with weighted KNN to extract similar postures from the prior database. We then apply PCA to create an intermediary posture space where Simulated Annealing Algorithm is utilized to synthesize natural and kinematically valid postures. Kalman Filter is employed to help select the perspective posture candidates as well as to ensure temporal motion continuity across frames. The preliminary results show that the implemented method could remarkably improve the quality of reconstructed driver’s postures from Kinect and the framework has a great potential to be used to monitor the driver.
CitationZhao, M., Beurier, G., Wang, H., and Wang, X., "In Vehicle Diver Postural Monitoring using a Depth Camera Kinect," SAE Technical Paper 2018-01-0505, 2018, https://doi.org/10.4271/2018-01-0505.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- Hancock, P.A., Simmons, L., Hashemi, L., Howarth, H. et al. , “The Effects of in-Vehicle Distraction on Driver Response during a Crucial Driving Maneuver,” Transportation Human Factors (4):295-309, 1999.
- Casner, S.M., Hutchins, E.L., and Norman, D. , “The Challenges of Partially Automated Driving,” Communications of the ACM 59(5):70, 2016.
- Norman, D.A. , “The Human Side of Automation,” . In: Road Vehicle Automation 2. (Springer International Publishing, 2015), 73-79.
- Stanton, N.A., Young, M., and Mccaulder, B. , “Drive-by-wire: The case of driver workload and reclaiming control with adaptive cruise control,” Safety Science 27(2-3):149-159, 1997.
- Dick de Waard, et al. "Driver Behavior in an Emergency Situation in the Automated Highway System." Transportation Human Factors 1.1: 87-89, 1999.
- Winter, J. C. F. De, and D. Dodou . "Preparing Drivers for Dangerous Situations: A Critical Reflection on Continuous Shared Control." IEEE International Conference on Systems, Man, and Cybernetics IEEE, 1050-1056, 2011.
- Seppelt, B.D. and Victor, T.W. , “Potential Solutions to Human Factors Challenges in Road Vehicle Automation,” . In: Road Vehicle Automation 3. (Springer International Publishing, 2016).
- Cheng, S.Y., Park, S., and Trivedi, M.M. , “Multi-Spectral and Multi-Perspective Video Arrays for Driver Body Tracking and Activity Analysis,” Computer Vision and Image Understanding 106(2):245-257, 2007.
- E. Ohn-Bar, A. Tawari, S. Martin, and M. M. Trivedi , “Predicting Driver Maneuvers by Learning Holistic Features,” in IEEE Intelligent Vehicles Symposium, 2014.
- S. Klauer, F. Guo, J. Sudweeks, and T. Dingus , “An Analysis of Driver Inattention Using a Case-Crossover Approach on 100-Car Data: Final Report,” National Highway Traffic Safety Administration, Washington, D.C., Tech. Rep. DOT HS 811 334, 2010.
- Klomark, Marcus . "Occupant Detection Using Computer Vision." Master Thesis, Electrical Engineering Department, Institute of Technology, Linköping University, 2000.
- Faber, P. “A System Architecture for an Intelligent Airbag Deployment,” no Mi.: 70-75, 2000.
- Sammonds, G.M., Fray, M., and Mansfield, N.J. , “Effect of Long Term Driving on Driver Discomfort and its Relationship with Seat Fidgets and Movements (SFMs),” Applied Ergonomics 58:119-127, 2017.
- Wahlstrom, Eric, Osama Masoud, and Nikos Papanikolopoulos , “Vision-Based Methods for Driver Monitoring.” In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2:903-8. Institute of Electrical and Electronics Engineers Inc. 2003, doi:10.1109/ITSC.2003.1252617.
- Smith, Paul, Mubarak Shah, and Niels da Vitoria Lobo . "Determining Driver Visual Attention With One Camera." IEEE Transactions on Intelligent Transportation Systems 4.4: 205-218, 2003.Tran, Cuong, and Mohan M. Trivedi. 2010. “Towards a Vision-Based System Exploring 3D Driver Posture Dynamics for Driver Assistance: Issues and Possibilities.” In IEEE Intelligent Vehicles Symposium, Proceedings, 179-84. doi:10.1109/IVS.2010.5547957.
- Kondyli, Alexandra, Virginia Sisiopiku, and Angelos Barmpoutis . "A 3D Experimental Framework for Exploring drivers' Body Activity Using Infrared Depth Sensors." Connected Vehicles and Expo (ICCVE), 2013 International Conference on. IEEE, 2013.
- Kondyli A, Sisiopiku VP, Zhao L, Barmpoutis A . “Comparative Analysis of Driving Maneuvers Using 3D Body Posture Data,” In Proceedings of the 5th International Conference on Applied Human Factors Ergonomics AHFE 2014.
- Kondyli, A., Sisiopiku, V.P., Zhao, L., and Barmpoutis, A. , “Computer Assisted Analysis of Drivers' Body Activity Using a Range Camera,” IEEE Intelligent Transportation Systems Magazine 7(3):18-28, 2015.
- Kean, S., Hall, J., and Perry, P. , “Meet the Kinect: An Introduction to Programming Natural User Interfaces,” (Apress, 2011).
- Freedman, B.; Shpunt, A.; Machline, M.; Arieli, Y. “Depth Mapping Using Projected Patterns,”U.S. Patent 2010/0118123, 13 May 2010.
- Berliner, T., Z. Hendel, A. Shpunt, J. Danowitz, et al. , “Modelling of Humanoid forms from depth maps,”US Patent Application Publication n. US 2010/0034457 A1.
- Shotton, J., A. Fitzgibbon, M. Cook, T. Sharp, et al. , “Real-Time Human Pose Recognition in Parts from Single Depth Images.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, June 20-25, edited by Nicolas Pinto, 1297-1304, 2011. Providence, RI.
- Auvinet E, Multon F, Meunier J , “Lower Limb Movement Asymmetry Measurement with a Depth Camera,” IN: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp 6793-6796, 2012
- Auvinet, E., Multon, F., Aubin, C.E., Meunier, J. et al. , “Detection of Gait Cycles in Treadmill Walking Using a Kinect,” Gait & Posture, 2014.
- Cassola F, Morgado L, de Carvalho F, Paredes H, et al. , “Online-Gym: A 3d Virtual Gymnasium Using Kinect Interaction,” Procedia Technology 13(0): 130-138, 2014.
- Diego-Mas, J.A. and Alcaide-Marzal, J. , “Using Kinect Sensor in Observational Methods for Assessing Postures at Work,” Applied Ergonomics 45(4):976-985, 2014.
- Galna, B., Barry, G., Jackson, D., Mhiripiri, D. et al. , “Accuracy of the Microsoft Kinect Sensor Formeasuring Movement in People with Parkinson’s Disease,” Gait & Posture 39(4):1062-1068, 2014.
- Huang, S.Y., Kuo, K.P., and Lin, Y.H. , “A golf swing analysis system using wii balance board and kinect sensors for novice players,” Multimedia Tools and Applications 74(23):10679-10696, 2015.
- Vignais, N., Miezal, M., Bleser, G., Mura, K. et al. , “Innovative system for real-time ergonomic feedback in industrial manufacturing,” Applied Ergonomics 44(4):566-574, 2013.
- Kurillo, G., Chen, A., Bajcsy, R., and Han, J.J. , “Evaluation of Upper Extremity Reachable Workspace Using Kinect Camera,” Technology and Health Care 21:641-656, 2013.
- Bonnech ere, B., Jansen, B., Salvia, P., Bouzahouene, H. et al. , “Validity and Reliability of the Kinect within Functional Assessment Activities: Comparison with Standard Stereo-Photogrammetry,” Gait & Posture 39(1):593-598, 2014.
- Xu, X. and McGorry, R.W. , “The Validity of the First and Second-Generation Microsoft Kinect™ for Identifying Joint Center Locations during Static Postures,” Applied Ergonomics 49:47-54, 2015.
- Plantard, P., Auvinet, E., Le Pierres, A.-S., and Multon, F. , “Pose Estimation with a Kinect for Ergonomic Studies: Evaluation of the Accuracy Using a Virtual Mannequin,” Sensors 15:1785-1803, 2015.
- Shum, H.P.H., Ho, E.S., Jiang, Y., and Takagi, S. , “Real-Time Posture Reconstruction for Microsoft Kinect,” Cybernetics. IEEE Transactions 43(5):1357-1369, 2013.
- Zhou, L., Liu, Z., Leung, H., Shum, H.P.H. , “Posture reconstruction using Kinect with a probabilistic model,” In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, 117-125, 2014.
- Plantard, P., Shum, H.P.H., and Multon, F. , “Filtered Pose Graph for Efficient Kinect Pose Reconstruction,” Multimedia Tools and Applications 1-22, 2016.
- Galna, B., Barry, G., Dan, J., Mhiripiri, D. et al. , “Accuracy of the Microsoft Kinect Sensor for Measuring Movement in People with parkinson's Disease,” Gait & Posture 39(4):1062, 2014.
- Herrera C. , D., Kannala, J., Heikkilä, J. , " Joint Depth and Color Camera Calibration with Distortion Correction", TPAMI, 2012.
- Ingber, L . Adaptive simulated annealing (ASA): Lessons learned. Invited Paper to a Special Issue of the Polish Journal Control and Cybernetics on “Simulated Annealing Applied to Combinatorial Optimization” 1995. Available from http://www.ingber.com/asa96_lessons.ps.gz
- Kalman, R.E. , “A New Approach to Linear Filtering and Prediction Problems,” Transactions of the ASME - Journal of Basic Engineering 82:34-45, 1960.
- G Bishop, G Welch , “An introduction to the Kalman filter,” in SIGGRAPH 2001, Course 8, Los Angeles, CA, USA (August 2001).
- Pagliari, D. and Pinto, L. , “Calibration of Kinect for Xbox one and Comparison between the two Generations of Microsoft Sensors,” Sensors 15(11):27569-27589, 2015.