Image-Based Driver Status Monitoring System for Determining the Transfer of Dynamic Driving Tasks in Autonomous Vehicles

Autonomous driving technology has advanced significantly in recent years, thanks to the innovations in vehicle design and artificial intelligence. This technology provides a more convenient and comfortable driving experience for both drivers and passengers. However, current autonomous driving systems still face some difficulties, such as the need for human intervention in critical situations. Switching control from the system to the driver without assessing their alertness or health status could pose a risk. Therefore, autonomous driving systems should be able to monitor and evaluate the driver's condition before handing over the control. This paper thus proposes an in-cabin detection system that monitors driver status using a camera. The system consists of software modules that use deep learning techniques to analyze the input from an IR camera. The modules include face detection, head pose estimation, eye state detection, and gaze estimation, which are used to evaluate the driver's condition. The system runs on an edge computing platform, with an average processing speed of about 10 FPS, 99% accuracy for driver availability detection, about 100% accuracy for driver unavailability detection and 100% warning rate for driver presence. The detection results are displayed visually on screen and applied to the testing of self-driving electric bus. It allows the vehicle to perform conditional autonomous driving, which is Level 3. This means that the system can assume the responsibility of driving the vehicle and comply with the UN R157 regulations that require the driver to be present and ready to intervene.