Towards Standardized Performance Evaluation of Camera-Based Driver State Sensing Technologies

Driver state sensing technologies start to be widely used in vehicular systems developed from different manufacturers. To optimize the cost and minimize the intrusiveness towards driving, majority of these systems rely on in-cabin camera(s) and other optical sensors. With their great capabilities of detecting and intervening driver distraction and inattention, these technologies might become key components in future vehicle safety and control systems. However, currently there are no common standards available to compare the performance of these technologies, thus it is necessary to develop one standardized process for the evaluation purpose. In this paper, we propose one standardized systematic evaluation process after successfully addressing three difficulties: (1) defining and selecting the important influential individual and environmental factors, (2) countering the effects of individual differences and randomness in driver behaviors, and (3) building a reliable in-vehicle driver head motion tracking tool to collect ground-truth motion data. We have completed one large-scale data collection towards a commercial driver state-sensing platform. For each subject, 30 to 40 minutes of head motion data were collected covering one full factorial design of variables of lighting conditions, head/face features, and camera locations. The collected data were analyzed based on a proposed performance measure. The whole process can not only efficiently evaluate individual camera-based driver state sensing products, but also builds one common base for comparing performance of different systems.