As intelligent cockpit technology continues to evolve, the ways in which
information is presented and interacted with within vehicle systems are becoming
increasingly diverse, driving the development of driver-machine interaction
toward multi-modal integration, proactive sensing, and personalized responses.
As the core perception object of the intelligent cockpit, the accuracy of driver
state recognition directly impacts the intelligence level of cockpit interaction
and driving safety. In response to the increasing trend of task diversity and
behavioral response complexity in natural driving scenarios, there is an urgent
need to develop a driver multimodal data collection and processing tool with
high timeliness, non-intrusiveness, and multi-source synchronization
capabilities, serving as the key foundation for driver state modeling and
intelligent interaction support. Based on multiple resource theory (MRT) and
driver status perception mechanisms, this study designs and develops a
multi-modal driver behavior and vehicle driving state data collection and
processing apparatus tailored for natural driving scenarios. The apparatus
adopts a model-view-view model (MVVM) architecture to achieve functional module
decoupling, integrates hardware and software co-design, and incorporates key
modules such as visual attention detection, hand operation tracking, and vehicle
longitudinal and lateral driving state perception. It supports synchronous
collection, real-time processing, and instantaneous/task-level feature
extraction of multi-source heterogeneous data. The apparatus boasts excellent
scalability, deployment flexibility, and interface visualization capabilities,
making it suitable for typical intelligent cockpit application scenarios such as
driver behavior modeling, risk identification, distracted driving detection, and
human-machine interaction research. It enables comprehensive driver state
perception across the entire process of “information acquisition—operation
execution—behavior output,” providing high-quality data foundations and
methodological support for cognitive decision-making and personalized control
strategies in intelligent driving systems.