Multimodal Intelligent Transportation Data Fusion and Application Methods in Heavy-Duty Vehicles

Heavy-duty commercial vehicles (HDCVs) are the key mobile nodes in intelligent transportation systems (ITS). However, their complex operating conditions and the diversity of data sources (such as road conditions, driver behavior, traffic signals, and on-board sensors) present considerable difficulties for accurately estimating the state and perceiving the environment using a single modality of data. This requires effective multi-modal data fusion to enhance the control and decision-making capabilities of HDCVs. This paper addresses this need by proposing a customized multi-modal intelligent transportation data fusion framework for intelligent HDCVs. This paper presents a solution for establishing a multi-modal intelligent transportation data collection platform, including real-scene collection methods and simulation scene collection methods based on the SUMO-MATLAB joint simulation platform. Through three representative case studies, the application methods of multi-modal traffic data are demonstrated: vehicle speed prediction, vehicle power demand prediction, and trajectory planning. The hyperparameter optimization using an enhanced LSTM neural network with the Sparrow Search Algorithm (SSA) is achieved, resulting in more adaptable, safer, and more efficient multi-modal intelligent transportation data applications.