Neural Surrogates and Reinforcement Learning for Heavy-Duty Diesel Torque Control

Accurate torque-trace reproduction on regulatory drive cycles is central to heavy-duty diesel certification and development testing. Conventional controllers such as Proportional Integral Derivative (PID or PI) can be enhanced with gain scheduling and feedforward (FF) maps to satisfy requirements but require extensive calibration and are sensitive to nonlinearities and delay. This paper evaluates a data-driven control framework comprising a recurrent neural surrogate of engine torque (specifically an LSTM – long short-term memory) trained on engine/dynamometer data and a reinforcement learning (RL) policy trained using this surrogate (“world model”) to track requested torque while regularizing control effort. The RL policy (specifically TD3 – twin delayed deep deterministic) is benchmarked against tuned PID and PID+FF baselines on the Environmental Protection Agency’s Heavy Duty Federal Test Procedure (HD-FTP) segments using EPA regression criteria (slope, |intercept|, R²) and tracking metrics (mean absolute error - MAE, root mean square error - RMSE). TD3 reduced mean absolute error (MAE) by 58% (from 91.34 to 38.69 N·m) and root mean square error (RMSE) by 54% (from 123.5 to 56.73 N·m), improved regression to slope = .9978, |intercept| = 7.13 N·m, R² = .9844, and cut the 95th-percentile absolute error by 60% compared to the PID+FF controller (the next best performing controller – in all categories). Results show the RL controller improves responsiveness and accuracy relative to autotuned PID+FF on the surrogate model, while reducing manual calibration effort. The approach is modular and engine-agnostic (retrain surrogate and policy) and is amenable to multi-objective extensions that incorporate emissions proxies in the reward.