Neural Surrogates and Reinforcement Learning for Heavy-Duty Diesel Torque Control: A Data-Driven Alternative to PID

Accurately reproducing the torque trace of regulatory drive cycles is central to heavy-duty diesel emissions certification and repeatable development testing. Conventional controllers—PI/PID with gain scheduling and feedforward mapping—can meet requirements but demand extensive hand-tuning and struggle with strong nonlinearities and time-varying delay. Building on prior work that quantified the limits and benefits of gain-scheduled and feedforward strategies under the FTP transient cycle, this work investigates a fully data-driven control framework. First, a recurrent neural network (LSTM) torque surrogate is trained on dynamometer data to capture engine dynamics from readily available signals (e.g., engine speed, commanded pedal/torque, recent torque history) and actuator constraints. Second, this learned “world model” is used to train a reinforcement learning (RL) policy that outputs pedal commands to minimize torque-tracking error while regularizing control effort and respecting torque slew limits. The resulting policy is benchmarked against tuned PI and gain-scheduled/feedforward baselines on segments of the FTP cycle, using the EPA’s regression-based criteria (slope, intercept, R2) and standard tracking metrics. Results show that (i) the neural surrogate provides high-fidelity next-step torque predictions on unseen transients, and (ii) the RL controller improves torque-trace reproduction relative to baseline PID-style controllers while reducing manual calibration effort. The approach is modular and engine-agnostic (retrain the surrogate; retrain the policy), and it naturally extends to multi-objective control where emissions proxies are added to the reward.