Deep Reinforcement Learning Based Application of Exhaust Gas Aftertreatment Control Using the Example of a Hydrogen Engine

Growing environmental concerns drive the increasing need for a more climate-friendly mobility and pose a challenge for the development of future powertrains. Hydrogen engines represent a suitable alternative for the heavy-duty segment. However, typical operation includes dynamic conditions and the requirement for high loads that produce the highest NO_x emissions. These emissions must be reduced below the legal limits through selective catalytic reduction (SCR). The application of such a control system is time-intensive and requires extensive domain knowledge.

We propose that almost human-like control strategies can be achieved for this virtual application with less time and expert knowledge by using Deep Reinforcement Learning. A proximal policy optimization (PPO) -based agent is trained to control the injection of Diesel exhaust fluid (DEF) and compared with the performance of a manually tuned controller. The performance is evaluated based on the restrictive emission limits of a possible EURO7-framework and DEF consumption. Applied to a standardized driving cycle (WHTC) and compared with the conventional application, the agent reaches similar emission values with a equally high DEF consumption. In addition, a long short-term memory (LSTM) network is trained to substitute the 1D-SCR-model and then used to train a PPO-based agent. The results of the agent interacting with the conventional 1D-model are compared to the results with the LSTM-network as environment.

The results demonstrate, that the control of an exhaust gas aftertreatment system using Reinforcement Learning is very satisfactory. Further work is required to refine the proposed methodology into a fully-fledged tool for application in powertrain development.