The two-wheel system design is widely used in various mobile tools, such as
remote-control vehicles and robots, due to its simplicity and stability.
However, the specific wheel and body models in the real world can be complex,
and the control accuracy of existing algorithms may not meet practical
requirements. To address this issue, we propose a double inverted pendulum on
mobile device (DIPM) model to improve control performances and reduce
calculations. The model is based on the kinetic and potential energy of the DIPM
system, known as the Euler-Lagrange equation, and is composed of three
second-order nonlinear differential equations derived by specifying Lagrange. We
also propose a stable feedback control method for mobile device drive systems.
Our experiments compare several mainstream reinforcement learning (RL) methods,
including linear quadratic regulator (LQR) and iterative linear quadratic
regulator (ILQR), as well as Q-learning, SARSA, DQN (Deep Q Network), and AC.
The simulation results demonstrate that the DQN and AC methods are superior to
ILQR in our designed nonlinear system. In all aspects of the test, the
performance of Q-learning and SARSA is comparable to that of ILQR, with some
slight improvements. However, ILQR shows its advantages at 10 deg and 20 deg. In
the small deflection (between 5 and 10 deg), the DQN and AC methods perform 2%
better than the traditional ILQR, and in the large deflection (10–30 deg), the
DQN and AC methods perform 15% better than the traditional ILQR. Overall, RL not
only has the advantages of strong versatility, wide application range, and
parameter customization but also greatly reduces the difficulty of control
system design and human investment, making it a promising field for future
research.
