Vehicle speeds, in both longitudinal and lateral directions, are vital signals for vehicular electronic control systems. In in-wheel motor-driven vehicles (IMDVs), because no slave wheel can be used for reference, it becomes more challenging to conduct velocity estimation, especially when all wheels turn to slip. To reduce the dependence of speed estimation on physical plant parameters and environment perception, in this work, we develop a new method that estimates the longitudinal and lateral velocities of an IMDV by using the kinematic model with the Kalman Filter. For longitudinal velocity measurement, we propose a hybrid approach combining Long-Short Term Memory (LSTM) networks and the kinematic rules to obtain a reliable estimation. More specifically, when at least one effective driven wheel is available, that is, no-slip happening, the longitudinal velocity can be derived using the average of those effective wheels' rotational speeds. When all driven wheels slip, the information of longitudinal velocity can be provided by the LSTM network. The previous longitudinal velocity is augmented in the input features to improve the learning accuracy of the estimation. Moreover, the cumulative errors caused by acceleration integration are avoided, and the precision is thus improved. Finally, hardware-in-the-loop (HiL) tests are carried out on dSPACE/SCALEXIO to verify the feasibility and effectiveness of the proposed method.