Battery terminal voltage modelling is crucial for various applications, including electric vehicles, renewable energy systems, and portable electronics. Terminal voltage models are used to determine how a battery will respond under load and can be used to calculate run-time, power capability, and heat generation and as a component of state estimation approaches, such as for state of charge. Previous studies have shown better voltage modelling accuracy for long short-term memory (LSTM) recurrent neural networks than other traditional methods (e.g., equivalent circuit and electrochemical models). This study presents two new approaches – sequence training and data shuffling – to improve LSTM battery voltage models further, making them an even better candidate for the high-accuracy modelling of lithium-ion batteries. Because the LSTM memory captures information from past time steps, it must typically be trained using one series of continuous data. Instead, the proposed sequence training approach feeds a fixed window of prior data (e.g., 100 seconds) into the LSTM at each time step to initialize the memory states properly and then only uses the output at the current time step. With this method, the LSTM just requires the prior data window to be continuous, thereby allowing the handling of discontinuities. This also means that during the training process, the data can be shuffled randomly, enabling mini-batches to speed up the training significantly. When these approaches were applied, LSTM voltage estimation error was reduced by 22%, from 28.5 mV to 22.3 mV RMS error over four drive cycles and temperatures from -20 to 25°C.