Range anxiety and lack of adequate access to fast charging are proving to be important impediments to electric vehicle (EV) adoption. While many techniques to fast charging EV batteries (model-based & model-free) have been developed, they have focused on a single Lithium-ion cell. Extensions to battery packs are scarce, often considering simplified architectures (e.g., series-connected) for ease of modeling. Computational considerations have also restricted fast-charging simulations to small battery packs, e.g., four cells (for both series and parallel connected cells). Hence, in this paper, we pursue a model-free approach based on reinforcement learning (RL) to fast charge a large battery pack (comprising 444 cells). Each cell is characterized by an equivalent circuit model coupled with a second-order lumped thermal model to simulate the battery behavior. After training the underlying RL, the developed model will be straightforward to implement with low computational complexity. In detail, we utilize a Proximal Policy Optimization (PPO) deep RL as the training algorithm. The RL is trained in such a way that the capacity loss due to fast charging is minimized. The pack’s highest cell surface temperature is considered an RL state, along with the pack’s state of charge. Finally, in a detailed case study, the results are compared with the constant current-constant voltage (CC-CV) approach, and the outperformance of the RL-based approach is demonstrated. Our proposed PPO model charges the battery as fast as a CC-CV with a 5C constant stage while maintaining the temperature as low as a CC-CV with a 4C constant stage.