Low earth orbit (LEO) satellite communication system is expected to play a key
role in the 6G wireless networks. Due to the limited communication resource and
inter-user interference (IUI), the user equipments (UEs) must be well scheduled
to maximize spectral efficiency. However, rapid channel variations and high
mobility in LEO system make the conventional user scheduling strategies relying
on instantaneous channel state information impractical. To address this
challenge, this paper proposes a deep reinforcement learning (DRL) framework
based on batch normalization in CrossQ algorithm to optimize the downlink user
scheduling in a LEO satellite massive multi-input multi-output (MIMO) system.
The scheduling task is modeled as a Markov decision process (MDP), where the
observation space integrates high-dimensional features including large-scale
fading (LSF) coefficients, channel matrix, and temporal features capturing the
orbital progress of the satellite. Unlike traditional DRL methods, the
CrossQ-based algorithm applies batch normalization (BN) within the critic
networks to eliminate the need for slowly-moving target networks, fostering
training stability and sample efficiency. A continuous priority score vector is
mapped to discrete scheduling decisions through a top-z selection logic,
allowing the agent to effectively identify spatially orthogonal user subsets to
suppress the IUI. To guide the agent toward the true optimization direction, a
differential reward mechanism is designed. Simulation results demonstrate that
the proposed CrossQ-based scheduler consistently achieves nearly optimal
performance at different signal-to-noise ratios (SNR), reaching at least 91% and
94% of the Monte Carlo search ceiling under different user densities. Moreover,
the trained model provides scheduling strategy in millisecond inference time,
making it a practical and robust solution for future 6G non-terrestrial
networks.