Optimal Low-Thrust GEO-to-DRO Transfers via a Multi-Layer Homotopy Strategy for Cislunar Applications

Low-thrust transfers from Geostationary Earth Orbit (GEO) to Distant Retrograde Orbit (DRO) are numerically difficult to compute because the underlying two-point boundary value problem is strongly affected by nonlinear cislunar dynamics, discontinuous control behavior, and the high sensitivity of indirect methods to the initial costates. Within the framework of the Earth-Moon Circular Restricted Three-Body Problem (CR3BP), this paper formulates an indirect optimal control model for GEO-to-DRO low-thrust trajectory design and develops a multi-layer homotopy strategy to improve the solvability of the problem. In the minimum-fuel formulation, the optimal throttle generally follows a bang-bang structure, which makes the associated shooting equations highly sensitive and often causes convergence failure. To reduce this difficulty, a smoothing homotopy is first introduced into the performance index, allowing the solution to evolve continuously from a minimum-energy problem with smoother control characteristics toward the target minimum-fuel problem. On this basis, a thrust continuation procedure is adopted to transfer the converged solution from an auxiliary high-thrust case to the desired low-thrust regime. However, when the conventional discrete continuation scheme becomes ineffective in a local thrust interval, a nonlinear intra-interval thrust homotopy combined with pseudo-arclength continuation is further employed to pass through the problematic region and maintain continuous tracking of the solution branch. Numerical results show that the proposed method can successfully overcome the local breakdown encountered in classical continuation and eventually obtain the minimum-fuel optimal transfer trajectory from GEO to DRO. The study indicates that the proposed framework enhances the robustness and continuity of indirect optimization for low-thrust transfers in the CR3BP, and it provides an effective computational approach for the design of cislunar transfer trajectories.