Stuck-Assist: A Large Multimodal Model-Based Stuck Recovery Assist for Driving on Off-Road Deformable Terrains

2026-01-7513

9/22/2026

Authors
Abstract
Content
Recent advancements in off-road autonomy have shown significant progress in perception, planning, and control frameworks, including end-to-end learning approaches. Comprehensive results have been demonstrated in both simulation and real-world experiments; however, there are significant challenges in critical cases that need further evaluation. One such challenge is the immobilization of autonomous ground vehicles (AGVs) in unstructured off-road environments, which can significantly impact agriculture, space exploration, military operations, and search and rescue missions. Addressing this problem requires recovery strategies that are context-sensitive, adaptable to terrain and vehicle conditions, and effective in integrating multimodal inputs. To this end, this paper investigates the use of a large multimodal model (LMM) providing higher-level planning assistance with human-in-the-loop evaluations for vehicle recovery after immobilization in unstructured off-road terrain. The experimental simulation platform developed was based on the Algoryx (AGX) Dynamics engine for high-fidelity terramechanics interaction and vehicle physics combined with Unreal Engine 5. This platform was further integrated with a driving simulator equipped with steering wheel and pedal interfaces for human-in-the-loop experiments. We evaluated ten representative unstuck scenarios across two deformable terrains (loose sand and compact sand) under two modes: an unskilled baseline, where participants attempted recovery unaided, and a co-intelligence mode, where participants used LMM advisory instructions. The results show that LMM assistance improved stuck recovery rates by 70% compared to unaided and unskilled human driving.
Meta TagsDetails
DOI
https://doi.org/10.4271/2026-01-7513
Citation
Bhosale, M., Whitson, J., Vahidi, A., and Jia, Y., "Stuck-Assist: A Large Multimodal Model-Based Stuck Recovery Assist for Driving on Off-Road Deformable Terrains," 2026 NDIA Michigan Chapter Ground Vehicle Systems Engineering and Technology Symposium, Novi, Michigan, United States, August 11, 2026, https://doi.org/10.4271/2026-01-7513.
Additional Details
Publisher
Published
Sep 22
Product Code
2026-01-7513
Content Type
Technical Paper
Language
English