Designing an Interactive Mixed Reality Cockpit for Enhanced Soldier-Vehicle Interaction

This paper discusses the design and implementation of an interactive mixed reality cockpit that enhances Soldier-vehicle interaction by providing a 360-degree situational awareness system. The cockpit uses indirect vision, where cameras outside the vehicle provide a video feed of the surroundings to the cockpit. The cockpit also includes a virtual information dashboard that displays real-time information about the vehicle, mission, and crew status. The visualization of the dashboard is based on past research in information visualization, allowing Soldiers to quickly assess their operational state. The paper presents the results of a usability study on the effectiveness of the mixed reality cockpit, which compared the Vitreous interface, a Soldier-centered mixed reality head-mounted display, with two other interface and display technologies. The study found that the Vitreous UI resulted in better driving performance and better subjective evaluation of the ability to actively monitor the environment compared to the other interface and display technologies. Soldiers also reported that the Vitreous interface allowed them to maintain the highest level of situational awareness.

Citation: J. Hansberger, R. Wood, T. Conner, J. Hansen, J. Nix, M. Torres, “Designing an Interactive Mixed Reality Cockpit for Enhanced Soldier-Vehicle Interaction,” In Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium (GVSETS), NDIA, Novi, MI, Aug. 16-18, 2023.