Mind the Gap: Quantifying Behavioral Fidelity in CARLA using Naturalistic Drone Data

The rapid advancement of autonomous driving systems, particularly SAE Levels 3 and 4, requires rigorous validation strategies to ensure safety during handover and fallback procedures. Although virtual environments provide the necessary scalability, the simulation-reality gap poses a significant risk to the reliability of human-in-the-loop (HITL) assessments. This paper presents a comprehensive, empirical investigation into the transferability of human driving behavior from the Car-Learning-to-Act (CARLA) simulation environment to real-world traffic situations. Using the exiD dataset, which comprises high-precision, drone-recorded, naturalistic trajectories from German highways, we systematically reconstructed distinct traffic scenarios. These scenarios included lane changes, cut-in maneuvers, and curve navigation. Twenty-five participants navigated these scenarios in a driving simulator. The study employed a robust, multidimensional analytical framework to compare the simulated trajectories with real-world reference data. This included classifying driver profiles using the Mini-Driving Behavior Questionnaire (DBQ) and employing quantitative metrics such as dynamic time warping (DTW) and time-to-collision (TTC) clustering. The results indicate distinct variances in transferability across scenario types. Reactive scenarios demonstrated high behavioral validity. Notably, cut-in situations yielded realistic braking reactions in 89% of cases and received a high TTC silhouette score of 0.62. Simple lane changes also showed robust reproducibility, with DTW explaining over 94% of the variance. In contrast, strategic maneuvers exhibited the lowest transferability. These maneuvers were characterized by systematic early triggers (an average of 27 frames early) and a 26% non-execution rate. These discrepancies are largely due to limitations in the simulator's sensory feedback. Specifically, the fields of view are restricted, and there is currently no rear-view visualization. In conclusion, CARLA is already effective for validating time-critical reactive systems inside the provided driving simulator. However, closing the reality gap for strategic decision-making requires enhancing the visual and haptic fidelity of the simulation interface in future work.