Mind the Gap: Quantifying Behavioral Fidelity in CARLA Using Naturalistic Drone Data

2026-01-0771

7/1/2026

Authors
Abstract
Content
Rigorous validation of SAE Levels 3 and 4 autonomous systems increasingly relies on simulation. However, the simulation-reality gap remains a challenge for human-in-the-loop assessments. This study empirically quantifies the behavioral fidelity of the Car-Learning-to-Act (CARLA) simulator by recreating specific real-world traffic scenarios using the high-precision exiD drone dataset. Twenty-five participants performed a series of maneuvers, including lane changes and time-critical cut-ins. Their performance was analyzed using Dynamic Time Warping (DTW), driver profiling, and Time-to-Collision (TTC) metrics. The findings reveal a clear distinction between relative and absolute behavioral validity. In strategic decision-making tasks, the simulation demonstrated remarkably high temporal fidelity. DTW analysis explained 94% of the trajectory variance. Participants initiated lane changes with an average lag of -9 frames (0.36 s) compared to naturalistic references. These results indicate that, despite the absence of peripheral optical flow, the simulator successfully elicits temporally correlated decision-making patterns suitable for assessing strategic driver intent. However, physical execution in reactive scenarios revealed significant absolute discrepancies. Although the high Pearson correlation (r ≈ 0.89) in velocity profiles proves that drivers recognize and react to hazards with realistic timing, their physical inputs were exaggerated. Participants displayed digital, over-modulated braking responses and maintained a negative safety bias of -11.26 m, a deviation attributed to the lack of vestibular g-force feedback and geometric minification. Furthermore, distinct driver profiles emerged. Risk-oriented participants exhibited a gaming effect by neglecting safety margins. In conclusion, while CARLA is highly valid for testing the temporal logic of driver interactions, absolute dynamics require calibration functions, such as force-feedback (pedal) tuning and visual deceleration cues like camera shake, to compensate for sensory limitations before it can be used for safety-critical validation.
Meta TagsDetails
DOI
https://doi.org/10.4271/2026-01-0771
Citation
Rebling, P., Alphan, M., and Nenninger, P., "Mind the Gap: Quantifying Behavioral Fidelity in CARLA Using Naturalistic Drone Data," 2026 Stuttgart International Symposium, Stuttgart, Germany, July 8, 2026, https://doi.org/10.4271/2026-01-0771.
Additional Details
Publisher
Published
Jul 01
Product Code
2026-01-0771
Content Type
Technical Paper
Language
English