Quantitative Resilience Assessment of GPS, IMU, and LiDAR Sensor Fusion for Vehicle Localization Using Resilience Engineering Theory

Practical applications of recently developed sensor fusion algorithms perform poorly in the real world due to a lack of proper evaluation during development. Existing evaluation metrics do not properly address a wide variety of testing scenarios. This issue can be addressed using proactive performance measurements such as the tools of resilience engineering theory rather than reactive performance measurements such as root mean square error. Resilience engineering is an established discipline for evaluating proactive performance on complex socio-technical systems which has been underutilized for automated vehicle development and evaluation. In this study, we use resilience engineering metrics to assess the performance of a sensor fusion algorithm for vehicle localization. A Kalman Filter is used to fuse GPS, IMU and LiDAR data for vehicle localization in the CARLA simulator. This vehicle localization algorithm was then evaluated using resilience engineering metrics in the simulated multipath and overpass scenario. These scenarios were developed in the CARLA simulator by collecting real-world data in an overpass and multipath scenario using WMU’s research vehicle. The absorptive, adaptative, restorative capacities, and the overall resilience of the system was assessed by using the resilience triangle. Simulation results indicate that the vehicle localization pipeline possesses a higher quantitative resilience when encountering overpass scenarios. Nevertheless, the system obtained a higher adaptive capacity when encountering multipath scenarios. These resilience engineering metrics show that the fusion systems recover faster when encountering disturbances due to signal interference in overpasses and that the system is in a disturbed state for a shorter duration in multipath scenarios. Overall these results demonstrate that resilience engineering metrics provide valuable insights regarding complicated systems such as automated vehicle localization. In future work, the insights from resilience engineering can be used to improve the design and thus performance of future localization algorithms.