Visual Sensor Fusion and Data Sharing across Connected Vehicles for Active Safety

The development of connected-vehicle technology, which includes vehicle-vehicle and vehicle-infrastructure communications, opens the door for unprecedented active safety and driver-enhanced systems. In addition to exchanging basic traffic messages among vehicles for safety applications, a significantly higher level of safety can be achieved when vehicles and designated infrastructure-locations share their sensor data. In this paper, we propose a new system where cameras installed on multiple vehicles and infrastructure-locations share and fuse their visual data and detected objects in real-time. The transmission of camera data and/or detected objects (e.g., pedestrians, vehicles, cyclists, etc.) can be accomplished by many communication methods. In particular, such communications can be accomplished using the emerging Dedicated Short-Range Communications (DSRC) technology. In our proposed system the vehicle receiving the visual data from an adjacent vehicle fuses the received visual data with its own camera views to create a much richer visual scene. We conducted several experiments across a pair of vehicles equipped with DSRC devices and our proposed system. These experiments demonstrated that our system achieve high accuracy, low delay and improved safety.