Vehicle to Pedestrian Relative State Estimation via Fusing Ultrawideband Radios and a Monocular Camera

As sensors, particularly Ultra-Wideband (UWB) Radios and monocular cameras, becomes less cost prohibitive and the need for self contained relative position solutions rises, algorithms to provide a relative position solution under these constraints have gained more attention in research. An Extended Kalman Filter (EKF) was developed to fuse an array of UWB radios and a monocular camera together. This filter combined the sensors described with a relative motion model to estimate Vehicle to Pedestrian (V2P) relative motion states. A significant challenge in using cameras to perform relative ranging is the mapping between camera detections in the image domain and ranging information in the position domain. It is shown that the filter is capable of learning it’s own mapping between the position and image domains to avoid the requirement of an a-priori image to position mapping model. This system was deployed on experimental hardware collecting real world data in a variety of test scenarios. Performance metrics were generated evaluating how the filter performed in different experimental datasets compared to ground truth. It was found under optimal operating conditions, the inclusion of the camera makes a marginal increase in estimate quality. However, in situations where the UWBs on their own struggle, the monocular camera provides the necessary redundancy to keep the relative state estimate constrained.