Recently, four-dimensional (4D) radar has shown unique advantages in the field of
odometry estimation due to its low cost, all-weather use, and dynamic and static
recognition. These features complement the performance of monocular cameras,
which provide rich information but are easily affected by lighting. However, the
construction of deep radar visual odometry faces the following challenges: (1)
the 4D radar point cloud is very sparse; (2) due to the penetration ability of
4D radar, it will produce mismatches with pixels when projected onto the image
plane. In order to enrich the point cloud information and improve the accuracy
of modal correspondence, this paper proposes a low-cost fusion odometry method
based on 4D radar and pseudo-LiDAR, 4DRPLO-Net. This method proposes a new
framework that uses 4D radar points and pseudo-LiDAR points generated by images
to construct odometry, bridging the gap between 4D radar and images in
three-dimensional (3D) space. Specifically, the pseudo-LiDAR point cloud is
obtained by back-projecting the depth map generated by the image into 3D space,
which changes the way the image is represented and effectively alleviates the
problem of sparse 4D radar point cloud. In order to fully integrate the two
modalities, we designed a cross-attention-based grouped fusion module, which
uses the accurate spatial measurement of radar to restore the scale of
pseudo-LiDAR points, and groups based on ball queries in the metric spatial
scale to achieve accurate association and bidirectional fusion of features.
Finally, we conducted experiments on the View-of-Delft (VOD) dataset and
compared and verified the excellence of the method and the effectiveness of each
module.