This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Distance Map Estimation of Stereoscopic Images Using Deep Neural Networks for Autonomous Vehicle Driving
Technical Paper
2017-01-0071
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
While operating a vehicle in either autonomous or occupant piloted mode, an array of sensors can be used to guide the vehicle including stereo cameras. The state-of-the-art distance map estimation algorithms, e.g. stereo matching, usually detect corresponding features in stereo images, and estimate disparities to compute the distance map in a scene. However, depending on the image size, content and quality, the feature extraction process can become inaccurate, unstable and slow. In contrast, we employ deep convolutional neural networks, and propose two architectures to estimate distance maps from stereo images. The first architecture is a simple and generic network that identifies which features to extract, and how to combine them in a multi-resolution framework. The second architecture is a more specialized one that extracts local similarity information from two images, which are used for stereo feature matching, and fuses them at multiple resolutions to generate the distance map. We generate several synthetic data sets for an end-to-end training of the networks. Note that the pavements are excluded from the ground truth distance maps in the synthetic data sets. This helps the networks put more emphasis on the surrounding objects than the pavements, and yield more accurate predictions for the objects, which is crucial for path planning and active safety systems during operation of autonomous vehicles. The evaluation results on 500 stereo images show that both networks are able to estimate the distance to the surrounding objects accurately in close to real time, which is a pivotal characteristic for operation of autonomous vehicles. However, the second network is especially more accurate for distance estimation of the distant objects than the first one.
Recommended Content
Authors
Citation
Taimouri, V., Cordonnier, M., Lee, K., and Goodman, B., "Distance Map Estimation of Stereoscopic Images Using Deep Neural Networks for Autonomous Vehicle Driving," SAE Technical Paper 2017-01-0071, 2017, https://doi.org/10.4271/2017-01-0071.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 | ||
Unnamed Dataset 2 | ||
Unnamed Dataset 3 | ||
Unnamed Dataset 4 |
Also In
References
- Scharstein , D. and Richard S. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms International journal of computer vision 47 1-3 7 42 2002 10.1023/A:1014573219977
- Birchfield , S. , and Carlo T. Multi-way cut for stereo and motion with slanted surfaces Proceedings of the Seventh IEEE International Conference on Computer Vision 1 489 495 1999 10.1109/ICCV.1999.791261
- Li , Y. and Huttenlocher , D. P. Learning for stereo vision using the structured support vector machine Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1 8 2008 10.1109/CVPR.2008.4587699
- Zhang , L. and Seitz , S. M. Estimating optimal parameters for MRF stereo from a single image pair IEEE trans. on pattern analysis and machine intelligence 29 2 331 342 2007 10.1109/TPAMI.2007.36
- Haeusler , R. , Nair. R. , and Kondermann. D. Ensemble learning for confidence measures in stereo vision Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 305 312 2013 10.1109/CVPR.2013.46
- Liu , W. , Anguelov , D. , Erhan , D. , Szegedy , C. et al. SSD: Single Shot MultiBox Detector 2015 arXiv preprint arXiv:1512.02325
- Fischer , P. , Dosovitskiy , A. , Ilg , E. , Häusser , P. et al. Flownet: Learning optical flow with convolutional networks IEEE International Conference on Computer Vision (ICCV) 2758 2766 2015 arXiv preprint arXiv:1504.06852
- Zbontar , J. and LeCun , Y. Stereo matching by training a convolutional neural network to compare image patches J. Machine Learning Research 17 1 32 2016
- Mayer , N. , Ilg , E. , Häusser , P. , Fischer , P. et al. A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016 arXiv preprint arXiv:1512.02134
- Arndt , P. A. , Hanspeter , A. M. and Heinrich , H. B. Human stereovision without localized image features Biological cybernetics 72 4 279 293 1995 10.1007/BF00202784
- Fleet , D. J. , Jepson , A. D. and Jenkin , M. R. Phase-based disparity measurement CVGIP: Image understanding 53 2 198 210 1991 10.1016/1049-9660(91)90027-M
- Memisevic , R. and Christian C. Stereopsis via deep learning NIPS Workshop on Deep Learning 1 2011
- Zagoruyko , S. and Nikos K. Learning to compare image patches via convolutional neural networks Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4353 4361 2015
- Zbontar , J. and LeCun , Y. Computing the stereo matching cost with a convolutional neural network Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1592 1599 2015
- Chen , Z. , Xun S. , Liang W. , Yinan Y. et al. A deep visual correspondence embedding model for stereo matching costs Proceedings of the IEEE International Conference on Computer Vision 972 980 2015 10.1109/ICCV.2015.117
- Luo , W. , Alexander G. S. and Raquel U. Efficient deep learning for stereo matching Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 5695 5703 2016
- Mostegel , C. , Rumpler , M. , Fraundorfer , F. and Bischof , H. Using Self-Contradiction to Learn Confidence Measures in Stereo Vision Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016 arXiv preprint arXiv:1604.05132
- Geiger , A. , Lenz , P. and Urtasun , R. Are we ready for autonomous driving? The KITTI vision benchmark suite Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3354 3361 2012
- Menze , M. and Geiger , A. Object scene flow for autonomous vehicles Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3061 3070 2015
- Farabet , C. , Couprie , C. , Najman , L. and LeCun , Y. Learning hierarchical features for scene labeling IEEE transactions on pattern analysis and machine intelligence 35 8 1915 1929 2013 10.1109/TPAMI.2012.231
- Eigen , D. , Puhrsch , C. and Fergus , R. Depth map prediction from a single image using a multi-scale deep network Advances in neural information processing systems 2366 2374 2014
- Long , J. , Shelhamer , E. and Darrell , T. Fully convolutional networks for semantic segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3431 3440 2015
- Jia , Y. , Shelhamer , E. , Donahue , J. , Karayev , S. et al. Caffe: Convolutional architecture for fast feature embedding Proceedings of the 22nd ACM international conference on Multimedia 675 678 2014
- Yamaguchi , K. , McAllester , D. and Urtasun , R. Efficient joint segmentation, occlusion labeling, stereo and flow estimation Proceedings of the European Conference on Computer Vision (ECCV) 756 771 2014
- Hirschmuller , H. Stereo processing by semiglobal matching and mutual information IEEE Trans. on pattern analysis and machine intelligence 30 2 328 341 2008 10.1109/TPAMI.2007.1166