This content is not included in your SAE MOBILUS subscription, or you are not logged in.
AUREATE: An Augmented Reality Test Environment for Realistic Simulations
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 03, 2018 by SAE International in United States
Annotation ability available
Automated driving is currently one of the most active areas of research worldwide. While the general progress in developing specific algorithms for perception, planning and control tasks is very advanced, testing and validation of the resulting functions is still challenging due to the large number of possible scenarios and generation of ground-truth. Currently, real world testing and simulations are used in combination to overcome some of these challenges. While real world testing does not suffer from imperfect sensor models and environments, it is expensive, slow and not accurately repeatable and therefore unable to capture all possible scenarios. However, simulation models are not sophisticated enough to fully replace real world testing. In this paper, we propose a workflow that is capable of augmenting real sensor-level data with simulated sensor data. With this approach we are able to generate scenarios which are as realistic as possible while also being flexible with the ability to insert arbitrary objects. This sensor-level based approach enables testing of the whole algorithm chain for automated driving, including perception, object-detection, scene understanding, path planning, decision making, and control.
CitationKoduri, T., Bogdoll, D., Paudel, S., and Sholingar, G., "AUREATE: An Augmented Reality Test Environment for Realistic Simulations," SAE Technical Paper 2018-01-1080, 2018, https://doi.org/10.4271/2018-01-1080.
- Kalra, N. and Paddock, S.M., “Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?” (RAND Corporation, 2016), 1-16, doi:10.1016/j.tra.2016.09.010.
- Sutherland, I.E., “A head-mounted three dimensional display,” AFIPS Conference Proceedings 33(1):757-764, 1968, doi:10.1145/1476589.1476686.
- Matsumoto, T., Watanabe, R., Eguchi, S., Aritake, H. et al., “Heads-up display,” U.S. Patent 5.210.624, May 11, 1993.
- Pokam, R., Chauvin, C., Debernard, S., and Langlois, S., “Augmented reality interface design for autonomous driving,” presented at FAST-zero’15, Sweden, September 9-11, 2015.
- Huang, W., Wang, K., Yisheng, L., and Zhu, F., “Autonomous vehicles testing methods review”, presented at ITSC 2016, Brazil, November 1-4, 2016, doi:10.1109/ITSC.2016.7795548.
- Gerkey, B. P., Vaughan, R. T., and Howard, A., “The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems,” Proceedings of the 11th International Conference on Advanced Robotics: 317-323, 2003.
- Carpin, S., Lewis, M., Wang, J., Balakirsky, S., and Scrapper C., “USARSim: a robot simulator for research and education,” presented at Robotics and Automation 2017, Italy, April 10-14, 2007, doi:10.1109/ROBOT.2007.363180.
- Durst, P. J., Goodin, C., Cummins, C., Gates, B. et al., “A Real-Time, Interactive Simulation Environment for Unmanned Ground Vehicles: The Autonomous Navigation Virtual Environment Laboratory (ANVEL),” presented at ICIC 2012, United Kingdom, July 24-25, 2012, doi:10.1109/ICIC.2012.5.
- Brand, J.G., “Graphics for a 3D Driving Simulator,” Bachelor thesis,” (Institute for Real-Time Computer Systems, Technical University of Munich, Munich, 2008).
- Lee, S., Cho, J., and Kim, S., “A 3-D real-time simulation for autonomous driving with V2V communications,” presented at ICCVE 2013, United States of America, December 2-6, 2013, doi:10.1109/ICCVE.2013.6799900.
- Zhang, C., Liu, Y., Zhao, D., and Su, Y., “RoadView: A Traffic Scene Simulator for Autonomous Vehicle Simulation Testing,” presented at ITSC 2013, China, October 8-11, 2014, doi:10.1109/ITSC.2014.6957844.
- Jayaraman, A., Micks, A., and Gross, E., “Creating 3D Virtual Driving Environments for Simulation-Aided Development of Autonomous Driving and Active Safety,” SAE Technical Paper 2017-01-0107, 2017, doi:10.4271/2017-01-0107.
- MCity, “Mcity Test Facility,” https://mcity.umich.edu/our-work/mcity-test-facility/, accessed September 2017.
- Zhao, D., and Pend, H., “From the Lab to the Street: Solving the Challenge of Accelerating Automated Vehicle Testing,” White Paper, Mcity, University of Michigan, 2017.
- The Antlantic, “Inside Waymo’s Secret World for Training Self-Driving Cars,” https://www.theatlantic.com/technology/archive/2017/08/inside-waymos-secret-testing-and-simulation-facilities/537648/, accessed September 2017.
- Michigan IT, “MCity uses AR to help test automated vehicles,” https://michigan.it.umich.edu/news/2017/06/27/mcity-uses-ar/, accessed September 2017.
- Gechter, F., Dafflon, B., Gruer, P., and Koukam, A., “Towards a Hybrid Real/Virtual Simulation of Autonomous Vehicles for Critical Scenarios,” presented at ICSEA 2014, France, October 12-16, 2014.
- Geiger, A., Lenz, P., Stiller, C., and Urtasun, R., “Vision meets Robotics: The KITTI Dataset,” International Journal of Robotics Research 32(11):1231-1237, 2013, doi:10.1177/0278364913491297.
- Self Racing Cars, “PolySync Dataset Available”, http://selfracingcars.com/blog/2016/7/26/polysync.
- Fischler, M.A. and Bolles, R.C., “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Communications of the ACM 24(6):381-395, 1981, doi:10.1145/358669.358692.
- Geschwandtner, M., Kwitt, R., Uhl, A., and Pree, W., “BlenSor: Blender Sensor Simulation Toolbox”, Proceedings of the 7th ISVC Part II: 199-208, 2011, doi:10.1007/978-3-642-24031-7_20.
- Majek, K. and Bedkowski, J., “Range Sensors Simulation Using GPU Ray Tracing,” Proceedings of the 9th International Conference on Computer Recognition Systems CORES: 831-840, 2015, doi:10.1007/978-3-319-26227-7_78.
- Lai, K. and Fox, D., “Object Recognition in 3D Point Clouds Using Web Data and Domain Adaption,” SAGE Journals 29(8):1019-1037, 2010, doi:10.1177/0278364910369190.
- Woods, J.O. and Christian, J.A., “GLiDAR: An OpenGL-based, Real-Time, and Open Source 3D Sensor Simulator for Testing Computer Vision Algorithms,” Journal of Imaging 2(1): Article 5), 2016, doi:10.3390/jimaging2010005.
- Peinecke, N., Lueken, T. and Korn, B. R., “LiDAR simulation using graphics hardware acceleration,” presented at DASC 2008, United States of America, October 26-30, 2008, doi:10.1109/DASC.2008.470283.
- Wang, S., Heinrich, S., Wang, M., and Rojas, R., “Shader-based sensor simulation for autonomous car testing,” presented at ITSC 2012, United States of America, September 16-19, 2012, doi:10.1109/DASC.2008.4702838.
- Sluys, M.v.d., Kan, P.v., and Sonneveld, P., “CPV in the Built Environment,” AIP Conference Proceedings 1679(1), 2015, doi:10.1063/1.4931544.
- Phong, B.T., “Illumination for Computer Generated Pictures,” Communications of the ACM 18(6):311-317, 1975, doi:10.1145/360825.360839.
- OpenStreetMap, Map, https://www.openstreetmap.org/, accessed September 2017.
- Google Maps, Map, https://www.google.com/maps/, accessed September 2017.
- Ester, M., Kriegerl, H.-P., Sander, J., and Xu, X., “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise,” Data Mining and Knowledge Discovery 2(2):226-231, 1998.
- Redmon, J., and Farhadi, A., “YOLO9000: Better, Faster, Stronger,” presented at CVPR 2017, United States of America, July 22-25, 2017, arXiv:1612.08242.