This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Development of an Autonomous Vehicle Control Strategy Using a Single Camera and Deep Neural Networks
Technical Paper
2018-01-0035
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Autonomous vehicle development has benefited from sanctioned competitions dating back to the original 2004 DARPA Grand Challenge. Since these competitions, fully autonomous vehicles have become much closer to significant real-world use with the majority of research focused on reliability, safety and cost reduction. Our research details the recent challenges experienced at the 2017 Self Racing Cars event where a team of international Udacity students worked together over a 6 week period, from team selection to race day. The team’s goal was to provide real-time vehicle control of steering, braking, and throttle through an end-to-end deep neural network. Multiple architectures were tested and used including convolutional neural networks (CNN) and recurrent neural networks (RNN). We began our work by modifying a Udacity driving simulator to collect data and develop training models which we implemented and trained on a laptop GPU. Then, in the two days between car delivery and the start of the competition, a customized neural network using Keras and Tensorflow was developed. The deep learning network algorithm predicted car steering angles using a single front-facing camera. Training and deployment on the vehicle was completed using two GTX 1070s since a cloud GPU computing instance was neither available nor feasible. Using the proposed methods and working within the competition’s strict requirements, we completed several semi-autonomous laps and the team remained competitive. The results of the competition indicated that autonomous vehicle command and control can be achieved in a limited form using a single-camera with a short engineering development timeline. This approach lacks robustness and reliability and therefore, a semantic segmentation network was developed using feature extraction from the YOLOv2 network and the CamVid dataset with a correction for the unbalanced occurrence of the different classes. Currently 31 classes can be reliably detected and classified allowing for a more complex and robust decision making architecture.
Recommended Content
Authors
Citation
Navarro, A., Joerdening, J., Khalil, R., Brown, A. et al., "Development of an Autonomous Vehicle Control Strategy Using a Single Camera and Deep Neural Networks," SAE Technical Paper 2018-01-0035, 2018, https://doi.org/10.4271/2018-01-0035.Also In
References
- Charette , R.N. This Car Runs on Code IEEE Spectrum 46 3 2009
- Bishop , R. Intelligent Vehicle Technology and Trends trid.trb.org 2005
- Gusikhin , O. , Filev , D. , and Rychtyckyj , N. Intelligent Vehicle Systems: Applications and New Trends Informatics in Control Automation and Robotics Berlin Heidelberg Springer 2008 3 14
- National Highway Traffic Safety Administration National Motor Vehicle Crash Causation Survey U.S. Department of Transportation 2008
- National Highway Traffic Safety Administration 2016 Fatal Motor Vehicle Crashes: Overview U.S. Department of Transportation 2017
- U S House of Representatives 2012
- Bengler , K. , Dietmayer , K. , Farber , B. , Maurer , M. et al. Three Decades of Driver Assistance Systems: Review and Future Perspectives IEEE Intelligent Transportation Systems Magazine 6 6 22 2014
- Mahajan , H.S. , Bradley , T. , and Pasricha , S. Application of Systems Theoretic Process Analysis to a Lane Keeping Assist System Reliability Engineering and System Safety 167 177 183 2017
- Kukkala , V.K. , Tunnell , J. , Pasricha , S. , Bradley , T.
- Fagnant , D.J. and Kockelman , K. Preparing a Nation for Autonomous Vehicles: Opportunities, Barriers and Policy Recommendations Transportation Research. Part A, Policy and Practice 77 167 181 2015
- Thrun , S. , Montemerlo , M. , Dahlkamp , H. , Stavens , D. et al. Stanley: The Robot that Won the DARPA Grand Challenge Journal of Field Robotics 23 661 692 2006
- Urmson , C. , Andrew Bagnell , J. , Baker , C.R. , Hebert , M. et al. 2007
- Cameron , O. Race Self-Driving Cars With Udacity! - Udacity Inc - Medium https://medium.com/udacity/race-self-driving-cars-with-udacity-42ae12e545c1
- Gundling , C. Our Very Own Grand Challenge - Udacity Inc - Medium https://medium.com/udacity/our-very-own-grand-challenge-b004a9863024
- Sun , Z. , Bebis , G. , and Miller , R. On-Road Vehicle Detection: A Review IEEE Transactions on Pattern Analysis and Machine Intelligence 28 694 711 2006
- Rosenblatt , F. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain Psychological Review 65 386 408 1958
- Hopfield , J.J. Neural Networks and Physical Systems with Emergent Collective Computational Abilities Proceedings of the National Academy of Sciences of the United States of America 79 2554 2558 1982
- Rumelhart , D.E. , McClelland , J.L. PDP Research Group Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations Cambridge MIT Press 1986
- Demuth , H.B. , Beale , M.H. , De Jess , O. , and Hagan , M.T. Neural Network Design USA Martin Hagan 2014
- Sallab , A.E.L. , Abdou , M. , Perot , E. , and Yogamani , S. Deep Reinforcement Learning Framework for Autonomous Driving Electronic Imaging 2017 70 76 2017
- Ohn-Bar , E. and Trivedi , M.M. Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles IEEE Transactions on Intelligent Vehicles 1 90 104 2016
- Self Driving Car Engineer Nanodegree https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013
- Bojarski , M. Del Testa , D. , , Dworakowski , D. , Firner , B. et al. End to End Learning for Self-Driving Cars http://arxiv.org/abs/1604.07316 2016
- Udacity Udacity’s Self-Driving Car Simulator https://github.com/udacity/self-driving-car-sim
- PolySync Core https://polysync.io/core/
- LeCun , Y. LeNet-5, Convolutional Neural Networks http://yann 2015
- Krizhevsky , A. , Sutskever , I. , and Hinton , G.E. ImageNet Classification with Deep Convolutional Neural Networks Pereira F. , Burges C.J.C. , Bottou L. , and Weinberger K.Q. Advances in Neural Information Processing Systems 25 Curran Associates, Inc. 2012 1097 1105
- PolySync Open Source Car Control https://github.com/PolySync/OSCC/wiki
- PolySync Hardware https://github.com/PolySync/oscc/wiki/Hardware-Main
- Redmon , J. Farhadi , A. and 2016
- Brostow , G.J. , Fauqueur , J. , and Cipolla , R. Semantic Object Classes in Video: A High-Definition Ground Truth Database Pattern Recognition Letters 30 88 97 2009
- Brostow , G.J. , Shotton , J. , Fauqueur , J. , and Cipolla , R. Segmentation and Recognition Using Structure from Motion Point Clouds Computer Vision - ECCV 2008 Berlin, Heidelberg Springer 2008 44 57