This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Real-Time Image Recognition System Based on an Embedded Heterogeneous Computer and Deep Convolutional Neural Networks for Deployment in Constrained Environments
Technical Paper
2019-01-1045
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Computer vision (CV) represents the idea of giving machines the capacity to make meaning out of images frames, and for decades it consisted mostly of laborious and complex techniques that provided poor performance. With the advent of Deep Convolutional Neural Networks (DCNN), CV systems reached levels of accuracy to become industry-relevant. A major challenge, however, resides in deploying real-time CV systems in environments (such as driverless cars) that impose a series of constraints in terms of energy supply, weight and space. This technical paper describes how a real-time embedded image recognition system was developed and how the design guidelines were specified in terms of functionality and performance. A minimum rate of 30 frames per second (fps) was identified as a real-time boundary. Alongside the decision for a DCNN topology, system architecture and software technology, the Nvidia’s Jetson TX2 embedded computer was chosen as the evaluation board. It is described how the image recognition pipeline was benchmarked and evaluated in terms of throughput, power consumption and energy efficiency. The test set-up consisted of remote cameras producing input video streams and a HDMI monitor for presenting the system’s output. Optimizations techniques like reduced precision and batching were implemented to obtain successive improvements of the system’s throughput, while the impact on the other metrics were considered. The best achieved performance was 47,7 fps at a resolution of 1080x720. The numerous intermediate results compose a comprehensive design landscape for different operational scenarios of the system.
Recommended Content
Authors
Citation
da Silva Carvalho, M., Koark, F., Rheinländer, C., and Wehn, N., "Real-Time Image Recognition System Based on an Embedded Heterogeneous Computer and Deep Convolutional Neural Networks for Deployment in Constrained Environments," SAE Technical Paper 2019-01-1045, 2019, https://doi.org/10.4271/2019-01-1045.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 | ||
| Unnamed Dataset 3 |
Also In
References
- Patternson , J. and Basu , K. https://www.oreilly.com/ideas/solving-real-world-business-problems-with-computer-vision 2017
- Kopetz , H. and Puscher , P. https://ti.tuwien.ac.at/cps/teaching/courses/real-time-systems/slides/rts01_definitions.pdf 2017
- Krizhevsky , A. , Sutskever , I. , and Hinton , G.E. Imagenet Classification with Deep Convolutional Neural Networks Advances in Neural Information Processing Systems 2012 1097 1105
- Szegedy , C. , Liu , W. , Jia , Y. , Sermanet , P. et al. Going Deeper with Convolutions Cvpr 2015
- Simonyan , K. and Zisserman , A. 2014
- Moor Insights and Strategy A Machine Learning Application Landscape and Appropriate Hardware Alternatives 2017
- Nielsen , M.A. Neural Networks and Deep Learning Determination Press 2015 http://neuralnetworksanddeeplearning.com/index.html
- Goodfellow , I. , Bengio , Y. , and Courville , A. Deep Learning MIT Press 2016 http://www.deeplearningbook.org
- Albanie , S. https://github.com/albanie/convnet-burden
- Franklin , D. Nvidia Jetson TX2 Delivers Twice the Intelligence to the Edge https://devblogs.nvidia.com/jetson-tx2-delivers-twice-intelligence-edge/ 2017
- Jetson Hacks Nvpmodel Nvidia Jetson TX2 Development Kit http://www.jetsonhacks.com/2017/03/25/nvpmodel-nvidia-jetson-tx2-development-kit/
- Jia , Y. , Shelhamer , E. , Donahue , J. , Karayev , S. et al. 2014
- Abadi , M. , Agarwal , A. , Barham , P. , Brevdo , E. et al. 2015 tensorflow.org
- Redmon , J. http://pjreddie.com/darknet/ 2013
- Redmon , J. and Farhadi , A. 2016
- Liu , W. , Anguelov , D. , Erhan , D. , Szegedy , C. et al. SSD: Single Shot Multibox Detector ECCV 2016
- Howard , A.G. , Zhu , M. , Chen , B. , Kalenichenko , D. et al. 2017
- Google https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md 2018
- Bahrampour , S. , Ramakrishnan , N. , Schott , L. , and Shah , M. 2015
- Stallkamp , J. , Schlipsing , M. , Salmen , J. , and Igel , C. The German Traffic Sign Recognition Benchmark: A Multi-Class Classification Competition IEEE International Joint Conference on Neural Networks 2011 1453 1460
- Prasanna , S. , Kashinkunti , P. , and Milletari F. https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/ 2017
- Nvidia , F.D. Deep Vision Tutorial https://github.com/dusty-nv/jetson-inference
- Nvidia Corporation GPU-Based Deep Learning Inference: A Performance and Power Analysis 2015
- Vanholder , H. Efficient Inference with TensorRT http://on-demand.gputechconf.com/gtc-eu/2017/presentation/23425-han-vanholder-efficient-inference-with-tensorrt.pdf 2016
- Shelhamer , E. Alexnet in Caffe’s Repository https://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/readme.md
- Eisenbach , M. , Seichter , D. , Wengefeld , T. , and Horst-Michael Gross , H.M. Cooperative Multi-scale Convolutional Neural Networks for Person Detection Neural Networks (IJCNN), 2016 International Joint Conference on 2016 267 276
- Eisenbach , M. , Stricker , R. , Seichter , D. , Vorndran , A. et al. Speeding Up Deep Neural Networks on the Jetson TX1 CAPRI 2017 2017 11
- Han , Y. and Erdal Oruklu , E.
- Rungsuptaweekoon , K. , Visoottiviseth , V. , and Takano , R. Evaluating the Power and Efficiency of Deep Learning Inference on Embedded Gpu Systems Information Technology (INCIT), 2017 2nd International Conference on IEEE 2017 1 5
- Bhandary , N. , MacKay , C. , Richards , A. , Tong , J. et al.
- Han , Y. and Oruklu , E. Real-Time Traffic Sign Recognition Based on Zynq FPGA and ARM SoCs Electro/Information Technology (EIT), 2014 IEEE International Conference on IEEE 2014 373 376
- Farhat , W. , Faiedh , H. , Souani , C. , and Besbes , K. Real-Time Hardware/Software Co-Design of a Traffic Sign Recognition System Using Zynq FPGA Design & Test Symposium (IDT), 2016 11th International IEEE 2016 302 307
- Qiu , J. , Wang , J. , Yao , S. , Guo , K. et al. Going Deeper with Embedded FPGA Platform for Convolutional Neural Network Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays ACM 2016 26 35
- Chin , C. , Si , J.T. , Clare , A.S. , and Ma , M. Intelligent Image Recognition System for Marine Fouling Using Softmax Transfer Learning and Deep Convolutional Neural Networks Complexity 2017