Real-Time Image Recognition System Based on an Embedded Heterogeneous Computer and Deep Convolutional Neural Networks for Deployment in Constrained Environments

Computer vision (CV) represents the idea of giving machines the capacity to make meaning out of images frames, and for decades it consisted mostly of laborious and complex techniques that provided poor performance. With the advent of Deep Convolutional Neural Networks (DCNN), CV systems reached levels of accuracy to become industry-relevant. A major challenge, however, resides in deploying real-time CV systems in environments (such as driverless cars) that impose a series of constraints in terms of energy supply, weight and space. This technical paper describes how a real-time embedded image recognition system was developed and how the design guidelines were specified in terms of functionality and performance. A minimum rate of 30 frames per second (fps) was identified as a real-time boundary. Alongside the decision for a DCNN topology, system architecture and software technology, the Nvidia’s Jetson TX2 embedded computer was chosen as the evaluation board. It is described how the image recognition pipeline was benchmarked and evaluated in terms of throughput, power consumption and energy efficiency. The test set-up consisted of remote cameras producing input video streams and a HDMI monitor for presenting the system’s output. Optimizations techniques like reduced precision and batching were implemented to obtain successive improvements of the system’s throughput, while the impact on the other metrics were considered. The best achieved performance was 47,7 fps at a resolution of 1080x720. The numerous intermediate results compose a comprehensive design landscape for different operational scenarios of the system.