Recent trends in autonomy have emphasized end-to-end deep-learning-based methods that have shown a lot of promise in overcoming the requirements and limitations of feature-engineering. However, while promising, the black-box nature of deep-learning frameworks now exacerbates the need for testing with end-to-end deployments. Further, as exemplars of systems-of-systems, autonomous vehicles (AVs) engender numerous interconnected component-, subsystem and system-level interactions. The ensuing complexity creates challenges for verification and validation at the various component, subsystem- and system-levels as well as end-to-end testing. While simulation-based testing is one promising avenue, oftentimes the lack of adequate fidelity of AV and environmental modeling limits the generalizability. In contrast, full-scale AV testing presents the usual limitations of time-, space-, and cost. Hence in this paper, we explore the opportunity for using experiential learning possible with a scaled vehicle-based deployment to overcome the limitations(e.g. simulation fidelity or experimentation costs) of scaled vehicles to lower the barriers especially at the early stages of testing of autonomy algorithms.
In recent times, several efforts have emerged for testing deep-learning-based autonomy algorithms on scaled vehicles - the Nvidia Jet racer, Amazon Deep racer, and Donkey car are being widely used. In this paper, we examine a deployment of the Donkey car Behavior Cloning software stack on a 1/10th scaled vehicle (F1tenth) and the issues faced while deploying the other software stacks. In particular, we explored the effectiveness of: (i) mixing and matching frameworks; and (ii) use of scaled vehicles in an academic set up to support testing and deployment of supervised learning (behavior cloning) technique to achieve lane-keeping and obstacle-avoidance. We showcase that the use of this scaled-vehicle framework permitted the rapid exploration of many different test tracks (challenging with full-scale vehicle tests) while retaining realistic environmental conditions (challenging with simulation-alone testing).