This paper describes the development and simulation of a voice and pointing gesture interaction system for on-route update of autonomous vehicles’ path. The objective of this research is to provide users of autonomous vehicles a human vehicle interaction mode that enables them to make and communicate spontaneous decisions to the autonomous car, modifying its pre-defined autonomous route in real-time. For example, similar to giving directions to a taxi driver, a user will be able to tell the car «Stop there» or «Take that exit». In this way, the user control/spontaneity vs interaction flexibility dilemma that current autonomous vehicle concepts have, could be solved, potentially increasing the user acceptance of this technology.
The system was designed following a level structured state machine approach. The simulations were developed using MATLAB and VREP, a robotics simulation platform, which has accurate vehicle and sensor models. The simulation results demonstrate the main functions of the proposed system and set potential challenges that should be addressed prior to a real car implementation. The resolution of these challenges constitute our team’s on-going efforts in this research area.