Gesture-Based Controls for Robots: Overview and Implications for Use by Soldiers

Developing a more effective means to communicate with robotic devices.

Army Research Laboratory, Aberdeen Proving Ground, Maryland

A future vision of the use of autonomous and intelligent robots in dismounted military operations is for soldiers to interact with robots as teammates, much like soldiers interact with other soldiers. Soldiers will no longer be operators in full control of every movement, as the autonomous intelligent systems will have the capability to act without continual human input. However, soldiers will need to use the information available from, or provided by, the robot. One of the critical needs to achieve this vision is the ability of soldiers and robots to communicate with each other. One way to do that is to use human gestures to instruct and command robots.

The use of gestures as a natural means of interacting with devices is a very broad concept that encompasses a range of body movements, including movements of the hands, arms, and legs, facial expressions, eye movements, head movements, and/or 2-dimensional (2-D) swiping gestures against flat surfaces such as touch screens. Gesture-based technology is already in place and commonly used without special instruction required for effective use. A common example of a well-designed gestural command is the use of hands to “wave” to activate devices (e.g., public bathroom faucet). This concept is also common to gaming interfaces and is now extending to other private and public domains such as automobile consoles.