Vision-Based Maneuvering and Manipulation by a Mobile Robot

A small mobile robot equipped with a stereoscopic machine vision system and two manipulator arms that have limited degrees of freedom has been given the ability to perform moderately dexterous manipulation autonomously, under control by an onboard computer. The approach taken in this development has been one of formulating vision-based control software to utilize the mobility of the vehicle to compensate for the limitation on the dexterity of the manipulator arms. Although the goal was selected visually, it is tracked onboard using information about its shape; in particular, the target is assumed to be a local elevation maximum (i.e., the highest point within a small patch of area).