Measuring the Benefits of a Complex System: The Rotorcraft Pilot’s Associate Evaluation Program

The Rotorcraft Pilot’s Associate (RPA) program is developing a cognitive decision aiding system that will be an intelligent associate and aid to helicopter crew members. RPA builds on future avionic subsystems, fuses battlefield information, reasons about threat intent, develops and continuously updates mission plans, provides recommendations and takes authorized actions. The evaluation team’s charter is to create and apply an evaluation methodology to measure the benefits of the RPA technology. We believe that the benefits of RPA can best be measured at multiple levels of the battlefield, from the individual crew through the higher echelons. During our early analysis of scout/attack missions and the RPA concept, we focused on how RPA will help the crew to do their job better. Benefits of RPA are reflected in more timely and accurate information for commanders, flexibility and responsiveness to changing situations, and efficient and effective mission task execution. These improvements are expected to be magnified at higher echelons of command. Maneuver force commanders will receive a clear view of the battlefield from a more responsive asset. Our evaluation plan encompasses several platforms, facilities, and evaluation methods. Our approach to human-centered design, the definition of an associate, and the linking of RPA behaviors to measurements is described. The process of mapping RPA concepts into the aviation mission down to actions taken by RPA will also be discussed. Preliminary design and diagnostic information is provided.