Using Deep Learning based Computer Vision in Helicopter Cockpits for Cognitive Decision Aiding

Collins Aerospace, through its Common Avionics Architecture System (CAAS) and Flight2 avionics management systems for rotary wing aircrafts, provides extensive video processing, internal graphics generation, and overlay capabilities on real time video streamed from onboard EO/IR cameras providing situational awareness to the pilot in clear day-light and reduced visibility/night conditions. This capability has served our customers well in their cargo, assault, and multi-mission roles, improving the effectiveness of their missions. We now realize that more can be done to reduce pilot workload and enhance mission effectiveness by extracting visual intelligence from the video feed using machine vision. In this paper we explore the use of deep learning based computer vision to extract visual intelligence from onboard video feed and use it to automate low risk pilot actions, such as automatic detection and tracking of objects of interest, panning to maintain focus on objects, zooming on to a chosen object and providing contextualized data link message recommendations.