EXTRACTING ACTIONABLE INFORMATION FROM HETEROGENEOUS SENSORS IN THE FIELD: A DISTRIBUTED HYBRID AI APPROACH IN CONSTRAINED DOMAINS

We present a modular architecture that enables advanced surveillance functions exploiting data collected from heterogeneous sensors dispersed over multiple, often mobile platforms in the field. Examples of such functions are red forces tracking with surveillance gaps, detection of different types of anomalies, search and rescue operation monitoring, and threat alerting. This novel approach combines a distributed fusion engine, an intelligent process manager, and a system of ruggedized computers, enabling information processing in the tactical domain. The hybrid AI-based heterogeneous fusion engine consists of different algorithms, including various detectors and classifiers, represented as services in a light-weight information management and interoperability layer. This architecture layer enables context-dependent discovery of the right sensing and processing services at runtime that are combined using a robust Bayesian fusion layer exploiting complex correlations in the data. The discovered services are distributed over a network of computing nodes by an intelligent process manager, which optimizes network resource allocation according to communication and processing capacities. The fusion engine and the process manager are delivered to the tactical domain using the ruggedized SOTAS computing and communication infrastructure, achieving efficient, actionable, timely, and consistent situation awareness in constrained domains, such as military vehicles.

Citation: G. Pavlin, R. Boudreault, A. Penders, M. de Graaf, D. Lafond, A. Swiebel, “Extracting Actionable Information From Heterogeneous Sensors in the Field: A Distributed Hybrid AI Approach in Constrained Domains,” In Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium (GVSETS), NDIA, Novi, MI, Aug. 15-17, 2023.