A Cooperative Hand Gesture Recognition System for Human Machine Interface to Control Robotic Arms

2026-01-0252

4/7/2026

Features
Event
Authors
Abstract
Content
In the context of Industry 5.0, effective human–machine collaboration requires seamless and natural interaction. Hand-Gesture Recognition (HGR) has emerged as a promising technology for developing human–machine interfaces (HMI) that enable users to control robotic systems without physical controllers or wearable devices. This research presents a real-time HGR system designed to control a 6-Degree-of-Freedom (DoF) robotic arm using YOLOv10, a state-of-the-art deep learning model for hand gesture detection and classification. While YOLOv10 delivers high recognition accuracy, its computational demands surpass the capabilities of edge devices typically mounted on robotic platforms, creating a hardware bottleneck. To address this challenge, a cooperative client–server architecture is proposed, distributing computational workload between the edge device and a more powerful remote server. An RGB camera attached to the robotic arm captures hand gesture images and transmits them to the server via the User Datagram Protocol (UDP). The server performs real-time inference using YOLOv10 and returns the detection results to the edge device, which translates the recognized gestures into corresponding robotic arm movements. Experimental evaluation demonstrates an interfacing speed of approximately 15.7 frames per second and an 11.54 times improvement in performance compared with standalone edge-based processing. The proposed cooperative HGR system successfully integrates advanced computer vision techniques with robotic control to deliver a responsive, touch-free interface, enabling smooth, natural HMI. By overcoming edge-computing limitations, this research contributes to the advancement of Industry 5.0, supporting applications in healthcare, assistive robotics, industrial automation, and collaborative robotics, and promoting effective and safe human–machine collaboration.
Meta TagsDetails
Citation
DeHaven, A. and Park, J., "A Cooperative Hand Gesture Recognition System for Human Machine Interface to Control Robotic Arms," WCX SAE World Congress Experience, Detroit, Michigan, United States, April 14, 2026, https://doi.org/10.4271/2026-01-0252.
Additional Details
Publisher
Published
10 hours ago
Product Code
2026-01-0252
Content Type
Technical Paper
Language
English