A Cooperative Hand Gesture Recognition System for Human Machine Interface to Control Robotic Arms

In the context of Industry 5.0, effective collaboration between humans and machines demands seamless and natural interaction. Hand-Gesture Recognition (HGR) is a promising technology for creating intuitive human-machine interfaces (HMIs) that allow users to control robots without physical devices or wearables. This research introduces a real-time HGR system designed to control a 6-Degree-of-Freedom (DoF) robotic arm using YOLOv10, a state-of-the-art deep learning model for hand gesture recognition. While YOLOv10 offers high accuracy, its computational demands exceed the capabilities of edge devices typically attached to robotic arms, posing a significant hardware challenge. To overcome this, we propose a cooperative client-server architecture that distributes the workload between the edge device and a more powerful remote server. An RGB camera mounted on the robotic arm captures hand gesture images and sends image data to the server using User Datagram Protocol (UDP). The server performs real-time inference with YOLOv10 and returns the detection results to the edge device. The edge device then translates the recognized gestures into corresponding robotic arm movements, achieving an interfacing speed of 20 Frames Per Second (FPS). This system successfully merges advanced computer vision techniques with robotic control to provide a responsive, touch-free interface. The proposed cooperative HGR architecture enables smooth and natural human-robot interaction, which is crucial for applications in healthcare, assistive robotics, industrial automation, and collaborative robotics. By addressing the limitations of edge computing through a cooperative architecture, this research contributes significantly to advancing Industry 5.0’s vision of human-machine collaboration.