Hand Gesture Control System for Robotic Hand Using Computer Vision

Human–machine interction plays an important role in modern industrial automation systems. In many industrial environments, operators are required to interact directly with machines through switches, buttons, and control panels. In hazardous environments such as chemical plants, radioactive facilities, and high-voltage power stations, this direct interaction may expose operators to serious safety risks. This paper presents a computer vision–based hand gesture control system that enables contactless interaction with machines. The proposed system uses a camera to capture hand gestures, which are processed using Python with OpenCV and MediaPipe libraries. The system detects hand landmarks and interprets gestures in real time. The recognized gesture is converted into control signals and transmitted to an Arduino microcontroller through serial communication. The Arduino generates PWM signals to control servo motors connected to a robotic hand mechanism that replicates the user’s gesture. The developed prototype demonstrates the feasibility of integrating artificial intelligence and embedded systems for safer human–machine interaction in industrial environments.

Hand Gesture Recognition, Computer Vision, Open CV, Media Pipe, Arduino, Human Machine Interface.

1. J. Qi, L. Ma, Z. Cui, and Y. Yu, “Computer vision-based hand gesture recognition for human–robot interaction: A review,” Complex & Intelligent Systems, 2023. [Google Scholar] [Crossref]

2. H. H. Li, “Dynamic Hand Gesture Recognition Using MediaPipe and Transformer Model,” MDPI Engineering Proceedings, vol.108, no.1, 2025. [Google Scholar] [Crossref]

3. M. Altayeb, “Hand Gestures Replicating Robot Arm Based on MediaPipe,” International Journal of Electrical Engineering and Informatics, 2023. [Google Scholar] [Crossref]

4. S. Chavan et al., “Real-Time Hand Gesture Recognition Based Control of Arduino Robot,” International Journal of Trend in Scientific Research and Development, vol.6, no.4, pp.79-83, 2022. [Google Scholar] [Crossref]

5. A. S. Amirtha Varshini et al., “Real-Time Hand Gesture Recognition for Robotic Arm Control,” ACM International Conference Proceedings, 2021. [Google Scholar] [Crossref]

6. I. Darmawan et al., “Hand Gesture Recognition for Collaborative Robots Using Lightweight Deep Learning Models,” arXiv, 2025. [Google Scholar] [Crossref]

7. V. Chaudhari, P. Mhatre, and J. Bind, “Hand Gesture Control Robot Using Arduino,” VIVA-Tech International Journal for Research and Innovation, 2024. [Google Scholar] [Crossref]

8. A. Ahmed et al., “Tracked Robot Control with Hand Gesture Based on MediaPipe,” ResearchGate Technical Paper, 2024. [Google Scholar] [Crossref]

9. M. Oudah et al., “Hand Gesture Recognition Based on Computer Vision: A Review,” Sensors Journal, 2020. [Google Scholar] [Crossref]

10. “Real-Time Robotic Hand Interaction Control Using MediaPipe for Gesture-Based Applications,” Robotics Research Article, 2025. [Google Scholar] [Crossref]

• Investigating the Performance of Colour Oxide on the PS/CNSO Oil Paint Production
• Real-Time Fabric Defect Detection Using a Lightweight Deformable YOLO Network
• Recent Trends in the Stock Market: An Analytical Study of Market Dynamics, Investor Behaviour, and Technological Influence
• Evaluation of Structural Dynamics and Equilibrium State, with Case Study of Large Cantilever Projection for a 10- Storey Reinforced Concrete Building in Lagos, Nigeria.
• Multi-Criteria Evaluation of AI-Based Adaptive Learning Platforms in Global Higher Education: A Fuzzy AHP Perspective