Human–machine interction plays an important role in modern industrial automation systems. In many industrial

environments, operators are required to interact directly with machines through switches, buttons, and control

panels. In hazardous environments such as chemical plants, radioactive facilities, and high-voltage power

stations, this direct interaction may expose operators to serious safety risks. This paper presents a computer

vision–based hand gesture control system that enables contactless interaction with machines. The proposed

Interface.

Industrial automation systems rely heavily on human–machine interaction for monitoring and controlling

equipment. Conventional control systems commonly use mechanical switches, push buttons, and touch-based

human–machine interfaces (HMIs). Although these systems provide reliable and precise control, they require

direct physical interaction between operators and machines. In hazardous environments such as chemical

processing plants, nuclear facilities, and high-voltage electrical stations, direct interaction may expose workers

to dangerous substances, toxic gases, or electrical hazards.

Recent advancements in artificial intelligence and computer vision have enabled the development of gesture-

control signals to an Arduino microcontroller to actuate servo motors.

Traditional industrial control systems rely on mechanical switches, push buttons, and touch-based control panels

to operate machinery. These systems have been widely used in industries due to their reliability, accuracy, and

predictable performance. Operators interact with these interfaces to send commands to control units such as

programmable logic controllers (PLCs) or relay-based systems. The control unit processes the input signals and

activates actuators such as motors, valves, or relays.

Despite their reliability, traditional control interfaces require direct physical interaction between the operator and

the machine. In hazardous environments, such as chemical plants or power generation facilities, this interaction

may expose operators to dangerous substances or unsafe working conditions. Therefore, there is a need for

The Media Pipe model detects multiple landmarks on the hand and determines the position of fingers and joints.

Based on these landmarks, the system identifies specific gestures performed by the user. Once a gesture is

recognized, the system converts it into a digital command that is transmitted to an Arduino microcontroller

through serial communication.

The Arduino receives the gesture command and generates pulse width modulation (PWM) signals to control

servo motors attached to the robotic hand mechanism. Each servo motor controls the movement of a finger in

the robotic hand. As a result, the robotic hand mimics the gesture performed by the user, demonstrating a

contactless method for controlling robotic systems

Fig. 1. System Architecture of the Proposed Hand Gesture Control System

Arduino microcontroller through serial communication.

The Arduino program receives the gesture data and maps it to appropriate PWM signals. These signals drive the

servo motors connected to the robotic hand mechanism. Each servo motor controls a specific finger, allowing

the robotic hand to replicate the user’s gesture. The process continues continuously to detect new gestures in real

time.

The Proposed Hand Gesture Control System Integrates Both Hardware and Software Components to Achieve

Real-Time Gesture Recognition and Robotic Hand Control. the Hardware Setup Mainly Consists of an Arduino

frames are transmitted to the computer where the gesture recognition algorithm processes the images. The

camera plays a crucial role in capturing high-quality visual data that enables accurate gesture detection. On the

software side, the system is developed using the Python programming language due to its strong support for

computer vision and machine learning libraries. The OpenCV library is used for image processing tasks such as

capturing video frames, resizing images, and performing basic pre-processing operations. The MediaPipe

framework is used for detecting hand landmarks and tracking finger positions in real time. MediaPipe provides

a pre-trained hand tracking model capable of identifying multiple key points on the human hand.

Once a gesture is recognized by the software, the corresponding control data is transmitted to the Arduino

microcontroller through serial communication. The Arduino program, written using Embedded C in the Arduino

IDE environment, interprets the received command and generates pulse width modulation (PWM) signals. These

acquisition, gesture processing, and robotic actuation. These modules work together to achieve real-time hand

hand gestures through a camera, which continuously records video frames during system operation. These frames

The gesture processing module performs several computer vision operations to identify the gesture performed

by the user. Initially, each captured frame is converted into a format suitable for processing using the OpenCV

library. Image preprocessing operations such as resizing, color conversion, and noise reduction are performed to

improve detection accuracy. After preprocessing, the MediaPipe framework is used to detect hand landmarks.

MediaPipe provides a machine learning-based hand tracking model that identifies 21 key points on the human

hand. These key points represent important finger joints and palm locations that allow the system to understand

the orientation and position of the hand.

Once the hand landmarks are detected, the system analyzes the relative positions of the fingers to determine the

gesture performed by the user. For example, when all fingers are extended, the system recognizes an open-hand

gesture. When all fingers are folded, the system interprets the gesture as a closed fist. Similarly, when two fingers

to specific control commands that correspond to the movement of the robotic hand.

The recognized gesture is converted into digital control signals and transmitted to the Arduino microcontroller

then received by the Arduino microcontroller. The Arduino program interprets this data and determines the

appropriate motor control signals required to replicate the gesture.

The robotic actuation module is responsible for converting the digital control commands into physical movement

enabling smooth and responsive interaction between the user and the robotic hand. This architecture

intuitive and contactless human–machine interface.

To evaluate the performance of the proposed system, a prototype experimental setup was developed using

commonly available hardware and software components. The system was implemented on a computer running

Python, where the OpenCV and MediaPipe libraries were used for gesture detection and processing. A standard

webcam was used to capture hand gestures, while an Arduino UNO microcontroller controlled the robotic hand

mechanism through servo motors.

During experimentation, different hand gestures were performed in front of the camera to test the system’s

recognition capability. The system successfully detected the hand landmarks and classified gestures in real time.

signals to control the servo motors. The robotic hand replicated the gestures performed by the user,

demonstrating the effectiveness of the system in translating visual gestures into physical robotic movement.

The experimental setup confirms that the proposed system is capable of achieving real-time gesture recognition

and robotic actuation with minimal delay. The results indicate that the combination of computer vision

techniques and embedded systems can provide an efficient and reliable approach for gesture-based control

Experimental results demonstrated that the recognized gestures were correctly transmitted to the Arduino

microcontroller through serial communication. The Arduino successfully generated PWM signals corresponding

to each gesture command. As a result, the servo motors controlling the robotic hand responded appropriately and

reproduced the user’s gestures.

The response time of the system was found to be sufficiently fast for real-time interaction. The use of optimized

computer vision libraries such as OpenCV and MediaPipe contributed to efficient image processing and gesture

detection. The results confirm that the proposed system can provide an effective contactless control mechanism

suitable for applications in industrial automation and human–machine interaction.

However, certain limitations were observed during testing. Variations in lighting conditions and camera

positioning can affect gesture detection accuracy. Additionally, the current system operates as an open-loop

3. M. Altayeb, “Hand Gestures Replicating Robot Arm Based on MediaPipe,” International Journal of