Capture
OpenCV mirrors a 1280 × 720 webcam feed for natural interaction.
Selected workHand
Computer vision × physical computing
Hand
Gesture LEDs
Five fingers become five bits. Five bits become five physical lights.
10101
Project overview
A visible bridge between software perception and hardware response.
The project uses a laptop webcam to recognize which fingers are open, reduces that hand pose to a five-bit message, and sends the state to an Arduino Nano. Each bit controls the LED in the same physical finger position.
Opening the middle finger produces 00100 and lights the center LED. Opening several fingers produces several active bits, so the matching lights respond together.
BitFingerPhysical output
0ThumbLeft LED
1IndexSecond LED
2MiddleCenter LED
3RingFourth LED
4PinkyRight LED
The control loop
From a moving hand to a stable binary command.
Track
MediaPipe returns 21 three-dimensional landmarks for one detected hand.
Classify
Joint angles and palm-relative distances determine each finger’s state.
Stabilize
A five-frame majority vote prevents small tracking changes from flickering LEDs.
Transmit
Only changed five-bit commands are sent over serial at 9,600 baud.
Respond
The Arduino Nano maps each bit to the corresponding physical LED.
Detection logic
Geometry, not screen position.
A simple rule such as “tip above knuckle” breaks when the hand rotates. This implementation checks two 3D joint angles for every regular finger. The thumb gets its own test because its motion is different: two angles plus its distance from the index knuckle, normalized by palm width.
Finger PIP> 155°
Finger DIP> 150°
Thumb MCP> 145°
Thumb distance> 0.75× palm
hand_led.pyFINGER CLASSIFICATION
pip_angle = angle_3d(mcp, pip, dip)
dip_angle = angle_3d(pip, dip, tip)
return (
pip_angle > 155
and dip_angle > 150
)
# Five classifications become one command
command = "".join(str(state) for state in states)
connection.write(f"{command}\n".encode("utf-8"))Signal stability
No flicker from one uncertain frame.
Landmark detection shifts slightly between camera frames. A deque retains the most recent five classifications, then each finger gets an independent majority vote. This keeps a brief tracking error from becoming a visible hardware change.
FrameTIMRP
0110101
0210101
0310001
0410101
0510101
Reliability decisions
Small safeguards make the prototype feel immediate.
Transmit only changes
The previous command is retained, so identical frames do not flood the serial link with redundant messages.
Fail closed
If the hand disappears, history is cleared and 00000 turns every LED off instead of preserving a stale gesture.
Release cleanly
On exit, the app makes one final all-off write, releases the camera, closes the interface window, and closes serial.
Diagnose connection errors
If the configured port cannot open, the program lists detected serial devices to make setup failures easier to resolve.
Outcome
A gesture becomes a protocol.
A protocol becomes light.
This prototype demonstrates a complete real-time physical computing loop across computer vision, geometric classification, temporal filtering, serial communication, microcontroller output, and safe cleanup.