AI Virtual Painter
π Overview
Draw in mid-air in real time: raise your index finger and move it around the frame to draw a line, hold up different numbers of fingers to switch colour, and use a fist or similar gesture to clear the canvas.
On top of hand tracking it adds canvas state management (draw / don't draw / clear), which makes it the best beginner project for understanding a gesture state machine.
π§° What you need
- Computer + camera
- Python 3.7+ environment
- pip install: opencv-python, mediapipe, cvzone
- Optional: a large monitor makes drawing much nicer
π§ Step by step
Hand-tracking basics
Use HandTrackingModule to get the 21 landmarks
Pick out the index, middle, and ring fingertip coordinates
Design the canvas
Create an all-black canvas with numpy and blend it over the camera frame
Keep the canvas the same size as the frame so coordinates line up
Decide the drawing state
Index finger only = draw mode, and record the fingertip trail
Index and middle finger together = don't draw (move mode)
Draw the line
Join the previous frame's fingertip to the current one with cv2.line
The trail stays continuous and feels natural
Switch colour
Hold up 3-4 fingers in the colour strip at the top to pick a colour
Make colour selection a 'point and hold on the swatch' interaction
Clear the canvas
Assign a gesture (a fist, for example) to reset the canvas
Refactor the code into a reusable drawing-board function
π‘ Tips
- Verify the fingertip coordinates line up with the frame before drawing, or your strokes land in the wrong place
- Use a uint8 array for the canvas and add colour gradients for a nicer look
- Debounce the gesture state with a small delay; quickly changing finger counts makes it flicker