Hand Tracking

📌 Overview

Hand tracking is the classic entry point into CV Zone: a camera detects hands in real time and locates the 21 hand landmarks (fingertips, knuckles, wrist, and so on).

Under the hood it uses Google's MediaPipe hand model plus OpenCV for the video feed, and the video author wrapped it into the cvzone HandTrackingModule so it runs in just a few lines of code. Later projects such as the virtual mouse, virtual painter, and finger counter all build on this module, which is why it is considered the foundation of hand-gesture applications.

🔬 How it works

1. How to read the 21 hand landmarks

MediaPipe outputs 21 landmarks per hand with fixed indices: 0 is the wrist; 1-4 the thumb (4 is the tip); 5-8 the index finger (8 is the tip); 9-12 the middle finger (12); 13-16 the ring finger (16); 17-20 the little finger (20).

Remember the rule 'fingertip indices = 4 / 8 / 12 / 16 / 20' - the virtual mouse, finger counter, and volume control all read these points and reason about their geometry.

2. What the HandDetector wraps

cvzone's HandTrackingModule packs three things into a few lines of API: running the MediaPipe Hands inference, converting normalised coordinates back to frame pixels, and tidying the result into a dictionary.

The dictionary returned for each hand has four keys: lmList (a list of 21 [x, y, z] points), bbox (the hand bounding box x/y/w/h), center (the palm centre), and type (Left/Right).

It is the shared foundation of the rest of the gesture projects on this site - learn this one and you already have the detection half of the virtual mouse, virtual painter, and finger counter.

3. How recognition works (the plain-English version)

MediaPipe Hands works in two steps: a lightweight network first locates the hand in the whole frame (palm detection), then keypoint regression runs on the cropped hand region; once a hand is found, later frames take the faster tracking path, so it comfortably hits real-time frame rates.

The trackCon and detectionCon confidences control whether it tracks or re-detects: when the hand is lost (confidence drops below trackCon) it automatically falls back to full detection.

🧰 What you need

• Computer + camera (a built-in laptop camera is fine)
• Python 3.7+ environment
• pip install: opencv-python, mediapipe, cvzone
• Optional: external USB camera for better image quality

🔧 Step by step

💻 Full code examples

# Dependencies: pip install opencv-python mediapipe==0.10.14 cvzone
# Note: cvzone still uses the older MediaPipe Hands API,
#       installing the latest mediapipe breaks the solutions module - pin 0.10.14
import cv2
from cvzone.HandTrackingModule import HandDetector

cap = cv2.VideoCapture(0)                # open the camera (0 is the default camera)
detector = HandDetector(detectionCon=0.8, maxHands=1)   # detection confidence 0.8, at most 1 hand

while True:
    ok, img = cap.read()                 # read frame by frame
    if not ok:
        break
    img = cv2.flip(img, 1)               # mirror horizontally, like looking in a mirror

    hands, img = detector.findHands(img) # detect and draw the 21-point skeleton, returns the list of hands

    if hands:
        hand = hands[0]
        lmList = hand["lmList"]          # 21 landmarks [[x, y, z], ...], z is depth
        bbox   = hand["bbox"]            # (x, y, w, h)
        cx, cy = hand["center"]          # palm centre
        tip    = lmList[8][:2]           # index fingertip coordinate (index 8)
        cv2.circle(img, tip, 8, (0, 255, 255), cv2.FILLED)  # highlight the index fingertip in yellow

    cv2.imshow("Hand Tracking", img)
    if cv2.waitKey(1) & 0xFF == ord("q"):  # press q to quit
        break

cap.release()
cv2.destroyAllWindows()

import cv2
from cvzone.HandTrackingModule import HandDetector

cap = cv2.VideoCapture(0)
detector = HandDetector(detectionCon=0.8, maxHands=2)   # detect up to 2 hands at once

while True:
    ok, img = cap.read()
    if not ok:
        break
    img = cv2.flip(img, 1)
    hands, img = detector.findHands(img)   # findHands already draws the skeleton and box by default

    if hands:
        for hand in hands:
            # five fingertip indices: thumb 4, index 8, middle 12, ring 16, little 20
            for i in [4, 8, 12, 16, 20]:
                x, y = hand["lmList"][i][:2]
                cv2.circle(img, (x, y), 7, (0, 255, 0), cv2.FILLED)
            x, y = hand["bbox"][:2]
            cv2.putText(img, hand["type"], (x, y - 10),
                        cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 255), 2)

    cv2.imshow("Hand Tracking", img)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

⚙️ Key parameters

💡 Tips

• The model is most stable when the whole hand is in frame; fingers together or occluded reduce detection confidence
• The larger the camera frame, the more CPU detection costs; 640x480 is the best value
• If no hand is detected, check that the lighting is even and avoid strong backlight

🧯 Troubleshooting

📦 Course & resources

• Official course (free, 2 lessons) ↗The original course page with the official video and written steps; the video requires a login and the written content is copyright CV Zone (Murtaza's Workshop).
• cvzone official GitHub ↗Full HandTrackingModule source plus more official examples to read and learn from.
• MediaPipe Hands official docs ↗Google's official hand landmark documentation, useful for the latest API and the definition of the 21 points.
• Search on Bilibili ↗If the original site or YouTube is unavailable, search Bilibili for a video walkthrough of the same tutorial.