Assignment Title

CS 585 HW 2
Rahul Ramachandran
Raj Vipani

Problem Definition

The goal of this assignment is to implement algorithms that can recognise shapes and gestures made by the hand.
The program should take video from the webcam as input, analyze and process it to recognise the hand gesture.
Parts of the assignment:
1. Skin detection
2. Template Matching
3. Recognize hand gestures.

Method and Implementation

Task 1 - Skin Detection

Task 2 - Hand Shape Recognition using Templates

Task 3 - Temporal Gesture Recognition


The following are the resulting images from the program:


Trial Template Gesture
Thumbs up
Peace Out

A confusion matrix can be obtained by changing the handshape slightly to see what will the program recognize it as. Here is the result.

Hand ShapePalmPeaceThumbs UpWave
Thumbs Up0190


  • Skin Detection is dependent on every individual but we found out that lighting makes a big difference, along with camera quality.
  • We were also able to detect multipile gestures in a frame and displayed it on top.
  • What we would like to improve is the multiple bounding boxes being displayed for the same gesture.
  • We implemented this assignment in C++, since python is a lot slower than C++ because of video processing.