Assignment Title

CS 585 HW 2
Rahul Ramachandran
Raj Vipani
12/2/2020


Problem Definition

The goal of this assignment is to implement algorithms that can recognise shapes and gestures made by the hand.
The program should take video from the webcam as input, analyze and process it to recognise the hand gesture.
Parts of the assignment:
1. Skin detection
2. Template Matching
3. Recognize hand gestures.


Method and Implementation

Task 1 - Skin Detection

Task 2 - Hand Shape Recognition using Templates

Task 3 - Temporal Gesture Recognition



Results

The following are the resulting images from the program:

Results

Trial Template Gesture
Palm
Thumbs up
Peace Out
Wave

A confusion matrix can be obtained by changing the handshape slightly to see what will the program recognize it as. Here is the result.

Hand ShapePalmPeaceThumbs UpWave
Palm9100
Peace2710
Thumbs Up0190
Wave00010

Conclusion

  • Skin Detection is dependent on every individual but we found out that lighting makes a big difference, along with camera quality.
  • We were also able to detect multipile gestures in a frame and displayed it on top.
  • What we would like to improve is the multiple bounding boxes being displayed for the same gesture.
  • We implemented this assignment in C++, since python is a lot slower than C++ because of video processing.