Assignment Title

CS 585 HW 2
Yicheng Li
Zilin Zhang
Fangya Xu
Date Feb. 2020

Problem Definition

Design and implement algorithms that recognize hand shapes (such as making a fist, thumbs up, thumbs down, pointing with an index finger etc.) or gestures (such as waving with one or both hands, swinging, drawing something in the air etc.) and create a graphical display that responds to the recognition of the hand shapes or gestures.

Method and Implementation

1. Read a new frame from video stream and detect skin color of this frame.

2. Firstly get greyscale of this picture and then we make it to be a binary image.

3. Label the connected fields of the template picture by seed filling algorithm and then cut the largest one as the template.

4. Scan the original image through a window with template size(once in two seconds).

5. Sum the amount of different pixels between origin image and the template image and calculate the error rate.

6. Use a rectangular box to locate the hand shape with the smallest error rate at most 0.2 in the original image.


Confusion Matrix


Hand Shapes Result Image



  • Firstly, we find our algorithm would be greatly affected by different aspects but some of them make us really confusing.For example, the light condition and the difference between cameras. So it really raise our confusion that how can we make algorithms that can be applied in different cameras since we can't know the condision of cameras and environments before we really see it.
  • Secondly, we find it really important that how we choose our threshold. If the threshold is too low, the algorithm would not be able to regonize gestures since the environment is constantly changing. And if the threshold is too high, irrelevant backgrounds like a head may be recognize as a fist.