CS 585 HW 2
Yize Xie U14485891
Design and implement algorithms that recognize hand shapes (such as making a fist, thumbs up, thumbs down, pointing with an index finger etc.) or gestures (such as waving with one or both hands, swinging, drawing something in the air etc.) and create a graphical display that responds to the recognition of the hand shapes or gestures.
We use template matching to recognize different hand shapes. For the criteria, we choose normalized correlation that is introduced during lecture, which is a robust and accurate method. Our algorithm has several stages.
we choose 5 templates: thumb-up, thumb-down, yay, palm and fist. They are converted to grayscale image by our skin detection algorithm (introduced on lab, use RGB value to decide whether a pixel belongs to human skin).
That is , we convert each skin pixel to grayscale, but background pixel to 0(black). For example:
We capture image from user camera per 1/30s in order to have 30FPS ideally. For each captured image
I conduct following experiment: test the program with 4 different true gestures, each 10 times, and record the detected result.
The confusion matrix is drawn as following. Columns are true labels, and rows are detected labels. The order is: thumb-up, thumb-down, palm and fist.
Besides, we compare our calculation with OpenCV library: matchTemplate, and they give same output. However, the algorithm of OpenCV is much faster because it uses Discrete Fourier Transformation to accelerate the multiplication process.
The interesting points are, our algorithms is somewhat negative and pessimistic. It tends to recognize everything as thumb-down, which may be caused by light condition or our template.
My teammates are Wenxing Liu and Weifan Chen.