CS 585 HW 3
Guangxing REN
TEAMMATE Jiafei Xue You Xi Oct 10th

Tmeplate Matching, Skin Color Differencing, Background Differencing and Frame to Frame differencing

This program is to analyze the camera captured video. Based on our understanding, we are supposed to do skin color differencing to finde the hands in the video. Then through the background differencing and frame to frame differencing, we can analyze the video, get some usful information like movement energy (the moving object in the video), the objects' orientation and even the changing of the background.
The most interesting in this case is the template matching problem. And our team basically follow the process as:
1. Find the hands.
2. Get binary image that only contians hands and the background.
3. Resize the template image based on the height of the hand object.
4. Do template matching and finde the most matching location (correlation coefficient).
5. Out put the result.

Method and Implementation

We used a lot of API of Opencv since it is hard to implement them by ourselves, but we do make sure that we know exactly what they are doing. Actually, during the process, we got a strong feeling that, the most difficult part of treating images with computer is still the "object detecting." Bascially, if we know what pixels exactly describe an object, it is very easy for us to get other information about the object including doing template matching.

Function involved:
void MatchingMethod(Mat &frame, vector &tmps, Mat &img_display, int type) - template matching.
void processCamera(VideoCapture &cap, Mat &fg) - background and frame differencing.
void mySkinDetect(Mat& src, Mat& dst) - skin color detecting.
int Read_Rz_Img(std::string name, Mat &img) - resize the template.


We have experiment with one hand and both hand.

We want to find out if we successfully recognize the hand shape that we make as well as the individual template matching score we received


One hand result


Trial Result Image
trial 1
trial 2
trial 3
trial 4

In Generally, the results looks OK

two hand result


Trial Result Image
trial 1
trial 2
trial 3

Basic HTML Table

matrix plam fist thumb victory
plam 5 3 1 1
fist 2 4 2 2
thumb 3 0 5 2
victory 2 2 3 3



The process of developing this program has brought us a lot of experiences of computer vision analysis. We also noticed that it is very time-consuming to approach a good algorithm in this field. We will take more efforts in the future study.