Tmeplate Matching, Skin Color Differencing, Background Differencing and Frame to Frame differencing
This program is to analyze the camera captured video. Based on our understanding, we are supposed to do skin color differencing
to finde the hands in the video. Then through the background differencing and frame to frame differencing, we can analyze the
video, get some usful information like movement energy (the moving object in the video), the objects' orientation and even the
changing of the background.
The most interesting in this case is the template matching problem. And our team basically follow the process as:
1. Find the hands.
2. Get binary image that only contians hands and the background.
3. Resize the template image based on the height of the hand object.
4. Do template matching and finde the most matching location (correlation coefficient).
5. Out put the result.
Method and Implementation
We used a lot of API of Opencv since it is hard to implement them by ourselves, but we do make sure that we know exactly what they are doing. Actually, during the process, we got a strong feeling that, the most difficult part of treating images with computer is still the "object detecting." Bascially, if we know what pixels exactly describe an object, it is very easy for us to get other information about the object including doing template matching.
void MatchingMethod(Mat &frame, vector
void processCamera(VideoCapture &cap, Mat &fg) - background and frame differencing.
void mySkinDetect(Mat& src, Mat& dst) - skin color detecting.
int Read_Rz_Img(std::string name, Mat &img) - resize the template.
We have experiment with one hand and both hand.
We want to find out if we successfully recognize the hand shape that we make as well as the individual template matching score we received
One hand result
Basic HTML Table
- The matching score of our template matching is not that stable, we think this is due to the imperfect of object detecting. As mentioned before, all more sophisticated computations are based on the result of object detecting. Thus, we need to dig more about this very fundamental procedure.
- We noticed that the template matching API of Opencv is unpredicable (the return value). And as a lib function, its accurency and efficient are limited. Therefore, maybe it is a good idea to implement our own template matching algorithm to adapt variant conditions.
The process of developing this program has brought us a lot of experiences of computer vision analysis. We also noticed that it is very time-consuming to approach a good algorithm in this field. We will take more efforts in the future study.