Homework Assignment 2

Summary

For this assignment, I implemented template matching "from scratch" in openCV (using supporting datastructures, but not the built-in template matching functionality). I also attempted to perform rudimentary gesture detection by applying template matching to the input image after masking it against the regions I believed to be skin. This proved largely ineffective, most likely due to the noisiness of my skin detection algorithm and inefficiencies in my template matching code that did not permit attempting matches over a sufficiently large set of rotations.

I also attempted to implement detection of moving skin, with a moderate degree of success; I accomplished this by subtracting the detected skin in adjacent frames. Much as in the template matching case, the noisiness of my skin detection algorithm made this less effective than I'd have hoped, although I managed to compensate somewhat by only designating pixels as moving skin if they also appeared in the non-skin diff of the frames.

Sample output

Template matching on a tea packet:

tea

Frame difference on skin:

skinmotion

Skin-detection masked image:

skinimage

Confusion Matrix:

Gesture
Present
Gesture
Not Present
Gesture
Detected
0 5
Gesture
Not Detected
0 0

As I mentioned above, my gesture detection algorithm always identifies the "most similar" location to the gesture (ie, it never gives a negative result), and in my trials it only very rarely correlated with my actually having made the gesture (in the trial runs I did for the table, it never did, in some earlier testing I observed occasional success).

Note as of 9/23: I used a color detection algorithm from the following paper: http://arxiv.org/pdf/1212.2692.pdf, and discussed the written and programming portions of the assignment (and freely shared code examples with): marcocai@bu.edu, Megan Van Welie, Kyle Savidge, and Daniel Shih.