Homework Assignment 2: Object tracking and Gesture Detection

Part 1: Tracking a red Eraser Holder

The Algorithm

My algorithm is very simple. For every frame that is read in, the getRed function is run on it. This is a function that I created to extract all the red objects in the image, since my eraser holder is read. This makes every pixel that is red a binary white pixel, and every pixel that is not red a binary black pixel. This then makes it very easy to run the matchTemplate function, which is defined by opencv, which slides a box the size of the template image from left to right and top to bottom, creating a heatmap of what it thinks are template matches. I then use a normalization function, provided by the opencv library, to normalize the image to account for static or outliers. Finally, I use the point with the highest match percentage as the corner of a bounding box of set size that I draw over the origional camera frame.

Part 2: Template Matching Hand Gestures

A Quick Summary

Frankly, I found this part of the assignment extreemely difficult. I was unable to isolate my hand, let alone detect gestures. The closest I got was isolating all the parts of the image that are skin color, and then highlighting the borders with adaptive thresholding. This created extremely distinctive binary shapes that should be able to be deciphered into different parts of the body, however, I was unsuccesful at doing so. An example of the images created can be seen to the left in the debugging panels I created.