CS585 IVC HW1

CS 585 HW 1
Wei Jiang
2017-09-11


Problem Definition

Use OpenCV to manipulate image pixels, tranfer an RGB colorful image into grayscale. Adding blur to the grayscale image. Interesting: Find a bounding box of my face based on skin color.


Method and Implementation

1. How to make it grayscale?

Using Mat datastructure in OpenCV, calculate the gray value using: grayValue = (R+G+B)/3. Then assign the grayValue to every channels in the image.

2. How to make it blur?

Build a temp mat that is a copy of input image. Average the gray value on horizontal direction of the temp mat. Average the gray values on vertical direction of the input map based on the temp mat. Average twice, and you will get a image that is averaged from 3*3 kernel. The weights are the same for every pixel in the kernel. As for the pixels on the edge of the image, I only consider the pixels inside the kernel. I blured the gray scale image 5 times to get better result.

3. How to find the face bounding box based on skin color?

First find the skin color value, of the input image, using Digital Color Meter, I got the hard coded skin color is RGB : (223, 190, 172) I try to find all the pixels that compares with (223, 190, 172) has the RGB space distance smaller than 60. Than I will get the left, top, and right boundary. I hard coded the face ratio to 1.3. Then I can draw the bounding box using OpenCV functions.

1. Code for making it grayscale

int index = col*channels + 0;
int grayValue = (rowPtr[index] + rowPtr[index+1] + rowPtr[index+2])/3;
rowPtr[index] = grayValue;
rowPtr[index + 1] = grayValue;
rowPtr[index + 2] = grayValue;
The 'rowPtr' is the pointer to the start of every row, and the 'index' is the index of the blue channel of the column in this row. Then I assign the gray values back to the image.

2. Code for making it bulr

for (int row = 0; row < gray.rows; row++)
{
	for (int col = 0; col < gray.cols; col++)
	{
		int BChannelIndex = row*gray.cols * channels + col * channels + 0;
		//corner case
		if (col == 0)
		{
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels]) / 2;
			//change to g channel
			BChannelIndex++;
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels]) / 2;
			//change to r cahnnel
			BChannelIndex++;
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels]) / 2;
			continue;
		}
		if (col == gray.cols - 1)
		{
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex]) / 2;
			//change to g channel
			BChannelIndex++;
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex]) / 2;
			//change to r cahnnel
			BChannelIndex++;
			blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex]) / 2;
			continue;
		}
		blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels])/3;
		//change to g channel
		BChannelIndex++;
		blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels])/3;
		//change to r cahnnel
		BChannelIndex++;
		blurPtr[BChannelIndex] = (imgPtr[BChannelIndex - channels] + imgPtr[BChannelIndex] + imgPtr[BChannelIndex + channels])/3;
	}
}

for (int row = 0; row < gray.rows; row++)
{
	for (int col = 0; col < gray.cols; col++)
	{
		//swap frames
		int BChannelIndex = row*gray.cols * channels + col * channels + 0;
		//corner case
		if (row == 0)
		{
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 2;
			BChannelIndex++;
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 2;
			BChannelIndex++;
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 2;
			continue;
		}
		if (row == gray.rows - 1)
		{
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex]) / 2;
			BChannelIndex++;
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex]) / 2;
			BChannelIndex++;
			imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex]) / 2;
			continue;
		}
		imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 3;
		BChannelIndex++;
		imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 3;
		BChannelIndex++;
		imgPtr[BChannelIndex] = (blurPtr[BChannelIndex - gray.cols * channels] + blurPtr[BChannelIndex] + blurPtr[BChannelIndex + gray.cols * channels]) / 3;
	}
}
I first do a covolution for every row, with 3 pixels, central, left and right. Then I do a convolution for every colmun, with central, up and down pixels. As for the edges of the image, I only consider the pixels inside the kernel.

3. Code for finding the face bounding box based on skin color

for (int row = 0; row < myface.rows; row++)
{
	for (int col = 0; col < myface.cols; col++)
	{
		int BChannelIndex = row*myface.cols * channels + col * channels + 0;
		int b1 = imgPtr[BChannelIndex];
		int g1 = imgPtr[BChannelIndex+1];
		int r1 = imgPtr[BChannelIndex+2];
		float distance = RGBdistance(b1, g1, r1, 172, 190, 223);
		//hard coded threshold
		if (distance < 60)
		{
			imgPtr[BChannelIndex] = 255;
			//imgPtr[BChannelIndex+1] = 0;
			//imgPtr[BChannelIndex+2] = 0;
			upper = row < upper ? row : upper;
			left = col < left ? col : left;
			right = col > right ? col : right;
		}
	}
}

rectangle(
	myface,
	cv::Point(left, upper),
	//hard coded face ratio.
	cv::Point(right, upper + (right-left)*1.3),
	cv::Scalar(0, 0, 0)
	);
In the for loop, I found all the pixels whose RGB value is within a distance threshold (60) compared with the harded coded skin value. Then I draw a bounding box based on the upper, left and right boundaries, with the hard coded face ratio (1.3).


Experiments

I blured gray scale image 5 times. My skin color is RGB:(223, 190, 172), my RGB distance threshold is 60, my face ratio is 1.3.


Results

Results

Trial Source Image Result Image
Gray scale
Blue
Face bounding box based on skin color


Discussion

Discuss your method and results:


Conclusions

We need to pay attention to the index when access to the image pixels data. And the order in OpenCV is BGR. The coordinate start from left upper corner, and herizontal is X, vertical is Y.


Credits and Bibliography

Discussed with Youcun Liu.