Painting Analysis

Final Project Report

CS 585 Fall2018
Kaiyuan Fan
kaikang Zhu

Project Problem Definition

Motivation: Help art historians to analyze paintings. There are paintings that unlabelled with their year of production. We can estimate the painting production period by computer vision.

  1. Detect, count and label the people in the painting. Analysis if they are naked or not.
  2. Training with classified dataset. Then given a test painting, we should recognize it's color scheme and estimate the year of production.
Painting Year Classification:
Early Years (1500-1529)
Middle Years (1530-1550) - In 1525 he married Cecilia (who tragically died in 1530). Titian's meeting with the Holy Roman Emperor Charles V in Bologna in 1530 would be a determining event in his life.
Late Years (1550+) - Starting from the late 1550s, Titian developed a much freer use of the brush and a less descriptive representation of reality.

We analyzed and studied the Italian painter Titian through searching his autobiography and art style. We used computer vision techniques to extract attributes from images, such as skin color, number of people shown in the paintings, the nude area of the people, the color schemes of the images and so on. After acquiring the attributes, we used machine learning techniques Support Vector Machine to train and test the attributes.

Methods and Implementation

We are collecting attributes from Titian's paintings. Firstly, we are calculating the color scheme of the paintings. We convert the RGB image into HSV image.
We divided the Hue value into six sections which stand for six different color ranges. Then we compute the average value of Saturation and brightness.
After that, we applied the skin detect and use connected blobs approaches to count the number of people in the image and the nudity(The proportion of naked body).To acquire the skin color, We partly borrowed the skin color detection code taught in the class. However, the skin color of painting has more yellow pixels and seem darker than the normal skin color. So we adjusted the thresholds for red and blue a bit higher and took the brightness of image into consideration. The results can be shown like:

orignial image skin color in class skin color after change
Then dilated and eroded the image and turn it into binary image. Use built-in function ConnectedComponent to get all the connected areas. If one area is larger than the average value, we considered it as one people. It served as one of our attribute. Then we summed up all the skin pixels and diveded it by the number of people, it is nudity which also served as one of our attributes.
Basically, the number of people and nudity can be correctly detected in images with 1-2 people. However, for those images with over 6-7 people, the methods don't work well.

At this moment, we have a total of ten attributes before we implement the classification with SVM.

  • 1-6. Color Distribution Percentages
  • 7.Saturation(colorfulness)
  • 8.Brightness
  • 9.Number of People
  • 10.Nudity
  • Below shows an example data collected from one image:

    example dataset

    We normalized data before we use them into SVM. We implemented the SVM by Scikit-learn package in Python. We used linear kernal and one vs one models. We have 50 training dataset for each class of production year period. We tested 17 from early, 18 from middle and 26 from late period. Total accuracy is 28/61=45.9%
    example dataset

    Related Paper and Algorithms

  • Machine Learning Algorithms That Can Analyze Works of Art
  • Using Machine Learning for Identification of Art Paintings
  • Detect People in Cubist Art

  • Conclusion

    The resulting accuracy is not ideally high, but it's in our expectation and we are satisfied with it. The quality of datasets are inconsistent, they have different resolutions. Also, we have only around 200 of the dataset which we believe are not enough for completely classify different periods of Titian's work. Meanwhile, few paintings from different periods have very close attributes.

    For future works, we will add more attributes to estimate the production year. We hope to get a more precise result for the number of people and nudity. So we need to adjust the skin color detection and come up with a better way to find people in connected blobs. Our implementation gave us a fair accuracy to classify Titian's work. We believe our project can deliver better results if we apply to compare with different artists since the painting style will have greater differences than that we only apply to Titian's paintings.