Art Painting Age Approximation Based on Colour Attribute

Presentation

CS 585 Project
Ding Jin
December 1 2017


Intuition

Colour usage in art changes from culture to culture and season to season and is often thought of as reflecting or inspiring mood and ambience. The painting techniques has been evolved throughtout the history; aside all the changes, the colours that used in painting remain almost the same. I believed there are certain roles specifically on colours that will make better painting recoginition and image knowledge extraction.The motive for this project is that the traditional chemical method to detect the painting age could damage the oil and the brushstroke, thus I am interested finding a way to do it without touching the painting itself.


Proposal


Progressions

The original thought is to have a independent style dectection algorithm, however when tesing the art style dectection and the showed the first 25 results to the custodial staff at Boston museum of fine arts, I found two problems:

However, it did give me the idea that the conception of art style is actually tied with the art period; and it will be more of a "tag" with age instead of a indepdent detection. Thus, the approach is to extract certain parameters from each image and build the training set based on those vector-formed parameters, then predict a new image's age(class). I can, of course, use the HSL value as the parameter which would gain too little information to build a vector; or record and use every pixel overwhelmingly which would take too much time with a large amount of data. Thus, I chose to extract certain parameter that can represent certain features of that image but would not take too much time for computing. And that is what I did later.

Based on the approximate art period, I found five breakpoints that related to the color analysis to separate the age(to be the class later) starting from Renaissance:

From the features I had above, I can translate parameters into the more quantifible data for the classification later: Below are the reasons why I choose those features and how I extracted them:

I used the scikit-learn Python package for my classfication algorithm and I first choose the nearest neighbour as the classification criterion. However, the problem is if I do not attempt to construct a general internal model, it will simply stores instances of the training data. This model building could be a real hard one. Thus I choose SVM as the classification algorithm; for several advantages, it peroforms realatively well:

Fortunately, SVM support multi-class classification. Assuming the vectors and the age are linear related(It is questionable to assume so but as far as I can think of, large data amount could somehow mitigate this disvantage), it will then contruct 5*(4-1)/2=10 classifiers(I labeled 5 classes) and each classifer would trains data from two classes.(See the Mathematical formulation here)


Methods

We contruct the vector as [R1,R2,R3,R4,R5,R6,P,M,F1,F2] with class C; the training set and the test set would be converted to 2D vector lists: V_training, V_test and 1D class lists: C_training, C_test.
SVM_Fit(V_training, C_training) and then predict from V_test which produces 1D predicted class list: C_predict
Compare C_test and C_predict, the accuracy A will be: (Number of same elements in C_test and C_predict)/(Number of test data points)


Results

The result was plotted by the point (x,y) where
x = The painting age
y = The difference value between the test class and the predicted class
(e.g. P_1=(1622,0) means a painting from 1622 was predicted the same age period with its actual age period; P_2=(1746,1) means a painting from 1746 was predicted 1 age period ahead of its actual age period)

Result 1: 540 training images; 152 test images:

Result 2: 33 training images; 89 test images (images are all low resolution for the purpose of faster demo):


Conclusion

The motive for this project is that the traditional chemical method to detect the painting age could damage the oil and the brushstroke, thus I was thinking if there is a way to do it without touching thepainting itself. As far as I see, I managed to finish the part of approximate age detection combining the art period identifcation. There are some possible improvment that I could not do due to my knowledge limitation:

Also, if considering the art history and the painting techniques evolution, then there would be some interesting observations from the results:


References

Using Machine Learning for Identification of Art Paintings http://cs229.stanford.edu/proj2010/BlessingWen-UsingMachineLearningForIdentificationOfArtPaintings.pdf

Pigments through the ages: Renaissance and Baroque (1400-1600) http://www.webexhibits.org/pigments/intro/renaissance.html

Image Processing for Artist Identification --Computerized analysis of Vincent van Gogh’s painting brushstrokes https://people.ece.cornell.edu/johnson/SPM-7-08.pdf

Mathematical Models of Perception and Generation of Art Works by Dynamic Motions https://www.psychologie.uni-heidelberg.de/ae/allg/mitarb/jf/Schubert%20etal%202014%20art%20works.pdf

RGB Color Cube-Based Histogram Specification for Hue-Preserving Color Image Enhancement http://www.mdpi.com/2313-433X/3/3/24/pdf

Searching patterns in painting images with computer vision techniques (January 2013) http://openaccess.uoc.edu/webapps/o2/bitstream/10609/19561/1/xescricheTFM0113.pdf

Computational Approaches in the Transfer of Aesthetic Values from Paintings to Photographs: Beyond Red, Green and Blue, Book by Jinze Yu, Junyan Wang, Kap Luk Chan, Martin Constable, and Xiaoyan Zhang: Chapter 2 The Colour Attributes of Paintings http://www.springer.com/cda/content/document/cda_downloaddocument/9789811035593-c2.pdf?SGWID=0-0-45-1612011-p180539341

A Study on Paintings Art Style Using Multi-Cues http://download.atlantis-press.com/php/download_paper.php?id=16439

Computer image analysis of paintings and drawings:An introduction to the literature https://pdfs.semanticscholar.org/2e11/17eaeda45cde46f5a1d50eb5e4692cb6030a.pdf