Art Painting Age Approximation Based on Colour Attribute
CS 585 Project
December 1 2017
Colour usage in art changes from culture to culture and season to season and is often thought of as reflecting or inspiring mood and ambience. The painting techniques has been evolved throughtout the history; aside all the changes, the colours that used in painting remain almost the same. I believed there are certain roles specifically on colours that will make better painting recoginition and image knowledge extraction.The motive for this project is that the traditional chemical method to detect the painting age could damage the oil and the brushstroke, thus I am interested finding a way to do it without touching the painting itself.
- Desired Results:
- Input: ⇒ Output:1889/19th C.E./Impressionism
- Approxmiate age(period) recognition
- Conclusion of patterns over saturation, bightness...
- Conclusion(or model) of colour's continuity over art periods
- Art style or Artist recognition
- Datasets and Tools:
- Large amount of painting tobe processed
- Most of the patterns could be hard to catgorized which may lead to vague and implicit sub-conclusion
The original thought is to have a independent style dectection algorithm, however when tesing the art style dectection and the showed the first 25 results to the custodial staff at Boston museum of fine arts, I found two problems:
- The paintings from different ages has the same art style is sometimes academicly impossible and the style itself can be hardly quantified based on the colour attributes
- The opinions on art style for a specific amount of works are controvarisal and it usually depends on content rather than techniques.
However, it did give me the idea that the conception of art style is actually tied with the art period; and it will be more of a "tag" with age instead of a indepdent detection.
Thus, the approach is to extract certain parameters from each image and build the training set based on those vector-formed parameters, then predict a new image's age(class). I can, of course, use the HSL value as the parameter which would gain too little information to build a vector; or record and use every pixel overwhelmingly which would take too much time with a large amount of data. Thus, I chose to extract certain parameter that can represent certain features of that image but would not take too much time for computing. And that is what I did later.
Based on the approximate art period, I found five breakpoints that related to the color analysis to separate the age(to be the class later) starting from Renaissance:
From the features I had above, I can translate parameters into the more quantifible data for the classification later:
- 1400-1575(Renaissance)--Limited pigments palette
- 1575-1749(Mannerism and Baroque)--Strong light and dark contrast
- 1750-1799(Rococo and Neoclassicism)--Creamy colours with a penchant for landscapes
- 1800-1849(Romanticism and Realism)--Distortion of light and the mosaic pigments
- 1850-1900(Impressionism and Post-Impressionism)--Strong colour inclination
Below are the reasons why I choose those features and how I extracted them:
- 1400-1575--Specific RYB colour range
- 1575-1749--Linear intensity change over on x-axis or y-axis
- 1750-1799--Deepth centre-corner contrast
- 1800-1849--Similiarity between original work and mosaic work
- 1850-1900--Avoidance of hue imbalance and balance
- Parameter:Hue range--Most of the Renaissance art painting has a inclination to use certain range of palette.Due to the limited resource for building pigment, although painting techniques improved immeasurably during the Renaissance, the Renaissance palette mirrored that of the Medieval Age but for three pigments: Naples yellow, smalt and carmine lake (cochineal). Other reds were vermilion and madder lake, which brought to Europe by crusaders in the 12th century. This shows, at least, the painting from the early 15th century has a vastly different range(with much lower upper bound) from other paintings. In order for "purer" result, I choose the HSL value instead of RYB. Although using RYB value here seems to be authentic; it, however,would be drastically affected by age while HSL would not lose much. I separated uniformly the hue spectrum into 6 parts and generally calculate the fraction of each part.
- Parameter:Linear change of intensity--Mannerism arts or the painting with religious, political theme tends to have stronger contrast of light and dark. This could be even found in many monochrome paintings and reason is that the lightness(L), being perceived intensity of a colour, is self-sufficient from chromaticity. In other words, it is easy to imagine an image without a chromatic component, but practically impossible to imagine the inverse. This it would be considered as a good invariance thought age. This intensity could be represented by the monotonic change of intensity in the horiziontal or vertical direction
Parameter:Represenation of background--Landscape is a very important component of canvas from early 16th century. It somehow reflects the development of the perspective technique. There are many ways to represent the landscape or the background in the painting; and my method is to calculate the overlapped region bewteen the center of image and the segmented background(most likely to be the landscape with rather low intensity)
Parameter:Distorion brushstroke--Most of the oil painting has heavy pattern of brushstroke. Although, my focus is not on the pattern checking, it still a nice parameter. For this, I simply retrieve the simliarity between the original painting and the mosaic painting
Parameter:Hue balance--Certain amount of paintings,especially the paintings from late 19th century, employ hue values from only one half of the RYB hue wheel or equal amount of hue values from one sied of the RYB wheel as from the other. The idea is not just the predominate value, but also a "peak" hue that will have draw the nieghbouring hue values.(The hue wheel images were from Computational Approaches in the Transfer of Aesthetic Values from Paintings to Photographs by Beyond Red, Green and Blue, Book by Jinze Yu, Junyan Wang, Kap Luk Chan, Martin Constable, and Xiaoyan Zhang: Chapter 2 The Colour Attributes of Paintings)
I used the scikit-learn Python package for my classfication algorithm and I first choose the nearest neighbour as the classification criterion. However, the problem is if I do not attempt to construct a general internal model, it will simply stores instances of the training data. This model building could be a real hard one. Thus I choose SVM as the classification algorithm; for several advantages, it peroforms realatively well:
- Effective in high dimensional spaces. A single vector that created from the algorithm contains 10(6 hue fraction number plus the rest 4 parameters) vectors.
- Still effective in cases where number of dimensions is greater than the number of samples. This comes very handy when testing a rather small amount of datas
- Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
Fortunately, SVM support multi-class classification. Assuming the vectors and the age are linear related(It is questionable to assume so but as far as I can think of, large data amount could somehow mitigate this disvantage), it will then contruct 5*(4-1)/2=10 classifiers(I labeled 5 classes) and each classifer would trains data from two classes.(See the Mathematical formulation here)
We contruct the vector as [R1,R2,R3,R4,R5,R6,P,M,F1,F2] with class C; the training set and the test set would be converted to 2D vector lists: V_training, V_test and 1D class lists: C_training, C_test.
- Compute 6 hue fractions ranged from 0 to 360: R1,R2,R3,R4,R5,R6
- Count the number of hue "peak" from the hue statistics: P
- Cut the image into 6 bars separately along x-axis and y-axis, count the number of monotonic region of 4 consecutive bars: M
- Compute the fraction between the middle area and the area of larggest contour within the middle: F1
- Compute the fraction of same pixel value between the origional image and the eroded image: F2
- Assign class(1 to 5) based the age period: C
SVM_Fit(V_training, C_training) and then predict from V_test which produces 1D predicted class list: C_predict
Compare C_test and C_predict, the accuracy A will be: (Number of same elements in C_test and C_predict)/(Number of test data points)
The result was plotted by the point (x,y) where
x = The painting age
y = The difference value between the test class and the predicted class
(e.g. P_1=(1622,0) means a painting from 1622 was predicted the same age period with its actual age period; P_2=(1746,1) means a painting from 1746 was predicted 1 age period ahead of its actual age period)
Result 1: 540 training images; 152 test images:
Result 2: 33 training images; 89 test images (images are all low resolution for the purpose of faster demo):
The motive for this project is that the traditional chemical method to detect the painting age could damage the oil and the brushstroke, thus I was thinking if there is a way to do it without touching thepainting itself. As far as I see, I managed to finish the part of approximate age detection combining the art period identifcation. There are some possible improvment that I could not do due to my knowledge limitation:
- Since the age did not scale with the aesthetic value people possessed, there should a better mathemethic relation between the parameters and the age instead of simple linear.
- Color itself cannot cover most of a painting and it should be combined with pattern recogniation or actuall content detection.
Also, if considering the art history and the painting techniques evolution, then there would be some interesting observations from the results:
- The algorithm becomes relatively more accurate when detecting the works after mid 19th century. This could be explained that the artists starts to view the works perceptually. In 1839, Michel-Eugene Chevreul(1786-1889) wrote 'The Laws of Contrast of Colour'. Though it was intended for use by the design and pringint industires, it had a siginificant impact upon the way that painters thought about colour. Taking the lead in this new thinking, artists start knowingly and aggressively unlink saturation from lightness and part of them creates the so-called "Impressionism movement". This eventful address matches the citerion on impressionism and post-impressionism(although there are many differences between the impressionism and post-impressionism, both two trends have some simliarity over the colour attribute).
- Romanticism and Realism shares the simliarity colour attribute; on the other hand, it means they are majorly catgorized by the content itself.
- The result from Renaissance was not good and it may be because that the range I build is not accurate. However, it might also be natually unpredictable becuase the art development during Renassiance over European was not uniform; actually some medieval art painting beared features from the works of Renaissance, but no much, because of the one-point perspective. Additionally, there are also some paintings, especially the early 16th century Itilian works that using the muliti-point perspective, have almost the same features with paintings from 100 or even 200 years after. So in this way of thinking, Leonardo da Vinci was indeed a genius, at least in painting!
Using Machine Learning for Identification of Art Paintings http://cs229.stanford.edu/proj2010/BlessingWen-UsingMachineLearningForIdentificationOfArtPaintings.pdf
Pigments through the ages: Renaissance and Baroque (1400-1600) http://www.webexhibits.org/pigments/intro/renaissance.html
Image Processing for Artist Identification --Computerized analysis of Vincent van Gogh’s painting brushstrokes https://people.ece.cornell.edu/johnson/SPM-7-08.pdf
Mathematical Models of Perception and Generation of Art Works by Dynamic Motions https://www.psychologie.uni-heidelberg.de/ae/allg/mitarb/jf/Schubert%20etal%202014%20art%20works.pdf
RGB Color Cube-Based Histogram Specification for
Hue-Preserving Color Image Enhancement http://www.mdpi.com/2313-433X/3/3/24/pdf
Searching patterns in painting images with
computer vision techniques (January 2013) http://openaccess.uoc.edu/webapps/o2/bitstream/10609/19561/1/xescricheTFM0113.pdf
Computational Approaches in the Transfer of Aesthetic Values from Paintings to Photographs: Beyond Red, Green and Blue, Book by Jinze Yu, Junyan Wang, Kap Luk Chan, Martin Constable, and Xiaoyan Zhang: Chapter 2
The Colour Attributes of Paintings http://www.springer.com/cda/content/document/cda_downloaddocument/9789811035593-c2.pdf?SGWID=0-0-45-1612011-p180539341
A Study on Paintings Art Style Using Multi-Cues http://download.atlantis-press.com/php/download_paper.php?id=16439
Computer image analysis of paintings and drawings:An introduction to the literature https://pdfs.semanticscholar.org/2e11/17eaeda45cde46f5a1d50eb5e4692cb6030a.pdf