Computer learns on how to become an art critic

Being a genius has almost meant to be misunderstood. Who observes a genius work or reads a successful book is thrilled for the necessary time to understand that yes, this artist is different, this picture or this book are telling me very interesting things. Maybe they're talking about me.

And it will take time to assimilate the true meaning of that work, because you don't yet have the knowledge and necessary experience to recognize as true what lies ahead, be this one a painting, or rather a book. At a shallow surface, the work of genius comes at human being and part of that is certainly recognized, but at a deeper level are miss those required associations to understand what the genius has seen and he wants us to tell. What is missing in people to recognize an author or an artistic movement, is currently being studied by researchers.

"So I was in this museum and I thought, it would be nice to instruct a computer to recognize the name of an artist only by one of his paintings, but not a known painting, but a new one at least unknown to the computer."

So says the Dr.Ahmed Elgammal, associate professor in the Department of Computer Science at Rutgers University in New Jersey.

"We're talking about machine learning. Our algorithm instructs computers to recognize art works. More precisely, our research's purpose was to instruct a machine to understand the artistic influences between an author and another in painting matter. "

on left Frederic Bazille - Bazille's Studio, 1870
on right Norman Rockwell - Shuffleton's Barber Shop, 1950

Prof.Elgammal's team worked on this project for three years . We are in Computer Vision, so we talk about recognizing objects algorithms, able to distinguish faces and people inside pictures, just like already do some photograph machines today or the Google and Facebook software during the uploading process of a photo.

Prof.Elgammal says that a picture can be represented generally by seven elements; space, form, shape, color, tone, line and texture. At the beginning of their research, however, they found that these parameters were not enough, so they had to introduce newer ones such as movement's recognition, unity representation, contrast and proportion. In addition they have also considered the subject of paintings, the represented objects, the type of strokes and the material. Finally, the historical context.

"Currently, there are some paintings databases all around the world, some others are shared projects and continue to grow. The classification is possible in this way, if each of us contributes to enrich the database itself. But make this work carried out by a machine makes things more interesting. It 's interesting because the machine can not only recognize the painter's style, but it can also recognize past painters who influenced him, that is an exclusive prerogative of an art critic. You see then, that the details of a picture can be so many that could also be lost by a very good critic of art history. "

I asked the professor to explain in general something about how his team has come to the algorithm.

"At the beginning of our study, we had to figure out which was the best mathematical representation of a painting in order to determine the artistic influences of the painter. Then, we keep notes of both similarities, between paintings and between artists. These three elements have been applied to our dataset of 1710 high resolution pictures. We are talking about works dating from 1412 to 1996, representing 13 different styles of 66 artists. In our dataset we have an average of 132 paintings for style and we can go from a maximum of 140 works by Paul Cézanne, up to only one work from Hans Hoffmann."

The styles examined by the team of professor Elgammal are expressionism, impressionism, renaissance (336 paintings), the period of romanticism, cubism, baroque, surrealism, post impressionism, american modernism (23 paintings), symbolism, pop art, neo-classicism and abstract art.

above Joan Miro - The Farm, 1922

on the right Vincent van Gogh  - Old Vineyard with Peasant Woman, 1890 

same scenarios, same objects, but different styles
Mirò was influenced by Van Gogh

"What we had to do - continues the professor - was first of all figure out what other scientists have produced about a similar problem. We looked at Thomas Lombardi studies for the recognition of a painter by different types of strokes. We studied the Bag of Words model used for example by Maria Khan to figure out who have made a painting looking at a possible choice of eight painters. Gustavo Carneiro studies instead, have been helpful to understand which was the best way to identify similarities between two paintings. So in short, Computer Vision have done so much in the past and is doing a lot now, but the study we have focused on in these three years has not been addressed by anyone. Finding out which artist has influenced a particular painter uploading a painting picture on a computer it was, and it is, frontier science in Artificial Intelligence field."

"And I imagine that this could become also a business."

"Yes, I've been already contacted by a couple of companies, but it was like more as a short talk on the model itself. I do not exclude however, that in the future they can get out applications using our algorithm. Let's think for example to those existing apps for smartphones able to recognize works at a museum or objects, as does, for example, the Google Goggles app. Perhaps our applications will be able to tell who were main contributors of the painter we are looking at. "

"And which results did you get since now?"

"For example Donatello was influenced by Mantegna and the flemish Van Eyck. But it is also true that Mantegna was in turn influenced by Donatello too. Caravaggio instead by Rubens, Raphael and Leonardo. Then it came out that Warhol had some influences of Degas and Rubens to Velazquez. Always Mantegna seems to have influenced Munch and Klimt. All these results have come out with a semantic abstraction level of our model. The method in fact which gave us the most reliable results was the so called "classeme". It's a model which encode the painting in a semantic space learned from today's everyday images found on the Internet.

Bag of Words model in computer vision

"Once we produced our own unique vector representing each painting, we applied Euclid to find out similarities between a painting and another. At this point, the accuracy of the results was satisfactory. We produced a series of graphs by grouping artists who most influenced each other. The concentration of painters in an area is a movement, an artistic current, where teachers guided other artists."

artists map

"Prof.Elgammal, considering the current state of research, do you believe is possible that these algorithms come to tell us if a person is a genius or if a child will be a genius in a particular field who may not yet know?"

"This is a good question." Professor Elgammal thinks about it for a while. "I must say that it would be a very interesting field of research, but it is not so easy to answer. Of course, if we talk about the genius in a particular field the chances increase. At a more general level maybe, I would try to work on brain's image. "

I conclude my interview with Dr.Elgammal asking him if he paints or has any passion in arts.

"Well, if I had time to do it, I'd love painting, yes. But I comfort myself with the fact that these studies are meeting the humanities field I wanted to learn in 80's. I was born in Alexandria, Egypt, and when I was young I wanted to be an archaeologist. Then, just before beginning university, I discovered computers and I started programming. So it was born a passion and that passion is now meeting the humanistic one through painting. As you can see, the distance between two such fields as art and computer is basically inversely proportional to our own curiosity. I am one of the many examples that this union is truly possible."


(END OF FIRST PART)
italian version