Creating artwork with an algorithm : An interview with Leon Gatys
Written by Celia Foster
Leon Gatys is a PhD student in the lab of Prof . Matthias Bethge at the Centre for Integrative Neuroscience in Tübingen . In his PhD work , he uses deep convolutional neural networks , computational algorithms inspired by our knowledge of how the visual cortex is organized . They contain multiple layers of ‘ neurons ’ that learn connections between the layers after training with input signals , for example the pixels of images . Leon uses these networks to explore the organisation of visual perception in tasks such as object recognition . In addition , he ’ s discovered that art style and content can be separated from the deep convolutional neural network layers , allowing art styles to be transferred from one image to another . This method allows anyone to turn their own pictures into artwork , and is available for all to enjoy on the popular website deepart . io .
Why are deep convolutional neural networks so much better at perceptual inference tasks compared to other kinds of algorithms ?
That is a good question . There is of course the obvious answer that deep CNNs are very powerful function approximators and if you train them with enough labeled data they can learn almost anything . But in fact , I think we don ’ t really understand well why they perform so much better than all previous algorithms . From the perspective of computational neuroscience , they are not very different to algorithms like the Neocognitron or HMAX that have been proposed many years ago . I actually believe that it is one of the most important tasks of Computational Neuroscience to develop theories of neural computation that can capture the vast difference in performance between something like HMAX and a modern deep CNN and thus provide a satisfying answer to your question .
When you began working on the deepart algorithm did you realise its potential for artwork or was it a by chance finding ?
Back then we had just figured out that image features from pre-trained CNNs are great for texture modelling and synthesis . I remember we were discussing to combine the texture of artworks with photographs and I wasn ’ t really sure if it would work . But when I played around with it and we saw the first results – even though they were so much worse than what we can do now – it was immediately clear to us that this is very exciting and we are on to something exceptional .
Before you began your PhD how much did you know about the project ? What inspired you to move to Tübingen ?
Actually we hadn ’ t decided on a project yet when I moved to Tuebingen . I had done an internship with Matthias the previous summer and I really enjoyed working with him and was generally fascinated by the question
14 | NEUROMAG | May 2017