Source-linked AI summary
A Neural Algorithm of Artistic Style
Leon A. Gatys, Alexander S. Ecker, Matthias Bethge
TL;DR
The algorithmic basis for combining image content and style remained unknown, with no artificial system matching this artistic capability. This paper introduces a deep-neural-network system that separates and recombines content and style through neural representations to generate high-quality artistic images.
Problem
An artificial system for creating images that mix photographic content with painting style had not yet been established.
Method
The system uses convolutional-network feature responses, gradient descent, and a style representation to separately match content and style before combining them.
Results
The generated images produced more appealing results when average pooling replaced max pooling.
Takeaways & Limitations
Neural representations can support an algorithmic approach to creating artistic images by recombining the content of one image with the style of another.
Abstract
from arXiv · showhide
In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.
Methods
The method uses convolutional feature representations from a 19-layer VGG network to reconstruct content and style through gradient descent. Artistic images are generated by jointly optimizing content features from a photograph and style correlations from a painting.
- Network representation: Replacing max pooling with average pooling improves gradient flow and produces slightly more appealing results.The reported images were generated with average pooling.
- Content reconstruction: Content reconstruction uses gradient descent from white noise to match an original image’s feature responses in a selected CNN layer.The generated image is updated using gradients computed by standard error back-propagation.
- Style reconstruction: Style is represented by correlations between filter responses, and textures are synthesized by minimizing mean-squared distance between the original and generated Gram matrices.The correlations are computed over the spatial extent of the input image.