PaperlayerView exact revision on arXiv

Source-linked AI summary

A Powerful Generative Model Using Random Weights for the Deep Image Representation

Kun He, Yan Wang, John Hopcroft

arXiv:1606.04801v2cs.CVcs.LGcs.NE

TL;DR

The paper asks whether deep visualization can be performed with untrained, random-weight networks, rather than relying on training. It applies random-weight CNNs to inversion, texture synthesis, and style transfer; results include higher-quality reconstructions than the same architecture with trained weights and textures similar to trained-network outputs. The authors propose these networks as a tool for studying deep-network architecture and representation.

  • Problem

    The paper asks whether deep visualization depends on training or can use untrained networks, which would help separate training's contribution from that of network structure.

    Source: §1 · p. 2

  • Method

    The authors apply random-weight CNNs to feature inversion, texture synthesis, and artistic style transfer.

    Source: §1 · p. 2

  • Results

    Random-weight VGG reconstructs images qualitatively better than pretrained VGG of the same architecture, produces textures similar to originals and trained-VGG results, and creates artistic images.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors present random-weight networks as a tool for studying deep-network architecture and representation and a possible starting point for developing deep-learning theory.

    Source: §1 · p. 3

Abstract

from arXiv · show

To what extent is the success of deep visualization due to the training? Could we do deep visualization using untrained, random weight networks? To address this issue, we explore new and powerful generative models for three popular deep visualization tasks using untrained, random weight convolutional neural networks. First we invert representations in feature spaces and reconstruct images from white noise inputs. The reconstruction quality is statistically higher than that of the same method applied on well trained networks with the same architecture. Next we synthesize textures using scaled correlations of representations in multiple layers and our results are almost indistinguishable with the original natural texture and the synthesized textures based on the trained network. Third, by recasting the content of an image in the style of various artworks, we create artistic images with high perceptual quality, highly competitive to the prior work of Gatys et al. on pretrained networks. To our knowledge this is the first demonstration of image representations using untrained deep neural networks. Our work provides a new and fascinating tool to study the representation of deep network architecture and sheds light on new understandings on deep visualization.

1 Introduction

The paper asks whether deep visualization depends on learned weights or can instead reveal representations arising from network architecture. Random-weight CNNs produce reconstructions, textures, and artistic images, motivating their use to study architecture and deep-network representations.

1 Introduction

  • Training on large image datasets makes object information increasingly explicit across levels of deep-network representations.The introduction situates its question amid visualization techniques developed for trained networks.Source: §1 · p. 1
  • Prior trained-network visualization includes feature inversion [7], correlation-based texture synthesis [8] [12], and content-and-style image generation [13].Other approaches visualize images that maximize individual neurons or class scores [6] [9].Source: §1 · p. 2
  • A separate line trains feed-forward networks to invert representations for faster visualization, but focuses on the visualization task rather than understanding the original CNN.These trained inverse networks are reported as two or three orders of magnitude faster than methods requiring forward and backward passes through the pretrained network.Source: §1 · p. 2
  • The paper asks whether untrained networks can support deep visualization, separating potential contributions of training from network structure.The authors hypothesize that hierarchical nonlinear structure, rather than training, may underlie visualization success.Source: §1 · p. 2
  • Random-weight VGG reconstructs images qualitatively better than pretrained VGG of the same architecture, synthesizes textures similar to originals and trained-VGG results, and creates artistic images.The texture results use automatically scaled weighting factors.Source: §1 · p. 2
  • The authors present random-weight visualization as a tool for studying network representations and a possible starting point for developing deep-learning theory.They note that proving properties of well-trained networks may be difficult.Source: §1 · p. 3

2 Convolutional neural network

The section describes VGG-19 and the network variants used for comparison. The random-weight version retains the architecture while replacing its learned weights with Gaussian random values.

2 Convolutional neural network

  • VGG-19 has 16 convolutional and 5 pooling layers followed by 3 fully connected layers.Its filters are 3 × 3 × N_l, with 2 × 2 spatial down-sampling between convolutional layers.Source: §2 · p. 3, Figure 1 · p. 3
  • The trained VGG reference uses average pooling instead of maximum pooling and rescales weights so each filter's mean activation is 1.Gatys et al. suggest average pooling may improve gradient flow and produce slightly better results [8].Source: §2 · p. 3
  • ranVGG retains the VGG architecture but replaces its weights with Gaussian random values, using a standard deviation such as 0.015 in experiments.The random-weight network is the paper's reference network for its experiments.Source: §2 · p. 3

3 Methods

The methods apply three visualization tasks to random-weight CNNs, using layer-weight normalization and a greedy procedure to stabilize reconstruction. They invert feature representations, synthesize textures from feature correlations, and combine content and style objectives for artistic images.

3 Methods

  • The approach applies random rather than trained weights to image inversion, texture synthesis, and style transfer, with layer-impact weighting and a greedy stacked-network procedure.The weighting normalizes the effects of activation layers, while the stacked ranVGG is built by selecting layer weights in forward order.Source: §3 · pp. 3–4
  • Inversion reconstructs an image by minimizing the L2 distance between its layer representation and that of a target image.The method starts from white noise and uses L-BFGS with backpropagation to morph the input.Source: §3 · pp. 3–4
  • Layer weights normalize gradient impact on the input, and the paper adds this normalization for combined content-and-style optimization rather than single-layer reconstruction.The reciprocal of the absolute mean input gradient over pixels and RGB channels sets the layer weight; the loss-based factor μ_l is defined similarly using each layer’s loss contribution.Source: §3 · p. 4
  • The stacked ranVGG is built greedily by fixing earlier layers and selecting each next layer’s random weights according to reconstruction loss on a reference image.At each layer, several random-weight sets are sampled, and the set with the smallest reconstruction loss is chosen.Source: §3 · p. 4
  • Texture synthesis matches feature-map correlations across selected layers, using the correlations as the texture representation.The objective minimizes the L2 loss between correlations for the synthesized image and a ground-truth image.Source: §3 · p. 4
  • Style transfer combines a content-reconstruction objective with a texture objective to impose the style image’s correlations on the content image.A total-variation regularizer encourages spatial smoothness in the output.Source: §3 · p. 5

4 Experiments

The experiments evaluate representation inversion, texture synthesis, and style transfer with the untrained random-weight network ranVGG. Across these tasks, ranVGG produces high-quality results, with inversion especially strong against trained-network comparisons.

Inverting deep representations

  • RanVGG retains photographically accurate image information in all convolutional and pooling layers, while reconstructions from its first two fully connected layers still show invariance and rough contours.Source: §4 · p. 5, Figure 2 · p. 5
  • RanVGG reconstructions have higher perceptual quality than pretrained VGG reconstructions, especially at higher layers, and outperform the compared AlexNet results.The authors suggest that VGG's 3 × 3 filters and average pooling, versus AlexNet's larger filters and maximum pooling, may explain the difference; the latter can make inversion harder and produce spikes.Source: §4 · pp. 5–6, Figure 4 · p. 6, Figure 3 · p. 6
  • Stacked ranVGG has lower variation and reconstruction distances than purely random ranVGG, while both random-weight variants have lower values than trained VGG.The reported variation comparison uses the girl image as an example.Source: §4 · p. 6, Figure 6 · p. 7
  • RanVGG yields higher-quality reconstructions within 100 iterations than pretrained VGG does after 2,000 iterations, and its reconstruction converges more quickly.The pretrained VGG reconstruction remains unchanged even when its iteration limit is doubled to 4,000.Source: §4 · p. 6, Figure 5 · p. 7

Texture synthesis

  • With automatically calculated layer weights, ranVGG synthesizes complex natural textures almost indistinguishable from the originals and trained-VGG textures; trained VGG is slightly better on neatly arranged cargo textures.Matching conv1_1, conv2_1, and conv3_1 already produces high-quality textures; adding conv4_1 slightly improves natural textures.Source: §4 · p. 7

Style transfer

  • RanVGG produces high-perceptual-quality artistic images from content and artwork styles, with results comparable to Gatys et al.'s pretrained-VGG work.The authors report slightly smoother lines and textures in the pretrained-network results.Source: §4 · p. 7

5 Discussion

The discussion frames random-weight visualization as a testable way to study image-appearance representations through network structure. It also raises questions about what this success implies for architecture and future visualization tasks.

5 Discussion

  • The authors frame their results as a testable hypothesis about image-appearance representations based only on network structure.Source: §5 · p. 7
  • Adding deeper ranVGG layers progressively organizes local texture structure, with the third and fourth rows synthesizing the original textures well.Source: Figure 8 · p. 8
  • If untrained networks support similar visualizations, the authors argue that the resulting understanding concerns architecture rather than training, and identify image morphing as a promising next task.Source: §5 · p. 8

Related papers

Loading 1606.04801v2…