Source-linked AI summary
Learned Convolutional Sparse Coding
Hillel Sreter, Raja Giryes
TL;DR
Patch-based sparse coding can be slow and loses spatial information such as shift-invariance. The paper extends LISTA to task-driven convolutional sparse coding, reporting competitive denoising and inpainting results with substantially shorter runtimes.
Problem
Patch-based sparse coding tends to be slow, and learning dictionaries independently for each segment loses spatial information such as shift-invariance.
Source: §1 · p. 1Method
The paper extends LISTA to convolutional sparse coding and trains the model end to end in a task-driven fashion.
Source: §4 · p. 2Results
The approach is competitive with KSVD denoising and leading CSC inpainting methods, with an inpainting runtime speed-up of more than three orders of magnitude.
Source: §5.2 · p. 3, §5.3 · p. 4Takeaways & Limitations
The learned model combines approximate convolutional sparse coding with a task-driven convolutional dictionary, and the conclusion characterizes it as combining computational capabilities and approximation power.
Source: §6 · p. 4LISTA targets the optimal sparse code when attainable; otherwise, its target is the final solution after ISTA converges.
Source: §2.2 · p. 2
Abstract
from arXiv · showhide
We propose a convolutional recurrent sparse auto-encoder model. The model consists of a sparse encoder, which is a convolutional extension of the learned ISTA (LISTA) method, and a linear convolutional decoder. Our strategy offers a simple method for learning a task-driven sparse convolutional dictionary (CD), and producing an approximate convolutional sparse code (CSC) over the learned dictionary. We trained the model to minimize reconstruction loss via gradient decent with back-propagation and have achieved competitive results to KSVD image denoising and to leading CSC methods in image inpainting requiring only a small fraction of their run-time.
1. INTRODUCTION
Patch-based sparse coding can be slow and loses spatial information such as shift-invariance. The paper introduces task-driven convolutional dictionary learning that learns sparse codes and demonstrates it on image denoising and inpainting.
1. INTRODUCTION
- Patch-based sparse coding can be slow and independently learned segment dictionaries lose spatial information, including shift-invariance.Source: §1 · p. 1
- LISTA approximates sparse coding with a recurrent network mimicking ISTA, while convolutional dictionaries provide shift-invariant atoms learned from the whole signal.Source: §1 · p. 1
- The paper introduces task-driven convolutional dictionary learning and demonstrates the approach on image denoising and inpainting.Source: §1 · p. 1
2. THE SPARSE CODING PROBLEM
Sparse coding seeks a sparse representation, but its direct ℓ0 formulation is combinatorial. The section describes ℓ1 relaxation and ISTA, then presents LISTA as a learned recurrent approximation and situates related architectures.
2.1. Sparse coding and ISTA
- Directly solving the ℓ0 sparse-coding problem is combinatorial, with complexity growing exponentially in dictionary size m.Source: §2.1 · p. 1
- A common approximation relaxes the ℓ0 pseudo-norm to an ℓ1 norm, and ISTA is presented as a technique for minimizing the resulting objective.Source: §2.1 · p. 1
2.2. Learned ISTA (LISTA)
- LISTA derives a recurrent network from ISTA and learns its parameters to approximate an optimal sparse code when attainable, or the converged ISTA solution otherwise.Source: §2.2 · pp. 1–2
- Related LISTA variants train against the optimization objective, add nonnegative discrimination or decoding, or use cascaded blocks for super-resolution.Source: §2.2 · p. 2
3. CONVOLUTIONAL SPARSE CODING
Convolutional sparse coding replaces dictionary multiplication with convolution and represents the full input rather than separate patches. Prior approaches include iterative inference and training-set memory demands that can limit their use.
3. CONVOLUTIONAL SPARSE CODING
- CSC replaces dictionary multiplication with convolution, using local filters and sparse feature maps to represent an input signal.Source: §3 · p. 2
- Unlike patch-based coding, CSC operates on the full input and is spatially invariant, allowing an oriented atom to represent matching edges across an image.Source: §3 · p. 2
- Some prior CSC methods optimize over the whole training set, requiring it to fit in memory, while iterative inference may require many iterations and be less suitable for real-time tasks.Source: §3 · p. 2
4. LEARNED CONVOLUTIONAL SPARSE CODING
The section extends LISTA to convolutional sparse coding and trains an encoder–decoder to learn task-driven convolutional dictionaries. It reports competitive task performance with substantially fewer computations.
4. LEARNED CONVOLUTIONAL SPARSE CODING
- The approach achieves competitive performance to classical SC and CSC methods across different tasks with an order of magnitude fewer computations.Source: §4 · p. 2
4.1. Learning approximate CSC
- Naively materializing Dconv requires n1n2m MACs per input element, versus s^2m MACs with convolution when s ≪ {n1, n2}.Source: §4.1 · p. 2
- The approximate CSC encoder reformulates ISTA for convolutions before constructing its LISTA version.Source: §4.1 · p. 2
4.2. Learning the CD
- A learned filter array d appended after convolutional LISTA decodes its approximate CSC into x̂, enabling convolutional-dictionary learning.Source: §4.2 · p. 3
4.3. Task driven convolutional sparse coding
- Stochastic-gradient training minimizes reconstruction loss; soft thresholding biases the encoder toward sparse codes, avoiding a separate sparsity-inducing loss term.Source: §4.3 · p. 3
5. EXPERIMENTS
Experiments use a three-step learned CSC network trained on PASCAL VOC, then evaluate denoising and masked-pixel inpainting. The reported results show competitive reconstruction quality alongside substantial runtime gains.
5.1. Learned CSC network parameters
- Training used three recurrent steps and the reconstruction loss, with α = 0.8 on normalized PASCAL VOC images.The authors report that the loss proposed in gave the best image quality among tested reconstruction losses.Source: §5.1 · p. 3
5.2. Image denoiseing
- Denoising reconstruction is qualitatively and quantitatively competitive with KSVD; runtime is faster by one order on CPU and two orders on GPU.The denoising experiment adds Gaussian noise with σn = 20.Source: Table 1 · p. 3, §5.2 · p. 3, Table 2 · p. 4
5.3. Image inpainting
- Inpainting randomly retained half the pixels and used a denoising-pretrained ACSC network optimized for the inpainting task.The test images were preprocessed with local contrast as in.Source: §5.3 · p. 4
- Inpainting results are competitive with and, while runtime is more than three orders of magnitude faster.The reported comparison uses the same test images as.Source: Table 3 · p. 4, §5.3 · p. 4, Table 4 · p. 4
6. CONCLUSION
The conclusion presents approximate convolutional sparse coding as combining neural networks’ computational capabilities and approximation power, and notes that source code is available.
6. CONCLUSION
- Approximate convolutional sparse coding combines neural networks’ computational capabilities and approximation power.The conclusion presents this combination as a strength of the proposed approach.Source: §6 · p. 4
- The paper lists a GitHub location for its source code.The listed repository path is AppriximateConvolutionalSparseCoding.Source: §6 · p. 4