End-to-End Learning for Structured Prediction Energy Networks

David Belanger; Bishan Yang; Andrew McCallum

Source-linked AI summary

End-to-End Learning for Structured Prediction Energy Networks

David Belanger, Bishan Yang, Andrew McCallum

arXiv:1703.05667v2Mar 16, 2017stat.MLcs.LG

TL;DR

Structured prediction can involve global interactions and hard constraints, while structured-SVM training can be unreliable when exact energy minimization is intractable. This paper trains SPENs by back-propagating through unrolled gradient-based prediction; its experiments report substantial denoising gains and the best SPEN + AD3 performance on WSJ test data.

Problem
Structured-SVM learning can be unreliable for non-convex energies when inexact minimization fails to discover existing margin violations.
Source: §2.3 · p. 3
Method
The method back-propagates through an unrolled approximate energy-minimization algorithm to train SPENs for practical prediction performance.
Source: §1 · p. 2, §3 · p. 3
Results
The deep-prior SPEN shows a substantial denoising gain over elementary signal processing, and SPEN + AD3 performs best on WSJ test data.
Source: §6.1.3 · p. 6, §6.2.3 · p. 8
Takeaways & Limitations
Unrolling approximate minimization produces a test-time prediction procedure tailored to the learned energy, supporting high performance on more complex tasks.
Source: §7 · p. 8
The tested ICNN did not achieve reasonable denoising performance because positive convolution parameters hindered edge detection and local-smoothness priors.
Source: §6.1.3 · p. 7

Abstract

from arXiv · show

Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016). An energy function over candidate structured outputs is given by a deep network, and predictions are formed by gradient-based optimization. This paper presents end-to-end learning for SPENs, where the energy function is discriminatively trained by back-propagating through gradient-based prediction. In our experience, the approach is substantially more accurate than the structured SVM method of Belanger and McCallum (2016), as it allows us to use more sophisticated non-convex energies. We provide a collection of techniques for improving the speed, accuracy, and memory requirements of end-to-end SPENs, and demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005 semantic role labeling tasks. In both, inexact minimization of non-convex SPEN energies is superior to baseline methods that use simplistic energy functions that can be minimized exactly.

1. Introduction

Structured prediction can require global output interactions and hard constraints that feed-forward prediction may not capture. The paper addresses the difficulty of training SPENs for complex tasks and evaluates its end-to-end methods on image denoising and semantic role labeling.

1. Introduction

End-to-end SPEN training addresses the difficulty of extending structured-SVM training to complex domains with intractable exact energy minimization.The authors report that loss-augmented inference may fail to discover margin violations in this setting.Source: §1 · pp. 1–2
The method back-propagates through an unrolled gradient-based minimization procedure, optimizing the practical performance of approximate prediction rather than requiring exact minimization.The energy and its approximate prediction algorithm are trained together.Source: §1 · p. 2
The paper offers general-purpose techniques for vanishing gradients, faster gradient-based minimization, and reduced computation and memory overhead.It also recommends architectures, parameter tying, and pretraining methods intended to reduce overfitting and improve efficiency.Source: §1 · p. 2
40.4 PSNR versus 36.3 for Wang et al. (2016) on 7-Scenes denoising, using a deep convolutional prior.The comparison is notable because Wang et al. unrolls more sophisticated optimization, while this method uses a more expressive image prior.Source: §1 · p. 2
On CoNLL-2005 semantic role labeling, SPENs improve over strong deep-feature baselines whose simpler energies permit dynamic-programming constraints.The task has discrete, sparse outputs and rigid non-local constraints.Source: §1 · p. 2
The same gradient-based prediction and end-to-end learning code is used for both denoising and semantic role labeling.The authors describe this black-box model interaction as an opportunity for further SPEN use.Source: §1 · p. 2

2. Structured Prediction Energy Networks

SPENs use deep-network energies and gradient-based optimization to model structured outputs. This section contrasts black-box energies with factorized alternatives and reviews approaches to learning energy-based predictors, including their limitations under inexact minimization.

2. Structured Prediction Energy Networks

A SPEN uses a deep neural network as its energy and predicts by gradient-based optimization over continuous outputs.Differentiability requires the energy to be defined on continuous inputs.Source: §2 · p. 2

2.1. Black-Box vs. Factorized Energy Functions

Black-box SPEN energies learn interaction structure without a preset factorization, capturing global interactions that can be difficult to represent with factorized energies.This flexibility offers few guarantees, particularly for non-convex energies, and can complicate constrained optimization.Source: §2.1 · p. 2

2.2. Learning as Conditional Density Estimation

Conditional-likelihood training is intractable for black-box energies without factorization because computing likelihood requires an unavailable normalizing constant.Contrastive backpropagation circumvents this with contrastive-divergence training and Hamiltonian Monte Carlo sampling.Source: §2.2 · pp. 2–3

2.3. Learning with Exact Energy Minimization

Structured-SVM learning requires loss-augmented inference to find margin violations, but inexact minimization may miss them and destabilize training with non-convex energies.The authors report that this approach is unstable and difficult to tune beyond simpler multi-label tasks.Source: §2.3 · p. 3
Implicit-function training can use conjugate gradients for inverse-Hessian products, but reliable performance requires exact energy minimization and many iterations.A naive implementation would require inverting Hessians.Source: §2.3 · p. 3
Training methods that update energies near ground truths and current predictions may leave the rest of the energy surface unshaped, although rapid convergence from good initializations may be desirable.Convex ICNNs return global optima, but their convexity restrictions may constrain energy expressivity; the paper reports better performance from non-convex SPENs in general.Source: §2.3 · p. 3

3. Learning with Unrolled Optimization

With non-convex energies, methods based on exact minimization are unreliable when prediction uses an inexact optimizer. The paper instead trains a specific gradient-based prediction procedure by back-propagating through its unrolled iterations.

3. Learning with Unrolled Optimization

For non-convex energies, exact-minimization learning methods cannot reliably use the output of inexact minimization as a drop-in substitute for the exact minimizer.End-to-end learning instead specifies an approximate gradient-based algorithm and optimizes its empirical performance.Source: §3 · p. 3
Back-propagation through a fixed number of unrolled optimization steps trains the energy and can also train step-specific learning rates.The unrolled procedure can be treated like a feed-forward predictor and trained with standard methods.Source: §3 · p. 3
Back-propagating through gradient computations requires second-order terms, but Hessian-vector products avoid explicitly instantiating Hessians.The paper lists exact Pearlmutter computation, differentiable gradient graphs, and finite differences as ways to obtain them.Source: §3 · pp. 3–4
The unrolled approach extends to momentum and L-BFGS, and end-to-end learning also applies to specialized methods such as mean-field inference and belief propagation.Momentum and L-BFGS require an additional state vector evolved alongside the prediction across iterations.Source: §3 · p. 4

4. End-to-End Learning for SPENs

The section develops end-to-end SPEN training through unrolled gradient-based prediction and presents techniques for discrete outputs, faster optimization, and lower implementation overhead. It also describes how the approach can use black-box energy interactions and be tuned to a particular test-time minimization procedure.

4. End-to-End Learning for SPENs

The section organizes its techniques around learning SPENs for relaxed discrete problems, encouraging rapid optimization, and reducing implementation speed and memory overhead.The experiments apply the same general approach across these considerations.Source: §4 · p. 4
The end-to-end approach tunes the energy to a particular test-time minimization procedure and supports black-box energy interaction using finite-difference Hessian-vector products.The experiments unroll gradient descent or gradient descent with momentum.Source: §4 · p. 4

4.1. End-to-End Learning for Discrete Problems

For simplex-constrained problems, entropic mirror descent provides an alternative to Euclidean projected gradient descent.Its distance is measured by KL divergence.Source: §4.1 · p. 4
For discrete outputs, additive updates to unnormalized logits are less susceptible to vanishing gradients than sigmoid-like updates, while projection can yield zero backpropagation gradients.The approach relaxes discrete outputs to continuous constraints and rounds predictions back to discrete outputs; the rounding introduces poorly understood error sources.Source: §4.1 · p. 4

4.2. Learning to Optimize Quickly

Summing losses across iterates encourages high-quality predictions early and supplies learning signals at every timestep, reducing vanishing gradients.The experiments use weights w_t = 1/(T − t + 1).Source: §4.2 · p. 5
Adding entropy to simplex-constrained energies introduces strong convexity that helps convergence, while small T can yield accurate predictions before energy minimization converges.Some experiments achieve good performance with T = 3; the optimization configuration, such as T, should match at training and test time.Source: §4.2 · pp. 4–5

4.3. Efficient Implementation

For efficiency, iterations stop when convergence is detected, and checkpointing energy inputs and outputs reduces memory by using one energy copy and recomputing forward evaluations during backpropagation.The fixed unroll length T remains, but iterations need only run until convergence.Source: §4.3 · p. 5

5. Recommended SPEN Architectures for End-to-End Learning

The recommended architecture separates global and local energy terms and uses a differentiable initializer. Training includes pretraining and stabilization steps, with differentiability requirements on the energy function.

5. Recommended SPEN Architectures for End-to-End Learning

The feature function F(x) and initializer Init(·) can be networks sub-differentiable in their parameters, and test-time prediction can avoid backpropagation through F(x).The energy is written Ex(y) = E(y ; F(x)).Source: §5 · p. 5
The energy can split into an arbitrary global term and local terms that provide modeling benefits and can also implement the initializer.The initializer predicts the initial iterate y0.Source: §5 · p. 5
Training pretrains F(x) through the feed-forward initializer and stabilizes learning by initially clamping local terms while updating the global energy.The local terms are clamped for a few epochs.Source: §5 · p. 5
Backpropagation through the unrolled energy updates requires an energy twice differentiable with respect to y, so the authors use SoftPlus instead of ReLUs.F(x) and Init(·) can be arbitrary networks sub-differentiable with respect to their parameters.Source: §5 · p. 5

6. Experiments

The experiments test SPEN design choices on image denoising and semantic role labeling, using domain-informed architectures and controlled comparisons. Across the tasks, the results examine the trade-offs of expressive energies and approximate inference.

6. Experiments

The evaluations use controlled comparisons to isolate the effects of unrolled optimization and energy expressivity across denoising and SRL.The paper motivates domain-specific architectures as a way to select models for data suitability rather than compatibility with model-specific algorithms.Source: §6 · pp. 5–6

6.1.1. IMAGE PRIORS

The experiments compare an ℓ1 field-of-experts prior, smoothed with SoftPlus for differentiable optimization, with a deeper convolutional prior that Wang et al. (2016) cannot handle.The deeper prior maps an image to a scalar using convolutional layers, SoftPlus activations, and spatial average pooling.Source: §6.1.1 · p. 6

6.1.2. EXPERIMENTAL SETUP

Denoising uses 7-Scenes depth images, trains on crops from 200 images of one scene, and evaluates PSNR on 5,500 images from different scenes.The setup follows Wang et al. (2016), treats noise variance as trainable, and minimizes mean-squared error.Source: §6.1.2 · p. 6

6.1.3. RESULTS AND DISCUSSION

The DNN prior yields a substantial denoising gain, and the three-step DP-3 variant provides a modest further gain over DP-20.The DeepPrior is also reported to outperform SSVM training, whose performance is sensitive to energy-minimization hyperparameters.Source: §6.1.3 · pp. 6–7, Table 1 · p. 7

6.2. Semantic Role Labeling

SRL assigns semantic roles to predicate–argument pairs and is evaluated as a challenging task with complex non-local output constraints.The CoNLL-2005 experiments use standard splits and evaluation scripts, with preprocessing that includes part-of-speech tagging, dependency parsing, and candidate-argument generation.Source: §6.2 · p. 7, §6.2.1 · p. 7

6.2.2. SPEN MODEL

The SRL SPEN relaxes role labels to continuous distributions, combines pretrained local scores with global terms, and adds twice-differentiable soft penalties for incompatible arcs.SPEN + H and SPEN + AD3 apply constraint-enforcement methods to the soft predictions from energy minimization.Source: §6.2.2 · p. 8

6.2.3. RESULTS AND DISCUSSION

SPEN + AD3 performs best on WSJ test data, while the authors attribute weaker Brown results to source mismatch and overfitting from the expressive global term.The SPEN trained with an SSVM loss did not outperform Local systems; SPEN + AD3 and SPEN + H perform identically, which the authors attribute to the learned global energy encouraging constraint satisfaction during optimization.Source: §6.2.3 · p. 8, Table 2 · p. 8

7. Conclusion and Future Work

The paper presents end-to-end SPEN training by back-propagating through approximate energy minimization, and reports high performance on more complex tasks than earlier SPEN work. It also identifies several possible directions for future research.

7. Conclusion and Future Work

End-to-end training unrolls approximate energy minimization into a differentiable computation graph trained by gradient descent, supporting high performance on more complex tasks than Belanger & McCallum (2016).The resulting approach returns both an energy function and a test-time prediction procedure tailored to it.Source: §7 · p. 8
The tailored test-time prediction procedure makes the approach user-friendly in practice.The paper says it returns not just an energy function but also a prediction procedure tailored for it.Source: §7 · p. 8
Future directions include learned optimizer hyperparameters, optimization in learned feature space, and value-based reinforcement learning for gradient-based prediction.The paper presents these as possibilities for future work.Source: §7 · p. 8

A.1. General Learning Setup

The method supplies gradients of the loss with respect to model parameters, which can be updated with standard neural-network optimizers. The experiments use Adam and address overfitting with development-set early stopping.

A.1. General Learning Setup

The method provides a gradient of the loss with respect to model parameters, which can be updated using standard neural-network optimization methods.The experiments use Adam with default settings.Source: §A.1 · p. 12
Because expressive energy networks are vulnerable to overfitting, the authors select the model that performs best on development data through early stopping.Source: §A.1 · p. 12
Early stopping with a higher-capacity energy network is often superior to using a low-capacity energy network.The passage gives higher-dimensional hidden layers as an example of higher capacity.Source: §A.1 · p. 12

A.2.1. BASELINE ARC-FACTORED ARCHITECTURE

The appendix describes an arc-factored baseline and a global SPEN energy for semantic role labeling. The global energy combines predicate-, label-, argument-count-, and coherence-related terms.

A.2.1. BASELINE ARC-FACTORED ARCHITECTURE

The baseline models each predicate-argument arc label with a logistic-regression conditional distribution whose score is computed by an MLP over span, predicate, and dependency features.Features are mapped through 300-, 250-, and 47-dimensional layers; the embedding layer uses dropout at rate 0.5.Source: §A.2.1 · p. 12

A.2.2. GLOBAL ENERGY TERM FOR SPEN

For the global SPEN energy, predicate-argument features and predicate and argument features are defined from the pretrained model and average span embeddings.The passage specifies 50-dimensional hidden layers and two-layer MLPs with SoftPlus; parameters are trained discriminatively end-to-end.Source: §A.2.2 · p. 12
The global energy uses predicate-specific arc and label mass summaries and total arc-label mass to represent assignments under y.Source: §A.2.2 · p. 12
Its terms score each predicate’s argument set and arc labels, penalize disagreement with predicted argument counts, and model topical coherence.Additional terms apply MLPs to label and arc-feature averages across predicates and arcs.Source: §A.2.2 · p. 12