PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning the Pareto Front with Hypernetworks

Aviv Navon, Aviv Shamsian, Gal Chechik, Ethan Fetaya

arXiv:2010.04104v2cs.LG

TL;DR

The paper addresses the lack of a scalable and flexible way to obtain Pareto-optimal models for many preferences without knowing the trade-off beforehand. It formulates Pareto-front learning and uses a single preference-conditioned hypernetwork to generate models, achieving better solution coverage and faster training across diverse tasks. The approach also supports runtime preference selection and generation of large models such as ResNet18.

  • Problem

    Existing MOO methods require separate models for different preferences, while the trade-off may be unknown before training and covering many objectives scales poorly.

  • Method

    Pareto HyperNetworks condition one hypernetwork on a preference vector and generate a model tuned to the corresponding objective trade-off.

  • Results

    PHNs achieve better objective-space coverage and significant training-time improvements across diverse tasks, with evaluation reporting 10 ∼50 times faster runtime than baselines.

  • Takeaways & Limitations

    A unified PHN lets users select or change the operating preference at inference or deployment time and can generate large models such as ResNet18.

  • Takeaways & Limitations

    Linear scalarization reaches only the convex part of the Pareto front and does not clearly identify weights for a desired preference ray.

Abstract

from arXiv · show

Multi-objective optimization (MOO) problems are prevalent in machine learning. These problems have a set of optimal solutions, called the Pareto front, where each point on the front represents a different trade-off between possibly conflicting objectives. Recent MOO methods can target a specific desired ray in loss space however, most approaches still face two grave limitations: (i) A separate model has to be trained for each point on the front; and (ii) The exact trade-off must be known before the optimization process. Here, we tackle the problem of learning the entire Pareto front, with the capability of selecting a desired operating point on the front after training. We call this new setup Pareto-Front Learning (PFL). We describe an approach to PFL implemented using HyperNetworks, which we term Pareto HyperNetworks (PHNs). PHN learns the entire Pareto front simultaneously using a single hypernetwork, which receives as input a desired preference vector and returns a Pareto-optimal model whose loss vector is in the desired ray. The unified model is runtime efficient compared to training multiple models and generalizes to new operating points not used during training. We evaluate our method on a wide set of problems, from multi-task regression and classification to fairness. PHNs learn the entire Pareto front at roughly the same time as learning a single point on the front and at the same time reach a better solution set. Furthermore, we show that PHNs can scale to generate large models like ResNet18. PFL opens the door to new applications where models are selected based on preferences that are only available at run time.

1 INTRODUCTION

The paper introduces Pareto-front learning (PFL), which targets multiple possible preference directions with one conditional model rather than separate pre-trained models. Pareto HyperNetworks (PHNs) implement PFL by mapping a preference vector to a model, achieving broad coverage, runtime flexibility, and faster training across tasks.

  • Approach: A single PHN-EPO model maps preference rays to corresponding solutions across the Pareto front.The illustrative example depicts the Pareto front as a black line and preference rays as colored dashed lines.
  • Motivation: PFL addresses settings where trade-offs are unknown before training or where many preference directions may matter.Network routing is given as an example in which trade-offs vary across applications or over time.
  • Approach: PHN is a hypernetwork that receives a preference vector and produces a deep model tuned to that objective trade-off.Training samples preferences from the m-dimensional simplex, where m is the number of objectives.
  • Results: 10 ∼50 times faster: PHN achieves superior overall solutions while reducing runtime across a wide range of evaluated problems.The evaluation spans multi-class classification, fairness, image segmentation, and multi-task regression.
  • Results: PHNs generate better objective-space coverage than multiple baseline models, with significant improvement in training time.The paper identifies this as a contribution supported by empirical evaluations on various tasks and datasets.

2 MULTI-OBJECTIVE OPTIMIZATION

This section defines Pareto optimality and motivates MOO procedures that scale across objectives while controlling the selected trade-off. It describes linear scalarization's limitations and introduces methods such as PMTL and EPO for targeting Pareto-front regions or rays.

  • Definitions: A point is Pareto optimal when no other point dominates it, and the set of all such points is the Pareto front.In non-convex neural-network optimization, the paper also considers local Pareto optimality within an open neighborhood.
  • Definitions: Dominance means improving at least one objective without worsening any other objective.Dominated solutions are preferable to the dominated point under the paper's loss ordering.
  • Linear Scalarization: Linear scalarization can reach only the convex part of the Pareto front and does not clearly map weights to a desired preference ray.These are identified as two major limitations of the straightforward scalarization approach.
  • Preference-Directed Methods: PMTL partitions loss space into cones around selected reference rays and returns one solution per cone using constrained optimization.This supports targeting several points while emphasizing selected reference directions.
  • Preference-Directed Methods: EPO seeks the intersection of the Pareto front with a specified preference ray by balancing descent toward the front with approach to the ray.Its search uses a point in the convex hull of objective gradients with a maximal angle relative to a balancing vector.

3 RELATWED WORK

Related work spans multitask learning, multi-objective optimization, genetic algorithms, gradient-based methods, and hypernetworks. These approaches differ in how they combine objectives, maintain trade-off solutions, scale to neural networks, or generate model weights.

  • Multitask Learning: Multitask learning solves several learning problems simultaneously while sharing information among tasks.MTL models commonly map a loss vector into one loss term through fixed or dynamic weighting.
  • Multi-Objective Optimization: Multi-objective optimization seeks Pareto-optimal solutions for different objective trade-offs and has applications across machine learning.The cited applications include reinforcement learning and neural architecture-related problems.
  • Genetic Algorithms: Genetic algorithms maintain sets of solutions and can naturally extend to different preference rays, but scale poorly with parameter count.The paper identifies NSGA-III and MOEA/D as leading approaches in this area.
  • Gradient-Based Methods: Gradient-based MOO methods include approaches for training large-scale neural networks, while other work extends Pareto solutions locally or approximates the front.The section cites MDGA-based methods and extensions of PMTL and related approaches.
  • Hypernetworks: Hypernetworks use one network to generate weights for a second target network and have been applied across several domains.The cited domains include computer vision, language modeling, sequence decoding, and continual learning.

4 PARETO HYPERNETWORKS

Pareto HyperNetworks use a single conditional hypernetwork to generate target-network weights for preference vectors, learning multiple Pareto-front solutions simultaneously. The section presents linear-scalarization and EPO-based training, along with theoretical caveats and scalability extensions.

  • Hypernetwork architecture: Hypernetworks generate target-network weights conditioned on their input, allowing one model to represent a family of target networks simultaneously.The implementation uses a feed-forward network with shared features and multiple heads, each producing a target-network weight tensor.
  • PHN formulation: A PHN receives a preference vector r and outputs target-network weights θr, with preference vectors sampled during training and supplied at inference.The preference vector lies in an m-dimensional simplex, and training samples r from a Dirichlet distribution.
  • PHN-LS: PHN-LS trains the hypernetwork with linear scalarization, optimizing the expected weighted loss over sampled preferences and data batches.The weighted loss uses the preference components as objective weights.
  • Theoretical basis: Under a full-rank assumption, the mapping from preference vectors to optimal model weights is smooth and can be learned through universal approximation.The proposition establishes a smooth local mapping θ(r) around a local Pareto-optimal point.
  • Theoretical limitations: PHN-LS can have stationary points where the generated model is not locally Pareto optimal, although experiments find that PHNs avoid these bad solutions.This limitation parallels the existence of bad local minima in standard deep neural networks.
  • PHN-EPO: PHN-EPO targets Pareto-optimal points on specified inverse rays, while its required linear-programming step makes it slower than PHN-LS.EPO addresses the theoretical limitations of linear scalarization but does not optimize a single specific objective.

5 EXPERIMENTS

Across synthetic, classification, fairness, vision, and regression problems, PHN is evaluated against preference-specific and gradient-free MOO methods using hypervolume, uniformity, runtime, and qualitative Pareto-front coverage. PHN generally achieves stronger solution coverage with substantially lower runtime, while genetic algorithms fail to provide meaningful solutions at larger scales.

  • Experimental setup: PHN is evaluated on diverse problems including Multi-MNIST, fairness, NYUv2 pixel-wise tasks, and SARCOS multitask regression.The experiments compare PHN variants with LS, EPO, PMTL, and CPMTL where applicable.
  • Baseline limitations: Genetic algorithms fail to converge to meaningful solutions in larger-scale experiments and are therefore excluded from large-scale comparisons.On small networks, they can require up to 100,000 generations and still produce sub-par results.
  • Illustrative example: PHN-EPO generates a continuous cover of a known non-convex Pareto front in one run and maps each preference to its corresponding Pareto-optimal solution.LS misses the concave part of the front, while PMTL and EPO cover the front with different limitations.
  • Image classification: PHN outperforms competing methods in hypervolume and uniformity on the three Multi-MNIST datasets while substantially improving runtime.The evaluation uses five evenly spaced rays for each baseline and a reference point of (2, 2).
  • Fairness: PHN achieves the best hypervolume across all fairness datasets with reduced training time, while EPO achieves the best uniformity on its training rays.PHN-EPO also visualizes accuracy-fairness trade-offs across solutions on the Pareto front.
  • Pixel-wise classification and regression: PHN-EPO achieves the best hypervolume and uniformity on NYUv2 while being faster than the baselines.NYUv2 evaluates semantic segmentation and depth estimation with an ENet-based target network.
  • Multitask regression: On SARCOS with seven regression tasks, PHN obtains the best hypervolume while reducing overall runtime relative to preference-specific models.LS and EPO require 20 preference-specific models, illustrating the computation and storage burden as objective dimensionality grows.

6 THE QUALITY-RUNTIME TRADE-OFF

PHN learns the entire Pareto front in one model, avoiding the multiple-model runtime trade-off faced by competing methods while often improving hypervolume.

  • PHN learns the entire front in a single model, whereas competing methods train multiple models to cover it.
  • PHN-EPO achieves superior or comparable hypervolume while being an order of magnitude faster than LS and EPO.PHN-LS is faster still, but this can come with lower hypervolume.
  • PHN often outperforms models trained on specific rays when evaluated on those same rays.The authors attribute this gain to weight sharing across preference rays.

7 SCALING UP PHNS

PHNs scale to generating expressive ResNet18 models through chunked hypernetwork generation, improving hypervolume while reducing trainable parameters.

  • Chunking generates target-network parts from preference-conditioned representations, avoiding linear growth of hypernetwork parameters with target-network size.Each target-parameter block is computed from a representation produced by the preference and chunk descriptor.
  • PHN achieves significant hypervolume improvement over baselines while dramatically reducing trainable parameters for an 11.4M-parameter ResNet18 target.Baselines use 57M trainable parameters from five models, while PHN varies its parameter count through the number of chunk matrices.
  • PHNs achieve similar or improved hypervolume while reducing trainable parameters by ∼×50 compared with preference-specific LS and EPO.The parameter-count axis is logarithmic.

8 CONCLUSION

The paper introduces Pareto Front Learning and Pareto Hypernetworks for selecting objective trade-offs at inference time. Across diverse tasks, PHNs improve runtime and performance while enabling deployment-time preference changes.

  • Pareto Front Learning defines learning a model that operates on any given preference vector at inference time and returns a Pareto-optimal solution.
  • Pareto Hypernetworks implement PFL with a unified hypernetwork architecture.
  • Experiments across diverse tasks and datasets show significant gains in runtime and performance.
  • PHNs allow users to select and change the operating point at inference time or deployment.

A EXPERIMENTAL DETAILS

The appendix details training and evaluation settings, known-front benchmarks, unseen-ray generalization, and PHN interpretation across the reported experiments.

  • Experimental details: NYUv2 uses 150 training epochs, Adam, and evaluation over 25 evenly spaced rays for pixelwise classification and regression.Learning rates searched were 1e −3, 5e −4, and 1e −4.
  • Experimental details: Multi-MNIST uses a LeNet-based target network with two task-specific heads and evaluates PHN over 25 rays.All methods train with Adam at learning rate 1e −4, batch size 256, and 150 epochs before validation-based epoch selection.
  • Experimental details: Fairness experiments use a 3-layer feed-forward network and evaluate PHN over a sample of 150 rays.Training lasts 35 epochs with Adam and validation-based hyperparameter selection.
  • Experimental details: SARCOS uses a 4-layer feed-forward network with 256 hidden dimensions and evaluates PHN over 100 rays.The dataset contains 40,036 training, 4,448 validation, and 4,449 test examples.
  • Known Pareto fronts: On Fonseca’s illustrative problem, MOEA/D fails to reach the exact Pareto front after 100,000 generations, while MGDA covers only a small fraction.MGDA reaches Pareto-optimal solutions but does not generate preference-specific solutions.
  • Known Pareto fronts: PHN is evaluated on non-convex known fronts from Problem2 and ZDT2, where solution density across the Pareto-optimal region is non-uniform.
  • Known Pareto fronts: PHN-EPO converges to the entire Pareto front with one hypernetwork, whereas EPO fails to converge for all rays in ZDT2.
  • Unseen-ray generalization: On seven-task SARCOS, unseen rays reduce HV from 0.7337 to 0.7336 and uniformity from 0.163 to 0.158.The authors characterize these changes as minor and conclude that PHN generalizes well to unseen rays.

B.5 DESIGN CHOICE ANALYSIS

The analysis tests PHN hyperparameters on Multi-Fashion + MNIST, finding robustness to α and a stronger hidden-dimension effect on hypervolume.

  • B.5 DESIGN CHOICE ANALYSIS: The study varies α and hypernetwork hidden dimension using PHN-EPO on Multi-Fashion + MNIST.The hidden dimension is fixed at 100 for the α sweep, then α is fixed at 0.2 for the hidden-dimension sweep.
  • B.5 DESIGN CHOICE ANALYSIS: PHN remains robust to α values from 0.1 to 1 on this dataset.The α sweep uses uniform sampling.
  • B.5 DESIGN CHOICE ANALYSIS: Best HV is achieved with a hidden dimension of 100.Changing hidden-layer dimension has a stronger effect on HV than changing α in this analysis.

B.6 ADDITIONAL RUNTIME ANALYSIS

Additional runtime analysis covers Multi-MNIST, Multi-Fashion, and NYUv2, while hypervolume evaluates both dominated quality and solution diversity.

  • B.6 ADDITIONAL RUNTIME ANALYSIS: Figure 11 provides additional runtime analysis on the Multi-MNIST, Multi-Fashion, and NYUv2 datasets.The analysis extends the runtime evaluation from Section 5.5.
  • B.6 ADDITIONAL RUNTIME ANALYSIS: EPO is omitted for NYUv2 because LS outperformed it in both HV and runtime.This exclusion is specific to the NYUv2 runtime analysis.
  • B.6 ADDITIONAL RUNTIME ANALYSIS: Hypervolume measures the region of non-dominated points bounded above by a reference point.It can be interpreted as the union of boxes formed by non-dominated points.
  • B.6 ADDITIONAL RUNTIME ANALYSIS: Hypervolume captures both individual-solution quality and diversity through overlap between dominated regions.Figure 12 illustrates the colored hypervolume area for a Pareto set relative to a reference point.
Loading 2010.04104v2…