PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Continuous Pareto Exploration in Multi-Task Learning

Pingchuan Ma, Tao Du, Wojciech Matusik

arXiv:2006.16434v2cs.LGcs.CVstat.ML

TL;DR

Conflicting task objectives often leave no single MTL solution optimal for all tasks, while existing trade-off exploration typically returns finite, discrete solutions. This paper reconstructs locally continuous Pareto sets using second-order results and matrix-free solvers. Experiments report denser Pareto sets and fronts with small computational overhead, including on networks with millions of parameters.

  • Problem

    MTL trade-offs are common, but prior methods typically produce one solution or finite, discrete Pareto sets rather than continuous families.

    Source: §1 · p. 1

  • Method

    The method uses second-order multi-objective results and a sample-based sparse linear system solved with matrix-free iterative methods to reconstruct continuous local Pareto sets.

    Source: §1 · p. 1, §4 · p. 3

  • Results

    The method produces much denser Pareto sets and fronts with small computational overhead, and scales to networks with millions of parameters.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Continuous local sets can be reparameterized in a low-dimensional space for intuitive manipulation and traversal.

    Source: §1 · p. 2

    The special two-objective augmentation using Hessian null vectors is generally not applicable to real problems because it requires null vectors.

    Source: §5 · p. 5

Abstract

from arXiv · show

Tasks in multi-task learning often correlate, conflict, or even compete with each other. As a result, a single solution that is optimal for all tasks rarely exists. Recent papers introduced the concept of Pareto optimality to this field and directly cast multi-task learning as multi-objective optimization problems, but solutions returned by existing methods are typically finite, sparse, and discrete. We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems. We scale up theoretical results in multi-objective optimization to modern machine learning problems by proposing a sample-based sparse linear system, for which standard Hessian-free solvers in machine learning can be applied. We compare our method to the state-of-the-art algorithms and demonstrate its usage of analyzing local Pareto sets on various multi-task classification and regression problems. The experimental results confirm that our algorithm reveals the primary directions in local Pareto sets for trade-off balancing, finds more solutions with different trade-offs efficiently, and scales well to tasks with millions of parameters.

1. Introduction

Conflicting task objectives make a single universally optimal MTL solution unlikely, yet prior exploration of trade-offs was limited to finite, discrete solutions. This work reconstructs continuous local Pareto sets and reports denser solutions with low overhead across problems ranging up to millions of parameters.

1. Introduction

  • Conflicting objectives commonly prevent one solution from being optimal across all objectives, motivating a set of solutions with different preferences.The introduction cites competing objectives in model design, training, and evaluation.Source: §1 · p. 1
  • The paper addresses the scarcity of trade-off exploration in deep MTL by reconstructing a first-order accurate continuous approximation to Pareto sets.Earlier approaches typically sought one solution for all tasks, while diverse-solution methods were limited to finite, discrete outputs.Source: §1 · p. 1
  • A continuous local Pareto approximation uses second-order multi-objective results, but full Hessians make direct application poorly scalable to neural networks.The proposed approach uses Krylov subspace iteration methods, which are matrix-free linear solvers.Source: §1 · p. 1
  • The method generates much denser Pareto sets and fronts than prior work, with small computational overhead relative to the full MTL training process.The evaluated problems range from a 20k-parameter MultiMNIST classifier to UTKFace with millions of parameters.Source: §1 · pp. 1–2

2. Related work

MTL methods have addressed task trade-offs through weighted objectives and multi-objective optimization, but most produce one solution or a finite set. This work distinguishes itself by reconstructing continuous solution families using second-order analysis.

2. Related work

  • Adaptive task-weight methods simplify weight selection but still target one solution and are not designed to explore trade-offs.Choosing task weights is typically empirical, problem-specific, and tedious.Source: §2 · p. 2
  • MGDA-based approaches include methods for one optimal solution and discrete Pareto-front solutions, with each discrete candidate requiring full training from an initial network.The repeated training limits the ability to generate dense Pareto sets.Source: §2 · p. 2
  • Unlike prior first-order methods that return one or sparse finite solutions, this work uses second-order analysis to reconstruct continuous families for denser trade-off analysis.The approach is also related to Hessian-free methods, which use Hessian-vector products to solve Hessian systems efficiently.Source: §2 · p. 2

3. Preliminaries

The preliminaries define Pareto optimality and stationarity for multi-objective problems, then state a second-order result that locally approximates the continuous Pareto set. The result connects tangent directions to task gradients through a Hessian matrix.

3. Preliminaries

  • The setup is an unconstrained problem minimizing m objectives, with each task loss represented by an objective; the Pareto front is the image of the Pareto set.A point is Pareto optimal when no other point dominates it across the objectives.Source: §3 · p. 2
  • Pareto stationarity is presented as a necessary condition for Pareto optimality, and local optimality is framed as non-domination within a neighborhood.The stationarity definition assumes continuously differentiable objectives.Source: §3 · p. 2
  • For a smooth objective and Pareto-optimal point, the second-order result characterizes curves in the local Pareto set and relates their tangents to the task-gradient span through H(x*).H(x*) maps a tangent vector into the space spanned by the objective gradients; the full-Hessian requirement prevents direct application to MTL.Source: §3 · pp. 2–3

4. Efficient Pareto Set Exploration

The exploration procedure first finds Pareto-stationary networks and local tangent directions, then constructs and joins continuous local sets. Its scalable direction estimates use matrix-free iterative solvers, while the overall procedure recursively expands and refines candidate networks.

4. Efficient Pareto Set Exploration

  • The procedure has two phases: generate Pareto-stationary solutions and exploration directions, then build local subspaces and stitch colliding local Pareto fronts.The parameter space of neural networks makes explicit Hessian calculation computationally prohibitive.Source: §4 · p. 3

4.1. Gradient-Based Optimization

  • The algorithm can use any gradient-based local optimizer that returns a Pareto-stationary solution from an initial parameter vector.Options include weighted-sum SGD and methods that choose gradient directions by solving a small convex problem.Source: §4.1 · p. 3

4.2. First-Order Expansion

  • Exploration combines tangent directions into new points near a stationary solution, with the step size selected empirically.The directions are estimated from the first-order expansion.Source: §4.2 · p. 3
  • The first-order expansion estimates tangent directions by solving a sparse linear system; a correction vector addresses the fact that early-stopped solutions are rarely truly Pareto stationary.The correction seeks a minimal gradient modification so the point would be Pareto stationary.Source: §4.2 · p. 3
  • Krylov methods such as MINRES solve the system without full Hessians, and early termination still yields meaningful directions as residual error decreases monotonically.Hessian-vector products can be implemented in O(n) time, and k = 50 iterations were usually sufficient even for networks with millions of parameters.Source: §4.2 · pp. 3–4

4.3. The Full Algorithm

  • The full algorithm expands a queue breadth-first, refining child networks and collecting Pareto-stationary networks until the target count is reached.Each popped network generates K directions; the procedure stops after collecting M networks.Source: §4.3 · p. 4
  • Returned objectives, gradients, and parent references support local-subspace construction and help resolve the sign ambiguity of tangent directions.Predicted objective changes can rule out a direction that would retract rather than expand the Pareto set.Source: §4.3 · p. 4
  • Stochasticity, early termination, and local minima mean ParetoOptimize and ParetoExpand rarely return precise solutions.The paper notes that hyperparameter choices also matter for the algorithm.Source: §4.3 · p. 4

5. Continuous Parametrization

The section turns discrete Pareto solutions into continuous local families and describes stitching families that meet in objective space. In the two-objective case, Hessian null vectors can provide additional backbone directions, though this construction is generally not applicable to real problems because it requires null vectors.

5. Continuous Parametrization

  • A continuous local Pareto approximation is formed as the convex hull of a discrete solution and its K child solutions, using a continuous interpolation variable.Source: §5 · p. 4
  • For two objectives and K = 1, the local continuous approximation is a piecewise linear set in R^n.Source: §5 · p. 4
  • When continuous families collide in objective space, stitching points and cropping dominated solutions can combine them into a single continuous Pareto front.Source: §5 · p. 4
  • With two objectives, adding a Hessian null vector to a known tangent direction preserves the mapped curve’s value, tangent direction, and curvature at the Pareto point.The proposition states this for a smooth curve satisfying H(x*)x′(0) ≠ 0 and a null vector u of H(x*).Source: §5 · p. 5
  • Null-vector augmentation is generally not applicable to real problems, although neural networks are observed to have Hessian null spaces whose dimension is often much higher than m.Source: §5 · p. 5

6. Experimental Results

Across synthetic and multi-task settings, tangent-based expansion followed Pareto fronts more closely than gradients or weighted-sum SGD, and produced solutions that dominated WeightedSum on four of five datasets. The experiments also show how continuous parametrization and expansion settings affect the resulting solution families.

6.1. Datasets, Metrics, and Baselines

  • Experiments covered MultiMNIST and variants, UCI Census-Income, and UTKFace; efficiency was assessed by objective, gradient, and Hessian-vector-product evaluations and by hypervolume, where larger is better.The authors note that no prior work directly targets continuous Pareto-front recovery, so they use sufficiency and necessity tests rather than direct baseline comparisons.Source: §6.1 · p. 5

6.2.1. ZDT2-VARIANT

  • On ZDT2-variant, MINRES-predicted tangent directions tracked the analytic Pareto front better than gradients, which expanded into the dominated interior and required more correction.The experiment used MINRES with k = 1 to solve five tangent directions.Source: §6.2.1 · p. 6, Figure 2 · p. 6, Figure 3 · p. 6

6.2.2. MULTIMNIST SUBSET

  • With roughly equal time budgets, MINRES-50 explored a wider MultiMNIST subset Pareto front than WeightedSum SGD, using larger steps enabled by approximated tangents.The subset contained 2,048 images and used a simplified 1,500-parameter LeNet.Source: §6.2.2 · pp. 6–7, Figure 4 · p. 7

6.3. Pareto Expansion

  • On 4 of 5 datasets, Pareto expansion discovered solutions that dominated WeightedSum, while using only a small fraction of baseline training time to generate denser fronts.UCI Census-Income was the exception; the comparisons use f1 and f2 for the three-objective UCI Census-Income and UTKFace datasets.Source: Figure 6 · p. 7, §6.3 · pp. 7–8

6.4. Continuous Parametrization

  • On MultiMNIST, a continuous parameter t traversed solutions favoring small f1 through small f2, and stitching fronts from multiple seeds formed a larger continuous approximation.Expansions along a tangent and that tangent plus a null vector shared the same gradient and curvature at the starting point, allowing null-space bases to enrich the Pareto set without degrading its quality.Source: §6.4 · p. 8, Figure 7 · p. 8

6.5. Ablation Study

  • With s = 0.1, increasing MINRES iterations from 20 to 50 improved the approximated front, gains plateaued by 100, and the trajectory drifted after 500; k = 50 was used thereafter.With k = 50, conservative step sizes followed the front more closely, while aggressive steps moved into the dominated interior; the tested step sizes were 0.05, 0.1, 0.25, and 0.5.Source: §6.5 · p. 9, Figure 8 · p. 9

7. Conclusions

The paper presents an efficient, continuous Pareto-set and Pareto-front construction method for multi-task learning, combining second-order analysis with matrix-free iterative solvers. It argues that continuous reconstructions support analysis of local Pareto-set structure and dense trade-off solutions.

7. Conclusions

  • The method combines second-order analytical results with matrix-free iterative solvers to construct continuous Pareto sets and fronts and scale to networks with millions of parameters.Experiments also demonstrate the method’s efficiency on synthetic examples.Source: §7 · p. 9
  • Table 2 reports hypervolume ablations over k and s, with experiment cost proportional to k and inversely proportional to s.Source: Table 2 · p. 9
  • Continuous reconstruction enables analysis of local Pareto-set dimensionality and structure, interpretability tools, and dense fronts encoded with limited storage.The authors describe these as opportunities enabled by continuous rather than finite, sparse, discrete solutions.Source: §7 · p. 9

1. Proofs

The proofs connect the constrained formulation to an existing optimization problem, then establish how Hessian-null directions extend a local Pareto set without changing the curves’ curvature at the reference point.

1.1. Proof of Proposition 4.1

  • The last constraint links c and α; substituting c* reduces Problem (5) to Problem (3), so both share the same optimal α*.Source: §1.1 · p. 11

1.2. Proof of Proposition 4.2

  • Adding a Hessian-null vector u to a Pareto-set direction adds a new dimension, while the resulting curves have the same curvature at t=0.The curvature equality follows from matching the relevant denominators and numerators.Source: §1.2 · pp. 11–12
  • The proof establishes that u is orthogonal to both task gradients, using Hessian symmetry and the null-vector condition.Source: §1.2 · p. 12

2. Experimental Setup

The experimental setup spans an analytic synthetic problem and multi-task image and demographic datasets, with varied model sizes and task objectives. The experiments compare local expansion directions and specify the datasets, architectures, and solver settings used.

2.1. ZDT2-variant

  • The innermost analytic Pareto set in ZDT2-variant is a family of concentric cylinders; experiments compare MINRES tangent steps with perturbed weighted-gradient steps and MGDA line-search projection.Source: §2.1 · p. 12

2.2. MultiMNIST Subset

  • The MultiMNIST Subset uses 2,048 downsampled images and a 1,500-parameter modified LeNet; 101 weighted-loss solutions form a piecewise-linear empirical front.Source: §2.2 · pp. 12–13
  • Its expansion test compares 50-iteration MINRES tangent directions with 50-step gradient descent on individual and perturbed weighted losses from selected solutions.Source: §2.2 · p. 13

2.3. MultiMNIST and Its Variants

  • MultiMNIST, MultiFashion, and MultiFashionMNIST pair two image items for two classification tasks; modified LeNet baselines train for 30 SGD epochs, while expansion uses 50 MINRES iterations.Source: §2.3 · p. 13

2.4. UCI Census-Income

  • For UCI Census-Income, the imbalanced income task is omitted in favor of age prediction; three binary tasks use 487-feature inputs, with 100 MINRES iterations and sampled gradient sums.Source: §2.4 · pp. 13–14

2.5. UTKFace

  • UTKFace uses age, gender, and race objectives on cleaned face images; its ResNet18 has 11,180,616 parameters, and expansion uses 50 MINRES iterations.Source: §2.5 · p. 14

3. Synthetic Examples

On ZDT2-variant, tangent directions explored the Pareto front more consistently than gradients across random samples, and on MultiMNIST Subset they expanded the empirical front more accurately than gradient-descent or BFGS trajectories.

3.1. ZDT2-variant

  • The reported time saving on ZDT2-variant mostly came from near-optimal inputs to ParetoOptimize.Source: §3.1 · p. 14
  • Across 40 random samples, tangent directions explored ZDT2-variant’s Pareto front more consistently than gradients.Figure 2 compares tangent and weighted-sum expansion strategies; MGDA solutions sometimes clustered at Pareto-stationary points that were not Pareto optimal.Source: Figure 2 · p. 14, §3.1 · p. 14

3.2. MultiMNIST Subset

  • On MultiMNIST Subset, tangent directions expanded the empirical Pareto front more accurately and dominated the regions explored by 50-iteration gradient descent or BFGS.The experiments began from 26 evenly distributed Pareto-optimal points and used 50 MINRES iterations for tangent directions.Source: §3.2 · pp. 14–15

4. Pareto Expansion

The Pareto expansion experiments test whether the method can grow dense fronts from known Pareto-optimal solutions and whether its fronts compare favorably with weighted-sum SGD. Results cover MultiMNIST variants, UCI Census-Income, and UTKFace.

4. Pareto Expansion

  • The expanded fronts are presented as sufficiency tests, while comparisons against weighted-sum SGD test whether tangent directions are needed.The supplementary experiments extend earlier results across five datasets and more random seeds.Source: §4 · p. 15

4.1. MultiMNIST and Its Variants

  • On MultiMNIST and its variants, the method grew dense Pareto fronts from baseline seeds in a fraction of the time needed to train one Pareto-optimal solution.The seeds came from WeightedSum and ParetoMTL.Source: §4.1 · p. 15, Figure 4 · p. 16
  • On MultiMNIST variants, the expanded regions dominated SGD with varied learning rates and task weights across the reported experiments.The comparison used five Pareto-optimal starting solutions found by ParetoMTL; lower-left regions indicate better performance.Source: §4.1 · p. 15, Figure 5 · p. 17

4.2. UCI Census-Income

  • On three-objective UCI Census-Income, the method’s revealed Pareto fronts dominated SGD results in most cases.The comparison considered three pairs of losses and SGD baselines with different weight combinations and learning rates.Source: §4.2 · p. 15

4.3. UTKFace

  • On UTKFace, the method almost dominated all SGD baselines in five experiments, was clearly outperformed in one, and performed comparably in the rest.The reported comparison covered 15 experiments and four SGD baselines.Source: §4.3 · p. 17

5. Continuous Parametrization

The section explores how local continuous Pareto fronts can be combined into broader trade-off choices. Across the datasets, the experiments show front connectivity increasing with additional seeds and illustrate parametrizations from both many seeds and a single seed.

5. Continuous Parametrization

  • Local continuous Pareto fronts can be stitched together to cover a wider range of solutions with varying trade-offs.The section considers both local growth from Pareto-optimal solutions and global coverage by combining fronts.Source: §5 · pp. 17–18

5.1. MultiMNIST and Its Variants

  • On MultiMNIST and its variants, adding seeds made the reconstructed fronts more connected, and their union was reparametrized with a single scalar.The experiments increased the number of Pareto-optimal seeds from 3 to 25.Source: §5.1 · p. 18

5.2. UCI Census-Income

  • On UCI Census-Income, the continuous front was reconstructed from 36 Pareto-optimal seeds and visualized as a three-dimensional surface mesh.Samples from each seed’s continuous Pareto set were marked with a unique color.Source: §5.2 · p. 19

5.3. UTKFace

  • On UTKFace, the continuous Pareto front was reconstructed from a single Pareto-optimal seed.The visualization used one color for all samples.Source: §5.3 · p. 19

6. Ablation Study

The ablation study finds that the reported effects of hyperparameters k and s persist across five Pareto-optimal solutions with different trade-offs on MultiMNIST and its variants.

6. Ablation Study

  • Across five Pareto-optimal solutions on MultiMNIST and its variants, the reported effects of hyperparameters k and s were consistently observed.The seeds were distributed across the Pareto front, representing varied trade-offs.Source: §6 · p. 19

Related papers

Loading 2006.16434v2…