PaperlayerView exact revision on arXiv

Source-linked AI summary

Pareto Multi-Task Learning

Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qingfu Zhang, Sam Kwong

arXiv:1912.12854v1cs.LGstat.ML

TL;DR

Conflicting task objectives make a single MTL solution inadequate for representing all useful trade-offs. Pareto MTL decomposes MTL into preference-specific constrained subproblems and generates distributed Pareto solutions, outperforming some state-of-the-art MTL algorithms across applications.

  • Problem

    Conflicting tasks can make it impossible for one solution to optimize all tasks, while a single prior Pareto solution may not satisfy practitioners' different trade-off needs.

  • Method

    Pareto MTL formulates MTL as multi-objective optimization, decomposes it into constrained subproblems with different preferences, and solves them in parallel.

  • Results

    Pareto MTL generates well-representative Pareto solutions and outperforms some state-of-the-art MTL algorithms across different MTL applications.

  • Takeaways & Limitations

    Practitioners can select preferred solutions from a set of Pareto solutions with different trade-offs for a given MTL problem.

  • Takeaways & Limitations

    Pareto MTL uses simple hard parameter-sharing neural networks and has not been generalized in this paper to advanced soft parameter-sharing architectures.

Abstract

from arXiv · show

Multi-task learning is a powerful method for solving multiple correlated tasks simultaneously. However, it is often impossible to find one single solution to optimize all the tasks, since different tasks might conflict with each other. Recently, a novel method is proposed to find one single Pareto optimal solution with good trade-off among different tasks by casting multi-task learning as multiobjective optimization. In this paper, we generalize this idea and propose a novel Pareto multi-task learning algorithm (Pareto MTL) to find a set of well-distributed Pareto solutions which can represent different trade-offs among different tasks. The proposed algorithm first formulates a multi-task learning problem as a multiobjective optimization problem, and then decomposes the multiobjective optimization problem into a set of constrained subproblems with different trade-off preferences. By solving these subproblems in parallel, Pareto MTL can find a set of well-representative Pareto optimal solutions with different trade-off among all tasks. Practitioners can easily select their preferred solution from these Pareto solutions, or use different trade-off solutions for different situations. Experimental results confirm that the proposed algorithm can generate well-representative solutions and outperform some state-of-the-art algorithms on many multi-task learning applications.

1 Introduction

Multi-task learning improves related tasks jointly, but conflicting objectives make a single universally best solution difficult. Pareto MTL addresses this by generating distributed trade-off solutions for practitioner selection.

  • MTL learns multiple correlated tasks together, improving task performance and reducing inference time in many applications.
  • Conflicting tasks can prevent any single solution from optimizing all tasks simultaneously, forcing practitioners to make trade-offs.
  • A prior multi-objective method finds one Pareto optimal MTL solution, which may not satisfy practitioners when many optimal trade-offs exist.
  • Pareto MTL generates a set of well-representative Pareto solutions with different trade-offs instead of requiring exhaustive weight searches.
  • The method decomposes MTL into multiple preference-specific subproblems and solves them in parallel to obtain distributed Pareto solutions.

2 Related Work

Related work uses scalarization, adaptive weighting, evolutionary optimization, and gradient descent to address multi-task trade-offs. Pareto MTL extends gradient-based MTL toward systematic generation of diverse preference-specific solutions.

  • Shared-representation MTL methods mainly combine tasks, while trade-off selection among multiple tasks remains insufficiently addressed.
  • Random or grid searches over linear-scalarization weights can be extremely inefficient and do not target diverse trade-off solutions.
  • Population-based evolutionary methods can find distributed Pareto solutions, but they are unsuitable for large-scale optimization.
  • Multi-objective gradient descent is efficient with available gradients, while prior MGDA-based MTL balances tasks without systematically incorporating trade-off preferences.
  • Pareto MTL generalizes MGDA-based MTL to find well-representative solutions with different trade-offs among tasks.

3 Multi-Task Learning as Multi-Objective Optimization

MTL can be formulated as multi-objective optimization because task losses may conflict, yielding Pareto solutions rather than one universally optimal point. The section contrasts linear scalarization with gradient-based descent and motivates Pareto MTL.

  • An MTL problem minimizes a vector of task losses while exploiting shared structure and information among correlated tasks.
  • When no solution optimizes every objective simultaneously, the attainable alternatives are Pareto optimal solutions representing different trade-offs.
  • Pareto dominance requires no worse loss on every task and strictly better loss on at least one task.
  • The Pareto set contains nondominated parameter points, while the Pareto front is their image in loss space.
  • The paper seeks a representative set that approximates the Pareto front rather than a single trade-off solution.
  • Linear scalarization combines task losses into one weighted surrogate objective, but performance depends heavily on manually chosen weights.
  • Different linear-scalarization weights cover only convex Pareto-front regions and cannot handle concave fronts.
  • Gradient-based multi-objective descent updates parameters along a direction that decreases all task losses when the current point is not Pareto critical.

4 Pareto Multi-Task Learning

Pareto MTL decomposes multi-task learning into constrained subproblems with distinct preference regions, solving them in parallel to obtain diverse trade-off solutions. Its scalable gradient-based formulation and adaptive scalarization support large neural-network decision spaces.

  • MTL Decomposition: Pareto MTL decomposes an MTL problem into constrained multi-objective subproblems with different trade-off preferences, solved in parallel.The resulting solutions represent different trade-offs among the original tasks.
  • MTL Decomposition: Preference vectors divide the objective space into subregions, guiding each subproblem toward a representative Pareto solution.A solution belongs to the region whose preference vector has the smallest acute angle to its objective vector.
  • Constrained Optimization: Each subproblem seeks Pareto optimality restricted to its feasible region, using gradient-based directions that reduce objectives and activated constraints.A small threshold ϵ handles points near constraint boundaries; a zero direction indicates a restricted Pareto critical point.
  • Scalable Optimization: The algorithm reformulates the high-dimensional constrained direction problem into a lower-dimensional dual problem over objective and constraint space.With two objectives and five activated constraints, the dual dimension is 7 rather than potentially more than a million parameters.
  • Algorithm Framework: All subproblems can be solved independently in parallel using evenly distributed unit preference vectors as a default when no prior preference information exists.For two tasks, the paper gives K + 1 vectors along a quarter-circle of unit preferences.
  • Adaptive Scalarization: Pareto MTL can be interpreted as linear scalarization with dynamically adaptive task weights, extending the unconstrained MGDA formulation.The adaptive weights are determined during optimization rather than fixed through exhaustive weight search.

5 A Synthetic Example

The synthetic experiment compares Pareto MTL with fixed scalarization and MOO-MTL on a concave Pareto front. Pareto MTL finds well-distributed trade-off solutions, whereas the alternatives concentrate or reach only extreme regions.

  • Experimental Setup: The experiment compares Pareto MTL with linear scalarization and MOO-MTL on a synthetic problem with two objectives and a concave Pareto front.The objectives are minimized simultaneously over a d-dimensional decision variable.
  • Results: Pareto MTL successfully finds a set of well-distributed Pareto solutions with different trade-offs.This behavior is reported across multiple runs of the synthetic experiment.
  • Results: MOO-MTL produces solutions with similar middle-of-front trade-offs because it balances tasks during optimization.Its solutions therefore have worse coverage of distinct trade-offs in this example.
  • Results: Linear scalarization generates only extreme solutions on the concave Pareto front.This limits its ability to represent intermediate trade-offs in the synthetic problem.

6 Experiments

Across multi-task benchmarks and autonomous-driving localization, Pareto MTL produces diverse trade-off solutions and achieves the strongest reported overall performance among the compared MTL methods. Its extreme-preference solutions can match the best performance of individual tasks, while the base architecture remains limited to hard parameter sharing.

  • Experimental Setup: Pareto MTL is compared with Single Task, Grid Search, GradNorm, Uncertainty, and MOO-MTL across several multi-task learning applications.The experiments include MultiMNIST-like classification tasks and autonomous-driving self-localization.
  • Multi-Task Results: Pareto MTL generates multiple well-distributed Pareto solutions compatible with the strong single-task baseline while offering different task trade-offs.The paper reports this pattern across all experiments.
  • Multi-Task Results: Pareto MTL achieves the overall best performance among the evaluated MTL algorithms.Figure 4 reports significant outperformance of Grid Search, Uncertainty, and GradNorm; MOO-MTL has poorer diversity.
  • Multi-Fashion-MNIST: Extreme preference vectors produce the best performance for their corresponding tasks, including in the less-correlated Multi-Fashion-MNIST experiment.In that setting, Pareto MTL can still achieve the best performance for each task separately despite other MTL solutions being dominated by single-task baselines.
  • Scope and Limitation: The reported base model uses simple hard parameter sharing, leaving generalization to soft parameter-sharing architectures as an open direction.The paper also suggests task-relation learning for better trade-offs among less relevant tasks.
  • Self-Driving Car: Localization: In autonomous-driving self-localization, Pareto MTL provides different trade-offs between camera location and orientation estimation while outperforming other MTL approaches.The experiment uses a PoseNet with a ResNet18 encoder on the Zpark subset of the Apolloscape dataset.

7 Conclusion

The paper proposes Pareto MTL to generate well-distributed solutions representing different task trade-offs. Experiments report that it outperforms some state-of-the-art MTL algorithms across multiple applications.

  • Conclusion: Pareto MTL generates a set of well-distributed Pareto solutions with different trade-offs for a given MTL problem.Practitioners can select preferred solutions from this set.
  • Conclusion: Experimental results show that Pareto MTL outperforms some state-of-the-art MTL algorithms across different MTL applications.The reported results also confirm that it finds well-representative solutions.

Supplementary Material: Pareto MTL

The supplementary material provides detailed discussions, experimental results, limitations, and potential research directions for Pareto MTL.

  • The supplementary material expands the discussion and experiments on Pareto MTL.It also identifies limitations and proposes potential research directions.

A The Importance of Finding the Initial Solution

Initialization helps Pareto MTL locate diverse trade-off solutions, especially near Pareto-front endpoints. Without it, the method still finds trade-offs but produces less diverse solutions and can fail to cross sub-region boundaries.

  • Synthetic example: Without initialization, Pareto MTL still generates Pareto solutions with different trade-offs on the synthetic example.It outperforms MOO-MTL in this test.
  • Synthetic example: Initialization improves solution diversity and enables Pareto MTL to find solutions near the endpoints of the Pareto front.Without initialization, the method fails to cross boundaries between different sub-regions.
  • Initialization mechanism: With initialization, solutions are sequentially updated toward their assigned preference sub-regions as constraint values decrease.Constraints can become inactive after a solution crosses into the relevant sub-region.
  • Multi-FashionMNIST: The initialization step improves diversity on the Multi-FashionMNIST problem compared with running Pareto MTL without initialization.
  • Optimization steps: Pareto MTL uses task-only directions for convergence, constraint-only directions for initialization, and combined directions for its main optimization step.The combined step balances convergence and diversity across restricted Pareto sub-regions.
  • Potential extension: The authors suggest saving solutions after the main step, relaxing constraints, and applying a convergence-oriented step as a possible extension.This could combine diverse restricted solutions with potentially better-converged unrestricted solutions.

B MTL with Many Tasks

For multi-task problems with many tasks, Pareto MTL can produce preferred trade-off solutions, but broad objective-space coverage requires many preference vectors and becomes computationally and visually demanding.

  • Preference-based decomposition: Pareto MTL decomposes an MTL problem into constrained subproblems using preference vectors and solves them to obtain different task trade-offs.The approach can directly target practitioners’ preferred trade-offs.
  • Scalability: The number of required preference vectors increases exponentially with the number of tasks when the objective space must be fairly covered.Each preference vector corresponds to a constrained subproblem.
  • Census-income experiment: For the three-task census-income experiment, 25 preference vectors represent task trade-offs much better than 5 preference vectors.Five vectors provide varied trade-offs but do not properly represent the Pareto front.
  • Census-income experiment: Pareto MTL solutions dominate nearly all solutions from linear scalarization with random weight search on the census-income problem.
  • Decision support: Visualization of solutions becomes less clear for three tasks and much more difficult for more than three tasks.
  • UTKFace experiment: On UTKFace, a few specific preference vectors still produce representative Pareto solutions for practitioners’ preferred trade-offs.The experiment predicts gender, race, and age from face images.
  • Adaptive weights: Pareto MTL’s adaptive weights differ across preference vectors, whereas MOO-MTL and GradNorm aim to balance tasks during training.The preference term guides Pareto MTL toward its assigned preference vector.

D Pareto MTL with Different Tasks Difficulties

Pareto MTL performs well when task difficulties are balanced or moderately unbalanced, but its performance worsens under extreme difficulty differences and depends on preference-vector selection.

  • Difficulty imbalance: For high difficulty imbalance, (a1 = 50, a2 = 1), Pareto MTL’s performance gets worse.
  • Assumption: Pareto MTL implicitly assumes that tasks have similar difficulties and uses widely distributed unit vectors as preference vectors.Its performance may deteriorate when task difficulties differ extremely.
  • Difficulty imbalance: Pareto MTL finds widely distributed Pareto solutions under low or medium difficulty imbalance: (a1 = 2, a2 = 1) and (a1 = 10, a2 = 1).
  • Algorithmic bias: In the highly unbalanced case, MOO-MTL favors the easier Task 1, while most Pareto MTL solutions favor the harder Task 2.Evenly distributed preference vectors create highly unbalanced sub-regions when task difficulties differ.
  • Future directions: Combining Pareto MTL with adaptive weighting or learning-based methods is proposed as a direction for handling different task difficulties and adapting preferences.
  • Preference selection: The final solution distribution depends on both the preference vectors and the shape of the Pareto front.Even for similarly difficult tasks, preference vectors must be assigned appropriately.
  • Preference selection: Random preference-vector sets consistently produce well-distributed solutions, but their accuracy can be better or worse than the default uniform setting.

F The Gap between Optimization and Generalization

Pareto MTL can distribute solutions in training-loss space, but optimization outcomes may diverge from training and test accuracy, especially under overparameterization and non-convexity. These gaps expose limitations in using loss optimization to represent generalization trade-offs.

  • Experimental setup: The ResNet-18 experiments use an overparameterized model that can remember all training examples, creating a deliberate gap between training fit and generalization.MultiMNIST models use early stopping, whereas the other two datasets are trained until the end.
  • Optimization versus generalization: Pareto MTL generates widely distributed training-loss solutions, but this does not ensure similarly diverse or strong training and test accuracy.The method directly optimizes training losses rather than accuracy measures.
  • MultiMNIST: On MultiMNIST with early stopping, Pareto MTL solutions are well distributed in training loss and accuracy but are outperformed in training accuracy by the separate single-task baseline.Its test-accuracy solutions are less diverse, yet outperform the single-task baseline and provide different optimal trade-offs.
  • MultiFashionMNIST and Multi-(Fashion+MNIST): On MultiFashionMNIST and Multi-(Fashion+MNIST), training losses approach 0 and most training accuracies approach 100%, while Pareto MTL performs worse on test accuracy.Some Pareto MTL solutions still match the strong performance of separate single-task solutions.
  • Optimization limitation: Because deep-network losses are highly non-convex, constrained subproblems may converge to inferior local Pareto optima.Avoiding poor local Pareto optima is identified as an important extension for Pareto MTL.
Loading 1912.12854v1…