Source-linked AI summary
Multi-Objective Meta Learning
Feiyang Ye, Baijiong Lin, Zhixiong Yue, Pengxin Guo, Qiao Xiao, Yu Zhang
TL;DR
Multi-objective meta learning must handle potentially conflicting upper-level targets, but prior approaches either aggregate objectives with tuned weights or use evolutionary algorithms. MOML formulates the task as a multi-objective bi-level problem and solves it with alternating gradient-based methods; the authors report convergence results and effectiveness across several learning problems.
Problem
Multi-objective meta-learning approaches either combine objectives using tuned weights or use evolutionary algorithms, while a gradient-based MOBLP solution with a convergence guarantee is lacking.
Source: §1 · p. 1Method
MOML formulates meta learning as a multi-objective bi-level optimization problem and alternates lower-level gradient descent with upper-level gradient-based multi-objective optimization.
Source: §1 · p. 2, §3.2 · p. 3Results
Takeaways & Limitations
MOML generalizes bi-level meta learning by allowing multiple objectives in the upper-level subproblem.
Source: §3.2 · p. 3The MOBLP requires a unique lower-level minimizer for each α; without that condition, the problem can be ill-defined.
Source: §4.1 · p. 3, §4.3 · p. 4
Abstract
from arXiv · showhide
Meta learning with multiple objectives can be formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possible conflicting targets for the meta learner. However, existing studies either apply an inefficient evolutionary algorithm or linearly combine multiple objectives as a single-objective problem with the need to tune combination weights. In this paper, we propose a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, neural architecture search, domain adaptation, and multi-task learning.
1. Introduction
Meta learning adapts knowledge from multiple tasks, but many meta-learning methods optimize only one upper-level objective despite real applications requiring trade-offs among conflicting goals. The paper proposes a gradient-based MOML framework for multi-objective bi-level optimization, proves convergence properties, and evaluates it across several meta-learning problems.
Motivation
- Real-world meta learning may need to balance conflicting objectives, such as accuracy and robustness or network size and performance.The latter trade-off matters when architectures are deployed on resource-limited devices.Source: §1 · p. 1
Research gap
- Existing multi-objective meta-learning approaches use tuned weighted combinations or time-consuming evolutionary algorithms that do not integrate with gradient-based deep learning.The paper frames the problem as a Multi-Objective Bi-Level Optimization Problem (MOBLP), with multiple upper-level objectives and task adaptation at the lower level.Source: §1 · pp. 1–2
Contribution
- The proposed MOML framework solves the MOBLP by alternating lower-level gradient descent with gradient-based multi-objective optimization at the upper level.The authors describe this as the first gradient-based algorithm for MOBLPs and report proving its convergence properties.Source: §1 · p. 2
- Experiments apply MOML to few-shot learning, neural architecture search, domain adaptation, and multi-task learning.The paper reports that these experiments show the framework's effectiveness.Source: §1 · p. 2
2. Related Work
The related work presents meta learning as a bi-level approach to adapting knowledge across tasks, while multi-objective optimization addresses simultaneous targets that may conflict. It distinguishes gradient-based optimization as an approach that can be integrated with gradient-based machine-learning models.
2.1. Meta Learning
- Bi-level meta learning updates meta parameters through upper-level objectives while lower-level optimization adapts to new tasks using those parameters.MAML is given as an example: it adapts on task training data and updates the initialization according to validation performance.Source: §2.1 · p. 2
2.2. Multi-Objective Optimization
- Multi-objective optimization handles simultaneous, potentially conflicting targets, which machine-learning methods sometimes reduce to one objective by aggregation.The section distinguishes evolutionary, population-based, and gradient-based algorithms, and focuses on gradient-based methods for their integration with gradient-based models.Source: §2.2 · p. 2
3. The MOML Framework
MOML formulates meta learning as a multi-objective bi-level optimization problem, with task learning below and multiple meta objectives above. The framework generalizes the usual single-objective bi-level formulation.
3.1. Notations and Terminologies
- For a vector-valued objective, the minimal points form the Pareto frontier, and the corresponding efficient solutions are Pareto-optimal solutions.The section defines minimality relative to an ordering cone: a point is minimal when no other point dominates it.Source: §3.1 · p. 3
3.2. Formulation
- MOML's upper-level function F contains multiple objectives, while its lower-level function f defines the current task's objective and S(α) denotes its lower-level optimal solutions.For meta learning, the paper identifies f with a task objective such as training loss.Source: §3.2 · p. 3
- When the upper level has one objective, the formulation reduces to the bi-level optimization problem commonly used in meta learning.The paper therefore presents MOML as a generalization of that formulation.Source: §3.2 · p. 3
- Prior MOBLP solvers relied on evolutionary algorithms described as complex, lacking convergence guarantees, and incompatible with gradient-based models; the paper proposes a gradient-based solver.The authors state they were unaware of a gradient-based MOBLP algorithm with a convergence guarantee.Source: §3.2 · p. 3
4. Optimization
The optimization procedure requires a unique lower-level minimizer and approximates that lower-level solution iteratively before optimizing multiple upper-level objectives. Under stated assumptions, the authors prove set-convergence results and describe conditions associated with stronger solution guarantees.
4.1. Lower-Level Singleton Condition
- The MOBLP requires a unique lower-level minimizer for each α; without the lower-level singleton condition, the upper-level evaluation is unclear.The paper notes that the problem can be ill-defined when the lower-level solution is a set.Source: §4.1 · p. 3
4.2. Gradient-based Optimization Algorithm
- The algorithm approximates the lower-level solution with K iterative updates, using gradient descent such as SGD, instead of directly optimizing through a closed-form solution.The paper notes that closed-form lower-level solutions are usually unavailable, and optimality-condition constraints do not work for general learning models.Source: §4.2 · pp. 3–4
- MGDA solves the upper-level unconstrained multi-objective problem by selecting a descent direction from the convex hull of objective gradients.The method uses a quadratic program to determine objective weights; the paper adopts FISTA for that subproblem and updates α by SGD.Source: §4.2 · p. 4
- Algorithm 1 is presented as the first gradient-based MOBLP solver, but it returns one solution rather than an evolutionary population of solutions.Finding multiple Pareto-optimal MOBLP solutions is left to future work.Source: §4.2 · p. 4
4.3. Convergence Analysis
- Under Assumptions 1 and 2, the algorithm's minimal-point and efficient-solution sets converge in the Kuratowski-Painlevé sense.The theorem states convergence of both Min ϕK(α) to Min ϕ(α) and Eff ϕK(α) to Eff ϕ(α).Source: §4.3 · pp. 4–5
- The authors state that a strongly convex lower-level problem can satisfy the singleton condition with a uniform linear convergence rate; MGDA can reach a Pareto-optimal solution when the upper-level function is strictly P-convex.They separately describe MGDA's convergence to a Pareto-stationary point when MGDA converges.Source: §4.3 · p. 5
5. Use Cases of MOML
The use cases cast few-shot learning, neural architecture search, semi-supervised domain adaptation, and multi-task learning as MOML problems with multiple upper-level objectives.
5.1.1. PROBLEM FORMULATION
- For few-shot learning, MOML models clean classification and robustness as separate upper-level objectives, using perturbed query examples to measure robustness.The formulation fits MAML when α is the shared initialization and ω denotes task-specific parameters.Source: §5.1.1 · p. 5
5.1.2. EXPERIMENTS
- MOML trades around 5% lower clean classification accuracy than MAML for robustness gains of up to about 8.4 times on two few-shot benchmarks.Across 600 testing tasks, the reported comparison covers clean classification accuracy and PGD accuracy; MOML’s solution also falls well on the approximate Pareto frontier.Source: §5.1.2 · pp. 5–6, Table 1 · p. 5, Figure 1 · p. 6
5.2.1. PROBLEM FORMULATION
- The NAS formulation extends DARTS by adding robustness and parameter count to its accuracy objective, while retaining MOML’s bi-level optimization framework.The paper describes MOML as more efficient than NSGANetV2 and provides a convergence guarantee.Source: §5.2.1 · p. 6, §5.2 · p. 6
5.2.2. EXPERIMENTS
- MOML searches CIFAR-10 architectures for accuracy, adversarial robustness, and expected parameter count; at comparable size, MOML-V1 raises PGD accuracy by about 14% versus DARTS-C26 while clean accuracy drops around 0.5%.The reported comparison also finds more robust architectures with similar model size and comparable classification accuracy.Source: §5.2.2 · pp. 6–7, §5.2.1 · p. 6, Table 2 · p. 7
5.3.2. EXPERIMENTS
- For semi-supervised domain adaptation, MOML treats source-domain loss, target-domain loss, and MMD alignment as multiple upper-level objectives; it performs best among the compared methods on Office-31.The experiments use three labeled target samples per class, with remaining target samples unlabeled.Source: §5.3.2 · pp. 7–8, §5.3 · p. 7, §5.3.1 · p. 7
5.4.2. EXPERIMENTS
- In multi-task learning, MOML jointly learns task loss weights and model parameters through a bi-level formulation based on training and validation splits; on Office-Home it outperforms weighting-strategy baselines in many cases.Pr is an exception: all MTL methods trail STL there, while joint training improves Ar and Rw; MOML alone among the task-weight-learning MTL models beats equal weights on average.Source: §5.4.2 · p. 8, §5.4 · p. 8, §5.4.1 · p. 8
6. Conclusions
The paper concludes that MOML is a multi-objective bi-level framework with a gradient-based algorithm whose convergence is analyzed. It also investigates several use cases and identifies application to more learning problems as future work.
6. Conclusions
- The conclusion presents MOML as a multi-objective bi-level framework and reports a gradient-based optimization algorithm with convergence analysis.The paper also investigates several use cases to demonstrate the framework's effectiveness.Source: §6 · p. 8
A. Additional Definitions
- The appendix introduces P-convexity and strict P-convexity for vector-valued functions.P-convexity requires coordinatewise convexity; strict P-convexity requires strict inequalities for distinct inputs and interior mixture weights.Source: A. Additional Definitions · p. 12
- The appendix defines lower and upper limits of set sequences and denotes Kuratowski set convergence by A_n → A.The upper limit uses subsequential limits, while the lower limit concerns points eventually approximated by the sequence.Source: A. Additional Definitions · p. 12
B. Proofs of Theorems in Section 4
These proofs use stability results for multi-objective problems to relate minima under continuous convergence. They establish set-convergence claims for weak minima and minima under stated convexity conditions.
B. Proofs of Theorems in Section 4
- The appendix introduces weakly minimal points as points for which no feasible alternative improves every objective strictly.It also states that minima are a subset of weak minima.Source: B. Proofs of Theorems in Section 4 · p. 12
- Under continuous convergence, any limit of weak-minimum points remains a weak minimum of the limiting vector-valued function.The proof rules out a limit that is strictly worse in every objective than another feasible point.Source: B. Proofs of Theorems in Section 4 · p. 12
- When the functions are P-convex as well as continuously convergent, minima of the limiting function are included in the lower limit of the approximating minima.The proof cites prior stability results for this inclusion.Source: B. Proofs of Theorems in Section 4 · pp. 12–13
B.1. Proof of Theorem 1
The proof establishes continuity of the upper-level objective evaluated at the lower-level optimizer as the meta parameter varies. It uses uniqueness of the lower-level minimizer and continuity of the objective.
B.1. Proof of Theorem 1
- The proof reduces continuity to showing convergence of F evaluated at the lower-level optimizer along every convergent sequence of meta parameters.The lower-level solution is defined as the minimizer of f for each meta parameter.Source: §B.1 · p. 13
- Uniqueness of the lower-level minimizer identifies each solution along the sequence with the optimizer at that parameter.The proof then uses boundedness to obtain a convergent subsequence.Source: §B.1 · p. 13
- Joint continuity of F and convergence of the lower-level solutions yield convergence of the evaluated upper-level objective.The proof concludes that F(ω*(α_n), α_n) converges to F(ω*(ᾱ), ᾱ).Source: §B.1 · p. 13
B.2. Proof of Theorem 2
The proof of Theorem 2 establishes convergence of efficient solution sets as the lower-level iteration count increases. The supplied experiment passages specify few-shot learning, neural architecture search, and domain-adaptation settings, but do not report their results.
B.2. Proof of Theorem 2
- Uniform convergence of the lower-level iterates and the stated convexity conditions support convergence of the efficient solution sets.The proof also states that analogous results hold for weakly minimal points.Source: §B.2 · p. 13
C.1. Few-Shot Learning
- Few-shot experiments use 5-way 1- or 5-shot tasks on CUB and mini-ImageNet, with 600 novel-task tests averaged for final results.Each task uses k samples per class for adaptation and 16 per class for query or test; the comparison is with MAML.Source: §C.1 · pp. 13–14
C.2. NAS
- The NAS setup follows DARTS for search and trains the searched network for 600 epochs before testing on the full test set.Search uses an 8-cell network for 50 epochs; evaluation uses 20 searched cells.Source: §C.2 · p. 14
C.3. Semi-Supervised Domain Adaptation
- The semi-supervised domain-adaptation setup uses an ImageNet-pretrained ResNet-50 backbone with a fully connected layer and batches split across source, labeled-target, and unlabeled-target domains.Each domain contributes 32 images to the batch of 96.Source: §C.3 · p. 14
C.4. Multi-Task Learning
The multi-task models use a shared hard-sharing or multi-head architecture, with a pretrained ResNet-50 backbone and dataset-specific training settings.
C.4. Multi-Task Learning
- A pretrained ImageNet ResNet-50 extracts features for the multi-task models.The backbone is used for both Office-31 and Office-Home.Source: §C.4 · p. 14
- The hard-sharing or multi-head network shares its first fully connected layer across tasks and uses a second layer for task-specific outputs.The two-layer architecture uses ReLU activation.Source: §C.4 · p. 14
- Training uses Adam at learning rate 0.0001, with 60/20/20 train-validation-test splits and dataset-specific batch sizes.Office-31 uses batch sizes 32/32; Office-Home uses 16/32 for training/validation.Source: §C.4 · p. 14
D. Ablation Study on NAS
On CIFAR-10, MOML with MGDA generally has higher clean accuracy and robustness than the equal-weight single-objective variant at comparable architecture sizes.
D. Ablation Study on NAS
- At comparable architecture sizes, MOMLw/o MGDA has lower clean accuracy and robustness than MOML in most cases.This comparison replaces the upper-level multi-objective problem with an equally weighted single objective.Source: D. Ablation Study on NAS · p. 14
- For L values of 1 or 2, MOMLw/o MGDA searches smaller architectures than MOML.The evaluation adjusts initial channel counts to make the compared network sizes comparable.Source: D. Ablation Study on NAS · p. 14
- The ablation compares MOML with an equal-weight single-objective variant that omits MGDA from the upper-level problem.The variant uses the same experimental settings as MOML.Source: D. Ablation Study on NAS · p. 14
E. Experimental Results on the Office-31 Dataset for Multi-Task Learning
On Office-31 multi-task learning, MOML outperforms STL and DMTL weighting strategies in most cases, with task-specific differences across A, W, and D.
E. Experimental Results on the Office-31 Dataset for Multi-Task Learning
- For task A, MOML is the best MTL method, but all MTL methods perform worse than STL; for task D, all MTL methods outperform STL.The authors suggest learning tasks W and D may hinder task A, while training the three tasks together may improve task D.Source: E. Experimental Results on the… · p. 15
- MOML is the only MTL model that performs no worse than DMTL with equal weights on every task.The authors suggest this may benefit from using validation loss to estimate generalization loss more accurately.Source: E. Experimental Results on the… · p. 15
- MOML outperforms STL and DMTL with different weighting strategies in most Office-31 cases.Table 6 reports accuracy (%) on Office-31 for multi-task learning.Source: E. Experimental Results on the… · pp. 14–15, Table 6 · p. 15