PaperlayerView exact revision on arXiv

Source-linked AI summary

ParetoTransport: Generative Optimization by Mass Transport Toward The Pareto Front

Stephanie Holly, Sepp Hochreiter, Werner Zellinger

arXiv:2609.07706v1cs.LG

TL;DR

Offline MOO must balance objective-space movement toward the Pareto front with effective distribution of designs along it, while prior generative methods often guide samples individually. ParetoTransport uses training-free mass-transport guidance with Wasserstein matching for pre-trained flow models. It achieves state-of-the-art performance among training-free generative inverse methods across standard benchmarks and establishes a convergence result; its stated limitation is the many-objective setting.

  • Problem

    Offline MOO needs to move designs toward the Pareto front and distribute them along it, but existing generative methods largely control the induced distribution indirectly through sample-wise guidance.

    Source: §1 · p. 1

  • Method

    ParetoTransport iteratively transports the empirical offline distribution toward the Pareto front using training-free guidance that combines directional improvement with Wasserstein control of probability-mass allocation.

    Source: §1 · p. 2

  • Results

    ParetoTransport achieves state-of-the-art performance among training-free generative inverse methods and competitive performance relative to forward methods on standard offline MOO benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper treats the induced distribution over designs, rather than only individual candidates, as the object of optimization for generative MOO.

    Source: §8 · p. 9

    The many-objective setting (m ≥4) is left for future work because a single global CHIM normal may be insufficient for higher-dimensional Pareto-front geometry.

    Source: §8 · p. 10

Abstract

from arXiv · show

Offline multi-objective optimization requires not only moving the objective vectors of candidate designs toward the Pareto front, but also distributing them effectively along it. Generative methods have recently emerged as a natural approach because they learn a distribution over feasible designs while allowing generation to be steered toward promising designs. Existing methods, however, largely retain classical sample-wise guidance strategies, leaving the distribution-level modeling capability of generative methods underused. We propose ParetoTransport, a training-free guidance method for pre-trained flow-matching models that explicitly specifies and refines a population-level distribution in objective space. ParetoTransport guides a flow-matching sampler to iteratively transport the empirical offline distribution toward the Pareto front, with Wasserstein matching to intermediate proxy distributions. This directly controls distributional displacement and mass allocation along the front. We establish a convergence result and demonstrate state-of-the-art performance on standard offline MOO benchmarks, extending recent evaluations beyond hypervolume to generational distance, inverted generational distance, and Wasserstein distance.

1 INTRODUCTION

Offline MOO must find designs that balance conflicting objectives using only previously evaluated designs, while existing generative guidance often controls samples rather than the resulting population distribution. ParetoTransport uses mass transport and Wasserstein matching to guide that distribution toward the Pareto front, with a convergence result and strong benchmark performance.

1 INTRODUCTION

  • Offline MOO seeks Pareto-optimal designs balancing potentially conflicting objectives using only a fixed dataset, especially when new evaluations are expensive or impractical.Examples include molecular and materials design, where properties such as efficacy, toxicity, strength, conductivity, or cost must be balanced.Source: §1 · p. 1
  • Existing generative methods generally guide the induced objective-space distribution indirectly through sample-wise mechanisms rather than explicitly refining its geometry and mass allocation.The approaches described use scalarized objectives, preference models, repulsive interactions, or selected conditioning targets.Source: §1 · p. 1
  • Across standard offline MOO benchmarks, ParetoTransport is state of the art among training-free generative inverse methods and competitive with forward methods.Evaluation spans 24 datasets and includes HV, GD, IGD, and W2.Source: §1 · p. 2
  • ParetoTransport is training-free guidance for pre-trained flow-matching models that iteratively transports the empirical offline distribution toward the Pareto front.It combines directional improvement with Wasserstein matching to control probability-mass allocation.Source: §1 · p. 2
  • Under general assumptions, the method has a Wasserstein convergence result toward the Pareto-front distribution.The paper also motivates W2 by relating it to GD and IGD.Source: §1 · p. 2

2 PROBLEM: OFFLINE MULTI-OBJECTIVE OPTIMIZATION

Multi-objective optimization seeks designs that balance potentially conflicting objectives, so it generally targets a set of Pareto-optimal solutions rather than one minimizer. Offline MOO restricts optimization to previously evaluated designs, without further objective evaluations.

2 PROBLEM: OFFLINE MULTI-OBJECTIVE OPTIMIZATION

  • Because objectives may conflict, MOO generally seeks a set of Pareto-optimal designs rather than a unique minimizer.A design is Pareto-optimal when no other design dominates it; the images of all such designs form the Pareto front.Source: §2 · p. 2
  • Offline MOO relies on a fixed dataset of previously evaluated designs and does not permit further evaluations of the objective function..Source: §2 · p. 3

3 BACKGROUND: TRAINING-FREE GUIDANCE FOR FLOW MATCHING

Flow matching learns a time-dependent velocity field that transforms a simple initial distribution into a complex terminal distribution. Training-free guidance modifies a pre-trained sampler at inference time to favor task-specific outcomes while limiting departures from its learned dynamics.

3 BACKGROUND: TRAINING-FREE GUIDANCE FOR FLOW MATCHING

  • Flow matching learns a time-dependent velocity field that transforms a simple initial distribution into a complex terminal distribution.Sampling draws an initial point and integrates the learned field's ODE.Source: §3 · p. 3
  • Training-free guidance incorporates a task-specific loss into a pre-trained generative model's sampling path at inference time.Its guidance velocity steers the trajectory toward a favorable terminal sample, while a penalty limits deviations from the pre-trained dynamics.Source: §3 · p. 3
  • Recent methods approximate full-horizon guidance optimization with single-step subproblems, sometimes mapping terminal-space updates back to an intermediate time.This can avoid differentiation through the terminal estimator and learned model.Source: §3 · p. 3

4 OFFLINE MOO AS SAMPLING PROBLEM

The sampling view treats MOO as finding a design distribution whose objective-space image concentrates near the Pareto front while covering it uniformly. The paper motivates W2 as a quality measure because it upper-bounds continuous GD and IGD.

4 OFFLINE MOO AS SAMPLING PROBLEM

  • The sampling formulation regards offline designs as independent, identically distributed samples from an unknown data distribution..Source: §4 · p. 3
  • The sampling formulation treats convergence as concentration near the Pareto front and coverage as probability-mass allocation along it.The target is a design-space distribution whose objective-space image approximates the uniform distribution on the Pareto front.Source: §4 · pp. 3–4

4.1 A QUALITY MEASURE: THE WASSERSTEIN DISTANCE

  • W2 between objective-space distributions upper-bounds the continuous versions of GD and IGD.The paper states the bounds as GD(µ∥ν) ≤ W2(µ, ν) and IGD(µ∥ν) ≤ W2(µ, ν).Source: §4.1 · p. 4, §A.4 · p. 21

5 OUR METHOD: OFFLINE MOO BY MASS TRANSPORT

ParetoTransport casts offline MOO as mass transport, iteratively refining an objective-space distribution toward the Pareto front through probability-mass matching and directional improvement.

5 OUR METHOD: OFFLINE MOO BY MASS TRANSPORT

  • ParetoTransport approaches offline MOO by transporting the empirical offline-front distribution toward the uniform distribution on the Pareto front.The method formulates MOO as a sampling problem and iteratively updates intermediate proxy distributions.Source: §5 · p. 4, §5.1 · p. 4

5.1 APPROACH BY MASS TRANSPORT

  • Each iteration matches the induced objective-space distribution to a proxy, then moves it toward the Pareto front using a fractional projection displacement.The CHIM normal serves as a surrogate for the unknown projection direction.Source: §5.1 · pp. 4–5, Figure 2 · p. 5
  • Under exact matching, Assumptions (1)–(3), and W2(¯µ, µ∗) < ϵ for some small ϵ, where ¯µ is the projected offline-front distribution, Theorem 1 guarantees lim supk→∞ W2(µk, µ∗) < ϵ; Eq. 5’s first term decreases exponentially with k.Source: §5.1 · p. 5

5.2 OUR PARETOTRANSPORT ALGORITHM

  • ParetoTransport lifts guidance from individual trajectories to the induced probability path, combining Wasserstein control with directional transport during sampling.The directional term uses the CHIM normal as a global surrogate for the unknown projection direction.Source: §5.2 · pp. 5–6
  • The sampling procedure discretizes the path, delays guidance until an onset time, and jointly optimizes matching and improvement in design space to avoid infeasible objective targets.Particles approximate the path; the proxy update reallocates mass so sparsely represented regions receive more and densely represented regions less.Source: §5.2 · p. 6

6 RELATED WORK

Related work spans surrogate-based forward optimization and generative inverse approaches, including conditional modeling and training-free sampling guidance.

6 RELATED WORK

  • Surrogate-based forward methods map candidate designs to objective values and optimize the learned surrogates, commonly with evolutionary search (Xue et al., 2024; Kim et al., 2025b; Knowles, 2006; Jin, 2011; Wang et al., 2018; Yang et al., 2019).NSGA-II is one example of an evolutionary algorithm used for multi-objective search (Deb et al., 2002a).Source: §6 · p. 7
  • Generative inverse methods use desired objective information to generate designs, either through conditional training or by guiding an unconditional model during sampling (Kumar & Levine, 2020; Hotegni & Peitz, 2026; Shrestha et al., 2026; Yuan et al., 2025; Annadani et al., 2025).Source: §6 · p. 7
  • Training-free guidance separates generative modeling from task-specific optimization, allowing a pre-trained model to be reused across objectives without retraining.Source: §6 · p. 7

7 EXPERIMENTS

The experiments compare ParetoTransport with offline MOO baselines on synthetic and engineering benchmarks using HV, GD, IGD, and W2. They report strong average ranks across metrics and benchmark families, alongside ablations of the transport terms and directional weight.

7 EXPERIMENTS

  • The evaluation covers 12 synthetic ZDT and DTLZ tasks and 15 real-world engineering tasks, with 60k designs per task; models are trained as MLPs and sampling uses 100 Euler steps.The generated population and proxy support each have size 256; guidance is applied only for t > 0.8.Source: §7 · p. 7, §7.1 · p. 8

7.2 RESULTS

  • Across representative tasks from three benchmark families, ParetoTransport remains competitive in HV, with only limited differences among methods.HV can be strongly influenced by individual solutions, whereas GD and IGD measure convergence and coverage, and W2 also accounts for probability-mass allocation.Source: §7 · p. 7, §7.2 · p. 8, Table 1 · p. 8
  • ParetoTransport achieves the best average rank on all four metrics for ZDT and DTLZ, and the best average W2 and GD ranks across all three benchmark families.On RE, it ranks best in GD and W2, and third in HV and IGD.Source: §7.2 · pp. 8–9, Table 2 · p. 8

7.3 ABLATIONS

  • Combining directional improvement Ln with Wasserstein control gives the largest IGD, W2, and HV improvements on both benchmark families, while Ln alone reduces GD most.The combined terms trade a small amount of GD relative to Ln alone, indicating complementary effects on convergence, coverage, and mass allocation.Source: §7.3 · p. 9, Table 3 · p. 9
  • Increasing γ initially improves GD on DTLZ, but larger γ makes GD rise again.The study rescales the Ln gradient to match the W2 gradient norm.Source: §7.3 · p. 9, Table 4 · p. 9

8 CONCLUSION

The paper frames offline MOO as mass transport toward the Pareto front and introduces training-free guidance that refines the generated objective-space distribution. It also identifies many-objective optimization as future work and discusses generative models’ fit to low-dimensional feasible-design manifolds.

8 CONCLUSION

  • ParetoTransport uses training-free guidance for pre-trained flow-matching models to refine the geometry and mass allocation of generated objective-space distributions.The conclusion presents optimizing the induced distribution itself, rather than only generating candidate solutions independently, as a natural use of generative models.Source: §8 · p. 9
  • Generative methods are promising when feasible designs lie on low-dimensional manifolds, which ambient-space forward optimization may fail to preserve.The paper notes that such methods may move candidates outside the data-supported region.Source: §8 · p. 10
  • Many-objective settings with m ≥4 remain future work because a single global CHIM normal may not capture higher-dimensional Pareto-front geometry.The paper focuses on tasks with m ≤3.Source: §8 · p. 10

A.1 TRAINING-FREE GUIDANCE OF FLOW-MATCHING MODELS

This section introduces flow matching and explains how training-free guidance approximates full-horizon optimization during sampling. It also reviews terminal-sample estimation and reverse parameterization used in single-step guidance.

A.1 TRAINING-FREE GUIDANCE OF FLOW-MATCHING MODELS

  • Flow matching learns a time-dependent velocity field that transforms an initial distribution into a terminal distribution, then samples by integrating the learned field from an initial draw.A conditional probability path makes the training objective tractable, with its gradient matching the marginal regression objective’s gradient.Source: §A.1 · p. 15
  • Full-horizon guidance optimizes the sampling trajectory, while Euler discretization determines that trajectory from the guidance variables once initial samples are fixed.Training-free guidance approximates this problem with sequential single-step subproblems.Source: §A.1 · pp. 15–16
  • Prior methods ignore future guidance at each step and use a posterior-mean terminal estimate; reverse parameterization can instead optimize directly in terminal space.The terminal sample is mapped back to the sampling path, and the velocity regularizer can also be expressed in terminal-sample terms.Source: §A.1 · p. 16
  • For affine conditional paths, flow matching admits a posterior-mean estimator derivation that does not require explicit use of Tweedie’s formula.The cited lemma states posterior-mean identities under an affine path and a nonzero scheduler quantity; a linear scheduler further simplifies them.Source: §A.1 · p. 16
  • Recovering an intermediate sample from a terminal sample becomes a fixed-point problem, for which contraction guarantees convergence of iterates to a unique fixed point.Prior methods approximate the solution with a single iteration or define an alternative reverse parameterization.Source: §A.1 · pp. 16–17

A.2.1 PARTICLE-LEVEL APPROXIMATION OF DISTRIBUTIONAL GUIDANCE.

This appendix details ParetoTransport’s particle-level approximation of distributional guidance, including its proxy distributions, losses, implementation, and evaluation setup. It also presents theoretical bounds and conditions for proxy updates, alongside descriptions of component and transport-strength ablations without numerical outcomes.

A.2.1 PARTICLE-LEVEL APPROXIMATION OF DISTRIBUTIONAL GUIDANCE.

  • The particle approximation represents the probability path with empirical particle trajectories and applies single-step terminal-space guidance to each, while jointly optimizing terminal particles because the loss acts on their empirical distribution.Optimized terminal particles are mapped back to the sampling path using the local reverse parameterization.Source: §A.2.1 · pp. 17–18

A.2.2 INTERMEDIATE PROXY DISTRIBUTIONS

  • Objective vectors are min–max normalized using fixed offline-dataset statistics; proxy updates combine current support with new candidates and allocate capped crowding-based mass, using κ = 5.Crowding-based allocation gives more mass to sparsely represented regions and less to dense ones.Source: §A.2.2 · pp. 18–19

A.2.3 LOSS FUNCTIONAL

  • The loss combines displacement along a fixed CHIM-derived direction with Wasserstein matching to the proxy, which controls probability-mass allocation.The Wasserstein term uses entropically regularized optimal transport, with a log-domain Sinkhorn solver and ε = 10^-3.Source: §A.2.3 · pp. 19–20

A.3.1 TRAINING DETAILS

  • The implementation uses separate three-layer MLP surrogates per objective and a four-layer flow-matching MLP; sampling generates 256 solutions and guides only steps t > 0.8.The flow integrates over T = 100 Euler steps; guidance uses 10 Adam steps per guided step with learning rate 0.01 and γ = 0.5.Source: §A.3.1 · p. 20

A.3.2 MOO METRICS

  • HV measures dominated objective-space volume, while GD and IGD aggregate distances from solutions to the Pareto-front approximation and from that approximation to solutions, respectively.The passages characterize GD and IGD as complementary measures of convergence and coverage because HV can be strongly influenced by individual solutions.Source: §A.3.2 · pp. 20–21

A.3.3 DATASET DETAILS

A.4 THEORETICAL RESULTS

  • The theoretical analysis focuses on tasks with m ≤ 3 and leaves many-objective settings (m ≥ 4) for future work because of high-dimensional Pareto-front complexity.The convergence result assumes a nonempty closed Pareto front with unique nearest-point projections in a neighborhood, finite second moments and invariance under transport.Source: §A.4 · pp. 21–22
  • Under exact matching and Assumptions 1–3, proxy updates iteratively interpolate toward the nearest-point projection of the initial distribution, with weight ξ_k = 1−(1−η)^k.The corollary also bounds GD and IGD by W2.Source: §A.4 · p. 22
  • W2(µk, µ∗) is bounded by (1 − η)^k W2(µ0, ¯µ) + W2(¯µ, µ∗), with contraction toward ¯µ along the geodesic.The intermediate distance contracts as W2(µk, ¯µ) = (1 − η)^k W2(µ0, ¯µ).Source: §A.4 · p. 23

A.6.1 WASSERSTEIN AND DIRECTIONAL IMPROVEMENT COMPONENTS

  • Tables ablate the Wasserstein matching term W2 and directional improvement term Ln on ZDT and DTLZ.The captions specify five runs and evaluation on 256 designs for each experiment.Source: Table 19 · p. 26, Table 20 · p. 26

A.6.2 TRANSPORT STRENGTH γ

  • The transport-strength study rescales the Ln gradient to match the W2 gradient norm, so γ directly controls transport’s relative contribution.It varies γ over {0, 0.1, 0.2, 0.5, 1, 2, 5, 10} and reports GD, IGD, W2, and HV on ZDT and DTLZ tasks.Source: §A.6.2 · p. 27

Related papers

Loading 2609.07706v1…