PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditional Flow Matching for ML-Based Inverse Design Problems

Juliana Felder, Milad Habibi, Soheyl Massoudi, Mark Fuge

arXiv:2609.00863v1cs.LG

TL;DR

Engineering inverse design is constrained by expensive, initialization-sensitive PDE optimization. This paper adds conditional flow matching to EngiOpt and compares it with diffusion and cGAN warm starts on two EngiBench tasks, finding that CFM has the best measured overall metrics among the evaluated implementations while sampling faster than diffusion.

  • Problem

    Engineering inverse design requires expensive iterative physics-constrained optimization, motivating candidate designs that can serve as refinable warm starts.

  • Method

    The paper adds conditional flow matching to EngiOpt and compares it with conditional diffusion and cGAN models using a shared downstream optimization protocol.

  • Results

    CFM achieves the lowest measured COG, FOG, MMD, and volume-fraction deviation on both evaluated EngiBench tasks, with approximately 32×–66× diffusion throughput depending on solver budget and task.

  • Takeaways & Limitations

    Among the evaluated implementations, CFM provides warm starts with lower measured cumulative optimization gaps and fewer network evaluations than diffusion.

  • Takeaways & Limitations

    The study is restricted to two 2D EngiBench tasks and one shared U-Net family, so it does not establish a universal ranking of generative inverse-design models.

Abstract

from arXiv · show

Engineering inverse design is often limited by the high computational cost of iterative solvers for optimization problems constrained by partial differential equations (PDEs) and by their sensitivity to initialization. Deep generative models can produce candidate designs without rerunning the simulator at inference time. Generative adversarial networks (GANs) sample in one forward pass, whereas diffusion models require iterative reverse-time integration. In this work, we add conditional flow matching (CFM) to EngiOpt and compare it with a conditional diffusion model and a conditional generative adversarial network (cGAN) on structural (beams2d) and thermal (heatconduction2d) benchmarks from EngiBench using the same downstream optimization protocol. We use cumulative optimality gap (COG) and final optimality gap (FOG) as the primary metrics for evaluating the generated designs as warm starts for gradient-based refinement. On the evaluated EngiOpt implementations and two EngiBench tasks, CFM achieves the lowest measured COG, FOG, maximum mean discrepancy (MMD), and volume-fraction deviation on both tasks. CFM has mean volume-fraction deviations of 0.4% and 1.0% on beams2d and heatconduction2d, respectively, compared with 3.8% and 11.2% for diffusion. At Euler s = 16, CFM achieves 53.2 samples/s on beams2d, about 66 times the measured throughput of the evaluated diffusion baseline using 1000 network evaluations under the same timing protocol, with COG 1.182 +/- 3.126, compared with 1.173 +/- 3.100 for Euler s = 32. Across the two tasks, CFM produces warm starts with lower measured COG than both baselines and uses fewer network evaluations than diffusion.

1 Introduction

Engineering inverse design uses expensive, initialization-sensitive physics-constrained optimization, motivating generative warm starts. This work adds conditional flow matching to EngiOpt and evaluates it against diffusion and cGAN baselines.

  • Generative models produce candidate designs without rerunning the simulator at inference time, then leave physical refinement to downstream optimization.
  • CFM replaces stochastic denoising with deterministic velocity regression while sharing diffusion’s conditional U-Net backbone.
  • The study asks whether CFM can provide useful warm starts with fewer network evaluations than diffusion and lower seed-to-seed variability than cGAN.
  • The evaluation uses matched data splits, checkpoint selection, downstream optimization, and COG and FOG to measure warm-start utility through optimization trajectories.

2 Related Work

Related work spans physics-based topology optimization, learned warm starts, conditional generative models, diffusion, and flow matching. The study positions CFM as a matched comparison with existing EngiOpt diffusion and cGAN implementations.

  • Physics-based topology optimization repeatedly solves governing equations while updating material distributions, making many related design instances expensive.
  • Learned warm starts reduce reliance on costly optimization by generating candidate material distributions that conventional optimizers physically refine.
  • cGANs generate designs in one forward pass, but adversarial training can be unstable and susceptible to mode collapse.
  • Diffusion models avoid adversarial training but reverse a gradual noising process through many sampling steps.
  • Flow matching regresses a vector field along a prescribed probability path, with conditional variants using sampled source and target endpoints.
  • This study compares linear-path I-CFM with existing diffusion and cGAN implementations using matched data splits, checkpoint selection, optimizers, and metrics.

3 Method

The method learns conditional density-field generators for two EngiBench topology-optimization tasks and uses their outputs as warm starts for a fixed downstream optimizer. Evaluation combines optimization-trajectory metrics with distributional and feasibility measures.

  • Problem formulation: The model learns p(x | c), mapping problem conditions to candidate 2D material-density fields for downstream physics-based optimization.
  • Benchmarks: Experiments cover structural compliance minimization on Beams2D and thermal compliance minimization on HeatConduction2D.
  • Preprocessing: CFM normalizes designs using training-split min-max statistics and clips generated outputs to [10^-3, 1] before optimizer evaluation.
  • CFM architecture: The conditional U-Net receives noisy designs and conditions and outputs a velocity field, using four encoder-decoder stages with cross-attention conditioning.
  • CFM training: I-CFM independently pairs Gaussian noise x0 with target designs x1, interpolates linearly in time, and trains velocity predictions toward x1 − x0.
  • Inference: CFM inference integrates the learned velocity field with Euler, Midpoint, or RK4 solvers at NFE budgets of 16, 32, and 48.
  • Baseline comparison: CFM and diffusion share the same conditional U-Net and parameter count, while diffusion uses DDPM sampling and cGAN retains EngiOpt’s adversarial architecture.
  • Warm-start pipeline: Generated designs are passed to the benchmark optimizer, whose trajectories are evaluated against paired reference designs using shared splits, seeds, and budgets.

4 Results

Across both benchmarks, CFM provides the strongest measured warm-start optimization utility and feasibility, while substantially reducing diffusion’s sampling cost. Its solver-step results show similar beams2d COG at NFE 16 and 32.

  • CFM achieves the lowest measured COG and FOG on both beams2d and heatconduction2d.
  • CFM’s beams2d COG is 1.173 ± 3.100, versus 1.603 ± 1.794 for Diffusion.Its mean FOG is −1.647 ± 0.411, compared with −1.637 ± 0.505 for Diffusion.
  • The displayed CFM samples retain continuous members and sharp solid–void boundaries, whereas cGAN examples show discontinuities or fragmented material regions.The qualitative comparison uses matched conditions and seeds across methods.
  • CFM obtains the lowest MMD and volume-fraction violation on both tasks, with violations of 3.60 × 10−3 for beams2d and 9.96 × 10−3 for heatconduction2d.Diffusion has higher volume-fraction deviations of 3.82 × 10−2 and 1.12 × 10−1, respectively.
  • CFM at Euler s = 32 is approximately 33.0× faster than Diffusion on beams2d and 32.2× faster on heatconduction2d while achieving the lowest measured COG.The timed region covers only model sampling on a single NVIDIA RTX 4090 GPU.
  • Euler s = 16 yields beams2d COG of 1.182 ± 3.126, compared with 1.173 ± 3.100 at Euler s = 32.Across the evaluated NFE budgets, Euler and Midpoint obtain beams2d COG values ranging from 1.134 to 1.182.

5 Discussion

Across the evaluated implementations, CFM provides stronger warm-start utility and constraint adherence than the baselines, while solver sensitivity and study scope limit broader conclusions.

  • Optimization utility and constraint adherence: CFM achieves the lowest measured COG on both tasks and a slightly lower beams2d FOG than Diffusion.The beams2d FOG values are −1.647 for CFM and −1.637 for Diffusion, a small difference relative to seed variation.
  • Optimization utility and constraint adherence: 0.4% and 1.0% are CFM’s mean volume-fraction deviations on beams2d and heatconduction2d, versus 3.8% and 11.2% for Diffusion.The remaining Diffusion deviations were not isolated to a specific cause.
  • Diversity and architecture: Higher measured DPP for Diffusion does not correspond to better warm-start optimization trajectories in these experiments.CFM and Diffusion use the same conditional U-Net backbone and channel schedule, while differing in training objectives and sampling procedures.
  • cGAN baseline: 22.493 ± 819.282 is the cGAN beams2d COG, indicating substantial seed sensitivity in the evaluated adversarial baseline.On heatconduction2d, cGAN ranks differently by COG and DPP, and the experiments do not isolate the source of these differences.
  • Flow geometry and solver sensitivity: At Euler s = 16 and s = 32, beams2d COG is 1.182 ± 3.126 and 1.173 ± 3.100, respectively, while s = 16 has much higher measured throughput.Euler s = 16 achieves about 65.8× Diffusion throughput on beams2d and 64.4× on heatconduction2d; broader solver conclusions need a denser NFE sweep.
  • Limitations and future work: Two 2D tasks and one shared U-Net family constrain the study’s scope, so the results compare evaluated EngiOpt implementations rather than establish a universal model ranking.Future work is needed in 3D, multi-physics, and higher-dimensional condition spaces, with targeted ablations for Diffusion’s volume-fraction deviations.

6 Conclusion

This work adds CFM as a warm-start prior for two EngiBench inverse-design tasks and compares it with Diffusion and cGAN under the EngiOpt protocol. Among the tested implementations, CFM has the lowest measured optimization, distributional, and feasibility metrics, with substantially higher throughput than Diffusion.

  • CFM is evaluated as a warm-start prior for beams2d and heatconduction2d against conditional Diffusion and cGAN.
  • CFM achieves the lowest measured COG, FOG, MMD, and volume-fraction deviation on both tasks.
  • Approximately 32×–66× higher measured sampling throughput than Diffusion is reported for CFM, depending on solver budget and task.Reducing Euler from s = 32 to s = 16 approximately doubles throughput while changing beams2d COG from 1.173 ± 3.100 to 1.182 ± 3.126.
  • Diffusion has the highest DPP on both tasks, whereas CFM has the lowest COG, so metric rankings are not identical.

A Training Configuration

The models are trained for up to 500 epochs on a single RTX 4090 with batch size 32, using AdamW for CFM and Diffusion and Adam for cGAN.

  • 500 epochs is the maximum training duration for all models on one NVIDIA RTX 4090 GPU with batch size 32.
  • CFM and Diffusion use AdamW with learning rate 4 × 10^-4 and (β1, β2) = (0.9, 0.999).
  • cGAN uses Adam with generator and discriminator learning rates of 1 × 10^-4 and 4 × 10^-4, respectively, and (β1, β2) = (0.5, 0.999).

B ODE Solver and Step-Count Ablation

The ablation compares Euler, Midpoint, and RK4 at matched NFE budgets, showing that higher-order solvers do not consistently improve beams2d warm-start metrics and that Euler’s COG-speed trade-off is non-monotonic.

  • Experimental setup: 16, 32, and 48 are the matched NFE budgets used to compare Euler, Midpoint, and RK4.Step counts are adjusted because Midpoint and RK4 require two and four velocity evaluations per integration step.
  • Optimization metrics: 2.274 ± 1.174 is RK4’s beams2d COG at NFE = 16, higher than matched Euler and Midpoint settings.The same RK4 configuration also has the highest DPP and MMD among matched-NFE settings, so greater measured diversity does not coincide with lower COG.
  • Solver comparison: At low NFE, RK4 uses fewer, larger integration steps and produces higher beams2d COG than Euler.
  • Optimization metrics: 1.537 versus 1.173 is the beams2d COG for RK4 s = 8 and Euler s = 32 at NFE = 32, respectively.On heatconduction2d, COG ranges from 8.88 × 10^-5 to 1.70 × 10^-4 across evaluated configurations.
  • Euler trade-off: 1.182 ± 3.126 and 1.173 ± 3.100 are Euler’s beams2d COG values at s = 16 and s = 32, while s = 16 reaches 53.156 samples/s.The small COG change despite increased throughput illustrates a non-monotonic NFE–COG relationship.

Acronyms

This section defines the paper’s abbreviations for flow matching, optimization metrics, generative models, statistical measures, numerical methods, and PDEs.

  • CFM means conditional flow matching; I-CFM means independent conditional flow matching.
  • COG and FOG mean cumulative optimality gap and final optimality gap, while NFE means number of function evaluations.
  • cGAN, GAN, MMD, DPP, ODE, RK4, DDPM, and PDE denote conditional generative adversarial network, generative adversarial network, maximum mean discrepancy, determinantal point process, ordinary differential equation, classical fourth-order Runge–Kutta method, denoising diffusion probabilistic model, and partial differential equation.
Loading 2609.00863v1…