PaperlayerView exact revision on arXiv

Source-linked AI summary

Learn Feasibility Once, Optimize All Objectives: Derivative-Free Diffusion Models for Chance-Constrained Programming

Ziwen Liu, Yan Liu, Congying Han, Tiande Guo, Yao Yan, Weichen Zhao

arXiv:2610.03071v1cs.LGmath.OC

TL;DR

Chance-constrained optimization must handle difficult objectives, while objectives can change even when chance constraints remain fixed. D3Opt learns a reusable constraint-only diffusion prior and applies an annealed particle correction using objective evaluations. The paper establishes conditional feasibility preservation and reports optimization and objective-transfer results across several settings.

  • Problem

    Chance-constrained optimization can be computationally challenging over non-convex feasible regions, and adapting to changing objectives under fixed chance constraints remains a challenge.

    Source: §1 · pp. 1–2

  • Method

    D3Opt trains and freezes a risk-conditioned diffusion prior using constraint-only data, then uses annealed Feynman–Kac particle correction with objective evaluations for post-specified objectives.

    Source: §1 · p. 2

  • Results

    The correction preserves feasibility when the frozen prior is almost surely feasible, and experiments demonstrate solution quality and objective generalization across linear Gaussian CCPs, objective-transfer tasks, and economic dispatch.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A learned chance-feasible structure can be reused across objectives under fixed chance constraints without objective-specific retraining.

    Source: §1 · p. 2, §6 · p. 9

    FK reweighting cannot reach regions outside the learned prior’s support, and relaxed empirical data acceptance does not guarantee exact feasibility.

    Source: §6 · p. 9

Abstract

from arXiv · show

Chance-constrained programs (CCPs) optimize decisions under uncertainty by limiting the probability of constraint violation. Despite advances in traditional and learning-based approaches, optimizing non-convex or non-smooth objectives and adapting to different objectives under fixed chance constraints remain challenging. In this paper, we propose a \textbf{D}erivative-free \textbf{D}iffusion-based framework that \textbf{D}isentangles constraint modeling from objective optimization, termed \textbf{D$^3$Opt}. We learn the chance-feasible structure once, independently of any particular objective, by training a risk-conditioned diffusion model solely on constraint-filtered decisions and freezing it as a reusable prior for post-specified objectives. At inference time, we propose an annealed, particle-based Feynman--Kac correction along the frozen reverse diffusion process to optimize post-specified objectives using only function evaluations. This enables derivative-free optimization of non-convex and non-smooth objectives without objective-specific retraining. We prove that the correction preserves feasibility when this property holds for the frozen prior, and derive an optimization-error bound separating learned-prior coverage, finite-particle approximation, and finite-temperature effects. Experiments on linear Gaussian CCPs, objective-transfer tasks, and chance-constrained economic dispatch demonstrate effective optimization across smooth and non-smooth objectives, including non-convex cases, and objective generalization under fixed chance constraints without retraining.

1 INTRODUCTION

D3Opt separates learning the chance-feasible structure from choosing among feasible decisions, so one constraint-only diffusion prior can be reused across objectives. An annealed particle correction then optimizes post-specified objectives using function evaluations, with feasibility and optimization error analyzed theoretically.

1 INTRODUCTION

  • Constraint-only training decisions let the frozen risk-conditioned prior be reused when objectives change, without objective-specific data generation.The paper contrasts this with training on solutions of objective-dependent restricted problems.Source: §1 · pp. 1–2
  • The framework targets fixed chance constraints with objectives that may be non-convex, non-smooth, or accessible only through function evaluations.Traditional chance-constrained optimization can be computationally challenging over non-convex feasible regions.Source: §1 · pp. 1–2
  • D3Opt combines a reusable constraint-only diffusion prior with an annealed particle correction for optimizing post-specified objectives using function evaluations.The prior is frozen and reused across objectives; the correction does not require objective gradients or Hessians.Source: §1 · p. 2
  • Theoretical analysis establishes feasibility preservation when the frozen prior is almost surely feasible and separates error into prior coverage, finite-particle, and finite-temperature effects.The correction reweights and resamples particles according to objective values evaluated at denoised decision predictions.Source: §1 · p. 2
  • Experiments cover linear Gaussian chance constraints, objective-transfer tasks, and chance-constrained economic dispatch.The reported scope includes objective generalization under fixed chance constraints.Source: §1 · p. 2

2 RELATED WORK

Prior chance-constrained optimization methods address specified objectives, while diffusion approaches learn decision distributions and guide constrained search. D3Opt instead learns from constraint-only data and uses objective values to adapt a frozen prior to new objectives.

2 RELATED WORK

  • Classical chance-constrained methods include deterministic reformulations, conservative convex approximations, sample average approximation, and mixed-integer, difference-of-convex, or zero–one methods.Differential evolution and mesh adaptive direct search are derivative-free options for non-smooth objectives.Source: §2 · p. 2
  • Diffusion approaches learn decision distributions, while DiffOPT uses gradient-guided sampling for differentiable objectives and iterative importance sampling for non-differentiable ones.GGDOpt trains on objective-dependent restricted-problem solutions and applies first- or second-order guidance at inference.Source: §2 · p. 2
  • D3Opt learns a risk-conditioned prior from constraint-only data and applies annealed Feynman–Kac particle correction using objective values, without retraining for new objectives.Source: §2 · p. 2

3 PRELIMINARIES

The preliminaries define chance-constrained optimization and its low-temperature sampling perspective, then introduce the learned diffusion prior and Feynman–Kac changes of measure used for inference-time steering.

3.1 PROBLEM DEFINITION

  • The chance-constrained problem minimizes an objective subject to a prescribed admissible violation probability, and its reference distribution is supported on the chance-feasible set.With positive reference mass near feasible global minimizers, the Boltzmann distribution concentrates on those minimizers as inverse temperature grows.Source: §3.1 · p. 3

3.2 DIFFUSION MODEL

  • The learned reverse diffusion uses a score contribution in its drift, with a path law and terminal distribution conditioned on risk level ρ.The reverse process is written in generative time.Source: §3.2 · p. 3
  • An annealed sequence of increasingly sharp target distributions motivates gradually redistributing probability from the pretrained prior toward objective-favored regions.The low-temperature limit corresponds to approaching the global optimum.Source: §3.2 · p. 3

3.3 FEYNMAN–KAC CORRECTION

  • Feynman–Kac correction changes the baseline path measure through nonnegative trajectory weights, which may be implemented by drift modification or particle reweighting.Sequential Monte Carlo alternates particle propagation, reweighting, and resampling while retaining the baseline diffusion kernels as transport.Source: §3.3 · pp. 3–4

4 D3OPT

D3Opt separates learning chance-feasible decisions from choosing among them: it freezes a constraint-only diffusion prior and uses inference-time Feynman–Kac correction for post-specified objectives. The section also establishes when this correction preserves feasibility and analyzes sources of optimization error.

4 D3OPT

  • D3Opt combines a reusable constraint prior with inference-time Feynman–Kac correction to disentangle constraint modeling from objective optimization.The frozen prior is reused for post-specified objectives, without objective-specific training.Source: §4 · p. 4

4.1 OBJECTIVE-FREE FEASIBLE DIFFUSION MODEL

  • Training uses constraint-filtered decisions without evaluating the objective, and the frozen risk-conditioned diffusion model is reused when objectives change under fixed constraints.The objective is absent from data generation, diffusion training, checkpoint selection, and model parameters.Source: §4.1 · pp. 4–5

4.2 DERIVATIVE-FREE FEYNMAN–KAC CORRECTION

  • An annealed Feynman–Kac correction reweights particles using objective evaluations at denoised predictions, progressively concentrating search toward low-objective regions without gradients or Hessians.The correction operates along the frozen reverse diffusion process.Source: §4.2 · p. 5

4.3 FEASIBILITY PRESERVATION AND ERROR ANALYSIS

  • If the frozen diffusion is almost surely chance-feasible, Feynman–Kac correction preserves that feasibility by redistributing mass only within the prior's support.The feasibility premise is an ideal support assumption and need not hold exactly with finite-scenario construction and finite-data diffusion approximation.Source: §4.3 · p. 6
  • The practical optimization-error analysis separates learned-prior coverage, finite-temperature concentration, and finite-particle approximation.When the prior's coverage gap is zero, the infinite-particle limit followed by the low-temperature limit recovers the global chance-constrained optimal value in expectation.Source: §4.3 · pp. 6–7

5 EXPERIMENTS

Experiments test D3Opt on linear Gaussian chance-constrained problems, reuse its frozen checkpoint for ASQ and MW4 objectives, and evaluate valve-point economic dispatch.

5.1 LINEAR GAUSSIAN CHANCE-CONSTRAINED OPTIMIZATION

  • Adding FKC lowered the mean objective from −0.1172 to −0.6519, only 0.0067 above the exact SOC optimum of −0.6586.D3Opt also had the smallest standard deviation, 0.0015, among evaluated diffusion methods; gains used neither objective gradients nor objective-specific retraining.Source: §5.1 · p. 7, Table 1 · p. 8

5.2 OBJECTIVE GENERALIZATION UNDER A LINEAR GAUSSIAN CHANCE CONSTRAINT

  • FKC lowered mean objectives from 0.8549 to 0.1035 on ASQ and from 0.7856 to 0.1834 on MW4 using the same frozen model.The reported gaps to exact SOC references were 5.704% on ASQ and 4.078% on MW4, versus DiffOPT's 34.098% and 26.725%, respectively.Source: Table 2 · p. 8, §5.2 · p. 8

5.3 CHANCE-CONSTRAINED ECONOMIC DISPATCH WITH VALVE-POINT EFFECTS

  • Mean cost fell from 107183.587 to 104823.577 (2.20%); mean analytic satisfaction probability was 0.9123, above the 0.90 requirement.D3Opt's mean cost was 0.81% below DiffOPT and within 0.407% of the projected DE numerical reference using objective evaluations alone.Source: Table 3 · p. 9, §5.3 · p. 9

6 CONCLUSION AND LIMITATIONS

D3Opt separates constraint-distribution learning from objective optimization, reusing a frozen, constraint-only diffusion prior with inference-time Feynman–Kac correction. The paper also characterizes the correction, shows when it preserves prior feasibility, and describes objective transfer across smooth and non-smooth objectives.

6 CONCLUSION AND LIMITATIONS

  • D3Opt learns a risk-conditioned diffusion prior from constraint-only data, reuses it without retraining, and reports objective transfer plus further results on dispatch, risk adaptation, and waveform design.The reported error contributions are prior coverage, finite-particle approximation, and finite temperature.Source: §6 · p. 9

A.1 FROM THE FEYNMAN–KAC TRANSFORM TO THE FEYNMAN–KAC PDE

  • The Feynman–Kac transform changes measure on trajectories through terminal and running potentials; a gradient-based drift correction is one realization, not its definition.The transform can also be implemented through weighted particle systems without representing it solely as an additional gradient-based drift.Source: §A.1 · pp. 13–14

A.2 DISCRETE-TIME FEYNMAN–KAC MEASURES

  • In discrete time, multiplicative potentials modify a baseline Markov process, while interacting-particle methods approximate the resulting FK marginals.Sequential Monte Carlo propagates particles through Markov transitions and redistributes their mass using FK potentials.Source: §A.2 · pp. 15–16

B.1 FEASIBILITY PRESERVATION UNDER FEYNMAN–KAC TILTING

  • The objective-only Feynman–Kac correction preserves any almost-sure chance feasibility already encoded by the frozen diffusion prior.This guarantee is conditional on the frozen prior being chance-feasible almost surely.Source: §B.1 · p. 17

B.2 PROOF OF PROPOSITION 2

The proof bounds optimization error by separating finite-particle, finite-temperature, and learned-prior coverage contributions. It also establishes vanishing finite-temperature error as inverse temperature increases, under the stated assumptions.

B.2 PROOF OF PROPOSITION 2

  • The proof separates the objective gap into finite-particle, finite-temperature, and prior-coverage terms.The coverage term reflects the best objective available within the frozen prior's support.Source: §B.2 · pp. 17, 19
  • The finite inverse-temperature contribution vanishes as βK tends to infinity.The proof establishes this limit for each fixed positive tolerance.Source: §B.2 · p. 18
  • For a fixed finite inverse-temperature schedule, taking the particle count to infinity removes the finite-particle term before the inverse-temperature limit is taken.The proof relies on Assumption 1 for its fixed-horizon particle-approximation bound.Source: §B.2 · pp. 17, 19

C.1 CONSTRAINT-ONLY DATA GENERATION

The experiments use constraint-only data to train frozen diffusion models, then apply objective-value Feynman–Kac correction at inference. The appendix specifies model, problem, and evaluation settings across the tested tasks.

C.1 CONSTRAINT-ONLY DATA GENERATION

  • Each training dataset contains 10,000 retained decisions generated and filtered using constraint information alone, with training risk levels sampled uniformly from [0, 0.5].Linear Gaussian feasibility is checked analytically; CCED uses separate 100,000-scenario labeling and validation sets.Source: §C.1 · p. 19

C.2 DIFFUSION TRAINING

  • Conditional 1D U-Nets serve the linear Gaussian and waveform tasks, while CCED-VPE-10 uses a conditional 2D U-Net; models are frozen before downstream optimization.ASQ and MW4 reuse the linear Gaussian model, and a single CCED model is reused across tested risk levels.Source: §C.2 · p. 19

C.3 FEYNMAN–KAC INFERENCE

  • Inference uses 512 particles and a prespecified inverse-temperature schedule synchronized with reverse-diffusion steps; ESS determines resampling timing.Post-specified objectives enter through derivative-free Feynman–Kac correction using objective-function evaluations.Source: §C.3 · p. 19

C.4 PROBLEM-SPECIFIC SETTINGS

  • The linear Gaussian, ASQ, and MW4 tasks share a chance constraint evaluated by exact SOC reformulation, while CCED-VPE-10 specifies a ten-unit system and valve-point cost.Table 4 identifies the generator output limits and valve-point cost coefficients for CCED-VPE-10.Source: §C.4 · pp. 19–20, Table 4 · p. 20

C.5 EVALUATION

  • Stochastic methods are evaluated over 100 runs, with mean, sample standard deviation, and applicable median computed from returned decisions; CCED satisfaction is checked analytically.Generator bounds and the chance constraint are checked with a numerical tolerance of 10^-10.Source: §C.5 · pp. 20–21

D ADDITIONAL EXPERIMENTAL RESULTS

Table 5 reports CCED-VPE-10 risk-level adaptation at three risk levels, summarizing costs over repeated runs.

D ADDITIONAL EXPERIMENTAL RESULTS

  • Table 5 reports mean, sample standard deviation, and best cost at ρ values 0.05, 0.15, and 0.20, with PWL-MILP based on one solve per risk level.Bold costs mark the best available learning-based results.Source: Table 5 · p. 21

D.1 MULTI-RISK GENERALIZATION ON CCED-VPE-10

The frozen model is reused across four risk levels within its training interval, and D3Opt outperforms DiffOPT while traditional methods retain lower costs on this instance.

D.1 MULTI-RISK GENERALIZATION ON CCED-VPE-10

  • Evaluation compares DE with exact-feasibility projection and DiffOPT; SAA-DE is solved separately at each risk level, unlike the reused frozen checkpoints.Each stochastic method is evaluated over 100 independent runs per risk level.Source: §D.1 · p. 21
  • At every tested risk level, D3Opt achieves lower mean and best costs than DiffOPT, with smaller cost standard deviations, while traditional methods retain lower costs.All 100 returned solutions at each risk level satisfy the analytic chance constraint.Source: §D.1 · p. 21
  • The same frozen constraint-only model supports the four risk levels within the training interval without data regeneration or retraining.Inference changes only the supplied risk condition; the dispatch objective, generator parameters, and Beta wind distribution remain fixed.Source: §D.1 · p. 21

D.2 ABLATION STUDY

The ablations show that more particles improve solution quality and stability, while stronger objective tilting lowers costs; the default balances cost, quality, and stability.

D.2 ABLATION STUDY

  • As M rises from 64 to 2048 with βmax = 10, mean cost falls from 105161.669 to 104707.598 and sample standard deviation from 243.555 to 74.224.Best cost also improves from 104691.425 to 104521.263.Source: §D.2 · p. 22
  • With M = 512, raising βmax from 0 to 50 reduces mean cost from 107183.587 to 104759.376 and best cost from 106146.715 to 104519.511.Standard deviation falls substantially up to βmax = 10 and changes only slightly at 20 and 50; stronger tilting may trade stability for further objective improvement.Source: §D.2 · p. 22
  • The authors select M = 512 and βmax = 10 as the default, balancing computational cost, solution quality, and stability.Larger settings further improve mean or best cost but require more particles or produce slightly higher variability.Source: §D.2 · p. 22

D.3 ROBUST WAVEFORM EXPERIMENTS

In robust waveform design, D3Opt has the lowest mean and best power among evaluated learning-based methods at all four risk levels, with run-averaged satisfaction probabilities matching nominal thresholds at reported precision.

D.3 ROBUST WAVEFORM EXPERIMENTS

  • The experiment minimizes total transmission power under individual SINR chance constraints for 16 transmit antennas and 3 users.Learning-based power statistics use 100 runs; the channel uncertainty and tested risk levels are specified in the experimental setup.Source: §D.3 · p. 22, Table 7 · p. 23
  • D3Opt achieves the lowest mean and best transmission power among evaluated learning-based methods at all four risk levels.Relative to second-order GGDOpt, D3Opt reduces mean power by 0.20%, 1.48%, 2.29%, and 2.81% at ρ = 0.05, 0.10, 0.15, and 0.20, respectively.Source: §D.3 · p. 23, Table 7 · p. 23
  • For ρ = 0.05, 0.10, 0.15, and 0.20, the minimum run-averaged user satisfaction probabilities are 0.95, 0.90, 0.85, and 0.80, matching nominal 1 −ρ thresholds at two-decimal precision.These are averages across runs, not counts of individually feasible returned solutions.Source: §D.3 · p. 23

Related papers

Loading 2610.03071v1…