PaperlayerView exact revision on arXiv

Source-linked AI summary

Dual Diffusion Implicit Bridges for Image-to-Image Translation

Xuan Su, Jiaming Song, Chenlin Meng, Stefano Ermon

arXiv:2203.08382v4cs.CVcs.AIcs.LG

TL;DR

Existing image translation methods require joint source-target training and are difficult to adapt across domain pairs, creating data-separation and privacy concerns. DDIBs instead use independently trained domain diffusion models and two ODE-based translation steps, framing the process as Schrödinger Bridges and evaluating it across diverse tasks.

  • Problem

    Existing translation models require concurrent access to source and target datasets and are trained specifically for domain pairs, limiting privacy protection and reuse across new pairs.

  • Method

    DDIBs train one diffusion model per domain and translate by source-to-latent and latent-to-target probability-flow ODEs.

  • Results

    Experiments demonstrate DDIBs across synthetic, color-transfer, paired-translation, and conditional ImageNet tasks, including cycle-consistency behavior.

  • Takeaways & Limitations

    DDIBs provide a reusable, privacy-preserving image-translation framework whose ODE construction has an optimal-transport interpretation.

  • Takeaways & Limitations

    Color transfer is limited because one model is required for each image, demanding significant compute; higher-order ODE solvers remain future work.

Abstract

from arXiv · show

Common image-to-image translation methods rely on joint training over data from both source and target domains. The training process requires concurrent access to both datasets, which hinders data separation and privacy protection; and existing models cannot be easily adapted for translation of new domain pairs. We present Dual Diffusion Implicit Bridges (DDIBs), an image translation method based on diffusion models, that circumvents training on domain pairs. Image translation with DDIBs relies on two diffusion models trained independently on each domain, and is a two-step process: DDIBs first obtain latent encodings for source images with the source diffusion model, and then decode such encodings using the target model to construct target images. Both steps are defined via ordinary differential equations (ODEs), thus the process is cycle consistent only up to discretization errors of the ODE solvers. Theoretically, we interpret DDIBs as concatenation of source to latent, and latent to target Schrodinger Bridges, a form of entropy-regularized optimal transport, to explain the efficacy of the method. Experimentally, we apply DDIBs on synthetic and high-resolution image datasets, to demonstrate their utility in a wide variety of translation tasks and their inherent optimal transport properties.

1 INTRODUCTION

DDIBs address privacy and adaptability limitations in unpaired image translation by training diffusion models independently per domain. They translate through two probability-flow ODEs and are theoretically linked to Schrödinger Bridges, with experiments spanning multiple translation settings.

  • Motivation: Existing unpaired translation methods typically train with adversarial losses on a specific source-target dataset pair, requiring both datasets during training.This limits data separation and can be problematic for privacy-sensitive applications.
  • Motivation: Pairwise translation across many domains can require a quadratic number of bespoke models, and shared-domain alternatives must be chosen carefully.The cited example notes that a shared domain may contain less information than the target domain.
  • Method and contribution: DDIBs train domain-specific diffusion models independently, allowing models to be reused whenever their domain appears as a source or target.Training concentrates on one dataset at a time, supporting federated settings and avoiding concurrent access to both datasets.
  • Method and contribution: Translation uses a source probability-flow ODE to encode an image into latent space, followed by a target ODE to synthesize the translated image.The mapping is smooth, deterministic, and reversible under the DDIM parameterization.
  • Theory and experiments: DDIBs are interpreted as concatenated Schrödinger Bridges between data and latent distributions, providing an optimal-transport perspective and cycle consistency up to ODE discretization error.Experiments examine cycle consistency and translation across color transfer, paired translation, and conditional ImageNet tasks.

2 PRELIMINARIES

The preliminaries introduce score-based diffusion models, their stochastic and deterministic formulations, and Schrödinger Bridges as entropy-regularized transport between distributions. DDIBs use probability-flow ODEs to move between source data, latent variables, and target data.

  • Score-based generative models: Score-based generative models include SMLD and DDPMs, both formulated within the stochastic differential equation framework.SMLD uses a variance-exploding formulation, while DDPMs use a variance-preserving SDE that can be reparameterized equivalently.
  • Score-based generative models: Forward and reverse SDEs describe diffusion and generation using drift, diffusion, and the score of the noise-perturbed data distribution.At the endpoints, the process connects the data distribution p0 to an easy-to-sample prior p1.
  • Probability-flow ODE: A probability-flow ODE is a deterministic process with the same marginal densities as the corresponding diffusion process and enables identifiable data-to-latent encodings.DDIBs solve this ODE forward and backward for conversion between data and latent representations.
  • Schrödinger Bridge Problem: A Schrödinger Bridge minimizes KL divergence to a reference path measure among processes with prescribed endpoint distributions.Unlike basic Monge-Kantorovich transport, it includes an additional entropy term.
  • DDIB algorithm: DDIBs apply two separately trained diffusion models: ODESolve first maps a source sample from time 0 to latent time 1, then maps it from time 1 to target data at time 0.The algorithm returns the target-domain image produced by the second ODE solve.
  • Connection to DDIBs: Score-based models correspond to Schrödinger Bridges with linear or degenerate drifts, and their probability-flow ODEs are equivalent in the relevant construction.In DDIBs, one marginal is Gaussian, so flowing through the PF ODEs realizes special Schrödinger Bridges.

3 DUAL DIFFUSION IMPLICIT BRIDGES

DDIBs translate between domains using independently trained diffusion models and two probability-flow ODE steps, without domain-pair training. The method provides cycle consistency under zero discretization error, privacy-sensitive data exchange, and an entropy-regularized optimal-transport interpretation.

  • Method: DDIBs use separately trained source and target diffusion models, first encoding the source image into latent space and then decoding it with the target model.The source ODE produces a latent code, which becomes the initial condition for the target ODE.
  • Method: Higher-order ODE solvers could replace the DDIM-based solver, but investigating them is left to future work.The paper implements DDIMs while identifying DPM-solver, exponential-integrator, and second-order Heun alternatives.
  • Exact Cycle Consistency: Under zero discretization error, DDIBs enforce exact cycle consistency through probability-flow ODEs.In practice, the paper reports almost negligible cycle inconsistency despite discretization error.
  • Data Privacy in Both Domains: DDIBs support privacy-sensitive translation because each party trains on its own data and exchanges latent codes and target results rather than raw datasets.The procedure avoids directly revealing either party’s data while allowing translation between their domains.
  • DDIBs are Two Concatenated Schrödinger Bridges: DDIBs concatenate two Schrödinger Bridges—source to latent and latent to target—thereby forming an entropy-regularized optimal-transport translation.The paper states that this mapping is unique and minimizes a regularized optimal-transport objective.

4 EXPERIMENTS

The experiments evaluate DDIBs on synthetic distributions and diverse image-translation tasks, finding cycle consistency, competitive paired-domain performance, close agreement with OT methods, and content-preserving ImageNet translations.

  • 4.1 2D SYNTHETIC EXPERIMENTS: Synthetic 2D experiments use six normalized datasets to examine DDIBs’ translation behavior and optimal transport properties.The datasets are Moons, Checkerboards, Concentric Rings, Concentric Squares, Parallel Rings, and Parallel Squares.
  • 4.1 2D SYNTHETIC EXPERIMENTS: DDIBs approximately recover original points after translating from Moons to Checkerboards and back, with negligibly small normalized errors.The reported values endorse the cycle-consistent property of DDIBs.
  • 4.2 EXAMPLE-GUIDED COLOR TRANSFER: DDIBs perform example-guided color transfer by encoding input colors and decoding them with a reference-image diffusion model.One diffusion model is trained per image in normalized RGB space to reproduce the reference palette.
  • 4.2 EXAMPLE-GUIDED COLOR TRANSFER: DDIB color-transfer outputs have pixel-wise MSEs very close to Earth Mover’s Distance, Sinkhorn, linear mapping, and Gaussian mapping methods.The comparison uses images normalized to [−1, 1].
  • 4.3 QUANTITATIVE TRANSLATION EVALUATION: On paired Facades and Maps tests, DDIBs produce segmentation images that surpass alternative methods in MSE terms.Reverse translations also achieve decent performance despite independently trained diffusion models.
  • 4.4 CLASS-CONDITIONAL IMAGENET TRANSLATION: On ImageNet, DDIBs preserve animal poses, complexions, and emotions while translating images across species and classes.Conditional diffusion models support translation among arbitrary domain pairs without additional fine-tuning or adaptation.

5 RELATED WORKS

Related work situates DDIBs among diffusion-based image translation methods and optimal-transport approaches for connecting image distributions and building generative models.

  • Score-based Diffusion Models: Diffusion models reverse a noising process to generate samples and are connected to score-based methods through density-score estimation.Prior work applies diffusion models across image translation and multiple data modalities.
  • Diffusion Models for Image Translation: Recent image-translation methods use diffusion models for colorization, inpainting, restoration, and text- or image-guided style transfer.The cited approaches include Palette, DiffuseIT, and UNIT-DDPM.
  • Optimal Transport for Translation and Generative Modeling: Optimal transport connects image distributions through cost-optimal plans, while Schrödinger Bridges provide an entropy-regularized alternative used in generative modeling.Prior work also computes OT plans between image distributions for unpaired translation.

6 CONCLUSIONS

The conclusion presents DDIBs as a diffusion-based, Schrödinger-Bridge-grounded translation method that separates domain training and improves dataset privacy, while identifying compute, mass-moving, dimensionality, and speed limitations.

  • 6 CONCLUSIONS: DDIBs avoid coupled losses specific to domain pairs and do not require both datasets to be present during training.This supports reuse of domain-specific models across translation pairs and better dataset privacy.
  • 6 CONCLUSIONS: The method is theoretically grounded as Schrödinger Bridges and experimentally validated across color transfer, paired translation, and conditional ImageNet translation.The experiments are presented as evidence of DDIBs’ practical value in domain translation.
  • 6 CONCLUSIONS: Color transfer requires one model per image, concatenated ODEs are time-consuming, and mass-moving behavior may be problematic in some cases.The paper also identifies differing source and target dimensions as an area for future extension.

A ILLUSTRATION: PRIVACY-SENSITIVE TRANSLATION

DDIBs support privacy-sensitive translation by independently training domain-specific diffusion models and transmitting only latent codes and translated images between parties.

  • Figure 5 depicts privacy-sensitive translation between Alice’s tiger domain and Bob’s cat domain.
  • Alice trains a diffusion model on tiger images and converts a source tiger image into a latent code.
  • Alice sends the latent code to Bob, who independently trains a cat diffusion model and decodes it into a cat image.
  • Bob sends the translated cat image back to Alice after target-domain decoding.
  • Only the latent code and translated cat image traverse the public channel, while both datasets remain private.

B DETAILS OF SGM TRAINING AND DDIM ODE SOLVER

The diffusion models use latent-variable training and deterministic DDIM updates, which are interpreted as Euler steps for probability-flow ordinary differential equations.

  • Diffusion probabilistic models learn pθ(x0) to approximate data distribution q(x0) while retaining tractable sampling through latent variables.
  • The model parameters θ are trained by maximizing a variational lower bound involving an inference distribution q(x1:T | x0).
  • Gaussian conditional distributions with trainable means and fixed variances simplify the variational objective.
  • The score networks are equivalent to the parameterization used in the diffusion formulation through Tweedie’s formula, with an equivalent variance-exploding probability-flow ODE.
  • DDIM updates between adjacent diffusion variables are deterministic and correspond to Euler discretization of a probability-flow ODE.

C LIMITATIONS OF OPTIMAL TRANSPORT-BASED TRANSLATION

DDIB translation follows optimal-transport mappings that minimize Euclidean transportation distances, but this preference can produce user-perceived failure cases.

  • DDIBs form deterministic bridges between distributions and constitute entropy-regularized optimal transport.
  • The learned diffusion models summarize their datasets and generate target-domain images close to source images in optimal-transport distance.
  • Figure 6 presents optimal-transport mappings from source images to translated images, including failure cases.
  • The left-to-right translation processes minimize Euclidean transportation distances between images, making the OT objective both a feature and a limitation.

D PROOF OF PROPOSITION 3.2

The proof establishes the proposition by substituting the specified values of (zt, ˆzt) into Eq. (6), yielding Eq. (2).

  • The proof substitutes (zt, ˆzt) = (0, g(t)∇x log pt(x)) into Eq. (6).
  • After substitution, the resulting expression is exactly Eq. (2).

E.1 OPTIMAL TRANSPORT IN PAIRED DATASETS

The paired-dataset experiments address substantial color differences between real and segmentation images by applying an OT-based color conversion before diffusion-model training. Qualitative comparisons with common OT methods are also provided, while the conversion introduces a limited privacy compromise.

  • Color Conversion: Real and segmentation images have significantly different color configurations, creating a large transportation cost that may hinder DDIBs.In Maps, real satellite images are dark whereas segmentation images are light-toned; similar differences occur across datasets.
  • Color Conversion: An OT algorithm computes a color correspondence on a small training subset to minimize Sinkhorn color distances before training on the converted segmentation data.The target segmentation datasets undergo this conversion before being fed into the diffusion model.
  • Privacy Protection: The color-conversion procedure requires joint consideration of both datasets, partially compromising DDIBs’ dataset-privacy objective.For Maps, the reported mapping used approximately 1000 sampled pixels from each dataset, and the authors characterize the overall information leakage as minimal.
  • Color Conversion: During MSE evaluation, images are converted back to their original color space.
  • Qualitative Comparison: Additional qualitative comparisons between DDIBs and common optimal-transport methods are presented in Figure 8.
Loading 2203.08382v4…