Source-linked AI summary
ContraFM-S2O: Flow Matching-Based One-step SAR-to-Optical Image Translation Model with Contrastive Learning
Mingqian Yu, Wei-kuan Chiang, Qiurui Wang, Peilin Zhao
TL;DR
SAR-to-optical translation seeks to combine SAR’s all-weather imaging with optical imagery’s interpretability and spectral information, while existing approaches face inference-latency or detail-fidelity shortcomings. ContraFM-S2O uses average-velocity flow matching and contrastive learning for one-step translation. It reports state-of-the-art results across benchmark datasets and 4.4× inference acceleration over PCGL-GAN.
Problem
GAN- and diffusion-based SAR-to-optical methods have quality or training advantages, but can suffer from image-detail problems or high inference latency.
Source: §1 · p. 1Method
Results
ContraFM-S2O achieves state-of-the-art performance across QXS-SAROPT and SAR2Opt and 4.4× inference acceleration over PCGL-GAN, the second-best model.
Source: §1 · p. 1, §5.4 · p. 4Takeaways & Limitations
The reported results support a speed-quality synergy: contrastive learning enhances input discrimination and detail preservation, while average velocity reduces computation.
Source: §6 · p. 4
Abstract
from arXiv · showhide
In recent years, diffusion models and GAN-based models have become the mainstream approaches for SAR-to-optical image translation, owing to their advantages, such as high-quality generation and stable training. However, they have shortcomings such as high inference latency and the generated optical images suffer from low detail fidelity, often resulting in blurred edges and loss of fine textures. Thus, we propose ContraFM-S2O, which is a flow matching-based model for SAR-to-optical image translation. Unlike conventional diffusion models, ContraFM-S2O learns to predict the velocity field in training and solves ODE instead of SDE during inference to improve the sampling efficiency. In addition, ContraFM-S2O replaces instantaneous velocity with average velocity along the interpolation path to realize one-step SAR-to-optical image translation and uses contrastive learning to improve the quality of the generated optical images. Experiments show our model achieves state-of-the-art on SAR2Opt and QXS datasets, outperforming baselines, and reduces inference latency via one-step generation.
1. INTRODUCTION
SAR-to-optical translation combines SAR’s all-weather, day-and-night imaging with optical imagery’s interpretability and spectral information, with applications including emergency detection and geological monitoring. ContraFM-S2O targets shortcomings in prior methods through one-step generation and contrastive learning, and reports state-of-the-art performance with faster inference.
1. INTRODUCTION
- SAR-to-optical translation connects SAR’s all-weather, day-and-night imaging with optical imagery’s interpretability and rich spectral information.The passage also cites emergency detection and geological monitoring as applications.Source: §1 · p. 1
- GAN-based methods can suffer mode collapse and blurred or distorted details, while diffusion methods offer higher detail fidelity but have high inference latency.The passage also notes color discrepancies among GAN-generated images and says diffusion latency challenges real-time generation.Source: §1 · p. 1
- ContraFM-S2O targets one-step generation and sharp, detailed optical images by predicting average rather than instantaneous velocity and using contrastive learning to distinguish flows conditioned on different SAR images.Average velocity is defined as displacement per time interval, with displacement given by integrating instantaneous velocity.Source: §1 · p. 1
- ContraFM-S2O reaches state-of-the-art performance across baselines and achieves 4.4× inference acceleration over PCGL-GAN, the second-best model.Contrastive learning contributes more to generated-image quality, while average velocity contributes more to inference acceleration.Source: §1 · p. 1
2. RELATED WORK
Prior SAR-to-optical work includes GAN and diffusion approaches, while flow-matching research studies continuous velocity-field transport. ContraFM-S2O applies average velocity and contrastive learning to address the stated latency and detail limitations.
2.1. SAR-to-Optical Image Translation
- GAN and diffusion SAR-to-optical methods have reported image-quality strengths, but are limited by high inference latency and insufficient edge and texture clarity.ContraFM-S2O models average velocity for one-step generation and uses contrastive learning to distinguish transport flows between different SAR and optical images.Source: §2.1 · pp. 1–2
2.2. Flow Matching-Based Image Generation
- Flow Matching learns a continuous, deterministic transport map between source and target domains through an ODE by regressing velocity fields.Rectified Flow, Conditional Flow Matching, and MeanFlow are cited as representative approaches; MeanFlow uses average velocity for one-step generation.Source: §2.2 · p. 2
3. BACKGROUND
Flow Matching learns velocity fields to transport between probability distributions and generates samples by solving an ODE. The background distinguishes conditional velocities associated with interpolation paths from the marginal velocity field used for generation.
3. BACKGROUND
- Flow Matching learns velocity fields representing flows between probability distributions and generates samples by solving an ODE from a prior sample.The ODE solution integrates the marginal velocity field across time.Source: §3 · p. 2, §3.2 · p. 2
3.1. Conditional Velocity.
- Conditional velocity is the time derivative along an interpolation path, and the same intermediate state can arise from different data-prior pairs with different conditional velocities.The passage gives a_t = 1 − t and b_t = t as common interpolation coefficients.Source: §3.1 · p. 2
3.2. Marginal Velocity.
- Marginal velocity is the expectation over possible conditional velocities, and Flow Matching trains a neural network to fit this field.The conditional Flow-Matching loss can be evaluated in place of the marginal loss, and minimizing them is stated to be equivalent.Source: §3.2 · p. 2
4. CONTRAFM-S2O
ContraFM-S2O predicts average rather than instantaneous velocity for one-step generation and adds contrastive learning to prevent flow overlap and improve generated optical-image quality.
4. CONTRAFM-S2O
- ContraFM-S2O models average velocity instead of instantaneous velocity for one-step generation and uses contrastive learning to prevent flow overlap and improve generated-image quality.The authors report that contrastive learning contributes more to image quality, while average velocity contributes more to inference acceleration.Source: §4 · p. 3
4.1. Average Velocity
- Average velocity captures overall path migration and may help preserve ground-object contours and layouts; a JVP forms the target using conditional rather than marginal velocity.The total derivative is expressed as a Jacobian–vector product.Source: §4.1 · p. 3
4.2. Training with Contrastive Learning and Average Velocity
- Contrastive training distinguishes flows conditioned on different SAR images to prevent predicted average-velocity fields from overlapping.The training procedure concatenates the interpolation path with SAR images, compresses them with a frozen VAE encoder, and feeds the features to a U-Net.Source: §4.2 · p. 3
4.3. One-step Inference
- At r=0 and t=1, one-step inference maps prior noise to the optical output as x = ϵ − u(z_1, 0, 1).The method sets ϵ from the prior distribution for this one-step translation.Source: §4.3 · p. 3
5. EXPERIMENTS
ContraFM-S2O achieves state-of-the-art results on QXS-SAROPT and SAR2Opt and reports 4.4× inference speed acceleration over PCGL-GAN. Experiments also examine the datasets, baselines, implementation, and contributions of average velocity, contrastive learning, and λ.
5.1. Datasets and Evaluation Metrics
- The evaluation uses QXS-SAROPT and SAR2Opt, reporting SSIM, FID, PSNR, and LPIPS; Fig. 2's rows show SAR2Opt and QXS-SAROPT test examples.QXS-SAROPT has 20,000 paired images, while SAR2Opt has 2,076; both use an approximately 4:1 train-test split.Source: §5.1 · p. 3, Figure 2 · p. 4
5.2. Baselines
- Baselines include GAN-based, diffusion-based, and contrastive-learning-based methods.The listed methods are Pix2Pix, CycleGAN, PCGL-GAN, S2ODPM, cDDPM, CycleDiff, KANCUT, and CUT.Source: §5.2 · p. 4
5.3. Implementation Details
- Training used eight NVIDIA A100 GPUs for 400 epochs with learning rate 2e-4, batch size 128, and λ = 0.01; inference used one A100.Source: §5.3 · p. 4
5.4. Main Results
- ContraFM-S2O achieves state-of-the-art performance on QXS-SAROPT and SAR2Opt and 4.4× inference speed acceleration compared with second-best PCGL-GAN.Table 1 compares baseline performance; Fig. 2 compares PCGL-GAN and ContraFM-S2O on both test sets.Source: §5.4 · p. 4, Table 1 · p. 4, Figure 2 · p. 4
5.5. Ablation Studies
- All ablation studies used one-step inference to examine average velocity, contrastive learning, and λ.Source: §5.5 · p. 4
5.5.1. The Influence of Average Velocity and Contrastive Learning
- Average velocity primarily speeds inference, while contrastive learning improves image quality by preventing velocity-field overlap and enabling detailed edge generation.Table 2 compares ContraFM-S2O with variants without average velocity, contrastive learning, or both.Source: §5.5.1 · p. 4, Table 2 · p. 4
5.5.2. The Influence of λ during training
- Quality and speed are optimal at λ = 0.01; larger λ degrades quality by weakening SAR-optical feature association.Table 3 compares training with different λ values.Source: §5.5.1 · p. 4, Table 3 · p. 4
6. CONCLUSION
The conclusion presents ContraFM-S2O as a flow-matching model for SAR-to-optical translation, combining contrastive learning for image quality with average velocity modeling for faster inference. It reports state-of-the-art performance across benchmark datasets in both generation quality and inference speed.
6. CONCLUSION
- ContraFM-S2O is a flow-matching model for SAR-to-optical translation.Source: §6 · p. 4
- Contrastive learning improves generation quality, while average velocity modeling speeds inference.Source: §6 · p. 4
- The model achieves state-of-the-art performance across multiple benchmark datasets in generation quality and inference speed.Source: §6 · p. 4