ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation

• SAR offers reliable observation across illumination and most weather conditions, but speckle and geometric distortions make it less intuitive than EO imagery.Source: §1 · p. 1
• The SAR-to-EO mapping is inherently ambiguous because SAR backscatter and EO reflectance describe fundamentally different physical properties.Source: §1 · p. 1
• Existing latent methods reduce computation but typically adopt the pretrained Stable Diffusion VAE as a fixed codec without examining its reconstruction fidelity.Source: §1 · p. 1
• ReFlowSET audits modality-specific reconstruction ceilings, selects a codec with strong joint SAR–EO fidelity, and trains a substantially smaller conditional DiT from scratch.Source: §1 · p. 1
• Intermediate noisy-EO features are aligned with clean-EO representations from a frozen vision foundation model, adding neither annotations nor inference-time overhead.Source: §1 · p. 2
• Evaluations on QXS-SAROPT and SAR2Opt report leading performance in perceptual similarity and distributional realism.Source: §1 · p. 2

• Earlier image-translation methods used conditional or cycle-consistent GANs, with later models improving high-resolution, spatially controlled, or non-bijective synthesis.Source: §2.1 · p. 2
• SAR-specific GANs incorporated atrous context or EO segmentation labels to reduce blur and semantic errors.Source: §2.2 · p. 2
• Diffusion approaches addressed pixel-space denoising and one-step distillation, while cBBDM and C-DiffSET operated in the SD2.1 latent space.Source: §2.2 · p. 2
• ReFlowSET makes the autoencoder endpoint an explicit design variable and combines flow matching with label-free, training-only representation guidance.Source: §2.2 · p. 2

• A round-trip reconstruction upper-bound is defined as the fidelity of decoding an image after encoding it with a frozen autoencoder.Source: §3 · p. 2
• Higher EO and SAR reconstruction ceilings indicate less target distortion and greater retention of source structure for conditional generation, respectively.Source: §3 · p. 2
• Figure 1 compares six pretrained autoencoders across SAR and EO imagery from four datasets using encode–decode PSNR and representative reconstructions.Source: §3 · p. 2
• FLUX.2 achieves the highest PSNR in six of eight dataset–modality settings and raises the mean EO ceiling by 7.49 dB over SD2.1.Source: §3 · p. 3
• The selected FLUX.2 autoencoder is retained for SAR and EO, while its associated multi-billion-parameter pretrained generator is excluded to avoid substantially higher computational cost.Source: §3 · p. 3

4.1. Conditional Latent Flow Matching

• ReFlowSET encodes paired SAR and EO observations with a selected frozen encoder and trains a conditional DiT to predict latent flow velocities.The SAR latent remains a persistent condition during flow-matching training.Source: §4.1 · p. 3
• Independent noise is transformed toward the EO latent along a linear probability path, and the learned ODE is integrated from t = 0 to 1 for decoding.Starting from independent noise prevents the target velocity from becoming an analytic function of the model inputs.Source: §4.1 · p. 3

4.2. Dual-Stream SAR Conditioning

• The model uses independently parameterized SAR and noisy-EO streams for the first r blocks, then concatenates and jointly processes their features to predict the velocity field.This differs from a single-stream baseline that concatenates the latents only at the input.Source: §4.2 · p. 3

4.3. Training-Only EO Representation Alignment

• Because the DiT is trained from scratch without pretrained-generator semantic priors, intermediate noisy-EO features are aligned with clean EO representations from a frozen vision foundation model.A trainable projector maps DiT features into the frozen representation space, and cosine distance supplies the alignment objective.Source: §4.3 · pp. 3–4

4.4. Experiments

• Table 1 compares methods on QXS-SAROPT and SAR2Opt using identical training splits and test pairs after retraining baselines from official code.The table reports quantitative comparisons under a common evaluation protocol.Source: Table 1 · p. 4

4.3. Training-Only EO Representation Alignment

• The total objective combines flow matching with representation alignment, while the vision foundation model and projector are used only during training and discarded at inference.The fixed alignment weight is λ = 0.5, with no test-time overhead.Source: §4.3 · p. 4

Experimental Setup

• QXS-SAROPT contains 20,000 paired 256-pixel tiles, while SAR2Opt contains 2,077 paired 600-pixel images with specified training and testing splits.All baselines are retrained on the same training splits and evaluated on identical test items.Source: §5.1 · p. 4

Evaluation

• Table 1 evaluates distributional realism with FID, perceptual fidelity with DISTS and LPIPS, and pixel fidelity with SSIM and PSNR.Source: §5.2 · p. 4

Comparison with the State of the Art

• 7.7% and 12.3% relative gains are reported for FID and DISTS on SAR2Opt, respectively, alongside the lowest LPIPS.FID improves from 71.8 to 66.3 and DISTS from 0.211 to 0.185.Source: §5.2 · pp. 4–5
• Replacing the SD2.1 codec with FLUX.2 improves all five metrics on both benchmarks, reducing FID from 25.5 to 19.1 on QXS-SAROPT and from 84.5 to 66.3 on SAR2Opt.The comparison supports reconstruction-guided codec selection within the ReFlowSET framework.Source: §5.2 · p. 5
• ReFlowSET does not maximize PSNR or SSIM because pixel-aligned metrics are sensitive to ambiguity and local misalignment in paired SAR–EO observations.The study therefore emphasizes perceptual and distributional metrics while reporting pixel fidelity for completeness; Figure 3 shows sharper structures and more coherent land-cover appearance than competing models.Source: §5.2 · p. 5

Ablation and Efficiency Analysis

• The ablation compares SAR conditioning, stream fusion, and EO representation alignment using compact 12-block models, while the final model uses an 8-and-16 block allocation.The table reports deployed parameters, peak training memory, and per-image inference latency; training-only alignment modules are excluded from deployed parameters.Source: §5.3 · pp. 5–6, Table 2 · p. 5
• Channel-wise fusion avoids the token-wise fusion variant’s increase in training VRAM from 10.63 to 13.57 GB and latency from 1.34 to 2.06 s/image, with virtually no FID benefit from token-wise fusion.Under the dedicated SAR stream, adding EO representation alignment reduces FID from 71.851 to 70.436 without changing deployed parameters or inference latency.Source: §5.3 · p. 6

• ReFlowSET combines reconstruction-guided codec selection, conditional latent flow matching, delayed SAR–EO fusion, and training-only EO representation alignment.Source: §6 · p. 6
• ReFlowSET achieves the best FID and DISTS on both benchmarks and the best LPIPS on SAR2Opt among the evaluated methods.Source: §6 · p. 6
• The alignment branch introduces no inference-time overhead.Source: §6 · p. 6
• Future work will examine cross-sensor generalization and suppression of unsupported EO-like structures.Source: §6 · p. 6