PaperlayerView exact revision on arXiv

Source-linked AI summary

Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning

Martin Thißen, Elke Hergenröther

arXiv:2610.29581v1cs.CVcs.LG

TL;DR

Fine-grained spatial control often requires structural adapters or inference-time steering, even though VLM embeddings can represent spatial coordinates. The paper studies flow-matching models with a synthetic geometric benchmark and SNR-distribution tuning, reporting improved spatial grounding and transfer to real imagery without structural modifications.

  • Problem

    Fine-grained spatial control often requires adapters or inference-time steering, while generative models may fail to use the spatial information encoded by VLM text encoders.

    Source: §1 · pp. 1–2

  • Method

    The paper fine-tunes flow-matching models on text-coordinate prompts, uses a geometric-primitives benchmark, and tests how SNR-distribution shifts affect layout grounding.

    Source: §1 · p. 2

  • Results

    Biasing training toward high-noise regimes improves spatial grounding, and the paper reports transfer to real-world imagery through COCO without specialized architectural extensions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    High-noise stages are identified as a critical learning phase for internalizing geometric constraints without architectural modifications.

    Source: §1 · p. 2

    A strong high-noise bias can compromise high-frequency textural synthesis, motivating investigation of multi-stage or scheduled sampling distributions.

    Source: §6 · p. 14

Abstract

from arXiv · show

Despite the success of foundation models in image synthesis, achieving fine-grained spatial control remains a significant challenge, typically requiring auxiliary structural adapters or complex inference-time steering. While modern vision-language (VLM) text encoders possess the latent capacity to represent spatial coordinates, generative models often fail to leverage this information. We demonstrate that the signal-to-noise ratio (SNR) distribution is a critical determinant of spatial grounding in flow-matching models. Using a novel synthetic benchmark of geometric primitives, we isolate structural mechanics from textural complexity inherent in natural imagery and introduce a rigorous color-aware layout metric to quantify spatial fidelity. We show that training on a centered SNR distribution results in poor layout adherence (25.9%). Instead, we identify high-noise regimes as a "critical learning phase" for spatial grounding. Shifting the training focus toward these stages increases layout adherence to 80.8%, effectively reaching the 83.6% reconstruction ceiling of the underlying variational autoencoder. We find that the necessity for this shift scales with resolution and aspect ratio variance, and that the conservative noise schedules employed by models like FLUX.1 are insufficient for precise layout adherence. Finally, we show that these findings translate to real-world imagery via the COCO dataset (1.0 vs. 57.3 Average Precision), where state-of-the-art layout adherence is unlocked through distribution tuning alone, without architectural modifications.

1 Introduction

Fine-grained spatial control often relies on architecture-specific adapters or inference-time steering, despite VLM encoders’ capacity to represent spatial relationships. The paper investigates whether tuning the SNR distribution can unlock text-prompt spatial grounding without structural modifications.

1 Introduction

  • High-noise training is identified as critical for spatial grounding, and distribution tuning alone transfers to real-world imagery without specialized architectural extensions.The study uses controlled synthetic experiments before extending evaluation to real images.Source: §1 · p. 2, Figure 1 · p. 3
  • The work introduces a geometric-primitives benchmark for exact, pixel-level layout-adherence assessment, isolating grounding mechanics from real-image textural complexity.Color-filled regions provide exact ground truth in the controlled synthetic setting.Source: §1 · p. 2, Figure 1 · p. 3

C Extended Fine-Tuning on Real-World Images

Extended fine-tuning at µ = 3.0 improves COCO detection metrics across 10 epochs, while image-quality metrics follow more varied trajectories.

C Extended Fine-Tuning on Real-World Images

D Compositional Failures in Pre-Trained Model

Compositional artifacts in fine-tuned FLUX.2-9B-base images are also present in the pre-trained model, indicating that these examples were not introduced by fine-tuning.

D Compositional Failures in Pre-Trained Model

  • Elephant heads occasionally fuse into one shared head, while seagulls can appear with multiple heads or two beaks pointing in different directions.These are examples of compositional artifacts in the pre-trained model.Source: Figure 12 · p. 26, Figure 13 · p. 26

Related papers

Loading 2610.29581v1…