PaperlayerView exact revision on arXiv

Source-linked AI summary

Surflo: Consistent 3D Surface Flow Model with Global State

Antoine Guédon, Shu Nakamura, Nicolas Dufour, Jiahui Lei, Ko Nishino, Angjoo Kanazawa

arXiv:2606.13644v1cs.CV

TL;DR

Multi-view 3D reconstruction should recover one shared geometry, but existing feed-forward models either produce unaligned per-view outputs or fixed-resolution global outputs. Surflo compresses unposed views into one latent and decodes arbitrary-resolution oriented surfaces, matching or surpassing feed-forward baselines across eight benchmarks.

  • Problem

    Although multiple views redundantly encode one global geometry, existing representations do not consistently recover that shared state across views or support flexible output resolution.

  • Method

    Surflo compresses variable-number unposed RGB views into a fixed-size global latent and uses flow matching with photometric guidance to decode arbitrary numbers of oriented surface points.

  • Results

    Across 8 3D reconstruction benchmarks, Surflo matches or surpasses recent feed-forward baselines on surface metrics while decoding arbitrary resolution from one shared latent.

  • Takeaways & Limitations

    Surflo provides a feed-forward reconstruction model that combines a shared global representation with arbitrary-resolution surface decoding.

  • Takeaways & Limitations

    Surflo can degrade with very few views or extreme baselines, incurs guidance-related rendering cost, relies on imperfect Gaussian-Wrapped supervision, and does not model appearance.

Abstract

from arXiv · show

Geometry is invariant to viewpoint, which makes any collection of images a redundant encoding of a single 3D state. Existing feed-forward reconstruction models fail to exploit this: per-view methods emit overlapping, unaligned pointmaps that grow linearly with input count, while global-latent methods commit to a fixed, low-resolution output. We introduce Surflo, which compresses a variable number of unposed RGB views into K latent tokens-one global state-and decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching. This frees the output from any fixed grid or token budget: the same latent yields from a few thousand to a million points in a single forward pass. To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient during ODE integration. Surflo matches or surpasses feed-forward baselines on surface metrics, runs an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.

1 Introduction

Surflo treats multi-view images as redundant observations of one global 3D state, compressing variable unposed RGB views into a shared latent. It then decodes arbitrarily many oriented surface points and applies rendering-based guidance to improve consistency during integration.

  • Motivation: Multi-view images redundantly encode one scene geometry, so additional views refine a shared global state rather than adding independent geometric content.Raw pixel data grows linearly with views, while geometric content does not.
  • Global representation: Surflo compresses a variable number of unposed RGB views into one compact global representation independent of view count.Its encoder is built on a frozen VGGT backbone.
  • Results: Across 8 3D reconstruction benchmarks, Surflo matches or surpasses recent feed-forward baselines on surface metrics.It is also identified as the only feed-forward method decoding arbitrary resolution from a single shared latent.
  • Arbitrary-resolution decoding: Surflo transports oriented points independently from noise to decode arbitrary-resolution surfaces from the same shared latent.The flow-matching decoder operates on oriented points in R3 × S2.
  • Consistency guidance: Rendering-based guidance during ODE integration couples nearby point velocities and aligns the decoded surface with the available evidence.The guidance is applied during inference-time ODE integration.

2 Method

Surflo encodes a variable number of unposed views into a fixed-size global latent and decodes oriented surface points at arbitrary resolution through conditional flow matching. Inference-time rendering guidance couples independently decoded points to improve surface coherence and detail.

  • Global encoding: Surflo encodes any number of input views into a fixed-size latent z ∈ R^(K×D), decoupling conditioning size from the number of views.A Perceiver-style module compresses position-encoded patch tokens into K learned latent queries, with camera information concatenated separately.
  • Per-point flow: The flow-matching decoder transports individual queries in R3 × S2 from a source distribution to the oriented surface distribution, conditioned on the global latent.A transformer predicts time- and world-space-conditioned velocities from Fourier-encoded queries through cross-attention and Ada-LN blocks.
  • 3D positional encoding: Shared Fourier encoding of 3D patch centers and query points lets cross-attention localize scene information by spatial proximity.The encoding provides the decoder’s notion of query location relative to the encoded scene and makes coordinate-frame robustness structural.
  • Inference: 10^3 to 10^6 points can be decoded in a batched single-GPU pass because all queries share the same latent and use a relatively small decoder.The source distribution uses perturbed VGGT pointmap samples to concentrate flow near geometry while retaining coverage for occluded surfaces.
  • Guided ODE: Independent query decoding can produce scattered points and drift, so late-ODE rendering guidance couples nearby queries and improves consistency with input images and surface details.For t ≥ 0.95, predicted targets are optimized using a global rendering loss; optional monocular-depth guidance further strengthens the signal.

3 Experiments

Surflo is evaluated across few-view reconstruction benchmarks and consistently addresses the misalignment of per-view predictions through a shared latent that supports variable-resolution decoding. Ablations and efficiency results further assess its latent design, photometric guidance, and inference cost.

  • Main evaluation: Surflo is evaluated on in-distribution and out-of-distribution benchmarks against per-view feed-forward, latent, and optimization-based scene-reconstruction baselines.The evaluation uses common unposed input views and includes native and generated reference surfaces.
  • Main evaluation: Per-view baselines accumulate misalignments when merged, whereas Surflo produces coherent scene-level surfaces from the same shared latent.Qualitative comparisons show noisy, duplicated pointmaps for VGGT and a clean Surflo mesh from 16 unposed views.
  • Varying number of input views: Additional input views progressively fill missing geometry and sharpen detail, while decoding cost remains unchanged because the latent size does not grow.Surflo already produces a coherent surface in the sparse 2-view regime.
  • Multi-resolution decoding: The same latent decodes outputs from 8K to 128K points without changing the latent or GPU, supporting fast previews and denser geometry.The coarse output targets collision queries or previews, while dense output sharpens details and background geometry.
  • Ablations: Increasing K from 32 to 128 tokens and adding 3D positional encoding improve both metrics, while replacing the noisy-VGGT source with a pure Gaussian source degrades reconstruction.The model uses K = 128 tokens throughout.
  • Guidance and efficiency: Full photometric guidance is consistently best for visual quality, while disabling it can introduce noisy outliers despite high accuracy.The guidance term adds modest overhead, and decoding 105 points from a cached latent is two orders of magnitude faster than per-scene optimization baselines.

4 Related Work

Related work spans view-bound feed-forward reconstruction, global-latent models, flow-matching decoders, and per-scene optimization. Surflo combines a single global state with arbitrary-resolution oriented surface-point decoding and feed-forward scene reconstruction.

  • Per-view feed-forward 3D: Per-view models such as DUSt3R and VGGT predict accurate but redundant, misaligned geometry whose token count grows linearly with input views.Their view-bound outputs are difficult to fuse into a consistent global mesh.
  • Global latent models: Global-latent methods include CUT3R and NOVA3R, but NOVA3R decodes a fixed 10K points and trains on only two views.Surflo instead decodes an arbitrary number of oriented surface points from one global state.
  • Global latent models: More input views progressively complete scenes and sharpen detail, while decoder cost remains unchanged because reconstructions use one fixed-size latent.This supports varying capture densities without retraining or architectural changes.
  • Flow matching and guidance: Surflo uses flow matching to transport query points independently and adds guidance to regularize their local consistency.The decoder differs from fixed-grid or structured-latent methods and its guidance mechanism is reminiscent of loss-conditioned diffusion.
  • Per-scene optimization: Unlike NeuS and 3D Gaussian Splatting, which require hundreds of posed images and substantial per-scene computation, Surflo reconstructs scenes feed-forward from a reusable latent.Surflo decouples final reconstruction cost from the number of input images.

5 Conclusion

Surflo compresses arbitrary unposed RGB views into one fixed-size latent and decodes arbitrary-resolution oriented surface points through flow matching. It outperforms strong baselines on few-view surface metrics, while limitations arise from unreliable pointmaps and the cost of photometric guidance.

  • 5 Conclusion: Surflo compresses arbitrary unposed RGB views into a single fixed-size latent and decodes arbitrary numbers of oriented surface points through flow matching in R3 × S2.Per-point independence enables both flexible output resolution and scalability, from coarse previews to million-point reconstructions.
  • 5 Conclusion: Surflo outperforms strong baselines on few-view 3D reconstruction benchmarks using standard surface metrics.It is the only feed-forward model combining a fixed-size latent with arbitrary-resolution decoding.
  • 5 Conclusion: The meshed DL3DV dataset contains ∼10.5K scenes augmented with watertight surfaces and 107 oriented points each.The dataset enabled large-scale Surflo training and is intended to support future scene-level surface learning.
  • Limitations: Surflo can fail when VGGT produces poor pointmaps, especially with very few views or extreme baselines, because its source distribution and patch tokenization become unreliable.These failures reduce the decoder’s ability to recover surfaces.
  • Limitations: Photometric guidance improves reconstruction but adds inference cost through oriented-Gaussian rendering, although inference remains small on a single GPU at standard resolutions.The supervision is derived from Gaussian Wrapping surfaces.

A Related work

Related work spans per-view feed-forward geometry, global latent representations, per-scene surface optimization, object-level learned 3D, and diffusion or flow priors. Surflo is positioned around full-scene reconstruction from a few unposed views using a shared, queryable latent.

  • Per-view feed-forward 3D: Per-view feed-forward models predict pointmaps or depth maps per image, scaling from paired views to many-view transformers.DUSt3R regresses paired pointmaps in a common frame, while VGGT processes hundreds of images in one forward pass.
  • Latent and global feed-forward 3D: Global feed-forward methods maintain or tokenize view-agnostic representations but commonly retain per-view metric-pointmap or per-view decoding.CUT3R maintains persistent state through cross-attention, while 3DRAE and NOVA3R are described as global-tokenization approaches with per-view readouts or decodings.
  • Per-scene optimization for surfaces: Per-scene optimization remains dominant for fine-grained reconstruction, spanning radiance-field methods without clean geometry and explicitly geometry-focused methods.NeRF, 3D Gaussian Splatting, and related variants produce faithful renderings but no clean geometry.
  • Object-level learned 3D and shape latents: Learned feed-forward 3D methods are largely object-level, including mesh, NeRF, Gaussian-splatting, iso-surface, and structured-latent predictors.Examples include LRM, GS-LRM, InstantMesh, TRELLIS, Sam3D, 3DShape2VecSet, Michelangelo, and CLAY.
  • Object-level learned 3D and shape latents: None of the cited object-level methods targets full scenes from a few unposed views with a shared, queryable latent.This is stated as the setting most closely aligned with Surflo’s design.
  • Diffusion / flow priors and guidance: Diffusion and flow priors support novel-view synthesis or reconstruction through hallucinated views, 3D regularization, distilled priors, or score distillation.The passage lists GeNVS, CAT3D, ReconFusion, NeRFbusters, ReconViaGen, and score-distillation methods as examples.

B Architecture details

Surflo’s architecture uses a frozen multi-layer VGGT encoder projected to a 512-dimensional working space and a 12-layer flow-matching decoder that independently processes query points while sharing spatial information through encoder latents. Query points encode normals as residual offsets, enabling normal recovery by differencing and normalization at inference.

  • Encoder: The frozen VGGT-1B encoder extracts patch and camera tokens from layers 4, 11, 17, and 23, projecting them to working dimension D = 512.VGGT-1B has 24 transformer blocks, patch size 14, input resolution 518 × 280, 740 patch tokens per view per layer, and 1 camera token per view, each of dimension dv = 2048.
  • Decoder: The flow-matching decoder is a L = 12-layer transformer with hidden dimension D = 512, 16 heads, mlp_ratio = 4.0, and qk-norm.It has no self-attention over query points; spatial information sharing is delegated to encoder latents, while the first 6 blocks alternate latent cross-attention and MLPs with Ada-LN conditioning.
  • Decoder: A two-layer MLP head predicts each point’s normal in S2 alongside its velocity after time and coordinate features are encoded into the decoder’s working dimension.The time embedding uses 512 log-spaced frequencies over [0.1, 1000], outputs dimension 1024, and is projected to D = 512; coordinates use a 16-band Gaussian RFF encoder.
  • Query Formulation: Normals are represented as residual offsets by initializing each query as x0 = (m, m + ϵn) ∈R6, with ϵ = 10−3 multiplied by scene scale.At inference, the normal is decoded by subtracting the first half from the second and normalizing the resulting vector.

C Training

Surflo training combines a scene-level surface dataset with velocity-prediction flow matching initialized from noisy VGGT world-point predictions. Ground-truth surfaces are aligned to the VGGT coordinate frame before loss computation using an affine transform fitted from rendered depth maps.

  • Dataset: The Meshed DL3DV dataset enriches each scene with a watertight surface and an associated oriented point cloud, sampled uniformly with normals.The dataset is built using Gaussian Wrapping and is planned for release as an auxiliary contribution.
  • Flow-matching training: The decoder is trained as a velocity-prediction flow-matching model with a conditional optimal-transport scheduler.Each query starts from a VGGT world-point prediction plus isotropic Gaussian noise with standard deviation 0.1 in scene-normalized coordinates.
  • Coordinate alignment: Training aligns ground-truth COLMAP-frame points with VGGT-frame predictions before computing the flow-matching loss.The alignment estimates an affine transform mapping COLMAP world coordinates into the VGGT frame.
  • Coordinate alignment: Only the top 25% of pixels ranked by VGGT depth confidence are retained before alignment to improve robustness.A coarse similarity alignment is first estimated from the reference camera’s ground-truth depth points using COLMAP extrinsics and camera-baseline extents.

D Guidance

Surflo guides its final flow trajectory with differentiable Gaussian rendering, using photometric and geometric consistency with input views to refine generated surfaces. The procedure also supports optional monocular depth-order guidance and prunes low-opacity outliers during integration.

  • Time grid: A bi-phase time grid uses 50+100 steps, switches at t = 0.95, and samples P = 105 query points by default.For tk ≥0.95, predicted velocities recover clean point positions and, for the R6 parameterization, per-point normals.
  • Rendering guidance: Rendering guidance activates at t ≥τg = 0.95 by instantiating one anisotropic 3D Gaussian per generated point.Each Gaussian has learnable log-scales, rotation quaternions, opacity, and view-dependent spherical-harmonic color.
  • Inner loop: At every guided ODE step, M = 32 differentiable rendering iterations minimize a composite loss and convert Gaussian displacements into a guided velocity.RaDe-GS rasterizes the Gaussians into input views, steering the generative trajectory toward photometric and geometric consistency.
  • Monocular geometric experts: An optional monocular expert adds a scale-invariant depth-order loss that enforces only relative ordering, avoiding inconsistent absolute monocular depths.The loss compares rendered depth with a monocular depth prior while preventing multi-view inconsistency from affecting the flow trajectory.
  • Filtering outliers: Low-opacity points are pruned during ODE integration because noisy points and floaters that do not contribute to input views naturally lose opacity.Surflo follows the standard pruning threshold of 3D Gaussian Splatting.

E Evaluation protocol.

Surflo is evaluated under a standardized protocol using identical unposed views, official latent-method checkpoints, TSDF fusion for per-view predictions, and scale-invariant aligned point-cloud metrics. The protocol reports normalized Chamfer distance and F-score after clipping and common-density voxel downsampling.

  • Evaluation protocol: All baselines receive identical unposed views per scene, while latent methods use their official checkpoints; per-view pointmaps are fused with TSDF.The fusion uses three nested 100^3 voxel grids spanning 1, 3, and 10 scene diagonals, with a 1.25% diagonal truncation margin.
  • Point cloud alignment: Predicted point clouds are aligned to ground truth with a closed-form 7-DoF similarity transform, and thresholds are normalized by the ground-truth bounding-box diagonal d.This makes the protocol invariant to each scene’s absolute scale.
  • Metrics: After alignment, out-of-bounds predictions are clipped, both clouds are voxel-downsampled to voxel size 10^-3d, and symmetric Chamfer distance plus F-score are reported.The F-score uses distance threshold τ = 10^-2d, while Chamfer distances are normalized by d for cross-scene aggregation.
Loading 2606.13644v1…