PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision

Rui Wang, Haoyu Zhang, Ran Cheng

arXiv:2610.15280v2cs.CVcs.AI

TL;DR

Depth estimation and Gaussian reconstruction share multi-scale feature processing but have different prediction interfaces and objectives, leaving open whether decoder computation can be optimized once and reused. GeoBridge searches stage-internal topology and evaluates candidates with a training-free proxy, then tests selected structures with target-specific training. GeoBridgeNet-A3 leads four decoder baselines on the reported depth datasets, and the same topology achieves 27.11 dB PSNR in Gaussian novel-view synthesis.

  • Problem

    Depth estimation and Gaussian reconstruction share multi-scale feature processing but differ in prediction interfaces and objectives, raising whether decoder computation can be optimized in one setting and reused in another.

    Source: §I · p. 1

  • Method

    GeoBridge evolves residual, sequential, and parallel stage compositions in a fixed multi-scale scaffold using functional-tree variation and a training-free GSAP proxy balanced against computation and parameter cost.

    Source: §I · pp. 1–2

  • Results

    GeoBridgeNet-A3 achieves the best depth accuracy among four recent decoder baselines on NYU Depth V2 and KITTI, and the same topology reaches 27.11 dB PSNR on RealEstate10K.

    Source: §I · p. 1

  • Takeaways & Limitations

    The evaluated results support stage-internal decoder topology as a reusable design unit across depth estimation and Gaussian novel-view synthesis, with reconstructed scenes supporting downstream semantic processing.

    Source: §V · pp. 12–13

Abstract

from arXiv · show

Depth estimation and Gaussian reconstruction share a need for multi-scale feature processing, yet use different prediction interfaces and training objectives. Whether a decoder's internal computation can be optimized once and reused across these systems remains a structural design question. Search over primitive configurations alone leaves the organization of transformations fixed. To address this gap, we propose GeoBridge, an evolutionary framework for transferable decoder topologies. First, GeoBridge makes stage-internal topology searchable through residual, sequential, and parallel composition of configurable primitives. Shape-preserving composition retains the stage interfaces of a fixed multi-scale scaffold. Second, a functional-tree encoding supports stage-aware crossover and hierarchical mutation, while a gradient sensitivity-alignment proxy evaluates candidates without training. Multi-objective evolution balances this proxy with computation and parameter cost. Third, selected topologies are instantiated and trained in target systems with task-specific prediction interfaces. A single source search takes approximately 12 GPU hours. GeoBridgeNet-A3 achieves the best depth accuracy among four adapted decoder baselines under a common encoder and training protocol on NYU Depth V2 and KITTI. The same topology reaches 27.11 dB PSNR in Gaussian novel-view synthesis on RealEstate10K, and its refined representations support open-vocabulary segmentation on LERF. These results support decoder topology as a reusable design object across the evaluated geometric vision systems. Code and pretrained weights are available at https://github.com/EMI-Group/GeoBridge-NAS.

I. INTRODUCTION

GeoBridge asks whether decoder computation can be optimized in one geometric setting and reused in another, and makes stage-internal composition searchable while retaining the interfaces needed by the surrounding network. It evaluates this approach across depth estimation and Gaussian reconstruction, against a backdrop of visual-decoding NAS methods that search operations and topology.

I. INTRODUCTION

  • Decoder topology is proposed as a reusable design object because depth estimation and Gaussian reconstruction share multi-scale feature processing but have different prediction interfaces and objectives.The paper tests reuse through target-specific training and also examines the semantic utility of reconstructed scenes.Source: §I · pp. 1–2
  • GeoBridgeNet-A3 achieves the best depth accuracy among four recent decoder baselines under a common encoder and training protocol on NYU Depth V2 and KITTI; the same topology reaches 27.11 dB PSNR on RealEstate10K.Its Gaussian outputs also support open-vocabulary segmentation on LERF after scene refinement and semantic optimization.Source: §I · p. 1
  • GeoBridge exposes residual, sequential, and parallel stage organization to search within a fixed multi-scale scaffold, alongside primitive configuration.Its evolutionary search uses stage-aware crossover, hierarchical mutation, and GSAP, balancing the training-free proxy with computation and parameter cost.Source: §I · pp. 1–2

A. Architecture Search for Visual Decoding

  • Prior visual-decoding NAS methods search feature fusion, multi-resolution structures, or task-specific designs; GeoBridge combines topology and operation choice within decoder stages while keeping external multi-scale connections fixed.The related-work discussion situates these examples across object detection, segmentation, depth prediction, text recognition, pedestrian detection, gait recognition, and guided super-resolution.Source: A. Architecture Search for Visual… · p. 2

B. Architecture Transfer

GeoBridge treats stage-internal decoder topology as a transfer unit, retaining its composition and primitive configurations while target systems train their own weights and prediction interfaces. It combines functional composition with training-free candidate evaluation within a fixed multi-scale scaffold.

B. Architecture Transfer

  • GeoBridge studies decoder stage-internal topology as the transfer unit between dense depth prediction and Gaussian reconstruction, with each target training the instantiated network for its own objective.Prior transfer approaches reuse selected configurations, cells, adapted networks, or broader architectural principles.Source: B. Architecture Transfer · p. 2

C. Structural Encoding in EvoNAS

  • GeoBridge composes residual, sequential, and parallel nodes into executable feature transformations; complete trees and subtrees can vary without changing connections between decoder stages.This functional composition operates within a fixed multi-scale decoder scaffold.Source: C. Structural Encoding in EvoNAS · pp. 2–3

D. Efficient EvoNAS

  • Unlike weight-sharing, supernet, and inherited-weight approaches, GeoBridge independently instantiates candidates and uses a training-free proxy to control evaluation cost.GSAP combines a parameter-scaled sensitivity statistic with gradient–parameter alignment under the source-search regression stimulus.Source: D. Efficient EvoNAS · p. 3

A. Design Object and Source Scaffold

  • The training-based reference objective balances depth prediction error after source training with decoder computation and parameter costs, maintaining a non-dominated set of structures with different cost profiles.Evolution instead uses JGSAP as the quality objective over independently instantiated candidates.Source: A. Design Object and Source Scaffold · p. 3
  • The selected architecture retains its stage compositions and primitive configurations across target systems, while each target optimizes its own network weights and prediction interface.The source search chooses the computation structure; target training fits it to each prediction objective.Source: A. Design Object and Source Scaffold · p. 3
  • The source search uses a fixed multi-scale scaffold with DINOv2 ViT-S features, Tree-Mamba stages, and TartanAir inputs plus cached Depth Anything V2 teacher targets.The stage-internal computation is searchable; projections, cross-scale connections, upsampling operators, and the source head remain fixed.Source: A. Design Object and Source Scaffold · p. 3

B. Compositional Decoder Topologies

GeoBridge represents each decoder stage as a functional tree whose residual, sequential, or parallel composition changes internal processing while preserving the stage interface. Stage-specific bounds allocate more compositional capacity to the coarser stages.

B. Compositional Decoder Topologies

  • Each topology uses configurable Mamba primitives as leaves and residual, sequential, or parallel internal nodes, varying processing depth, branching, and residual paths without changing the stage interface.The trees map to executable computation.Source: B. Compositional Decoder Topologies · p. 3
  • Fig. 1 traces a single source search through Pareto selection to the GeoBridgeNet-A3 topology, then into depth estimation and Gaussian reconstruction, where each target trains its own weights and prediction interface.The source environment uses TartanAir images and cached teacher-derived targets.Source: Figure 1 · p. 4

1) Stage-Wise Tree Structure:

  • Four functional trees align with decoder resolutions 1/32, 1/16, 1/8, and 1/4, transforming each stage’s fused feature while returning the same external interface.The multi-scale scaffold governs information flow across stages; each tree governs transformations within its stage.Source: 1) Stage-Wise Tree Structure · p. 4
  • The coarse stages T32 and T16 allow at most two composition levels, while T8 and T4 allow at most one, yielding 15,376 four-stage topologies before primitive configurations.The stated bounds allocate greater compositional capacity to coarse stages, where spatial processing is less expensive.Source: 1) Stage-Wise Tree Structure · p. 4

2) Composition Nodes:

GeoBridge searches both the composition of transformations within decoder stages and the configurations of their primitive operations. Its training-free GSAP evaluator ranks candidates using gradient statistics from a shared regression stimulus.

2) Composition Nodes:

  • RES adds an identity path, SEQ extends the processing path, and PAR aggregates branches; each composition preserves its children’s input and output shape.Shape preservation lets subtrees vary within a stage without additional interface matching.Source: 2) Composition Nodes · p. 4

3) Primitive Leaves:

  • Primitive leaves configure Mamba-style spatial operators, while internal nodes choose how leaf and subtree outputs are combined.Evolution selects both the elementary transformations and their organization; coarse stages allow deeper primitives and more scan patterns, while fine stages use lighter configurations.Source: 3) Primitive Leaves · pp. 4–5

C. GSAP for Efficient Candidate Evaluation

  • GSAP combines parameter-scaled gradient sensitivity with gradient–parameter alignment, averages responses across three independent initializations, then applies a logarithm.The search minimizes this score alongside computation and parameter costs; an averaged raw response no greater than 10^-12 receives a penalty of 30.Source: C. GSAP for Efficient Candidate… · pp. 5–6
  • The auxiliary objective gives candidates a common stimulus derived from a fixed source batch and frozen teacher features, producing dense predictions for comparison.The teacher-derived scalar target is broadcast across prediction resolution.Source: C. GSAP for Efficient Candidate… · p. 5
  • GSAP ranks each initialized decoder using its response to a fixed regression stimulus, avoiding candidate-specific training during evaluation.The images and teacher-derived targets are cached and reused, and gradients from the auxiliary loss provide the ranking statistics.Source: C. GSAP for Efficient Candidate… · p. 5

D. Evolving Decoder Topologies

GeoBridge evolves stage-internal decoder compositions while preserving the fixed scaffold’s interfaces. It evaluates candidates with a three-initialization GSAP proxy and resource costs, then retains non-dominated structures for downstream evaluation.

D. Evolving Decoder Topologies

  • Each individual is a four-stage tree, and evolution balances JGSAP against MACs and parameter count.The search uses non-dominated sorting and crowding distance to maintain proxy–cost trade-offs.Source: D. Evolving Decoder Topologies · pp. 6–7
  • Stage trees combine residual, sequential, and parallel nodes with configurable Mamba primitives while preserving scaffold interfaces.Depth limits and primitive domains vary by spatial scale, allowing more flexibility at coarser resolutions and constraining finer ones.Source: D. Evolving Decoder Topologies · p. 6
  • Candidates receive a log-transformed JGSAP score from the mean of three raw GSAP responses generated by independent decoder initializations.Each response uses a forward–backward pass on a common source batch; parameters are not updated.Source: D. Evolving Decoder Topologies · p. 6, Figure 4 · p. 7
  • Stage-aware crossover exchanges same-scale trees, while hierarchical mutation regenerates subtrees, changes compositions, or resamples primitives.Tournament selection chooses parents by non-dominated rank and crowding distance.Source: D. Evolving Decoder Topologies · p. 7
  • The final non-dominated front supplies GeoBridgeNet-A1, A2, and A3, representing low-cost, intermediate, and higher-capacity designs.The surviving population is scored again using the same three-initialization rule before the front is extracted.Source: D. Evolving Decoder Topologies · p. 7

E. Instantiation in Target Systems

Target instantiation retains selected stage computations and connects them to each system’s prediction interface. The targets train their own weights using task-specific procedures.

E. Instantiation in Target Systems

  • Depth estimation uses an adaptive depth-bin head and ground-truth supervision, with decoder weights initialized randomly.The reused design comprises topology and primitive configurations, not trained decoder weights.Source: E. Instantiation in Target Systems · p. 7
  • Gaussian NVS places the selected decoder in a DepthSplat-based monocular branch, followed by pseudo-depth distillation and differentiable-rendering optimization.The selected topology remains fixed through both stages; scene-level Gaussian refinement and COS3D semantic optimization assess downstream utility.Source: E. Instantiation in Target Systems · p. 7

IV. EXPERIMENTS

The experiments evaluate searched structures across depth estimation and Gaussian novel-view synthesis, with controlled studies of decoder designs, evolutionary representations, and candidate evaluation. They also examine the semantic utility of reconstructed scenes.

IV. EXPERIMENTS

  • Fig. 5 tracks JGSAP, MACs, and parameter count across a 100-generation main run using three-repeat GSAP aggregation.Its projections show the objective space and non-dominated boundary, while the hypervolume plot marks generations 29 and 82.Source: Figure 5 · p. 8
  • The hypervolume improves beyond the early search before approaching a late-stage plateau.The marked generations are 29 and 82.Source: Figure 5 · p. 8
  • The evaluation covers depth estimation on NYU Depth V2 and KITTI, Gaussian NVS on RealEstate10K, and semantic utility on LERF.The study also includes controlled comparisons of decoder designs, evolutionary representations, and candidate evaluation.Source: §IV · p. 8

A. Topologies Selected by Source Evolution

One source search produces a non-dominated set with distinct proxy–cost trade-offs. Its selected topologies vary in how they compose transformations within the common scaffold.

A. Topologies Selected by Source Evolution

  • The source run evolves 64 decoders for 100 generations on the TartanAir source batch, and its selected structures are reused in subsequent target systems.A1, A2, and A3 are selected from this run.Source: A. Topologies Selected by Source… · p. 8
  • The final population contains 26 non-dominated solutions, and normalized hypervolume improves beyond generations 29 and 82 before nearing a plateau.The objective-space projections show trade-offs among JGSAP, MACs, and parameter count.Source: A. Topologies Selected by Source… · p. 8
  • A3 uses three sequential primitives with residual refinement at s32, parallel composition at s16, a residual transformation at s8, and one primitive at s4.A1 and A2 mainly use individual primitives.Source: A. Topologies Selected by Source… · p. 8

B. Reuse Across Geometric Prediction Systems

GeoBridgeNet-A3 retains strong depth accuracy across real indoor and outdoor benchmarks and transfers to Gaussian novel-view synthesis, with task-specific target training. The transferred topology also supports visible detail recovery in the rendering pipeline.

1) Depth estimation across domains:

  • A3 achieves the best AbsRel, RMSE, and δ1 among four DINOv2-adapted baselines on NYU Depth V2 and KITTI; its AbsRel is 0.0876 on NYU and 0.0555 on KITTI.Metric3D v2 has lower latency, while A3 provides higher depth accuracy on both datasets.Source: 1) Depth estimation across domains · p. 8
  • Across A1 to A3, NYU AbsRel falls from 0.0918 to 0.0876 as latency rises from 36.9 to 40.7 ms; KITTI AbsRel falls from 0.0582 to 0.0555 as latency rises from 53.9 to 59.1 ms.The transferred variants provide different accuracy–cost profiles for deployment.Source: 1) Depth estimation across domains · p. 8
  • A3 preserves visible object boundaries and local depth transitions in the reported NYU examples of thin objects, cabinet edges, and a glass shower door, and KITTI examples of distant vehicles and roadside structures.The comparisons include IEBins, PixelFormer, and NeWCRFs.Source: 1) Depth estimation across domains · p. 8, Figure 7 · p. 9, Figure 8 · p. 10

2) From depth decoding to Gaussian reconstruction:

  • A3 reaches 27.11 dB PSNR on RealEstate10K, exceeding pixelSplat by 1.22 dB and MVSplat by 0.72 dB.At 62.0M parameters, A3 is close in PSNR to the ViT-S two-scale DepthSplat variant; A2 has the best SSIM and LPIPS within the GeoBridgeNet family.Source: 2) From depth decoding to Gaussian… · pp. 8–9
  • In a matched DepthSplat-based control, A3 reaches 27.11 ± 0.05 dB PSNR versus 27.03 ± 0.09 dB for DPT, while DPT has better mean SSIM and LPIPS and fewer parameters.A3 exceeds DPT in PSNR in all three paired runs, with margins of 0.13, 0.09, and 0.02 dB.Source: 2) From depth decoding to Gaussian… · p. 9
  • In the rendering-based pipeline, A3 recovers visible texture and edge structure in wall decoration, foliage behind a screen door, and bedding.The RealEstate10K comparison is against pixelSplat and MVSplat.Source: 2) From depth decoding to Gaussian… · p. 9, Figure 9 · p. 10

C. Decoder Design and Evolutionary Representation

The controlled studies compare decoder designs, topology-search controls, and tree-aware versus flat evolutionary representations. They report accuracy and structural-coverage differences after transfer, alongside trade-offs between metrics.

Comparison of Decoder Designs

  • On NYU Depth V2, A3 reaches mean AbsRel 0.0876±0.0002 across three seeds, below the searchable FPN, LocalMamba, DPT, and NeWCRFs comparisons.The decoders share DINOv2 ViT-S, encoder adaptation, and retraining protocol.Source: C. Decoder Design and Evolutionary… · p. 10

Topology Freedom and Evolutionary Selection

  • Joint topology evolution lowers RMSE by 0.0036 and raises δ1 by 0.0027 versus fixed-topology evolution at comparable MACs, with 15% more parameters.Against random sampling, it lowers AbsRel by 0.0009 and raises δ1 by 0.0032 with 16% fewer parameters.Source: C. Decoder Design and Evolutionary… · p. 11

Structural Exploration

  • The tree–flat exploration holds the topology grammar and search configuration fixed while changing encoding and variation operators: tree uses stage-aware crossover and hierarchical mutation, and flat uses one-point crossover and locus mutation.The comparison uses 15 paired runs.Source: C. Decoder Design and Evolutionary… · p. 11
  • Tree-aware evolution evaluates 839±17 genotypes and 276±21 four-stage topologies per run, versus 455±33 and 147±32 for flat encoding; topology entropy is 4.67 versus 4.17.Paired tests find significant differences in these coverage measures, but not in best GSAP or HV (p = 0.68 and p = 0.98).Source: C. Decoder Design and Evolutionary… · p. 11

Reuse of Selected Tree and Flat Structures

  • After transfer, tree-derived A3 has lower RMSE and higher δ1 on NYU while flat encoding has lower AbsRel; on KITTI, A3 is better on all three reported metrics.The comparisons use the same downstream protocols.Source: C. Decoder Design and Evolutionary… · p. 11
  • On RealEstate10K, the tree-derived structure improves PSNR by 0.18 dB, while flat encoding has slightly better SSIM and LPIPS.After LERF refinement, the tree initializer has 2.58 dB higher average PSNR and 0.80 percentage points higher semantic mIoU, but 4.35 points lower mAcc.Source: C. Decoder Design and Evolutionary… · p. 11

D. GSAP as a Search Signal

GSAP is evaluated as a low-cost ranking signal for decoder search, with ranking reliability and cross-task behavior examined alongside downstream use of transferred Gaussian representations. The reported LERF results show photometric refinement and open-vocabulary semantic evaluation.

D. GSAP as a Search Signal

  • Across 24 initialization seeds, GSAP's mean Spearman correlation between −JGSAP and retrained δ1 is 0.344, with 67% of draws significant at p < 0.05; it also has the highest Hit@10% and NDCG@10%.Its mean pairwise rank correlation across four source batches is 0.936, with a minimum of 0.893.Source: D. GSAP as a Search Signal · p. 12
  • GSAP takes about 3.3 s per candidate from three forward–backward measurements, and the 100-generation source search takes approximately 12 GPU hours.The 1,000-step short-training baseline has a Spearman correlation of −0.068 with retrained accuracy.Source: D. GSAP as a Search Signal · p. 12
  • On TransNAS-Bench-101, GSAP has the highest compared-proxy correlation for autoencoding and room-layout prediction, while GradSign leads on object classification and semantic segmentation.The study covers 7,352 benchmark architectures.Source: D. GSAP as a Search Signal · p. 12

E. Downstream Utility of the Transferred 3-D Representation

  • After 30,000 iterations of scene-level refinement, average PSNR for the A3-initialized LERF representation rises from 14.90 to 23.33 dB; vanilla 3DGS reaches 24.40 dB under the same budget.On Figurines, A3 reaches 24.29 dB versus 24.10 dB for vanilla 3DGS; on Ramen, it reaches 24.69 versus 24.75 dB.Source: E. Downstream Utility of the… · p. 12
  • In the COS3D semantic pipeline, the A3 initializer reports 50.54% mIoU and 71.32% mAcc, versus 50.76% and 72.08% for COS3D with kernel-regression inference.Among transferred structures, A1 has the highest mIoU and Acc@0.5, while A3 has the highest mAcc.Source: E. Downstream Utility of the… · p. 12
  • Five queried objects have IoUs between 86.3 and 97.6%, while the dark cup reaches 74.4% with scattered Gaussians around its rim and handle.The examples come from the A3-initialized representation.Source: E. Downstream Utility of the… · p. 12

V. CONCLUSION

GeoBridge makes stage-internal residual, sequential, and parallel computation searchable while preserving a multi-scale scaffold. The selected structures retain utility after target-specific training for depth estimation and Gaussian novel-view synthesis, and their reconstructed scenes support downstream semantic processing.

V. CONCLUSION

  • GeoBridge formulates decoder design as evolution of stage-internal topologies within a fixed multi-scale scaffold, using residual, sequential, and parallel composition while preserving stage interfaces.A functional-tree representation and GSAP support multi-objective evolution at low evaluation cost.Source: §V · p. 12
  • The selected structures retain utility after target-specific training for real-world depth estimation and Gaussian novel-view synthesis, and their reconstructed scenes support downstream semantic processing.The paper concludes that stage-internal topology is a reusable unit of decoder design.Source: §V · p. 13
  • Future work will examine which composition patterns account for transfer performance and how the source evaluation setting affects their selection.Source: §V · p. 13

Related papers

Loading 2610.15280v2…