PaperlayerView exact revision on arXiv

Source-linked AI summary

HiCo-GS: Hierarchical Context Aggregation and Geometric Consistency for Octree Gaussian Splatting

Wei Zhang, Shengkai Yu, Shiqiang Gong, Qi Zhang, Qiang Li, Qi Wang

arXiv:2608.14136v1cs.CVcs.AI

TL;DR

Octree Gaussian Splatting lacks communication between hierarchical levels and geometric supervision, limiting high-fidelity urban reconstruction. HiCo-GS addresses these gaps with bidirectional context aggregation and depth-normal consistency regularization, achieving state-of-the-art rendering and cleaner geometry across urban benchmarks.

  • Problem

    Existing octree-anchor methods isolate features across levels and lack geometric supervision, limiting coordinated detail representation and normal consistency in urban reconstruction.

  • Method

    HiCo-GS combines bidirectional Cross-Level Context Aggregation with Depth-Normal Geometric Consistency regularization to improve hierarchical features and surface geometry.

  • Results

    Across China-Pagoda, HiCo-GS achieves 22 of 24 first-place rankings, while attaining state-of-the-art performance across all four Mill19 and UrbanScene3D scenes and metrics.

  • Takeaways & Limitations

    HiCo-GS produces more detailed, cleaner surfaces with smoother planar normals, sharper structural edges, and fewer floating artifacts than baseline approaches.

Abstract

from arXiv · show

Octree-based anchor Gaussian Splatting has emerged as a scalable representation for city-scale novel view synthesis, where multi-level anchors adaptively capture scene content from coarse building structures to fine architectural details. However, we identify a fundamental limitation in existing methods: cross-level feature isolation, where each level's anchor features are optimized independently with no inter-level communication, causing color drift on building facades and over-smoothing in textured regions. We present HiCo-GS, a high-fidelity reconstruction framework with two complementary modules. Cross-Level Context Aggregation (CLCA) enables bidirectional hierarchical prior injection by leveraging the octree's spatial containment structure to aggregate per-level context vectors into parent-self-child triplets, fused via a lightweight MLP with residual connection. Coarse-level structural priors flow down to inform fine-level anchors, while fine-level detail statistics feed back to prevent over-smoothing, at negligible computational overhead. Depth-Normal Geometric Consistency (DNGC) regularization enforces agreement between rendered normals and depth-derived normals through an alpha-weighted consistency loss, complemented by edge-aware smoothness losses with progressive warmup that exploit the strong planar priors ubiquitous in urban geometry to suppress floating artifacts. We further introduce the China-Pagoda dataset comprising 8 ancient Chinese pagodas with over 1,200 images each, featuring dense ornamental carvings, curved multi-layer eaves, and repetitive fine-grained textures. Extensive experiments on Mill19, UrbanScene3D, MatrixCity, and China-Pagoda demonstrate that HiCo-GS achieves state-of-the-art rendering quality and substantially cleaner geometry across real-world and synthetic urban benchmarks.Code: https://github.com/WZ-CS/HiCo-GS.

1 Introduction

HiCo-GS addresses cross-level feature isolation and geometric inconsistency in octree-based Gaussian Splatting through bidirectional hierarchical context aggregation and depth-normal regularization. Together, these mechanisms target missing inter-level communication, noisy normals, depth discontinuities, and floating artifacts in urban scenes.

  • Problem: Existing octree-anchor methods optimize each level independently, preventing fine anchors from accessing coarse structural context and coarse anchors from sensing local detail density.This cross-level isolation leaves features without inter-level communication.
  • Problem: Inter-level cosine distances approaching 0.97 indicate near-orthogonal, structureless mixtures among independently optimized level features.The t-SNE visualization confirms the lack of meaningful cross-level feature organization.
  • DNGC: Geometric inconsistency in planar urban scenes produces noisy normals, depth discontinuities, and floating artifacts, motivating DNGC depth-normal consistency regularization.The contribution list identifies DNGC as depth-normal geometric consistency regularization.
  • CLCA: CLCA uses octree spatial containment to aggregate each anchor’s parent context with child features, enabling bidirectional communication across levels.Fine-level features receive structural priors, while coarse-level features receive detail statistics.

2 Related Work

Prior work established 3D Gaussian Splatting as a differentiable-rasterization representation and extended it for compactness, anti-aliasing, appearance modeling, and city-scale scenes. HiCo-GS builds upon anchor-based neural Gaussians introduced by Scaffold-GS.

  • 3D Gaussian Splatting and Large-Scale Extensions: 3DGS represents scenes with anisotropic Gaussian primitives rendered through differentiable rasterization.This work forms the foundation for subsequent improvements and large-scale extensions.
  • 3D Gaussian Splatting and Large-Scale Extensions: Subsequent 3DGS methods improve compactness, anti-aliasing, and appearance modeling.The passage cites multiple representative works across these three directions.
  • 3D Gaussian Splatting and Large-Scale Extensions: City-scale scene reconstruction has been addressed through spatial partitioning and hierarchical LOD.These strategies target scalability beyond individual scenes.
  • 3D Gaussian Splatting and Large-Scale Extensions: Scaffold-GS introduces anchor-based neural Gaussians, which HiCo-GS builds upon.The passage identifies Scaffold-GS as the anchor-based foundation for the presented work.

3 Method

HiCo-GS enhances octree Gaussian Splatting with bidirectional parent-self-child feature aggregation and depth-normal geometric regularization. Its pipeline combines hierarchical feature refinement, shared MLP Gaussian prediction, differentiable rasterization, and progressive geometric-loss warmup.

  • Depth-Normal Geometric Consistency (DNGC): DNGC regularizes agreement between rendered normals from Gaussian covariance and depth-derived normals from the rendered depth map.Differentiable rasterization produces RGB, opacity, depth, and both independent normal estimates used by the geometric regularization.
  • Pipeline: HiCo-GS constructs a multi-level octree from sparse SfM points, filters visible anchors by level of detail and view frustum, then predicts neural Gaussians with shared MLPs.The MLPs use enhanced anchor features, viewing direction, and optionally per-camera appearance embeddings to predict opacity, covariance, and color.
  • Cross-Level Context Aggregation (CLCA): CLCA injects parent, self, and child context into each visible anchor, enabling coarse structural priors and fine detail statistics to flow bidirectionally across octree levels.Parent context captures wall orientation, facade material, and building-scale color, while child context captures local texture density and micro-geometry.
  • Cross-Level Context Aggregation (CLCA): CLCA uses one-hop spatially contained parent-self-child triplets, fallback self-context at hierarchy boundaries, and a residual MLP with architecture 3d→d→d for d=32.Hash-based lookup has O(Nlog M) complexity, and the enhancement MLP adds 3,168 parameters.
  • Depth-Normal Geometric Consistency (DNGC): Edge-aware normal and distance smoothness preserve planar-region coherence while allowing boundary discontinuities and penalizing distance jumps associated with floating artifacts.The geometric terms use RGB-derived edge weights and a linear warmup so noisy early depth and normal estimates do not hinder RGB convergence.

4 Experiments

Experiments show that HiCo-GS achieves state-of-the-art rendering quality across standard urban benchmarks and delivers especially strong gains on the geometrically complex China-Pagoda dataset. Its DNGC regularization also improves normal smoothness on planar surfaces while preserving structural edges.

  • Novel View Synthesis: HiCo-GS achieves state-of-the-art performance across all four Mill19 and UrbanScene3D scenes and all evaluated metrics.The method outperforms reconstruction methods with and without geometric optimization, including the direct CityGS-X baseline.
  • Novel View Synthesis: HiCo-GS produces smoother normals on planar walls while preserving sharp transitions at structural edges, unlike noisy baseline normal maps.This comparison validates the effectiveness of DNGC regularization, particularly on building facades and other large planar regions.
  • China-Pagoda Benchmark: 22 out of 24 first-place results are achieved across all eight China-Pagoda scenes, with gains larger than on standard urban benchmarks.On Yuhuangta, HiCo-GS gains +6.62 dB PSNR and reduces LPIPS by 0.105 over CityGS-X; on Lingshan it reaches 32.95 dB PSNR and 0.965 SSIM.
  • Ablation Study: +5.18 dB is the largest single-scene improvement delivered by DNGC on Rubble, demonstrating its effectiveness for geometrically coherent surfaces.The result appears in the setting where multi-scale structural context is critical.

5 Conclusion

HiCo-GS is a high-fidelity reconstruction framework addressing two complementary limitations in octree-based urban scene reconstruction. It combines bidirectional hierarchical context flow with rendered/depth-derived normal agreement to improve feature representation and geometric consistency.

  • HiCo-GS addresses two complementary limitations in octree-based urban scene reconstruction with a high-fidelity reconstruction framework.
  • Cross-Level Context Aggregation: CLCA resolves cross-level feature isolation by using the octree’s spatial containment structure for bidirectional hierarchical context flow.It enriches fine-level features with structural priors and coarse-level features with detail statistics.
  • Depth-Normal Geometric Consistency: DNGC enforces agreement between rendered and depth-derived normals through edge-aware smoothness.

Supplementary File … B Adjacent-Level Distance Reduction

The supplementary analysis validates cross-level feature isolation across four fully converged urban-scene models and shows that CLCA chiefly reduces adjacent-level cosine distances at the finest octree levels. Baseline features are nearly orthogonal across levels, with isolation worsening toward finer depths.

  • Supplementary File: The analysis covers Building, Rubble, Residence, and SciArt, using fully converged models evaluated at 100K iterations.The scenes contain 8, 5, 6, and 7 octree levels, respectively.
  • A Cross-Level Cosine Distance Matrix: Anchor features and octree levels are analyzed through mean pairwise cosine distances for every level pair.For each model, the analysis extracts all anchor features and their corresponding octree levels.
  • A Cross-Level Cosine Distance Matrix: A cosine distance of 1.0 denotes perfect orthogonality between feature representations.This interpretation provides the basis for assessing cross-level feature incoherence.
  • A Cross-Level Cosine Distance Matrix: Baseline off-diagonal distances span 0.94–1.00 across all four scenes, indicating near-orthogonality between different octree levels.The result holds regardless of the anchors’ spatial relationship.
  • A Cross-Level Cosine Distance Matrix: In Building, adjacent-level distances rise from 0.949 at L0↔L1 to 0.999 at L6↔L7, worsening monotonically with level depth.The finest levels therefore exhibit the strongest measured isolation in this scene.
  • B Adjacent-Level Distance Reduction: CLCA reductions concentrate on the finest adjacent-level pairs: Building L5↔L6 falls 65.5% and L6↔L7 falls 25.0%.Earlier Building pairs, from L0↔L1 through L4↔L5, change by ≤3.9%.

C Per-Pair PCA Analysis

Per-pair PCA isolates adjacent octree levels to expose cross-level feature structure that global dimensionality reduction obscures. In the Building scene, CLCA separates coarse-to-fine features at the transition while leaving coarse pairs largely unchanged.

  • Analysis protocol: Per-pair PCA fits a shared 2D projection for each adjacent level pair after subsampling 3,000 anchors per level and annotates mean-feature cosine similarity.This avoids domination by the far more numerous coarse-level anchors.
  • Building scene results: In the baseline, most adjacent-level pairs form overlapping point clouds with cos_sim ≥0.78, indicating diffuse distributions without level-specific organization.The result is reported for the Building scene.
  • Building scene results: After CLCA, coarse pairs remain unchanged, while the L4-versus-L5 transition forms clearly separated clusters as cos_sim drops from 0.979 to 0.066.The two levels’ feature distributions become distinctly organized at the coarse-to-fine transition.
  • Building scene results: At the L4-versus-L5 transition, PC1 explained variance rises from 16.8% to 33.1% after CLCA.This accompanies the emergence of separated clusters.

D t-SNE Feature Visualization

Across all four scenes, baseline features form an unstructured embedding with levels uniformly intermixed, whereas CLCA produces a consistent hierarchical layout separating fine-level anchors into tight peripheral clusters around a shared coarse-level core.

  • Visualization setup: t-SNE uses stratified sampling across all levels, perplexity 30, and 1,000 iterations to visualize feature spaces across four scenes.Fine levels contain fewer anchors than coarse levels, motivating stratified sampling for adequate representation.
  • Baseline embedding: Baseline embeddings show a single diffuse blob in every scene, with all levels uniformly intermixed and no level-specific clustering.This visualizes the absence of organized inter-level structure and reflects near-orthogonality in cosine distance matrices.
  • CLCA embedding: After CLCA, coarse-level anchors form a shared core while fine-level anchors separate into distinct, tight peripheral clusters.The hierarchical layout remains consistent across scenes with varying depth and complexity.
  • Interpretation: The hierarchical organization emerges from rendering loss, with CLCA supplying shared context that enables coherent fine-level groups and scale-dependent attribute prediction.Separating fine and coarse levels gives the shared MLP decoder clearer scale-dependent signals.
  • Interpretation: CLCA transforms the feature space from uninformative orthogonality into functional specialization, as shown by the transition from a diffuse blob to hierarchical organization.This transition is presented as direct visual evidence of organized inter-level structure.

II Additional DNGC Ablation Results

Additional ablations qualitatively evaluate DNGC regularization on Building and Residence scenes by comparing normal and depth maps with and without DNGC.

  • II Additional DNGC Ablation Results: Qualitative ablations on Building and Residence compare normal maps and depth maps with versus without DNGC regularization.The comparisons are presented in Figs. 6 and 7.

III China-Pagoda Dataset Details · IV Large-Scale Urban Scene Comparison

The China-Pagoda dataset contains eight richly detailed ancient pagodas captured from over 1,200 viewpoints each, supporting reconstruction evaluation across complex architectural structures and textures. Large-scale comparisons show that HiCo-GS improves hierarchical feature organization, geometric coherence, and full-scene reconstruction quality.

  • III China-Pagoda Dataset Details: The China-Pagoda dataset contains 8 ancient Chinese pagodas, each captured with over 1,200 drone and ground-level images.The scenes include dense ornamental carvings, curved multi-layer eaves, repetitive brick textures, and weathered stone surfaces.
  • IV Large-Scale Urban Scene Comparison: 28.8% distance reduction occurs for the finest level pair L3↔L4 on Rubble, while coarse feature pairs remain stable after CLCA.Per-pair PCA shows selective organization of fine-level features without disrupting coarse-level representations.
  • IV Large-Scale Urban Scene Comparison: Up to 40.1% adjacent-level distance reduction occurs at Residence’s finest pair L4↔L5, with minimal change among coarse pairs.The adaptive coupling pattern is consistent with the other scenes.
  • IV Large-Scale Urban Scene Comparison: SciArt shows the largest fine-level reduction, −62.9% for L5↔L6, consistent with stronger cross-level communication for higher geometric complexity.Across scenes, t-SNE visualizations show fine-level features separating into peripheral clusters while coarse levels share a common core.
  • IV Large-Scale Urban Scene Comparison: With DNGC, rendered normals become consistent on planar rooftops and walls, while depth maps show smooth gradients, sharp boundaries, and fewer floating artifacts.Without DNGC, normals are noisier and depth maps contain discontinuities around facades and ground planes.
  • III China-Pagoda Dataset Details: Sample images span octagonal and hexagonal pagodas, tiered eaves, hanging bells, intricate brick carvings, arched doorways, and weathered repetitive surfaces.These architectural and textural variations create challenging reconstruction conditions across multiple scales.
  • III China-Pagoda Dataset Details: Reconstructions faithfully recover multi-layer eaves, symmetry, curved structures, ornamental details, rooftop geometry, and clean surface normals across the pagoda scenes.Examples include Duobaota, Yunjusi, Lingshan, Baoanta, and Beita, with meshes and top-down views demonstrating structural fidelity.
  • IV Large-Scale Urban Scene Comparison: HiCo-GS produces consistent facade coloring, clean planar surfaces, well-defined building edges, and smooth ground planes compared with CityGS-X.CityGS-X exhibits more floating artifacts, noisier mesh geometry, and surface noise around vegetation, overpasses, sports fields, and rooftops.
Loading 2608.14136v1…