RelateGS: Reliable Semantic and Relational Evidence for 3D Scene Understanding with Gaussian Splatting

• Relational expressions can leave same-category objects equally plausible, while weak semantic observations and limited viewpoint coverage can leave grounding evidence unreliable or incomplete.Source: §1 · pp. 1–2
• RelateGS selectively aggregates reliable Gaussian-level language evidence, consolidates object observations into a persistent graph, and re-queries informative existing frames for under-observed pairs.Source: §1 · p. 2
• RelateGS achieves competitive overall grounding performance on Sr3D+ and Nr3D, with semantic evaluations and controlled ablations further validating the representation design.Source: §1 · p. 2
• The approach treats scene-side evidence quality as central to relation-aware grounding and combines confidence-aware semantic uplifting with graph-guided relational refinement.Source: §1 · p. 2

2.1. Language-Embedded 3D Gaussian Splatting

• Language-embedded 3DGS methods [14] [18] [26] transfer language-aligned features to Gaussians, but observation reliability is not always explicitly considered during aggregation.Source: §2.1 · p. 2

2.2. 3D Scene Graph Generation

• 3DSSG [21], SceneGraphFusion [25], VL-SAT [23], SGAligner [16], and GaussianGraph [22] develop object-centric graphs, but relational evidence may remain incomplete.Source: §2.2 · p. 3

2.3. 3D Visual Grounding

• Relational grounding methods [20] use contextual, relational, or geometry-aware cues; RelateGS instead refines scene evidence before downstream reasoning [10] [22].Source: §2.3 · p. 3

3.1. Overview

• RelateGS constructs reliable Gaussian-level semantics and an initial object graph, then selectively re-queries informative observations to complement missing relational evidence.The refined graph is used for language grounding.Source: §3.1 · p. 3

3.2. Reliable Gaussian-Level Semantic Evidence

• Gaussian language features aggregate supporting-view observations, while rendering significance filters weak or occluded evidence before aggregation.The confidence signal comes from opacity contribution and accumulated transmittance; retained observations replace the original aggregation weights.Source: §3.2 · p. 3

3.3. Persistent Object-Level Relational Evidence

• Frame-level objects and relations are lifted and associated across views, then fused with geometric relations and filtered into a persistent object graph.Persistent identities are matched using spatial, semantic, and scale consistency; the graph can still lack evidence for insufficiently observed pairs.Source: §3.3 · pp. 4–5

3.4. Graph-Guided Relational Refinement

• The method identifies nearby object pairs with missing or low-confidence relations, selects a favorable existing frame, and re-queries it with a pair-specific prompt.New observations are processed through the initial graph-construction procedure and associated with the corresponding persistent pair.Source: §3.4 · p. 5

3.5. Relation-Aware 3D Grounding

• RelateGS combines language-aligned Gaussian features with persistent object identities and refined relational evidence to match referring expressions to objects.The graph matcher compares expressions with stored semantic attributes and relations.Source: §3.5 · p. 5

4.1. Experimental Setup

• The evaluation uses Sr3D+ and Nr3D grounding benchmarks, plus semantic segmentation on Replica and ScanNet and open-vocabulary localization on four LERF-OVS scenes.The grounding comparison uses the same eight ScanNet scenes as prior structured 3D representations.Source: §4.1 · p. 5, §4 · p. 5

4.2. 3D Visual Grounding

• 22.2/15.3 Acc@0.1/Acc@0.25 on Sr3D+ and 21.7/10.3 Acc@0.1/Acc@0.25 on Nr3D; RelateGS exceeds GaussianGraph on both benchmarks.On Sr3D+, the respective GaussianGraph scores are 18.2/7.4; on Nr3D, they are 17.2/8.6.Source: Table 1 · p. 6, Table 2 · p. 6, §4.2 · p. 6

4.3. Evaluation of Semantic Evidence

• RelateGS remains competitive on LERF-OVS, with best results across all reported metrics on ramen and mixed performance on figurines.It also matches the best localization accuracy on teatime and reports strong Acc@0.5 and mIoU on waldo kitchen.Source: §4.3 · p. 7, Table 4 · p. 7

4.4. Ablation Studies

• With the same additional VLM query budget, graph-guided re-querying reaches 22.22/15.28 Acc@0.1/Acc@0.25, while random frame re-querying falls to 19.78/11.62.Confidence-aware observation selection also raises Acc@0.1 from 21.34 to 21.72 and Acc@0.25 from 12.12 to 13.10.Source: §4.4 · p. 7, Table 5 · p. 7

4.5. Qualitative Results

• On ScanNet, the full model more reliably resolves relational expressions than language-embedded Gaussian representations or RelateGS without re-querying.Targeted re-querying complements under-observed relations, helping the same downstream matcher recover correct targets in challenging cases.Source: Figure 3 · p. 8, §4.5 · p. 8
• On LERF-OVS, confidence-aware uplifting suppresses weak multi-view observations and concentrates language responses around the referred object.The comparison visualizes activation maps and localized regions for the same query.Source: Figure 4 · p. 8

• RelateGS suppresses weak multi-view observations during semantic feature uplifting and uses a persistent object graph to identify and complement under-observed relations through targeted re-querying of informative existing frames.Source: §5 · p. 8
• Experiments on Sr3D+ and Nr3D demonstrate the effectiveness of RelateGS for 3D visual grounding.Source: §5 · p. 8
• Semantic segmentation and open-vocabulary localization further validate the underlying language-aware Gaussian representation.Source: §5 · p. 8