PaperlayerView exact revision on arXiv

Source-linked AI summary

PARC-Loc: Text-to-Point-Cloud Localization with Partial Assignment and Relational Consistency

Shengkai Ma, Zhenyu Hou, Weihua Cao

arXiv:2610.09761v1cs.CVcs.AIcs.CL

TL;DR

Coarse-to-fine text-to-point-cloud localization can select submaps with relationally inconsistent matches and leave out query-relevant evidence across submap boundaries. PARC-Loc applies partial assignment with relational consistency to candidate selection and expanded-context localization. On KITTI360Pose, it improves Top-1@5m recall from 0.50 to 0.67, a 34% relative gain over MNCL.

  • Problem

    Aggregate query–submap similarity does not explicitly verify relational consistency, and a selected submap may omit query-relevant instances distributed across neighboring cells.

    Source: §1 · p. 1

  • Method

    PARC jointly evaluates attribute compatibility and pairwise spatial consistency under partial correspondence, complementing neural similarity for coarse selection and producing object weights for expanded-context localization.

    Source: §1 · p. 2, §4.2 · p. 4, §4.3 · p. 5

  • Results

    Top-1@5m localization recall improves from 0.50 to 0.67 on KITTI360Pose, a 34% relative improvement over MNCL.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A single framework addresses candidate-level layout verification and fine localization with cross-boundary context, with CityLoc and gallery-growth tests also supporting transfer and robustness.

    Source: §6 · p. 8

    Coarse selection should validate matched instances as a joint layout, while fine localization should preserve the selected coordinate frame when admitting relevant cross-boundary evidence.

    Source: §3.2 · p. 3

Abstract

from arXiv · show

Text-to-point-cloud localization estimates a position in a city-scale 3D map from descriptions of surrounding objects. Existing coarse-to-fine methods retrieve submaps using aggregate learned compatibility and then localize within a selected submap. However, repetitive or similar urban objects can inflate the embedding similarity between the query and multiple submaps, even when the instance layout within a submap violates the query description. Meanwhile, query-relevant instances often span submap boundaries, leaving the retrieved submap with incomplete contextual evidence. We term these failure modes layout-inconsistent aliasing and boundary evidence incompleteness, respectively. To address them, we propose PARC-Loc, a coarse-to-fine localization framework built on Partial Assignment with Relational Consistency (PARC). PARC jointly models hint-object compatibility and pairwise spatial relations, allowing unmatched elements while favoring assignments consistent with the queried layout. At the coarse stage, its candidate-level assessment complements neural similarity for layout-consistent submap selection. At the fine stage, the context is expanded with query-relevant instances from adjacent submaps, while PARC yields object-level matching weights that guide cross-modal attention. Extensive experiments on KITTI360Pose and CityLoc show that PARC-Loc outperforms conventional coarse-to-fine baselines. On KITTI360Pose, our method improves Top-1 localization recall at 5 m from 0.50 to 0.67, achieving a 34% relative gain over the strongest baseline.

1 Introduction

PARC-Loc addresses layout-inconsistent aliasing in coarse retrieval and incomplete boundary context in fine localization with partial assignment and relational consistency. It applies PARC at both stages and reports improved KITTI360Pose recall alongside competitive CityLoc results.

1 Introduction

  • Aggregate query–submap similarity can favor candidates with individually plausible matches even when their joint layout violates the query relations.The pipeline also treats a selected submap as a complete evidence domain, although query-relevant instances may be distributed across neighboring submaps.Source: §1 · p. 1
  • Fine localization can lose relevant context when query-relevant instances span neighboring submaps, while added neighboring objects may also introduce relationally inconsistent distractors.The selected submap remains the coordinate anchor as relevant context is selectively incorporated.Source: §1 · pp. 1–2
  • PARC jointly evaluates hint–object attribute compatibility and pairwise spatial consistency while allowing missing or irrelevant elements to remain unmatched.At coarse selection it complements neural retrieval scores; at fine localization it provides object-level weights that bias cross-attention toward query-relevant context from neighboring cells.Source: §1 · p. 2
  • Top-1@5m localization recall rises from 0.50 to 0.67 on KITTI360Pose, a 34% relative improvement over MNCL.CityLoc-K, CityLoc-C, and controlled gallery-growth experiments further assess transfer and robustness.Source: §1 · p. 2
  • The paper diagnoses two structural failures in coarse-to-fine localization and proposes Layout-Consistent Submap Selection and Cross-Boundary Context Localization to address them.These are the framework’s named mechanisms for candidate-level layout verification and expanded-context fine localization.Source: §1 · p. 2

2 Related Work

Prior text-to-point-cloud localization work improves spatial representations, cross-modal alignment, and handling of partial correspondence or uncertainty. PARC-Loc differs by validating joint layouts during coarse selection and extending fine-stage evidence beyond the selected submap boundary.

2 Related Work

  • Existing T2P methods strengthen representation, training, spatial descriptors, or retrieval, while PARC-Loc validates joint layouts during coarse selection and extends fine-stage evidence beyond submap boundaries.The related work covers Text2Pos, RET, Text2Loc, MNCL, CMMLoc, PMSH, SpatiaLoc, SympLoc, Vehicle-SceneInteraction, and VLM-Loc.Source: §2 · p. 2
  • Unlike referred-object grounding, T2P localization may target a position inferred from surrounding objects rather than an observed instance, motivating joint hint–object validation.Prior grounding work spans indoor benchmarks, city-scale point clouds, and methods for relation modeling and spatial–language alignment.Source: §2 · p. 2

3 Preliminaries and Pipeline Diagnosis

The standard coarse-to-fine pipeline ranks submaps by aggregate similarity and localizes within one selected submap, without explicitly verifying joint layout or complete context. Pipeline analyses on KITTI360Pose identify partial-match aliasing and boundary-truncated evidence as consequential limitations.

3.1 Text-to-Point-Cloud Localization

  • Text-to-point-cloud localization estimates a target position from hints describing relations among surrounding objects, while common pipelines rank submaps by query similarity before localizing within one selected submap.The pipeline does not explicitly verify full query–instance layout consistency or account for context omitted by a submap’s fixed extent.Source: §3.1 · p. 3

3.2 Diagnosis of Pipeline Limitations

  • The diagnosis uses released Text2Loc, CMMLoc, and MNCL models with official checkpoints on KITTI360Pose.Figure 2 compares standardized score changes and promotion rates for relation-satisfied and relation-violated placements.Source: §3.2 · p. 3, Figure 2 · p. 3
  • Partial matches raise candidate scores and Top-1 promotion rates even when added instances violate queried relations, demonstrating layout-inconsistent aliasing.The analysis tests 911 partially matched candidates with one to six exact semantic matches; relation-satisfying placements provide an additional gain.Source: §3.2 · p. 3
  • 40.28% of instances are boundary-truncated, 64.49% of hints refer to them, and 99.75% of queries contain at least one such hint.Nearby non-ground-truth submaps account for 46.8–59.2% of Top-1 retrieval errors across three baselines.Source: §3.2 · p. 3, Table 1 · p. 3
  • The pipeline diagnoses motivate joint semantic and spatial compatibility under partial correspondence, with coarse candidate assessment and fine-stage context routing as distinct requirements.Coarse selection should validate a joint layout; fine localization should preserve the selected coordinate frame while admitting relevant cross-boundary evidence.Source: §3.2 · p. 3

4 Method

PARC-Loc uses partial assignment with relational consistency to assess candidate submaps, then expands fine-stage context across neighboring submaps and applies assignment weights in cross-attention before offset prediction.

4.1 Overview

  • PARC combines candidate-level assignment assessment with neural similarity to select a submap, then uses its frame as the anchor for fine localization.The fine stage reuses PARC after expanding the available context to neighboring objects.Source: §4.1 · pp. 3–4, Figure 3 · p. 4

4.2 Partial Assignment with Relational Consistency (PARC)

  • PARC combines attribute compatibility and pairwise layout consistency in a partial assignment that can leave missing hints and irrelevant objects unmatched.Its assignment matrix represents soft correspondence between query hints and map instances.Source: §4.2 · p. 4

4.3 Coarse Stage: Layout-Consistent Submap Selection

  • For each neural retrieval candidate, PARC scores whether matched objects form a coherent layout; combining this score with neural similarity suppresses layout-inconsistent aliases.The normalized PARC score is high when unary matches are low-cost and preserve the joint layout.Source: §4.3 · p. 5

4.4 Fine Stage: Cross-Boundary Context Localization

  • Neighboring-object aggregation expresses nearby instances in the selected-submap frame and prioritizes them by query compatibility and spatial context.This makes cross-boundary evidence available without assuming every retained object is relevant.Source: §4.4 · p. 5
  • PARC-derived object weights bias hint-to-object cross-attention, after which the fine decoder predicts a normalized offset in the selected-submap frame.The matching bias is additive in log space; setting β to zero recovers standard content-based attention.Source: §4.4 · p. 5

5 Experiments

Experiments evaluate PARC-Loc on KITTI360Pose and CityLoc using retrieval and localization metrics, ablations, transfer tests, gallery-growth tests, and qualitative cases. The reported results cover overall localization, stage-specific mechanisms, and robustness.

5.1 Experimental Setup

  • The official KITTI360Pose protocol uses five training scenes, one validation scene, three test scenes, and 30 m cubic submaps at a 10 m stride.Submap retrieval reports top-1/3/5 recall; fine localization reports recall within 5, 10, and 15 m for top-1/5/10 candidates.Source: §5.1 · p. 5

5.2 Quantitative Analysis

  • 0.71 validation and 0.67 test Top-1@5 m recall exceed MNCL by 17 percentage points on both KITTI360Pose splits.PARC-Loc obtains the best recall across all reported metrics on both splits.Source: §5.2 · p. 6, Table 2 · p. 6
  • PARC-Loc exceeds the strongest baseline in submap retrieval by 4–5 percentage points on validation and 6–7 points on test.This metric credits only the annotated ground-truth submap, although a neighboring non-ground-truth submap may support a correct final prediction with cross-boundary context.Source: §5.2 · p. 6, Table 3 · p. 6

5.3 Ablation Studies

  • Full PARC raises Top-1 retrieval recall from 0.49 to 0.54 on validation and from 0.46 to 0.51 on test.Removing partial assignment causes the largest test decline, to 0.48; removing relational consistency yields 0.50.Source: §5.3 · p. 6, Table 4 · p. 6
  • Coarse PARC raises test Top-1@5 m recall from 0.51 to 0.61; adding context, prioritization, and matching bias successively raises it to 0.64, 0.66, and 0.67.The reported order-conditioned gains attribute the largest increase to coarse PARC, with context construction and routing adding a further 6 percentage points.Source: §5.3 · p. 7, Table 5 · p. 7
  • Expanded contexts improve the strict 5 m endpoint over a single-submap input by 5.52 and 4.46 percentage points, while the larger distance-prioritized neighborhood does not outperform the five-cell context.Across three matched epoch-3 seeds, the β = 0.3 condition has higher mean recall at every reported candidate rank; the authors characterize this as a limited stability check.Source: §5.3 · p. 7, Table 6 · p. 7, Table 7 · p. 7

5.4 Further Analysis

  • On CityLoc-K, PARC-Loc achieves best R@15 m at 80.47% validation and 77.64% test, while VLM-Loc leads at stricter thresholds; on CityLoc-C, PARC-Loc achieves best R@15 m, 73.57%, in zero-shot transfer.With 2,373 fixed queries, PARC-Loc retains 96.7% of Top-1 recall at 500 m, decreasing from 54.11% to 52.34%, versus 89.7% for MNCL, 83.8% for Text2Loc, and 84.4% for CMMLoc.Source: §5.4 · p. 7

5.5 Qualitative Analysis

  • In two fixed cases, PARC replaces a distant layout-inconsistent neural Top-1 alias with a candidate within 15 m, while 3×3 cross-boundary context reduces localization error.These cases illustrate the mechanisms but do not estimate how often either corrects an error.Source: §5.5 · p. 8, Figure 4 · p. 8

6 Conclusion

PARC-Loc addresses layout-inconsistent aliasing and boundary evidence incompleteness in coarse-to-fine text-to-point-cloud localization. It applies partial assignment with relational consistency at coarse selection and fine localization, and achieves 0.67 Top-1@5m test recall on KITTI360Pose.

  • PARC-Loc addresses layout-inconsistent aliasing and boundary evidence incompleteness in coarse-to-fine text-to-point-cloud localization.Source: §6 · p. 8
  • PARC evaluates attribute and relational compatibility under partial correspondence, complementing neural similarity for coarse selection and producing object-level weights from cross-boundary context for localization in the selected-submap frame.Source: §6 · p. 8
  • PARC-Loc achieves 0.67 Top-1@5m test recall on KITTI360Pose, with ablations, CityLoc transfer, and gallery-growth tests supporting its effectiveness, transferability, and distractor robustness.Source: §6 · p. 8
Loading 2610.09761v1…