PaperlayerView exact revision on arXiv

Source-linked AI summary

HF-SID: High-Fidelity Semantic IDs for Generative Retrieval in Location-Based Services

Haowen Lin, Jing Li, Zhibin Hao, Fangye Wang, Lihui Su, Song Yang, Xiaojiang Zhou, Pengjie Wang

arXiv:2608.30479v1cs.IR

TL;DR

HF-SID targets fidelity losses in POI Semantic IDs that affect geographic, numerical, and hierarchical distinctions in LBS retrieval. It restores these signals before quantization through continuous coordinate and numerical representations, LLM continual pre-training, and residual structural contrastive learning. The resulting 3-token IDs reduce average intra-SID geographic distance by 95.9%, improve Hit@200 by 3.66 points, and deliver online conversion gains without extra decoding cost.

  • Problem

    POI Semantic IDs can lose geographic, scale-aware numerical, and hierarchical information because LLMs encode coordinates discontinuously, mix differently scaled attributes, and rely on surface text.

  • Method

    HF-SID restores fidelity before quantization using 3D Cartesian coordinates, unified numerical units with type-aware embeddings, Geo-CPT, Num-CPT, and residual structure-based contrastive learning.

  • Results

    95.9% lower average intra-SID geographic distance and a 3.66-point Hit@200 improvement are reported with a 3-token SID and no added decoding cost.

  • Takeaways & Limitations

    HF-SID improves offline SID quality and online business metrics while keeping identifier length unchanged.

  • Takeaways & Limitations

    Public datasets lacking dynamic numerical attributes or fine-grained category structure support only Geo-CPT evaluation.

Abstract

from arXiv · show

Generative retrieval has attracted increasing attention in Location-Based Services (LBS), where each Point-of-Interest (POI) is represented as a Semantic ID (SID). As the SID is the only channel through which POI information reaches the generative model, whatever it fails to preserve is irrecoverable at decoding time, and LBS retrieval is especially sensitive to the fine-grained differences that existing SIDs blur. Specifically, (1) LLMs embed continuous coordinates discontinuously, so their numeric differences do not reflect true geographic distance; (2) dynamic numerical attributes differ vastly in scale, so an identical gap may be decisive for one attribute yet negligible for another; and (3) short text cannot convey hierarchical affiliation, as text-similar POIs may belong to different hierarchies. We therefore propose HF-SID, which restores geographic, numerical, and structural fidelity at the representation stage, before any information is committed to a discrete code. It transforms coordinates into a continuous 3D Cartesian form and encodes each numerical value as a single unit, consolidated inside the LLM by Geo-CPT and Num-CPT with type-aware embeddings; a Structure-based Contrastive Learning objective, applied only to the last-layer residual, then separates co-located POIs that share a coarse tag but differ at the fine level. Because these mechanisms enrich the representation rather than lengthen the identifier, HF-SID uses a 3-token SID at no extra decoding cost. On a large-scale industrial

1 Introduction

HF-SID addresses three fidelity gaps in POI Semantic IDs—geographic, numerical, and hierarchical—at the representation stage before quantization. It preserves these signals without increasing the identifier length and improves offline and online retrieval outcomes.

  • Motivation: Existing POI SID methods lose geographic, numerical, and structural fidelity before quantization, while longer identifiers increase autoregressive decoding cost.These methods intervene after coordinate precision is already lost and leave numerical attributes and structure to plain text.
  • Challenges: A 0.1° coordinate difference can carry substantial business impact, but LLMs encode coordinates discontinuously and grid boundaries can separate geographically adjacent POIs.Longitude differences also represent different physical distances across latitudes.
  • Challenges: A 0.5 gap is meaningful for ratings on a 0–5 scale but negligible for visit counts on a 0–120K scale, exposing the need for type-aware numerical representations.Prior work serializes such attributes as plain text and forces them into one shared numeric subspace.
  • Challenges: Textually similar POIs can share a coarse tag yet differ in fine-grained hierarchy, as with Apple Store and Apple Market.Geographic proximity likewise cannot distinguish heterogeneous shops within a business district.
  • Approach: HF-SID restores geographic, numerical, and structural fidelity before discrete coding through Cartesian coordinates, numerical encoding, Geo-CPT, Num-CPT, and structure-based contrastive learning.The contrastive objective refines fine-level distinctions while preserving coarser geographic information.
  • Results: 95.9% lower average intra-SID geographic distance and +3.66 Hit@200 points are achieved with a 3-token SID and no extra decoding cost.Online deployment gains were +6.74% PV_CVR and +6.03% UV_CVR.

2 Related Work

Related work enhances Semantic IDs with spatial or attribute information, but POI methods commonly rely on discrete spatial prefixes or embedding modifications. HF-SID instead motivates preserving fidelity in the representation before quantization.

  • Generative Retrieval: Generative retrieval generates short structured item identifiers, making Semantic IDs responsible for both LLM generatability and semantic faithfulness.SIDs are commonly produced through residual-quantized VAEs or hierarchical K-Means over item embeddings.
  • POI Retrieval: Because quantization irreversibly discards embedding information, text-driven SIDs can confuse semantically similar but spatially distant POIs and separate adjacent POIs of different categories.These failures can cause geographical hallucination during generation.
  • Attribute Enhancement: Attribute-enhanced SID methods include discrete spatial tokenization with S2 Cell or Geohash prefixes, hierarchical geographic tokenization, and token-level continuous injection.These approaches represent different routes for adding attribute awareness to POI-oriented SIDs.

3 Methods

HF-SID builds POI representations that preserve geographic, numerical, and structural information before hierarchical quantization. Geo-CPT and Num-CPT align these representations with the LLM, while residual contrastive learning refines fine-grained structure.

  • Representation: HF-SID converts coordinates to 3D Cartesian form and represents every numerical value as a single unit before integrating attributes into the LLM input sequence.Type embeddings assign attribute types separate subspaces.
  • Continual Pre-Training: Geo-CPT and Num-CPT continually pre-train the LLM with geographic reasoning and scale-aware numerical objectives, producing the POI representation from the final hidden state.The training objective combines text-position cross-entropy with numerical multi-task loss.
  • SID Generation: Hierarchical quantization assigns geographic and numerical semantics to early layers, while Structure-based Contrastive Learning refines the residual before the final quantization layer.This separates co-located or coarsely similar POIs at fine levels without disturbing coarser geographic signals.

3.1 Coordinate and Numerical Representation

HF-SID represents coordinates continuously in Cartesian space and numerical values as unified polar units with field-aware embeddings. Carry-smoothing encodings make nearby values geometrically adjacent before the LLM produces the POI representation.

  • Coordinate Representation: Coordinates are transformed from spherical latitude–longitude into 3D Cartesian form before embedding.The resulting x, y, z coordinates are expressed in meters using the mean Earth radius.
  • Numerical Representation: Every coordinate and dynamic attribute is encoded as one numerical unit using sign, integer digits, and a fractional-part representation.The unified polar decomposition avoids treating values as ordinary digit strings.
  • Numerical Representation: Magnitude augmentation makes integer carries continuous, while unit-circle encoding makes fractional carries continuous across the 0.99-to-0.01 boundary.For example, 399 → 400 changes the augmented magnitude smoothly rather than changing three digits simultaneously.
  • Field-Aware Embedding: Field embeddings lift polar components into vectors and distinguish sign, integer, and fractional roles; the resulting encodings map numerically proximate values to adjacent input-space points.All integer positions share one embedding because their order is already encoded in the augmented magnitudes.

3.2 Numerical Encoder

The Numerical Encoder maps each scalar into a high-dimensional embedding while arranging its sign, integer digits, and fractional component for separate linear recovery. It is pretrained once and then frozen for downstream geographic and numerical attributes.

  • Numerical embedding: The Numerical Encoder maps a field-augmented polar representation through an MLP into a high-dimensional numerical embedding.The output dimension is larger than the input representation, so the encoder lifts rather than compresses a scalar.
  • Pretraining objective: Its pretraining objective jointly predicts the sign and each integer digit with classification heads and the fractional part with regression.This makes the scalar’s decomposed components recoverable from the embedding.
  • Pretraining objective: The objective arranges information so digit-wise magnitude structure is linearly decodable from separate embedding directions.It supervises independent 10-way digit classifications together with sign classification and fractional regression.
  • Pretraining objective: A positional weight ω_i = 1 + 0.2i mildly emphasizes higher-order digits while retaining supervision for lower-order digits.The design reflects the larger value shifts caused by higher-order digit errors and preserves low-order signals such as ratings.
  • Downstream use: After pretraining, the Numerical Encoder is frozen and reused across geographic coordinates and heterogeneous numerical attributes.This provides one consistent embedding space for both types of inputs.

3.3 Integrating Numerical Encoder into the LLM

HF-SID integrates single-unit numerical values into the LLM through projection and type-aware embeddings, then trains the model with geographic and numerical reasoning objectives. The hybrid training objective assigns text and numerical positions different losses.

  • Type embedding: A linear projection aligns frozen numerical embeddings with the LLM input space, and a learnable type embedding distinguishes attribute types.Type offsets preserve magnitude relations within an attribute while separating values from different attributes.
  • Fused embedding construction: Each numerical value is replaced by one [NUM] placeholder, so its sequence position is independent of digit count.This also shortens serialized inputs when multi-digit values would otherwise split into several sub-tokens.
  • Fused embedding construction: The fused sequence keeps ordinary token embeddings at text positions and injects type-aware numerical embeddings at [NUM] positions before entering the pretrained LLM.The final POI representation is the hidden state of the last token.
  • Two-stage continual pre-training: Geo-CPT trains pairwise distance computation and nearest-pair identification, using Cartesian inputs to supervise spatial reasoning.For pairs closer than 100 km, the chord-length target agrees with geographic distance within 10^-3%.
  • Two-stage continual pre-training: Num-CPT trains the model to identify which of two POIs has the larger value for a queried attribute type.Ties are discarded and labels are balanced so answers cannot be inferred from position; comparisons remain type-specific across different scales.
  • Training objective: The hybrid objective applies next-token prediction at text positions and a multi-task numerical loss at numerical positions.The numerical terms are averaged over their own positions, and Geo-CPT precedes Num-CPT because geographic fidelity is treated as dominant for LBS retrieval.

3.4 HF-SID Generation with Structure-based Contrastive Learning

HF-SID generates three-token SIDs through residual quantization, preserving geographic and numerical semantics in the leading tokens while using last-layer contrastive learning to encode fine-grained structure. The resulting prefix-sharing SIDs support constrained beam search.

  • Residual quantization: POI representations are residual-quantized with three K-means layers using codebooks of a common size, producing coarse-to-fine SID tokens.The first two layers yield coarse tokens reflecting geographic and numerical semantics.
  • Structure-based contrastive learning: Because geographically compact POIs can share a coarse prefix, HF-SID refines the second-order residual using two-level POI tags.Positive pairs match at both tag levels, while POIs differing at either level are negatives, making coarse-level matches hard negatives.
  • Structure-based contrastive learning: The projected residual is optimized with an InfoNCE objective while the LLM and first two codebooks remain fixed.Only the projection parameters are updated during this stage.
  • Final-layer refinement: The final quantization layer operates in projected space to trade reconstruction error for discriminative power because SIDs identify POIs rather than reconstruct representations.Restricting the projection to the last layer preserves geographic and numerical fidelity in the leading tokens.
  • SID structure: The resulting SIDs retain prefix sharing for similar POIs, while the last token separates POIs by fine-grained tag.Candidate generation can therefore use beam search constrained by the valid SID prefix tree.

4 Experiments

HF-SID is evaluated on industrial and public POI datasets against seven SID baselines, showing that representation-level fidelity improves geographic compactness, numerical coherence, hierarchical organization, and retrieval. These gains also persist in online deployment.

  • Experimental Setup: HF-SID is evaluated on two AMap datasets and two public Foursquare datasets, with baselines re-implemented using matched configurations.The evaluation uses Next POI recommendation and SID quantization metrics, including ICR, Avg.Dist, p90 Dist, and p95 Dist.
  • Overall Performance: 95.9% lower Avg. Dist. on AMap-L and 89.5% lower Avg. Dist. on AMap-S than plain RQ-KMeans demonstrate the strongest SID compactness.HF-SID reaches 0.25 km on AMap-L and 3.99 km on AMap-S; it also reduces distance-tail errors across p90 and p95 metrics.
  • Overall Performance: 81.24% and 48.98% Hit@200 on AMap-L and AMap-S exceed the strongest baselines by 3.66 and 6.78 points, respectively.Hit@300 improvements are 2.94 and 11.01 points, while HF-SID uses half the tokens of the 6-token GenPOI it surpasses.
  • Ablation Study: Removing geography causes the largest ablation drop, while structure and numerical pathways contribute smaller declines aligned with their positions in the SID hierarchy.Hit@200 falls from 81.24% to 68.34% without geography; the numerical pathway contributes drops of 4.32% and 3.25%.
  • Geographic Awareness: Geo-CPT raises geographic NMI from 0.1993 to 0.7454 at Province, 0.2615 to 0.9796 at City, and 0.3972 to 0.9399 at District.HF-SID forms tight, well-separated geographic clusters, linking representation geometry to SID-space compactness.
  • Numerical Awareness: HF-SID achieves the best dispersion on all five numerical attributes, with POI Visited Avg.Std reduced by 77.5% against the second-best baseline.The advantage widens for heavy-tailed attributes, while improvements on Rating are smaller but consistent.
  • Public Datasets: HF-SID’s third-level children are nearly tag-pure under a shared parent, while public-dataset Hit@5 exceeds Pro-GEO by 6.83 points on NYC and 8.10 points on TKY.On public datasets, HF-SID is the only method reported to improve both geographic compactness and ICR simultaneously.
  • Online A/B Test: Online A/B testing improves average PV_CVR by 6.74% and UV_CVR by 6.03% across Restaurant, Life Services, and Tourist Attraction scenarios.Tourist Attraction shows the largest PV_CVR gain at 11.51%.

5 Conclusion

HF-SID is a high-fidelity Semantic ID framework for generative POI retrieval that preserves geographic, numerical, and structural information at the embedding level without increasing identifier length.

  • HF-SID converts coordinates into continuous 3D Cartesian form and encodes numerical values as single units with type-aware embeddings.Geo-CPT and Num-CPT align these representations with the LLM, while structure-based contrastive learning separates fine-level POI categories.
  • HF-SID applies all mechanisms at the embedding level, adding no token overhead.The framework uses a 3-token Semantic ID.
  • On the AMap platform, HF-SID improves offline Semantic ID quality and online business metrics over the previous system.The paper also releases AMap-S as a large-scale real-world POI benchmark.

A.1 Case Study

The case study compares generated POI candidates from a baseline and HF-SID under the same user context, showing more spatially compact and category-consistent retrieval with HF-SID.

  • HF-SID concentrates generated candidates around the target region, while the baseline produces geographically scattered candidates.The comparison uses the same user context.
  • HF-SID reduces average target distance from 9.22 km to 2.40 km and improves category consistency from 29/30 to 30/30.The baseline misses the ground-truth target, whereas HF-SID ranks it first.
  • The case study links geographic and structural fidelity in the representation to candidate sets that are spatially compact and semantically reliable.

A.2 CPT Prompt Templates

The prompt templates support continual pretraining for spatial reasoning and heterogeneous numerical comparison, while the case study illustrates the resulting retrieval behavior.

  • Geo-CPT: Geo-CPT focuses on spatial representation and geographic reasoning.Its prompts include POI spatial-information serialization, pairwise distance prediction, and nearest-location identification.
  • Case study: Figure 5 compares baseline and HF-SID generated POI candidates under the same user context.The comparison concerns candidate generation behavior.
  • Num-CPT: Num-CPT focuses on comparing heterogeneous numerical attributes under a specified attribute type rather than raw numerical magnitude alone.
Loading 2608.30479v1…