H2CE: Modeling Geo-Semantic Interactions for POI Reranking with Heterogeneous Two-Stage Cross-Encoders

1 Introduction

• Local search reranking must refine hundreds or thousands of candidates while meeting strict latency and cost constraints.Expensive listwise or LLM reranking over the full candidate pool is impractical for real-time serving.Source: §1 · p. 1
• POI relevance depends on query-conditioned interactions among query and POI semantics, location, distance, rating, and review count.The same distance or quality signal can matter differently depending on query intent and semantic match.Source: §1 · pp. 1–2
• H2CE textualizes bucketized numerical descriptors for cross-encoder attention and processes exact values with MLPs before nonlinear latent-space fusion.The design preserves scalar magnitude while enabling semantic matching to use coarse spatial and quality concepts.Source: §1 · p. 2
• Stage 1 scores all retrieved candidates pointwise, then Stage 2 compares the top-K head-to-head and uses Copeland aggregation.Restricting pairwise comparisons to the top-K bounds their cost while making relative tradeoffs explicit.Source: §1 · p. 2
• 67.48% NDCG@5 on 5,743 local-search test queries improves over XGBoost LTR by +22.82% absolute and over RankGPT by +35.89%.The paper also reports +9.62% over a BGE reranker with local-search fine-tuning.Source: §1 · p. 2
• The study includes ablations of numerical features, training strategies, top-K sensitivity, quantization, and serving latency.Its contributions also include comparisons with XGBoost LTR, zero-shot neural rerankers, and fine-tuned cross-encoder variants.Source: §1 · p. 2

POI search, recommendation, and geographic relevance

• POI recommendation and trajectory methods model geographic or contextual signals, but generally target personalization or mobility rather than query-dependent reranking of retrieved candidates.Query–POI matching work incorporates geographic or spatio-temporal context, including semantic similarity with geographic correlation and dual graph attention.Source: §2 · p. 2

Neural reranking and heterogeneous features

• Efficiency methods prune or partially encode candidates, whereas H2CE spends additional computation on surviving top-K candidates to resolve relative ordering.Examples include a cheap ranker before a transformer, progressive pruning in a Cascade Transformer [31], and partial interaction in Poly-encoders [12].Source: §2 · p. 3
• H2CE extends score-as-text approaches by representing POI attributes as attention-readable descriptors while preserving exact values in numerical encoders.Prior approaches inject retrieval scores as text [2] or fuse real-valued context with [CLS]; H2CE applies the combined approach to distance, rating, and review count.Source: §2 · p. 3

Learning to rank with structured and multi-field evidence

• Structured-feature rankers and late-fusion systems do not provide H2CE's combination of semantic–numeric attention, scalar precision, and nonlinear conditional interactions.H2CE models cases where proximity matters more for “near me” queries or semantic match overrides distance.Source: §2 · p. 3

Pointwise, pairwise, and listwise ranking

• H2CE applies relative comparison only after pointwise scoring has reduced the candidate set, keeping cost proportional to O(N+K(K-1)).This targets the top of the ranking, where explicit comparison of semantics, distance, rating, and review count is retained.Source: §2 · p. 4

3.1 POI Reranking

• Each candidate supplies semantic fields such as name and categories alongside distance, star rating, and review count as quantitative attributes.The reranker produces a permutation of the retrieved candidate set.Source: §3.1 · p. 4
• A correct POI ordering can require balancing cuisine match against distance, rating, and review-count evidence, with the tradeoff depending on query intent.A few meters may be negligible while a gap over a kilometer may reverse preference, especially for walking-distance queries.Source: §3.1 · p. 4

3.2 Pointwise Formulation

• A pointwise ranker independently scores each query–POI pair and trains the score as a binary relevance classifier.At inference, candidates are sorted in decreasing score order.Source: §3.2 · p. 4
• Pointwise scoring cannot explicitly compare relevant candidates with one another to choose which better matches the query or has stronger relative proximity and review evidence.It can separate relevant from irrelevant candidates, but it scores each POI in isolation.Source: §3.2 · p. 4

3.3 Pairwise Formulation

• A pairwise ranker receives two candidates together and predicts which should rank above the other for the query.Its comparisons can incorporate semantic match, distance, rating, review evidence, and the query-specific tradeoff.Source: §3.3 · p. 4
• Pairwise training uses rank positions from a commercial provider's Golden Standard List, validated with an LLM-as-a-Judge procedure.Copeland aggregation scores a candidate by its net pairwise preference strength and ranks candidates by that score.Source: §3.3 · p. 4

3.4 Two-Stage Pointwise+Pairwise Ranking

• The two-stage scheme scores all N candidates pointwise, then compares the K selected candidates using K(K−1) ordered non-self comparisons and Copeland voting.With K=10, the pairwise stage requires 90 comparisons per query; the final output appends the remaining candidates in Stage 1 order.Source: §3.4 · p. 5, Figure 1 · p. 5

4 Methodology

• H2CE combines semantic descriptors, exact numerical values, and top-K pairwise comparisons to resolve query-conditioned tradeoffs among plausible POIs.Its supervision combines a smaller customer-labeled set with a larger LLM-annotated dataset built over search-engine candidate outputs.Source: §4 · pp. 5–6

4.1 System Overview

• The system fuses cross-encoder and numerical MLP embeddings, scores all candidates pointwise, then applies Copeland voting to pairwise scores among the top K.The final list concatenates the pairwise-reranked top K with remaining candidates in Stage 1 order.Source: §4.1 · p. 6

4.2 Heterogeneous Feature Encoder

• The Heterogeneous Feature Encoder places bucketized distance, rating, and review descriptors in cross-encoder text while dedicated MLPs preserve exact scalar values.Paired MLP inputs allow direct comparison; the importance of numerical evidence depends on query intent and POI semantics.Source: §4.2 · p. 6

4.3 Latent-Space Feature Aggregation

• Latent-space aggregation combines the CLS embedding and numerical embeddings so a classifier can learn nonlinear interactions in both pointwise and pairwise stages.This can favor a slightly farther POI when its semantic and review evidence is stronger.Source: §4.3 · pp. 6–7

4.4 Stage 1: Pointwise HCE

• Stage 1 scores candidates independently for scalable coarse ranking, but cannot resolve tradeoffs that depend on other candidates in the set.It serves as a high-recall filter; a relevant POI omitted from the top K cannot be recovered by Stage 2.Source: §4.4 · p. 7

4.5 Stage 2: Pairwise HCE

• Stage 2 compares candidate pairs using semantic, distance, rating, and review evidence, making relative preference explicit among plausible POIs.Copeland aggregation ranks candidates by their head-to-head wins.Source: §4.5 · p. 7

4.6 Inference Complexity

• Pointwise scoring costs O(N), while restricting pairwise comparison to top K costs O(K(K-1)); at K=10, this requires 90 ordered comparisons.The comparison set is limited to candidates already considered plausible.Source: §4.6 · p. 7

5.1 Experimental Setup

• Pairwise training samples top-10 candidates using Stage 1 scores, aligning its training distribution with inference.The pointwise model uses up to 16 positive and 16 negative POIs per query; pairwise runs sample K=4 from the Stage 1 top-10.Source: §5.1 · pp. 8–9
• Evaluation uses 5,743 queries from an enriched pointwise local-search dataset, isolating reranker performance from upstream routing effects.Training uses roughly 500,000 query–POI examples with a Golden Standard List validated by LLMaaJ.Source: §5.1 · p. 8
• The setup reports RDQ, NDCG, Precision, and MRR and compares H2CE with lexical, feature-based, zero-shot, and fine-tuned rerankers.The XGBoost LTR baseline uses five features; H2CE comprises pointwise HCE followed by top-K pairwise reranking with Copeland voting.Source: §5.1 · pp. 8–9

5.2 Component-Level Comparison

• 45.07% RDQ@1 is XGBoost LTR’s score using five structured features; zero-shot RankGPT and BGE reach approximately 22% RDQ@1.BM25-only reaches 9.00% RDQ@1.Source: §5.2 · p. 9
• 67.02% RDQ@1 is H2CE’s score, improving over XGBoost LTR by +21.95% absolute and over zero-shot BGE by +44.46%.The evaluation uses a 5,743-query component test set.Source: §5.2 · p. 9
• +34.73% RDQ@1 comes from domain fine-tuning over zero-shot BGE, followed by +4.36% from numerical MLPs, +2.64% from latent aggregation, and +2.73% from pairwise reranking.The reported component gains are cumulative.Source: §5.2 · p. 9

5.3 Numerical Feature Ablation Across Stages

• +10.99% RDQ@1 from adding numerical MLP inputs only to Stage 1 exceeds the +6.16% RDQ@1 gain from adding them only to Stage 2; the full model performs best.Stage 1 determines which candidates enter the top-K.Source: Table 4 · p. 9, §5.3 · p. 9
• 51.65% RDQ@1 from the semantic-only model shows that textual matching is substantial but insufficient.Source: Table 4 · p. 9, §5.3 · p. 9

5.4 Training Strategy

• Multi-pair pointwise training improves RDQ@1 by +2.64% and RDQ@5 by +3.14%.It samples up to 16 positives and 16 negatives per query, versus one of each for single-pair training.Source: §5.4 · p. 9, Table 5 · p. 9
• Pairwise training samples four candidates from Stage 1’s top 10, aligning its training distribution with inference.The sampled candidates form 12 ordered comparisons per query.Source: §5.4 · p. 9

• +2.73% RDQ@1, +2.09% RDQ@5, +2.26% NDCG@1, and +1.98% NDCG@5 result from adding pairwise reranking to Pointwise HCE at K=10.Deeper metrics improve less because only the top-10 order changes.Source: Table 6 · p. 9, §5.5 · p. 9

5.6 Top-𝐾Sensitivity

• Performance peaks at K=10 for RDQ@1 and P@1, so the paper uses K=10 throughout.At K=5, Stage 2 is recall-limited by Stage 1.Source: §5.6 · p. 9, Table 7 · p. 10
• Larger K marginally improves NDCG@10 but degrades top-biased metrics, likely as inference diverges from the top-10 training distribution while cost grows quadratically.Source: Table 7 · p. 10, §5.6 · p. 10

5.7 Latency Analysis

• 163 ms median latency for full H2CE with 90 pairwise comparisons falls to 120 ms with the half-matrix variant.Measurements use a single A100-40GB GPU, batch size 1, 100 measured queries, and 10 warmup queries; pointwise models have approximately 65 ms median latency.Source: Table 8 · p. 10, §5.7 · p. 10

5.8 Half-Matrix Optimization

• For K=10, half-matrix scoring evaluates 45 unordered pairs instead of 90 ordered non-self pairs, reconstructing the opposite direction by probabilistic complementarity.Source: §5.8 · p. 10
• Less than 0.3% relative loss on all metrics accompanies a 26.4% median-latency reduction, from 163 ms to 120 ms.Source: Table 9 · p. 10, §5.8 · p. 10

5.9 Runtime TPS

• The half-matrix variant is estimated at 8.3 TPS per GPU and 67 TPS on an 8-GPU node, versus full H2CE’s 6 TPS per GPU and 49 TPS per node.Pointwise HCE reaches 15.5 TPS per GPU.Source: Table 10 · p. 10, §5.9 · p. 10

6 Conclusion

• 67.02% RDQ@1 on 5,743 local-search queries improves over XGBoost LTR by +21.95% absolute and over a zero-shot LLM reranker by +44.83%.The pairwise stage adds +2.73% RDQ@1 over the pointwise model alone.Source: §6 · p. 10
• H2CE combines bucketized semantic descriptors and exact scalar inputs with latent-space aggregation, then applies pointwise scoring and top-K pairwise Copeland aggregation.This reduces pairwise cost from O(N^2) to O(N+K(K-1)).Source: §6 · p. 10
• Median latency rises from 65 ms for pointwise-only reranking to 120 ms for the two-stage system, making pointwise reranking potentially preferable under very tight budgets.Training combines a smaller customer-labeled set with a larger LLM-annotated dataset whose labels may contain duplicated POIs, ambiguous preferences, or imperfect judgments.Source: §6 · p. 10
• Although the dataset is proprietary, the authors release POISS, a multilingual POI search dataset built with the same methodology and privacy-preserving techniques.It provides comparable query–POI sets and annotations and should yield comparable, though not identical, results.Source: §6 · p. 10

GenAI Usage Disclosure

• Claude was used to generate manuscript sections, including text, graphs, and formatting assistance.Claude 3 Haiku was also used as a zero-shot LLM reranker baseline with RankGPT prompting.Source: GenAI Usage Disclosure · p. 11
• The authors state that they produced the scientific content, experimental design, and results.Source: GenAI Usage Disclosure · p. 11
• The disclosure identifies Claude 3 Haiku with RankGPT prompting as an experimental baseline in Section 5.Source: GenAI Usage Disclosure · p. 11