PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Geometry-Aware Positional Encodings Help Transformers in Spatial Imperfect-Information Games?

Wenji Fu

arXiv:2608.14982v1cs.LGcs.AIstat.ML

TL;DR

Transformers in spatial imperfect-information games must encode map geometry while tracking hidden opponents over time, but it is unclear whether positional structure improves both representation and play. This paper benchmarks geometry-aware encodings across geometry, exact-belief tracking, imitation, and gameplay, finding improved beliefs and low-data imitation but no reliable win-rate improvement.

  • Problem

    It is unclear whether positional encodings improve geometric representation, temporal hidden-target tracking, and actual play in spatial imperfect-information games.

    Source: §1 · p. 1

  • Method

    The paper builds a four-level hexagonal naval-pursuit benchmark including exact-Bayes belief labels, controlled imitation experiments, and fixed-seed gameplay evaluation.

    Source: §1 · p. 1, §3.3 · p. 2

  • Results

    Geometry-aware encodings improve exact-belief estimation and spatial-policy imitation, especially with limited data, but do not reliably improve closed-loop win rate.

    Source: §1 · p. 1, §6 · p. 5, §9 · p. 6

  • Takeaways & Limitations

    Representation and belief gains from geometric inductive bias do not automatically translate into stronger closed-loop play.

    Source: §6 · p. 5, §9 · p. 6

    External validity is limited because the benchmark uses one game, small maps, controlled belief dynamics, legacy-AI labels, and only three training seeds for costly studies.

    Source: §7 · p. 5

Abstract

from arXiv · show

Transformers applied to spatial imperfect-information games must represent map geometry while tracking hidden entities through time. We ask whether geometry-aware positional encodings improve these capabilities, without claiming a new positional encoding. We construct a four-level benchmark on a hexagonal naval pursuit game: controlled geometry and topology probes, an exact-Bayes hidden-target tracking task, offline policy imitation at 1k and 10k games, and 7,200 fixed-seed games against three legacy opponents. Across matched Transformer backbones, HexRoPE reduces exact-belief posterior cross-entropy relative to no positional encoding by 0.278 on D6-transformed test orbits and 0.329 on a larger map; both hierarchical-bootstrap confidence intervals exclude zero, and both Holm-adjusted p-values are below 0.001. At 1k games, HexRoPE improves policy action accuracy by 4.63 percentage points over no encoding and 2.05 points over rectangular relative bias; the gains shrink to 1.55 and 0.41 points at 10k games. However, HexRoPE does not improve aggregate gameplay win rate: its paired effect over no encoding is -1.56 percentage points (95% CI [-4.50, 1.17]). Rectangular relative bias is strongest on D6 belief consistency but fails sharply when extrapolating from radius 3 to radius 4, while graph bias provides only a small blocked-edge gain. The results show that geometric inductive bias improves belief estimation and data-efficient imitation, but those representation gains do not automatically produce stronger closed-loop play.

1 Introduction

The paper evaluates whether geometry-aware positional encodings help Transformers represent hex-map geometry and track hidden targets in spatial imperfect-information games. A four-level benchmark separates geometric reasoning, exact-belief tracking, policy imitation, and closed-loop gameplay without claiming architectural novelty.

Motivation

  • Spatial imperfect-information games require Transformers to represent directional geometry, distance, topology, and hidden-target beliefs from positive and negative observations.The introduction frames these as distinct challenges that flattening a board into tokens does not solve automatically.Source: §1 · p. 1
  • The benchmark separates geometric relations, calibrated hidden-target posteriors, spatial-action imitation, and closed-loop winning as distinct forms of spatial understanding.This separation is intended to test whether spatial advantages survive temporal belief tracking and affect actual play.Source: §1 · p. 1

Experimental scope

  • The study compares matched Transformer backbones using no positional encoding, rectangular relative bias, axial 2D RoPE, cube-coordinate HexRoPE, and graph-distance-biased HexRoPE.The aim is empirical evaluation rather than inventing HexRoPE.Source: §1 · p. 1

Benchmark design

  • The exact-belief task uses rule-enumerated labels, isolating hidden-target tracking from the quality of legacy policy imitation.The benchmark includes IID, D6-transformed, larger-map, and blocked-edge test sets, plus transform and probability-integrity tests.Source: §1 · p. 1
  • The four-level benchmark covers hex geometry, blocked-edge topology, exact hidden-state belief tracking, policy imitation, and gameplay with controlled 1k/10k experiments and 7,200 fixed-seed games.Per-game outputs and hierarchical paired-bootstrap inference support the gameplay evaluation.Source: §1 · p. 1

2 Related Work

Prior work develops relative, rotary, multidirectional, hexagonal, and group-representation positional encodings for spatial Transformers. Graph-based biases and factorized spatial-temporal attention provide complementary approaches for modeling relational structure and efficient video-like sequences.

Spatial positional encoding

  • Relative position biases add displacement-dependent terms to attention logits and are widely used in vision Transformers.RoPE-ViT adapts rotary embeddings to images, while LieRE formulates rotary encoding through group representations.Source: §2 · p. 1
  • Multidirectional and hexagonal variants motivate testing coordinates beyond rectangular axis decomposition in imperfect-information decision settings.These mechanisms are treated as existing approaches whose behavior is studied in this setting.Source: §2 · p. 2

Graph structure and factorized attention

  • Graphormer and GRPE use shortest-path or relation information as attention biases, but graph positional encoding usefulness depends on the task.This motivates separating continuous hex geometry from blocked-edge topology.Source: §2 · p. 2
  • Policy and belief models factor spatial and temporal attention, following the broad efficiency pattern of video Transformers.The related-work framing treats factorized attention as an efficiency-oriented design pattern for spatiotemporal modeling.Source: §2 · p. 2

3 Benchmark

The benchmark evaluates geometry-aware Transformers in an irregular-hex naval pursuit game through spatial-relation probes, exact hidden-target belief tracking, offline policy imitation, and fixed-seed gameplay. Its protocols isolate spatial reasoning, transform consistency, posterior estimation, data efficiency, and closed-loop behavior.

Environment

  • The environment is an independent research implementation of a two-player naval pursuit game on an irregular hex map with hidden German ships and side-visible observations.Play lasts at most 18 turns, with movement, search, and special actions available.Source: §3.1 · p. 2

Geometry probes

  • The spatial probes enumerate 1,521 query–target pairs on the irregular map and 1,369 pairs on a regular radius-3 hexagon, testing direction, hex distance, and blocked-edge BFS distance.Seven encodings use a shared 236k-parameter backbone across five seeds, with D6-orbit splits preventing transformed copies from crossing train/test boundaries.Source: §3.2 · p. 2

Hidden-target tracking

  • The belief task labels the exact posterior for hidden-target trajectories generated by a known random-walk transition with stay actions and noisy search observations.The benchmark uses a 0.7 detection probability in the searched region; failed searches assign likelihood 0.3 to searched cells before normalization.Source: §3.3 · p. 2
  • The tracking protocol trains on 10,000 radius-3 episodes and tests IID radius-3, full D6-orbit, radius-4, and blocked-edge episodes with disjoint base identifiers.Reported metrics include posterior cross-entropy, KL divergence, Brier score, calibration error, expected hex-distance error, and D6 consistency.Source: §3.3 · p. 2

Policy imitation and gameplay

  • The policy benchmark uses executed actions from reproducible non-expert policy pools, with approximately 211k records from 1,000 games and 2,112,926 records from 10,000 games.Gameplay evaluates four policy modes across three seeds against three opponents in 7,200 shared-initial-seed games, recording win rate and additional behavioral measures.Source: §3.4 · pp. 2–3

4 Models and Protocol

The study compares five positional-encoding variants across matched belief-tracking and policy-imitation Transformers for a hexagonal spatial game. It uses fixed model protocols and hierarchical paired-bootstrap evaluation, with principal comparisons corrected by Holm adjustment.

Encodings

  • Five encodings range from no spatial position to rectangular, axial, and cube-coordinate geometry, with HexRoPE+Graph adding graph-distance and edge-availability bias.HexRoPE distributes rotations across the dependent cube coordinates q, r, and s, where q + r + s = 0.Source: §4 · p. 3

Belief model

  • The exact-belief tracker processes eight observation frames through three factorized spatial/temporal blocks and predicts a 61-cell posterior.All encodings share hidden dimension 96, four heads, FFN dimension 192, and the same 15-epoch, three-seed training budget.Source: §4 · p. 3

Policy model

  • The policy model uses eight-step histories, four factorized blocks, legal-action context, invalid-cell masking, and approximately 2.02M parameters.All modes share six heads, dimension 192, FFN dimension 512, three epochs, AdamW, cosine decay, mixed precision, and auxiliary-loss settings.Source: §4 · p. 3

Inference

  • Evaluation reports per-game action accuracy, CE, value MAE, and Brier score using identical validation splits across training seeds.Principal comparisons use 5,000-replicate hierarchical paired bootstrap intervals that resample training seeds before paired games or episodes.Source: §4 · p. 3
  • Holm correction applies to preregistered HexRoPE–None and HexRoPE–RectRel comparisons, while graph-bias comparisons and gameplay remain explicitly exploratory.This separates confirmatory positional-encoding comparisons from exploratory analyses.Source: §4 · p. 3

5 Results

Geometry-aware encodings improve spatial representation, exact-belief tracking, and low-data policy imitation, but these gains do not reliably improve closed-loop win rate. Rectangular relative bias excels on D6 consistency yet fails on larger-map extrapolation, while graph bias offers only modest topology-specific gains.

Spatial probes

  • HexRoPE reaches 0.984 direction and 0.977 hex-distance accuracy on the irregular map, while graph bias leads blocked-edge BFS distance at 0.927 accuracy and 0.074 MAE.On the regular D6 split, HexRoPE reaches 1.000 direction and 0.996 hex-distance accuracy; no-position distance accuracy remains near chance.Source: §5.1 · p. 3

Exact-belief tracking

  • 0.278 and 0.329 CE reductions versus None on D6 and radius-4 exact-belief tracking, respectively, have hierarchical 95% CIs excluding zero and Holm-adjusted p < 0.001.All spatial encodings outperform no encoding, supporting improved hidden-target posterior estimation beyond imitation labels.Source: §5.2 · p. 3
  • RectRel has the lowest D6 consistency error at 0.0047 versus HexRoPE’s 0.0095, but its radius-4 CE reaches 11.008 after radius-3 training.Coordinate-based RoPE variants remain between 2.096 and 2.200, and AxialRoPE performs best among tested RoPE forms on the larger-map split.Source: §5.2 · p. 3
  • Graph bias improves blocked-edge CE by −0.00147 and expected-distance error by −0.00349 over HexRoPE, while Top-1 and Brier intervals include zero.The result indicates complementary topology information, but the benchmark effect is modest.Source: §5.2 · p. 3

Offline policy imitation

  • At 1k games, HexRoPE improves action accuracy by 0.0463 over None and 0.0205 over RectRel; at 10k, gains shrink to 0.0155 and 0.00407.All four primary accuracy tests remain significant after Holm correction, supporting a low-data benefit rather than a scaling-law claim.Source: §5.3 · p. 4

Fixed-seed gameplay

  • −0.0156 is HexRoPE’s paired win-rate effect versus None, with 95% CI [−0.0450, 0.0117], so offline improvements do not yield a reliable aggregate gameplay gain.Matchup results vary: HexRoPE is stronger as Germany against V11 but weaker as Britain against Yanfu; positional representation alone does not provide strategic diversity or opponent adaptation.Source: §5.4 · p. 4

6 Discussion

The four evidence levels distinguish geometric representation, exact-belief prediction, data-efficient imitation, and closed-loop gameplay, showing that representational gains do not reliably yield stronger policies. The discussion attributes this gap to remaining topology and strategy-learning demands while identifying belief-conditioned action learning as a next step.

Evidence levels

  • The four evidence levels show that geometry probes recover direction and distance, exact-belief tracking improves posterior prediction, and policy imitation benefits more at lower data.Gameplay supplies the boundary: the resulting policy is not reliably stronger in closed loop.Source: §6 · p. 5

Interpretation

  • Explicit geometric structure reduces the representational and sample burden of spatial prediction, but topology needs a separate bias and strategy needs additional learning signals.The discussion therefore rejects the interpretation that HexRoPE solves spatial games.Source: §6 · p. 5

Limitations

  • The sharp radius-4 failure of RectRel’s learned absolute components cautions against evaluating only on fixed-size boards.This limitation concerns extrapolation beyond the training geometry.Source: §6 · p. 5
  • RectRel’s strong D6 consistency cautions against treating cube-coordinate RoPE as automatically equivariant to physical rotations and reflections.Strong consistency on D6 does not by itself establish automatic physical equivariance.Source: §6 · p. 5

Future work

  • A proposed next step is to expose exact or learned beliefs to an action-consequence module and train expected-utility or counterfactual objectives.This targets the gap between representing where the hidden target may be and choosing a strategically useful action under uncertainty.Source: §6 · p. 5

7 Limitations

The study’s external validity and imitation conclusions are limited by its single small-map game, controlled exact-belief dynamics, legacy-AI labels, and only three training seeds. HexRoPE is evaluated as a prior method family rather than presented as a novelty claim, and recent 2026 preprints may require metadata updates.

Benchmark and evaluation limits

  • External validity is limited because the benchmark uses one game and small hex maps, while its exact-belief dynamics are simpler and intentionally controlled.The controlled tracking task does not represent the full game’s dynamics.Source: §7 · p. 5
  • Imitation accuracy measures fidelity to legacy-AI policy labels, not agreement with humans, experts, or a solved policy.The labels come from legacy AIs rather than human, expert, or solved-policy demonstrations.Source: §7 · p. 5
  • Fixed seeds align initial randomness, but model actions alter later states, so paired gameplay does not imply identical state trajectories.This limits direct trajectory-level comparability between paired gameplay evaluations.Source: §7 · p. 5
  • Only three training seeds are available for the expensive evaluations.The passage identifies the number of training seeds but is truncated before specifying the evaluations’ full description.Source: §7 · p. 5

Method positioning and references

  • HexRoPE is an evaluated prior method family, not a novelty claim, and recent 2026 preprints should be updated if their archival metadata changes.This limitation concerns method positioning and the stability of cited preprint metadata.Source: §7 · p. 6

8 Reproducibility

The study provides reproducibility artifacts by hashing the exact-belief dataset, recording split metadata and evaluator outputs, and preserving commands, checkpoints, hardware metadata, and live state.

Reproducibility

  • The exact-belief aggregate SHA-256 begins with b3927015cab6d7df, with the complete value recorded in the dataset manifest.The manifest is located at deeplearn/data/paper1_exact_belief/dataset_manifest.json.Source: §8 · p. 6
  • Each split manifest stores episode and map identifiers, transforms, generation seeds, legal masks, posteriors, and file hashes.Source: §8 · p. 6
  • Every evaluator emits per-game or per-episode JSON, while commands, checkpoints, hardware metadata, and live state are stored beside results.The passage also identifies the main entry points for reproducing the experiments.Source: §8 · p. 6

9 Conclusion

Geometry-aware positional encodings improve exact-belief estimation and spatial-policy imitation, especially with limited data. These gains are conditional and do not reliably translate into stronger win rates.

9 Conclusion

  • Geometry-aware positional encodings improve exact-belief estimation and spatial-policy imitation, with the largest policy gain under low data.The central empirical distinction is between representation, belief, imitation, and closed-loop play.Source: §9 · p. 6
  • Rectangular absolute-plus-relative encoding is strong in-distribution but fragile when extrapolating across map size.Its benefit depends on the geometry regime in which it is evaluated.Source: §9 · p. 6
  • Graph bias provides only a small topology gain, while better representations do not reliably improve win rate.Representation and belief improvements therefore remain separable from aggregate gameplay performance.Source: §9 · p. 6

Related papers

Loading 2608.14982v1…