Source-linked AI summary
Do Geometry-Aware Positional Encodings Help Transformers in Spatial Imperfect-Information Games?
Wenji Fu
TL;DR
Transformers in spatial imperfect-information games must encode map geometry while tracking hidden opponents over time, but it is unclear whether positional structure improves both representation and play. This paper benchmarks geometry-aware encodings across geometry, exact-belief tracking, imitation, and gameplay, finding improved beliefs and low-data imitation but no reliable win-rate improvement.
Problem
It is unclear whether positional encodings improve geometric representation, temporal hidden-target tracking, and actual play in spatial imperfect-information games.
Method
The paper builds a four-level hexagonal naval-pursuit benchmark including exact-Bayes belief labels, controlled imitation experiments, and fixed-seed gameplay evaluation.
Results
Geometry-aware encodings improve exact-belief estimation and spatial-policy imitation, especially with limited data, but do not reliably improve closed-loop win rate.
Takeaways & Limitations
Representation and belief gains from geometric inductive bias do not automatically translate into stronger closed-loop play.
Takeaways & Limitations
External validity is limited because the benchmark uses one game, small maps, controlled belief dynamics, legacy-AI labels, and only three training seeds for costly studies.
Abstract
from arXiv · showhide
Transformers applied to spatial imperfect-information games must represent map geometry while tracking hidden entities through time. We ask whether geometry-aware positional encodings improve these capabilities, without claiming a new positional encoding. We construct a four-level benchmark on a hexagonal naval pursuit game: controlled geometry and topology probes, an exact-Bayes hidden-target tracking task, offline policy imitation at 1k and 10k games, and 7,200 fixed-seed games against three legacy opponents. Across matched Transformer backbones, HexRoPE reduces exact-belief posterior cross-entropy relative to no positional encoding by 0.278 on D6-transformed test orbits and 0.329 on a larger map; both hierarchical-bootstrap confidence intervals exclude zero, and both Holm-adjusted p-values are below 0.001. At 1k games, HexRoPE improves policy action accuracy by 4.63 percentage points over no encoding and 2.05 points over rectangular relative bias; the gains shrink to 1.55 and 0.41 points at 10k games. However, HexRoPE does not improve aggregate gameplay win rate: its paired effect over no encoding is -1.56 percentage points (95% CI [-4.50, 1.17]). Rectangular relative bias is strongest on D6 belief consistency but fails sharply when extrapolating from radius 3 to radius 4, while graph bias provides only a small blocked-edge gain. The results show that geometric inductive bias improves belief estimation and data-efficient imitation, but those representation gains do not automatically produce stronger closed-loop play.
1 Introduction
The paper evaluates whether geometry-aware positional encodings help Transformers represent hex-map geometry and track hidden targets in spatial imperfect-information games. A four-level benchmark separates geometric reasoning, exact-belief tracking, policy imitation, and closed-loop gameplay without claiming architectural novelty.
- Motivation: Spatial imperfect-information games require Transformers to represent directional geometry, distance, topology, and hidden-target beliefs from positive and negative observations.The introduction frames these as distinct challenges that flattening a board into tokens does not solve automatically.
- Motivation: The benchmark separates geometric relations, calibrated hidden-target posteriors, spatial-action imitation, and closed-loop winning as distinct forms of spatial understanding.This separation is intended to test whether spatial advantages survive temporal belief tracking and affect actual play.
- Experimental scope: The study compares matched Transformer backbones using no positional encoding, rectangular relative bias, axial 2D RoPE, cube-coordinate HexRoPE, and graph-distance-biased HexRoPE.The aim is empirical evaluation rather than inventing HexRoPE.
- Benchmark design: The exact-belief task uses rule-enumerated labels, isolating hidden-target tracking from the quality of legacy policy imitation.The benchmark includes IID, D6-transformed, larger-map, and blocked-edge test sets, plus transform and probability-integrity tests.
- Benchmark design: The four-level benchmark covers hex geometry, blocked-edge topology, exact hidden-state belief tracking, policy imitation, and gameplay with controlled 1k/10k experiments and 7,200 fixed-seed games.Per-game outputs and hierarchical paired-bootstrap inference support the gameplay evaluation.
2 Related Work
Prior work develops relative, rotary, multidirectional, hexagonal, and group-representation positional encodings for spatial Transformers. Graph-based biases and factorized spatial-temporal attention provide complementary approaches for modeling relational structure and efficient video-like sequences.
- Spatial positional encoding: Relative position biases add displacement-dependent terms to attention logits and are widely used in vision Transformers.RoPE-ViT adapts rotary embeddings to images, while LieRE formulates rotary encoding through group representations.
- Spatial positional encoding: Multidirectional and hexagonal variants motivate testing coordinates beyond rectangular axis decomposition in imperfect-information decision settings.These mechanisms are treated as existing approaches whose behavior is studied in this setting.
- Graph structure and factorized attention: Graphormer and GRPE use shortest-path or relation information as attention biases, but graph positional encoding usefulness depends on the task.This motivates separating continuous hex geometry from blocked-edge topology.
- Graph structure and factorized attention: Policy and belief models factor spatial and temporal attention, following the broad efficiency pattern of video Transformers.The related-work framing treats factorized attention as an efficiency-oriented design pattern for spatiotemporal modeling.
3 Benchmark
The benchmark evaluates geometry-aware Transformers in an irregular-hex naval pursuit game through spatial-relation probes, exact hidden-target belief tracking, offline policy imitation, and fixed-seed gameplay. Its protocols isolate spatial reasoning, transform consistency, posterior estimation, data efficiency, and closed-loop behavior.
- Environment: The environment is an independent research implementation of a two-player naval pursuit game on an irregular hex map with hidden German ships and side-visible observations.Play lasts at most 18 turns, with movement, search, and special actions available.
- Geometry probes: The spatial probes enumerate 1,521 query–target pairs on the irregular map and 1,369 pairs on a regular radius-3 hexagon, testing direction, hex distance, and blocked-edge BFS distance.Seven encodings use a shared 236k-parameter backbone across five seeds, with D6-orbit splits preventing transformed copies from crossing train/test boundaries.
- Hidden-target tracking: The belief task labels the exact posterior for hidden-target trajectories generated by a known random-walk transition with stay actions and noisy search observations.The benchmark uses a 0.7 detection probability in the searched region; failed searches assign likelihood 0.3 to searched cells before normalization.
- Hidden-target tracking: The tracking protocol trains on 10,000 radius-3 episodes and tests IID radius-3, full D6-orbit, radius-4, and blocked-edge episodes with disjoint base identifiers.Reported metrics include posterior cross-entropy, KL divergence, Brier score, calibration error, expected hex-distance error, and D6 consistency.
- Policy imitation and gameplay: The policy benchmark uses executed actions from reproducible non-expert policy pools, with approximately 211k records from 1,000 games and 2,112,926 records from 10,000 games.Gameplay evaluates four policy modes across three seeds against three opponents in 7,200 shared-initial-seed games, recording win rate and additional behavioral measures.
4 Models and Protocol
The study compares five positional-encoding variants across matched belief-tracking and policy-imitation Transformers for a hexagonal spatial game. It uses fixed model protocols and hierarchical paired-bootstrap evaluation, with principal comparisons corrected by Holm adjustment.
- Encodings: Five encodings range from no spatial position to rectangular, axial, and cube-coordinate geometry, with HexRoPE+Graph adding graph-distance and edge-availability bias.HexRoPE distributes rotations across the dependent cube coordinates q, r, and s, where q + r + s = 0.
- Belief model: The exact-belief tracker processes eight observation frames through three factorized spatial/temporal blocks and predicts a 61-cell posterior.All encodings share hidden dimension 96, four heads, FFN dimension 192, and the same 15-epoch, three-seed training budget.
- Policy model: The policy model uses eight-step histories, four factorized blocks, legal-action context, invalid-cell masking, and approximately 2.02M parameters.All modes share six heads, dimension 192, FFN dimension 512, three epochs, AdamW, cosine decay, mixed precision, and auxiliary-loss settings.
- Inference: Evaluation reports per-game action accuracy, CE, value MAE, and Brier score using identical validation splits across training seeds.Principal comparisons use 5,000-replicate hierarchical paired bootstrap intervals that resample training seeds before paired games or episodes.
- Inference: Holm correction applies to preregistered HexRoPE–None and HexRoPE–RectRel comparisons, while graph-bias comparisons and gameplay remain explicitly exploratory.This separates confirmatory positional-encoding comparisons from exploratory analyses.
5 Results
Geometry-aware encodings improve spatial representation, exact-belief tracking, and low-data policy imitation, but these gains do not reliably improve closed-loop win rate. Rectangular relative bias excels on D6 consistency yet fails on larger-map extrapolation, while graph bias offers only modest topology-specific gains.
- Spatial probes: HexRoPE reaches 0.984 direction and 0.977 hex-distance accuracy on the irregular map, while graph bias leads blocked-edge BFS distance at 0.927 accuracy and 0.074 MAE.On the regular D6 split, HexRoPE reaches 1.000 direction and 0.996 hex-distance accuracy; no-position distance accuracy remains near chance.
- Exact-belief tracking: 0.278 and 0.329 CE reductions versus None on D6 and radius-4 exact-belief tracking, respectively, have hierarchical 95% CIs excluding zero and Holm-adjusted p < 0.001.All spatial encodings outperform no encoding, supporting improved hidden-target posterior estimation beyond imitation labels.
- Exact-belief tracking: RectRel has the lowest D6 consistency error at 0.0047 versus HexRoPE’s 0.0095, but its radius-4 CE reaches 11.008 after radius-3 training.Coordinate-based RoPE variants remain between 2.096 and 2.200, and AxialRoPE performs best among tested RoPE forms on the larger-map split.
- Exact-belief tracking: Graph bias improves blocked-edge CE by −0.00147 and expected-distance error by −0.00349 over HexRoPE, while Top-1 and Brier intervals include zero.The result indicates complementary topology information, but the benchmark effect is modest.
- Offline policy imitation: At 1k games, HexRoPE improves action accuracy by 0.0463 over None and 0.0205 over RectRel; at 10k, gains shrink to 0.0155 and 0.00407.All four primary accuracy tests remain significant after Holm correction, supporting a low-data benefit rather than a scaling-law claim.
- Fixed-seed gameplay: −0.0156 is HexRoPE’s paired win-rate effect versus None, with 95% CI [−0.0450, 0.0117], so offline improvements do not yield a reliable aggregate gameplay gain.Matchup results vary: HexRoPE is stronger as Germany against V11 but weaker as Britain against Yanfu; positional representation alone does not provide strategic diversity or opponent adaptation.
6 Discussion
The four evidence levels distinguish geometric representation, exact-belief prediction, data-efficient imitation, and closed-loop gameplay, showing that representational gains do not reliably yield stronger policies. The discussion attributes this gap to remaining topology and strategy-learning demands while identifying belief-conditioned action learning as a next step.
- Evidence levels: The four evidence levels show that geometry probes recover direction and distance, exact-belief tracking improves posterior prediction, and policy imitation benefits more at lower data.Gameplay supplies the boundary: the resulting policy is not reliably stronger in closed loop.
- Interpretation: Explicit geometric structure reduces the representational and sample burden of spatial prediction, but topology needs a separate bias and strategy needs additional learning signals.The discussion therefore rejects the interpretation that HexRoPE solves spatial games.
- Limitations: The sharp radius-4 failure of RectRel’s learned absolute components cautions against evaluating only on fixed-size boards.This limitation concerns extrapolation beyond the training geometry.
- Limitations: RectRel’s strong D6 consistency cautions against treating cube-coordinate RoPE as automatically equivariant to physical rotations and reflections.Strong consistency on D6 does not by itself establish automatic physical equivariance.
- Future work: A proposed next step is to expose exact or learned beliefs to an action-consequence module and train expected-utility or counterfactual objectives.This targets the gap between representing where the hidden target may be and choosing a strategically useful action under uncertainty.
7 Limitations
The study’s external validity and imitation conclusions are limited by its single small-map game, controlled exact-belief dynamics, legacy-AI labels, and only three training seeds. HexRoPE is evaluated as a prior method family rather than presented as a novelty claim, and recent 2026 preprints may require metadata updates.
- Benchmark and evaluation limits: External validity is limited because the benchmark uses one game and small hex maps, while its exact-belief dynamics are simpler and intentionally controlled.The controlled tracking task does not represent the full game’s dynamics.
- Benchmark and evaluation limits: Imitation accuracy measures fidelity to legacy-AI policy labels, not agreement with humans, experts, or a solved policy.The labels come from legacy AIs rather than human, expert, or solved-policy demonstrations.
- Benchmark and evaluation limits: Fixed seeds align initial randomness, but model actions alter later states, so paired gameplay does not imply identical state trajectories.This limits direct trajectory-level comparability between paired gameplay evaluations.
- Benchmark and evaluation limits: Only three training seeds are available for the expensive evaluations.The passage identifies the number of training seeds but is truncated before specifying the evaluations’ full description.
- Method positioning and references: HexRoPE is an evaluated prior method family, not a novelty claim, and recent 2026 preprints should be updated if their archival metadata changes.This limitation concerns method positioning and the stability of cited preprint metadata.
8 Reproducibility
The study provides reproducibility artifacts by hashing the exact-belief dataset, recording split metadata and evaluator outputs, and preserving commands, checkpoints, hardware metadata, and live state.
- Reproducibility: The exact-belief aggregate SHA-256 begins with b3927015cab6d7df, with the complete value recorded in the dataset manifest.The manifest is located at deeplearn/data/paper1_exact_belief/dataset_manifest.json.
- Reproducibility: Each split manifest stores episode and map identifiers, transforms, generation seeds, legal masks, posteriors, and file hashes.
- Reproducibility: Every evaluator emits per-game or per-episode JSON, while commands, checkpoints, hardware metadata, and live state are stored beside results.The passage also identifies the main entry points for reproducing the experiments.
9 Conclusion
Geometry-aware positional encodings improve exact-belief estimation and spatial-policy imitation, especially with limited data. These gains are conditional and do not reliably translate into stronger win rates.
- 9 Conclusion: Geometry-aware positional encodings improve exact-belief estimation and spatial-policy imitation, with the largest policy gain under low data.The central empirical distinction is between representation, belief, imitation, and closed-loop play.
- 9 Conclusion: Rectangular absolute-plus-relative encoding is strong in-distribution but fragile when extrapolating across map size.Its benefit depends on the geometry regime in which it is evaluated.
- 9 Conclusion: Graph bias provides only a small topology gain, while better representations do not reliably improve win rate.Representation and belief improvements therefore remain separable from aggregate gameplay performance.