ActionRoPE: Action as Coordinates for Spatially Consistent Control in Planar-Camera Game World Models

1 INTRODUCTION

• Feature-based action interfaces leave motion direction and magnitude to be learned from data, despite the geometric structure of planar-camera worlds.When the camera returns to a previously seen location, the scene may no longer look the same.Source: §1 · p. 2
• ActionRoPE writes action-driven camera displacement into world-coordinate rotary positions, so the same world location retains its positional address over time.The interface is parameter-free and turns action control from learned feature conditioning into explicit coordinate alignment.Source: §1 · pp. 2–3
• CNS assigns token-wise denoising schedules by known–new regions, stabilizing first-frame regions earlier while newly revealed regions retain the original schedule.The known–new partition is supplied through a 0.015M-parameter input channel.Source: §1 · pp. 2–3
• 66% less revisit drift than the strongest baseline accompanies more precise motion scaling and directional compliance.The evaluation compares four representative action-conditioning strategies under the same backbone and training setting.Source: §1 · p. 3
• The interface transfers across backbones from 1.3B to 33B parameters and to out-of-distribution scenes.The evaluation protocol covers motion fidelity, directional compliance, and closed-loop revisit.Source: §1 · p. 3

2.1 INTERACTIVE GAME WORLD MODELS

• Game world models typically inject actions as conditioning features, leaving the resulting visual motion to be learned from data.Prior approaches include latent or additive signals, language, adaptive normalization, and cross-attention.Source: §2.1 · p. 3

2.2 GEOMETRY-AWARE POSITIONAL ENCODING

• ActionRoPE applies geometry-aware positional encoding to action control, converting planar displacement into world coordinates rather than encoding an external 3D camera pose.The same world location retains the same positional address throughout camera motion.Source: §2.2 · p. 3

2.3 ADAPTIVE DIFFUSION SCHEDULING

• CNS defines known versus new regions from world coordinates, letting known regions stabilize earlier without copying or reinserting observed content.Earlier adaptive schedules distinguish regions by time, an external mask, or explicit observed content.Source: §2.3 · p. 4

3.1 PLANAR-CAMERA WORLDS

• In planar-camera worlds, calibrated actions determine camera displacement geometrically, while feature-conditioning interfaces leave motion direction and magnitude implicit in the training distribution.These worlds include top-down and isometric games where the camera follows the player and the static scene translates approximately rigidly in the image plane.Source: §3.1 · p. 4

3.2 ACTION CONTROL AS WORLD COORDINATES

• ActionRoPE compensates for cumulative camera displacement so the same world location keeps the same spatial positional address across time.It computes rotary angles from real-valued coordinates, without quantizing camera displacement to the token grid.Source: §3.2 · p. 5
• The knownness field κ marks tokens inside the first-frame footprint as observed and tokens outside it as newly exposed regions to synthesize.The partition comes from the action trajectory and is also provided to the network through a zero-initialized convolution added to patch embeddings.Source: §3.2 · pp. 5–6

3.3 COORDINATE-CONDITIONED NOISE SCHEDULING

• CNS assigns known and newly revealed tokens different noise schedules, stabilizing known regions earlier while generating new content with reference to them.New regions retain the original schedule; known regions use a lower shift and enter low-noise regimes earlier, with shared noisy and clean endpoints.Source: §3.3 · p. 6
• Training applies spatially varying noise and up-weights loss on newly revealed regions by a factor of 2.The first temporal cell remains a clean anchor and is excluded from the loss.Source: §3.3 · p. 6

3.4 TRAINING AND INFERENCE

• At inference, actions determine world coordinates and κ for sampling; the approach is also illustrated in revisit and out-of-distribution-style control figures.Training fine-tunes the backbone, and the method is distilled into a four-step causal student with a key–value cache for real-time use.Source: §3.4 · p. 6, Figure 3 · p. 7, Figure 4 · p. 7

4.1 EXPERIMENTAL SETUP

• The evaluation uses 18,528 training clips and 982 held-out clips from 50.9 hours of top-down gameplay, with each test generation conditioned only on its first frame.Models are compared on the same Wan backbone and training setting; metrics test motion magnitude, direction, and return-to-start consistency.Source: §4.1 · pp. 7–8

4.2 QUALITATIVE RESULTS

• Across three evaluation scenes, ActionRoPE with CNS returns close to the initial frame, while the four baselines show scene offsets and Cross-Attn also changes a layout.The offset is clearest along Scene 2’s bright path; Cross-Attn changes the lower-left layout in Scene 3.Source: §4.2 · p. 8

4.3 QUANTITATIVE RESULTS

• The same action interface controls backbones from 1.3B to 33B parameters; MiniMax-H3’s 73 generated frames are resampled to 81 for evaluation.Table 2 reports results across backbones.Source: Table 2 · p. 9

4.4 ABLATION STUDY

• Adding CNS improves all revisit metrics; without CNS, residual drift increases from 7.8 to 11.7 px.Both compared Wan models retain ActionRoPE.Source: Table 3 · p. 9, §4.4 · p. 9

4.3 QUANTITATIVE RESULTS

• ActionRoPE outperforms all four baselines on every metric; against the strongest revisit baseline, it improves PSNR by 2.8 dB, SSIM by 13%, and NCC by 21%, and reduces drift by 66%.The comparison uses the same Wan backbone; the control path adds no parameters.Source: Table 1 · p. 8, §4.3 · p. 9

• ActionRoPE writes action-driven camera motion into video-token rotary positions, while CNS lets known regions settle earlier and guides generation of newly revealed content.The conclusion reports more precise commanded motion and more consistent revisits than representative action-injection mechanisms, with transfer across backbones from 1.3B to 33B parameters.Source: §5 · p. 9

• The supplied title identifies supplementary material for ActionRoPE and its focus on coordinates for spatially consistent control in planar-camera game world models.Source: Supplementary Material for ActionRoPE:… · p. 13

A.1 REVISIT COMPARISON

• On six additional evaluation scenes, our revisit errors remain low over most of the frame, while four baselines show larger high-error regions.Our remaining errors concentrate on thin structure edges.Source: §A.1 · p. 14, Figure 6 · p. 15, Figure 7 · p. 15

A.2 MOTION SCALING

• Our method stays close to ideal commanded displacement at every speed, while Linear and AdaLN compress speed and Prompt and Cross-Attn move nearly the same amount at every speed.Prompt and Cross-Attn’s nearly speed-invariant movement explains their negative R2 in Table 1.Source: §A.2 · p. 14, Figure 8 · p. 16, §B.2 · p. 19

A.3 OUT-OF-DISTRIBUTION SCENES

• On three unseen scenes, baselines return with larger errors than our method, while on four unseen styles the camera returns close to its starting view and revealed regions match the first-frame style.The lower part of one out-of-distribution scene changes for all methods, including ours.Source: §A.3 · p. 14, Figure 9 · p. 16, Figure 10 · p. 16

A.4 CROSS-BACKBONE EXAMPLES

• All four backbones follow the rightward walk, turn back at f=40, and return close to the starting view at f=80, differing mainly in rendering newly revealed regions.New regions appear darker for LTX-2.5 and MiniMax-H3 in Figure 11.Source: §A.4 · p. 14, Figure 11 · p. 17, Figure 12 · p. 17

A.5 EFFECT OF CNS

• With CNS, floor tiles in the known region appear by sampling step 6 and the right-side wall and door by step 14, while the same region remains blurry without CNS.Figure 14 compares revisit error maps for the full model and a model trained without CNS on three evaluation scenes.Source: §A.5 · p. 14, Figure 13 · p. 17, Figure 14 · p. 18

B.1 OUT-OF-DISTRIBUTION SCENES

• 7.6 px residual drift is less than half the strongest baseline’s 16.5 px on 50 out-of-distribution first frames, where our method remains best on every metric.Revisit PSNR and SSIM are lower than on the evaluation set for all methods.Source: §B.1 · p. 19

B.2 MOTION SCALING PER SPEED

• Our measured-to-commanded displacement ratios stay between 0.96 and 0.99; baselines overshoot slow commands and undershoot fast ones.Prompt and Cross-Attn ratios fall in inverse proportion to commanded speed because their measured displacement does not change with speed.Source: §B.2 · p. 19

B.3 NOISE SCHEDULE DESIGN

• Every coordinate-conditioned noise schedule reaches higher revisit PSNR than the uniform schedule, and the shift-based form is used because it reuses each backbone’s shifted schedule with one hyperparameter.The hyperparameter is Sknown.Source: §B.3 · p. 19, Figure 15 · p. 19

B.4 TRAINING VERSUS SAMPLING

• CNS’s effect comes from training: changing only the sampling schedule leaves revisit PSNR essentially unchanged (22.82 vs. 22.81 dB; 24.06 vs. 23.94 dB).The first comparison samples a model trained without CNS using the CNS schedule; the second samples a CNS-trained model using the uniform schedule.Source: §B.4 · p. 19

C.1 ACTIONROPE AND CNS

• Across backbones, Sknown/S is kept at 0.4; Table 7 lists per-backbone settings including pixels per spatial token, mask parameters, CNS shifts, and guidance scale.For MiniMax-H3, whose spatial positions are not spaced by one, ∆k/s is scaled by the same spacing.Source: §C.1 · p. 20, Table 7 · p. 20

C.2 TRAINING AND SAMPLING

• Models are trained for 5,000 steps with AdamW, weight decay 0.01, bf16 precision, and gradient clipping at 1.0; evaluation fixes the first frames, commands, random seed, and scene-free prompt.The flow-matching loss excludes the first latent frame, newly revealed tokens receive weight 2 for the authors’ models, and sampling settings vary by backbone.Source: §C.2 · pp. 20–21

C.3 BASELINES

• All baselines share the Wan2.2 backbone, data, training settings, and per-frame camera displacement; Prompt and Cross-Attn quantize displacement to direction labels and use uniform loss weights.The baseline layers are zero-initialized, so each model starts from the pretrained backbone.Source: §C.3 · p. 21
• Prompt appends one of nine direction labels to the text prompt, while Cross-Attn uses four-key action vectors attended to over time; neither receives speed.Prompt adds no parameters; Cross-Attn adds 98.6M parameters.Source: §C.3 · p. 21
• Linear adds cumulative displacement through a bias-free linear layer in each block, whereas AdaLN encodes cumulative displacement and its per-frame increment into timestep modulation.Linear uses 0.18M parameters and AdaLN uses 66.5M parameters.Source: §C.3 · p. 21

D DATASET

• The recordings comprise 50.9 hours of exploration in two sessions; a calibrated map converts logged player positions to screen displacement, with R2 of 0.99 horizontally and 0.89 vertically.Predicted eight-way directions agree with optical-flow measurements on 87.7% of 42,456 samples, or 98.0% within one neighboring direction.Source: D DATASET · p. 21
• After filtering, the dataset contains 18,528 training clips and 982 held-out evaluation clips, split by entire exploration segments with 16 buffer segments excluded.The clips are 81 frames long, use stride 84, and are resized to 832×480.Source: D DATASET · p. 21

• At 1.0×, the commanded speed is 4.33 px per frame horizontally; motion fidelity tests five speed factors, while directional compliance tests all 982 evaluation first frames.Source: E EVALUATION PROTOCOL · p. 21
• Revisit consistency tests a 40-frame rightward walk followed by a 40-frame leftward walk, returning the commanded camera position to its start.Source: E EVALUATION PROTOCOL · p. 22
• Motion is measured by fitting affine movement from SIFT features with RANSAC, excluding the central 160×160-pixel player box; unmeasurable clips are omitted from medians.Source: E EVALUATION PROTOCOL · p. 22
• Motion fidelity is the through-origin regression slope, directional compliance is median absolute angular error, and revisit metrics compare final and initial frames outside the central box.Before PSNR, color-channel means and standard deviations are matched; SSIM, NCC, DINO similarity, and residual drift are also reported.Source: E EVALUATION PROTOCOL · p. 22
• The out-of-distribution test uses 50 first frames with unseen scenes and art styles, applying the same commands and metrics and measuring motion fidelity on the first 10.Source: E EVALUATION PROTOCOL · p. 22

• The method assumes the static world translates approximately rigidly in the image plane, as in top-down and isometric cameras that follow the player.Source: F LIMITATIONS · p. 22
• It does not model zoom or strong parallax, and moving objects do not follow the world coordinates.Source: F LIMITATIONS · p. 22
• Knownness κ covers only the first-frame footprint, so areas first revealed later are not treated as known when the camera returns.Source: F LIMITATIONS · p. 22