Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey

• When tracking fails, defence depends on information retained from earlier puck observations because the controller cannot know the puck’s current position or direction.Source: §1 · p. 1
• The study distils a history-dependent DreamerV3 teacher into smaller students that learn their own compact history representations without access to the teacher’s latent state.Source: §1 · p. 1
• The controlled comparison varies a diagonal linear recurrent update’s rank-k nonlinear innovation while holding the encoder, action head, 64-dimensional state, and training conditions fixed.The students are compared with feed-forward, finite-history, and GRU-64 policies.Source: §1 · p. 1
• Observation-alias pairs and an intervention on the teacher’s recurrent state at blackout onset test whether preceding trajectories and retained state contribute to defence.Source: §1 · p. 1
• Across five matched seeds, the k=0 model shows no clear clean-observation save-rate gain from nonlinear recurrence through 400 ms and uses 88% fewer recurrent-core parameters than GRU-64.A supplementary test found a higher save rate for k=0 than GRU-64 under tested 5 mm noise and 400 ms blackout.Source: §1 · pp. 1–2
• Under tested 5 mm noise and 400 ms blackout, k=0 retains a higher save rate than GRU-64 without retraining.This changes the observation condition, not the task or puck dynamics.Source: §1 · p. 2

Robot air hockey

• The study isolates temporary puck-tracking loss in a controlled one-shot defence, trading full-match realism for exact replay and paired evaluation.Source: §2 · p. 2

Memory under partial observability

• Across partial-observation tasks, recurrent policies and world-model agents use memory, but which kind of memory a given task needs remains unclear.The cited benchmark finds no memory architecture wins on every task [Morad et al., 2023].Source: §2 · p. 2

Behavioural policy transfer

• Policy distillation transfers a teacher’s actions to a compact student, with dataset aggregation adding teacher-labeled examples from states visited by the student.All student families receive the same aggregation budget.Source: §2 · p. 2

Structured recurrence

• Prior structured linear and low-rank recurrent models motivate a controlled comparison of recurrent nonlinearity, which this work makes by varying rank-k while fixing the teacher, dataset, budget, and remaining architecture.Source: §2 · p. 2

3.1 Teacher and policy boundary

• The frozen DreamerV3 teacher and students share a 19-value public interface; recurrent state and the previous command persist across steps, with state reset only at episode ends except for explicit ablation.Source: §3.1 · pp. 2–3
• At 400 ms blackout, the teacher outperforms the memoryless policy, and erasing teacher state sharply reduces performance; both comparisons use the same 225 shots.Source: Figure 1 · p. 3

3.3 Behavioural policy distillation

• Students imitate the frozen teacher’s clipped actions using MSE, then receive family-specific shadow trajectories relabelled by the teacher before five new models are trained from scratch.Training combines the shared teacher dataset with shadow episodes; checkpoint selection uses offline validation action error rather than closed-loop performance.Source: §3.3 · pp. 3–4

3.2 Structured recurrent student

• The 64-dimensional student varies a diagonal linear memory update with an optional rank-k nonlinear correction, while retaining nonlinear encoding and action heads.The diagonal channels propagate independently; the correction rank controls how many nonlinear combinations of recurrent state can influence the update.Source: §3.2 · p. 3, Figure 2 · p. 4

3.4 Comparisons and measurements

• Comparisons include structured ranks k ∈{0, 1, 2, 4}, a GRU-64, finite-history and memoryless distilled students, with common budgets, matched seeds, and paired evaluations.A memoryless PPO policy is included only as a task-validity control, not as a distilled family.Source: §3.4 · p. 4
• The principal evaluation uses a fresh 225-shot split opened only after all final checkpoints and analyses are frozen, with paired hierarchical bootstrap intervals over seeds and evaluation units.The main measures include save rate, teacher-relative gap, action error, contact-aware outcomes, parameter counts, multiply-adds, and batch-one CPU latency.Source: §3.4 · pp. 4–5
• Supplementary bootstrap intervals are pointwise and unadjusted, while latency is implementation-specific and reported separately from multiply-add cost.Latency timing excludes simulation and rendering and is summarized across five training seeds.Source: §3.4 · p. 5

• The MuJoCo set-piece launches reproducible pucks directly, isolating tracking loss from opponent behaviour and impact variation while representing less than complete air-hockey play.Source: §4 · p. 5
• After a 100 ms visible prefix, the puck is masked for 0–400 ms plus a predeclared 500 ms extrapolation condition; the mallet remains fixed during the prefix.During masking, puck position and visibility are zeroed.Source: §4 · p. 5
• Paired shots reach nearly the same puck position at blackout onset but approach from different trajectories, requiring different actions despite identical current public observations.Across 90 alias families, onset observations are bit-identical and privileged actions differ by at least 0.315 in normalized action space; the guarantee is limited to blackout onset.Source: §4 · p. 5
• The principal test uses a separate 225-shot split held out until evidence release, following independent training, validation, and task-validation shot splits.Source: §4 · p. 5
• Saves comprise returns, arrests, and safe deflections; calibration controls range from 208–209 concessions by inactive or fixed-centre defenders to 214 saves by a privileged controller.The teacher and PPO training reward is distinct from the distilled students’ supervision.Source: §4 · pp. 5–6
• The supplementary noise test evaluates frozen policies on fresh shots with 0, 1, and 5 mm synthetic visible-position noise, without retraining or changing puck dynamics.Noise is independently added across visible coordinates and time; it is not calibrated to a physical tracker.Source: §4 · p. 6

5.1 Memory requirement

• At 400 ms, the teacher saves 96.9% of shots while memoryless PPO saves 33.3%; resetting the teacher state lowers its save rate from 96.0% to 50.2%.The paired reset decrease is 45.8 percentage points (95% CI [38.7, 52.9]).Source: §5.1 · p. 6

5.2 Closed-loop imitation and distribution correction

• After shadow-teacher correction, the k = 2 pilot saves 98.7% without blackout and 98.4% across the paired validation grid.Before correction, a k = 2 pilot trained only on teacher-controlled trajectories saved 67.1% of non-blackout validation shots; the correction experiment diagnoses distribution shift, not a k = 2 advantage.Source: §5.2 · p. 6

5.3 Principal recurrent-complexity comparison

• At 400 ms, recurrent students save 97.8%–98.8% of shots, versus 44.4% for feed-forward, 86.0% for ten-step history, and 98.2% for the teacher.The k = 0 minus GRU-64 difference is 0.5 percentage points (95% CI [−1.2, 2.5]); increasing innovation rank provides no consistent improvement. The 500 ms result is an extrapolation beyond the main range.Source: §5.3 · pp. 6–7, Figure 3 · p. 7, Table 1 · p. 8
• On support shots at 400 ms, k = 0 and GRU-64 each save 91.6%; at 500 ms, their rates are 71.1% and 73.8%, respectively.The 500 ms paired difference is −2.7 percentage points (95% interval [−15.6, 10.7]); perfect observed alias-shot rates do not guarantee performance on unseen shots.Source: §5.3 · p. 7

5.4 Efficiency and compression

• The k = 0 policy uses 2,304 recurrent-core parameters versus 19,200 for GRU-64, and 11,776 rather than 28,352 multiply-adds per step.Batch-one CPU latency decreases from 40.06 µs for GRU-64 to 23.03 µs for k = 0; the paper identifies multiply-adds as the cleaner architecture-level comparison.Source: §5.4 · pp. 7–8, Table 1 · p. 8

5.5 Sensitivity to puck-position noise

• At 5 mm noise and 400 ms blackout, k = 0 saves 98.5% of shots versus 92.9% for GRU-64, a paired difference of 5.6 percentage points.The aggregate result does not establish that the linear update is preferable in every condition: without blackout at 5 mm, k = 4 saves more support shots than k = 0.Source: §5.5 · p. 8, Table 2 · p. 8

Discussion

• At 400 ms, paired comparisons show no clear save-rate gain from higher innovation rank or GRU-64; corrected recurrent students recover aggregate performance close to the teacher.The tested 5 mm noise extension also shows high k = 0 save rates, but does not establish why its performance changes less than GRU-64's.Source: §6 · p. 8
• Approximately ballistic hidden puck motion may favor propagating earlier positions over state-dependent nonlinear computation, but the paper presents this only as a possible explanation.The encoder and action head remain nonlinear; the experiments do not establish this mechanism or explain the noise difference.Source: §6 · pp. 8–9
• Structured k = 0 records 98.3% saves with 12,002 parameters and 23.0 µs latency, versus GRU-64's 97.8%, 28,898 parameters, and 40.1 µs.The paired save-rate difference is 0.5 points, with a 95% interval of [−1.2, 2.5].Source: Figure 4 · p. 9

Discussion and Limitations

• The ten-step stack loses visible puck observations beyond its approximately 200 ms window, despite saving 94.6% of alias shots and 51.6% of support shots at 400 ms.The breakdown cannot determine whether current robot state, earlier actions, or other correlations explain its post-window success.Source: §6 · p. 9
• Student actions create histories unlike the teacher-controlled training data, and imitation alone performs poorly; one shadow-teacher correction round largely closes the k = 2 pilot gap.The correction uses teacher-labelled student trajectories.Source: §6 · p. 9

Limitations

• Evidence is limited to one simulated set-piece, one teacher, a 64-dimensional state, and five seeds; similar aggregate saves do not establish formal equivalence.Correlated or biased errors, repeated occlusions, changing dynamics, and physical deployment remain untested.Source: §6 · p. 9

Conclusion

• At 400 ms, the k = 0 student is 0.1 percentage points above the teacher and 0.5 points above GRU-64; increasing innovation rank yields only small paired differences.The values are rounded to one decimal place, and the conclusion reports no clear save-rate advantage for nonlinear recurrence.Source: §7 · p. 9
• On fresh shots with 5 mm per-coordinate Gaussian noise and 400 ms blackout, frozen k = 0 saves 98.5%, compared with 92.9% for GRU-64.Support-shot differences remain unresolved.Source: §7 · p. 9
• At the same 64-dimensional state size, k = 0 uses 88.0% fewer recurrent-core parameters and 58.5% fewer whole-policy multiply-adds per step than GRU-64.The authors support testing linear recurrent memory as an efficient baseline in this setting, not formal equivalence or general sufficiency.Source: §7 · p. 9