PaperlayerView exact revision on arXiv

Source-linked AI summary

Delayed Supervision for Test-Time Language Models

Jinha Kim, Taksh Kothari

arXiv:2609.32312v1cs.AI

TL;DR

Next-token prediction does not explicitly require test-time memory to retain facts across many updates, even though later questions may require retrieving or revising them. The paper trains this memory with delayed, simulator-grounded questions on disposable branches alongside ordinary event prediction. Across reported LaCT, plain DeltaNet, and RWKV-7 comparisons, scores improve over event-only training; LaCT gains 5.48 percentage points on BABILong over event NTP.

  • Problem

    Next-token prediction need not strongly supervise facts that are not needed for local text, leaving open whether a model can answer delayed questions about its state.

    Source: §1 · pp. 1–2

  • Method

    The paper adds simulator-grounded questions after intervening events, supervises answers on disposable branches, and combines this with event prediction.

    Source: §1 · p. 2, §3.1 · p. 3, §3.3 · p. 4

  • Results

    Across reported LaCT, plain DeltaNet, and RWKV-7 comparisons, delayed supervision improves aggregate BABILong and single-needle RULER scores over event-only training; LaCT gains 5.48 percentage points over event NTP on BABILong.

    Source: §5.1 · p. 5, §8 · p. 8

  • Takeaways & Limitations

    The results support future semantic usefulness as an outer training target for test-time memory, including retaining the latest valid state after revisions.

    Source: §7 · p. 7

    The comparisons do not isolate delay’s causal effect, and answer-plus-EOS supervision also teaches formatting and stopping.

    Source: §7 · p. 8

Abstract

from arXiv · show

Test-time language models adapt a compact memory while processing the input sequence. This perspective encompasses nonlinear fast-weight learning in LaCT, associative delta-rule updates in DeltaNet, and generalized delta-rule state updates in RWKV-7. Training these models to predict the next token does not explicitly require a fact to remain accessible after many subsequent memory updates. We study delayed supervision for this test-time memory: during post-training, ask a simulator-grounded question only after a long interval of unrelated events, and supervise its answer alongside ordinary next-token prediction. Questions are evaluated on disposable branches, so their answers never enter the continuing event stream. The construction distinguishes retention from revision: a retained fact must remain valid throughout the delay, whereas a revised fact must be answered with its latest value. We evaluate this approach on LaCT-760M and plain DeltaNet-1.3B using TextWorld training trajectories and shared BABILong and RULER evaluation panels, and include a separately reported RWKV-7 comparison. Relative to event-only training, delayed QA improves BABILong by 5.48 percentage points for LaCT and 1.32 points for DeltaNet, and single-needle RULER by 1.45 and 3.27 points, respectively. The RWKV-7 comparison reports gains of 4.60 and 7.00 points on its own panels. These results support delayed semantic supervision as a practical outer training objective for usable test-time memory, while leaving open how much of the benefit derives specifically from delay rather than general question-answering and answer-termination supervision.

1 INTRODUCTION

Test-time memory must retain still-valid facts and revise facts when they change, yet next-token prediction need not teach retrieval after long intervening activity. The paper proposes delayed question supervision while preserving the event objective and isolating probe answers from the continuing stream.

1 INTRODUCTION

  • LaCT, DeltaNet, and RWKV-7 share context-adapted memory while retaining distinct update mechanisms.The paper frames these sequence models through their connection to test-time learning.Source: §1 · p. 1
  • Next-token prediction need not strongly supervise an old fact when it is not needed for local text, leaving a gap between continuing a stream and answering a delayed state question.A useful memory must retain a fact that remains valid and update one that changes.Source: §1 · pp. 1–2
  • Delayed question-answer supervision is added to event next-token prediction after substantial intervening activity, using the existing language-model head.Its gradients can shape earlier writes, intervening state transitions, and eventual readout without changing the native test-time update rule.Source: §1 · p. 2
  • The contribution combines validity-checked probes that handle revisions with isolated branches and transfer evaluation, but the comparisons do not identify delay as the sole cause of improvement.The evaluation transfers from TextWorld to long-context QA and retrieval using original pretrained and event-NTP baselines.Source: §1 · p. 2

2 TEST-TIME LANGUAGE MODELS

Test-time language models update sequence-specific memory while processing input, using distinct fast-weight or associative mechanisms. The paper applies delayed semantic supervision as an outer objective that preserves each backbone’s native state update.

2 TEST-TIME LANGUAGE MODELS

  • Test-time memory is sequence-specific fast weights or associative state updated from the current input during inference, while learned parameters remain fixed at benchmark inference.The memory state evolves from the token stream, with optional local context for architectures combining recurrence and windowed attention.Source: §2 · p. 2
  • In TTT layers, a context-dependent update changes a fast predictor’s parameters during sequence processing, and the fast weights serve as sequence-specific memory.Slow parameters learn how to construct the update and use its output; inner updates may operate over chunks rather than tokens.Source: §2 · pp. 2–3
  • Delta-rule memory corrects a previous key–value mapping using prediction error rather than merely accumulating an outer product; RWKV-7 adds vector-valued gating and distinct removal and insertion keys.The paper does not assume RWKV-7’s native update is the simple squared-error step shown for the delta-rule family.Source: §2 · p. 3
  • Delayed questions train the outer objective while preserving each backbone’s native state update, with no ground-truth probe labels or auxiliary optimizer at benchmark inference.Auxiliary question gradients teach slow parameters how to form useful adaptive memory.Source: §2 · p. 3

3 DELAYED SUPERVISION OF TEST-TIME MEMORY

The method creates simulator-grounded questions about a trajectory’s state, delays them until after intervening events, and combines their answer supervision with event next-token prediction. Independent disposable branches keep probe answers out of the continuing trajectory while training the model’s ordinary answer pathway.

3.1 A STATE-GROUNDED TASK

  • A probe attaches a question and target answer to a trajectory, with sufficient evidence at s and a delayed endpoint t where the simulator state determines the answer.Retention probes require the answer and dependencies to stay unchanged; revision probes begin after the last relevant change and target the current value.Source: §3.1 · p. 3
  • Accepted LaCT and DeltaNet probes span at least 4,096 native tokens and 16 intervening state-changing writes, measured separately as Dtok and Dwrite.TextWorld questions cover locations, inventories, attributes, counts, and relations; the core DeltaNet set has eight paired retention and revision questions per world.Source: §3.1 · p. 3

3.2 AN AUXILIARY ANSWER OBJECTIVE

  • The auxiliary objective appends each delayed question to its event-only prefix, scores the answer tokens through EOS, and combines answer loss with event NTP across worlds.Context and question tokens receive no auxiliary targets, but remain differentiable so answer loss can train evidence encoding and survival; separate token normalization does not guarantee equal gradient magnitudes.Source: §3.2 · p. 4

3.3 DISPOSABLE BRANCHES AND ORDINARY INFERENCE

  • Each probe branches independently from an event-only prefix, and its question, answer, and branch state are discarded after scoring.The target is the current semantic answer, including revisions, and inference uses ordinary generation without the training-only branch construction.Source: §3.3 · p. 4

4 EXPERIMENTAL SETUP

The experiments compare original checkpoints, event-only post-training, and delayed QA for LaCT and DeltaNet, with a separate RWKV-7 comparison. Evaluation uses official BABILong and RULER panels, with the RWKV-7 panels reported separately.

4.1 MODELS AND CONTROLS

  • The three conditions are Original, event-only NTP, and Delayed QA; LaCT-760M and plain DeltaNet-1.3B share the principal comparison, while RWKV-7 is reported separately.The RWKV-7 original-model score is unavailable and is not inferred.Source: §4.1 · p. 4
  • The architectures provide separate within-backbone tests, not a controlled architecture ranking, because pretraining, tokenization, probe budgets, delay placement, and loss weights differ.The shared-panel LaCT NTP checkpoint also used a newer execution environment than the retained QA checkpoint.Source: §4.1 · p. 4

4.2 EXTERNAL EVALUATION

  • LaCT and DeltaNet share official BABILong and single-needle RULER panels, whereas RWKV-7 uses separate BABILong and RULER panels that are not pooled with the shared panel.Evaluation uses official prompts and scoring with frozen examples; external benchmark answers do not supply training labels.Source: §4.2 · p. 5, Table 1 · p. 5

5 RESULTS

Delayed QA improves transfer to long-context benchmarks for LaCT and DeltaNet, while the context-length and complementary-task panels show that gains are not uniform. A separately reported RWKV-7 comparison also improves on its own panels.

5.1 TRANSFER BEYOND EVENT PREDICTION

  • LaCT rises from 25.60% to 31.08% on BABILong and from 87.39% to 88.85% on single-needle RULER; DeltaNet rises from 21.16% to 22.48% and 80.24% to 83.52%, respectively.Figure 2 reports positive gains over event NTP on both benchmarks for each backbone.Source: §5.1 · p. 5, Figure 2 · p. 6
  • DeltaNet's BABILong point estimate is positive but its paired example-bootstrap interval [−0.16, 2.84] points does not exclude zero.The intervals reflect evaluation-example variation, not variation across training seeds.Source: §5.1 · pp. 5–6
  • RWKV-7 increases from 22.60% to 27.20% on BABILong and from 68.83% to 75.83% on RULER in a separate comparison.Its different training budget and evaluation panel preclude direct magnitude comparisons with LaCT and DeltaNet.Source: §5.1 · p. 6

5.2 BEHAVIOR AS CONTEXT GROWS

  • LaCT exceeds event NTP at every evaluated BABILong length, while DeltaNet gains most at 4K and 8K but regresses at 1K and 2K.At 8K, LaCT scores 19.60% versus 14.60%; DeltaNet scores 9.60% versus 4.80%.Source: §5.2 · p. 6, Figure 3 · p. 6
  • All conditions remain substantially less accurate on longer BABILong inputs, so the auxiliary objective improves the evaluated range without eliminating long-context difficulty.The small absolute DeltaNet score at 8K should not be obscured by its large relative gain.Source: §5.2 · p. 6

5.3 BEYOND SINGLE-NEEDLE RETRIEVAL

  • The complementary DeltaNet panel shows selective transfer: variable tracking, frequent-word extraction, and document QA improve, while multi-value retrieval and common-word extraction decline.Table 2 reports task-specific official metrics, including partial-target credit for synthetic tasks, rather than whole-response exact match.Source: §5.3 · p. 7, Table 2 · p. 7

6 RELATED WORK

Related work spans learned test-time memory mechanisms, objectives beyond next-token prediction, and benchmarks for controlled environments and long-context evaluation. This paper distinguishes its approach through simulator-verified delayed semantic questions and transfer evaluation.

Learning at test time

Supervision beyond the next token

  • Unlike ReFINE (Hwang et al., 2026), the method uses simulator-verified delayed semantic questions, directly supervises answer tokens, and requires no sampled reward optimization.Its distinction is the state-grounded construction with retention and revision validity, causal branch isolation, and transfer evaluation.Source: §6 · p. 7

Controlled environments and long-context evaluation

  • TextWorld, BABILong, and RULER support evaluation across in-environment supervision and external QA or retrieval, though both settings can share semantic skills.TextWorld supplies generated environments with explicit state, while BABILong and RULER provide long-context evaluation resources.Source: §6 · p. 7

7 DISCUSSION AND LIMITATIONS

Delayed semantic supervision can target future use of test-time memory and accommodate revised facts. The reported comparisons leave the specific role of delay unresolved and have boundaries in computation, model settings, reproducibility, and training-distribution scope.

7 DISCUSSION AND LIMITATIONS

  • Delayed supervision rewards retaining the latest valid state rather than preserving every historical association indefinitely.A delayed answer uses the model’s ordinary readout, coupling storage to the behavior ultimately evaluated.Source: §7 · p. 7
  • The comparisons do not isolate delay’s causal effect, and answer-plus-EOS supervision also teaches formatting and stopping.The study does not claim delayed questions universally outperform otherwise equivalent immediate questions; benchmark gains alone do not prove a purely mnemonic mechanism.Source: §7 · p. 8
  • Training matches event exposures, not total computation, and cross-backbone recipes and execution environments differ.The reported RWKV comparison also requires prediction artifacts to be integrated into the repository for independent reproduction.Source: §7 · p. 8
  • TextWorld’s symbolic structure makes labels reliable but narrows training, leaving transfer to unstructured factual revision, dialogue, and natural long histories open.Source: §7 · p. 8

8 CONCLUSION

The conclusion presents delayed supervision as a way to train test-time memory for later semantic use, reporting improved benchmark scores over event-only training. It frames the approach as a complement to native test-time learning while calling for further controlled replication.

8 CONCLUSION

  • Delayed semantic supervision improves aggregate BABILong and single-needle RULER scores over event-only training across the reported LaCT, plain DeltaNet, and RWKV-7 comparisons.Source: §8 · p. 8
  • The objective preserves event NTP, supervises answer tokens on isolated branches, and distinguishes retained facts from revised values.Source: §8 · p. 8
  • Further controlled replication is needed to establish generality and isolate the role of delay.Source: §8 · p. 8

REPRODUCIBILITY STATEMENT

The paper provides code, pinned protocols, score records, machine-readable result provenance, and scripts for regenerating figures; RWKV-7 values are identified as transcribed from an author-supplied corrective report.

REPRODUCIBILITY STATEMENT

A TRAINING AND EVALUATION DETAILS

The appendix details model initialization, training recipes, probe validation, and benchmark scope. The LaCT and DeltaNet panels share frozen evaluation inputs, while the separately described complementary RULER panel is not the complete suite.

A TRAINING AND EVALUATION DETAILS

  • Table 3 reports QA-arm training settings, while its NTP controls match event exposure within each comparison and omit the auxiliary loss.Source: Table 3 · p. 9
  • LaCT uses a public 760M model with 2,048-token native updates and local attention; DeltaNet uses a pinned 1.3B checkpoint with its native architecture and recurrence.No architecture is retrained from random initialization; documented recipes use FP32 weights with BF16 autocast.Source: A TRAINING AND EVALUATION DETAILS · pp. 9–10
  • DeltaNet’s auxiliary coefficient was fixed from training-only gradient calibration, while LaCT retains equal scalar allocation; these are not a common cross-model weighting ablation.The observed DeltaNet ratios ranged from 12.45 to 15.78; the coefficient does not establish a universal optimal weight.Source: A TRAINING AND EVALUATION DETAILS · p. 10
  • The DeltaNet paired-probe set has 4,096 unique questions, with symbolic answers checked against an independent raw-fact oracle throughout the relevant interval.Retention versus revision is classified separately, and terminal LaCT probes are additional to its paired delayed-probe budget.Source: A TRAINING AND EVALUATION DETAILS · p. 10
  • LaCT and DeltaNet share 25 frozen BABILong task–length cells and 825 frozen single-needle RULER inputs; nominal lengths do not guarantee equal native token counts.The complementary RULER panel has 50 examples in each of ten cells and is not the complete suite.Source: A TRAINING AND EVALUATION DETAILS · p. 10

B SEPARATELY REPORTED RWKV-7 LENGTH RESULTS

The section reports RWKV-7 results as a distinct corrective comparison, documents evidence provenance and research-assistant involvement, and states copyright and disclosure terms.

B SEPARATELY REPORTED RWKV-7 LENGTH RESULTS

C EVIDENCE PROVENANCE AND RESEARCH-ASSISTANCE DISCLOSURE

  • LaCT and DeltaNet scores are checked against saved records, while RWKV-7 values remain report-level evidence pending prediction and verification files.These checks do not constitute a new execution of every benchmark prediction.Source: C EVIDENCE PROVENANCE AND… · p. 10
  • An AI coding and writing assistant helped inspect records, retrieve metadata, draft the manuscript, produce figures and scripts, and participate in implementation and diagnostics.The authors remain responsible for reviewing the experimental claims, citations, and publication materials.Source: C EVIDENCE PROVENANCE AND… · p. 10

COPYRIGHT AND DISCLOSURE NOTICE

  • The notice reserves reproduction and redistribution of original content except where applicable law, granted licenses, or prior permission allow it.It also states that the notice does not restrict arXiv’s rights under the submission’s selected license.Source: COPYRIGHT AND DISCLOSURE NOTICE · p. 11

Related papers

Loading 2609.32312v1…