PaperlayerView exact revision on arXiv

Source-linked AI summary

Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics

Zubin Zheng, Jiahao Wu, Shaofeng Zhang, Zhirui Zhang, Yew-Soon Ong, Shengcai Liu

arXiv:2609.36608v1cs.LGcs.CL

TL;DR

In multi-turn OPD, lengthy reasoning before each short action delays environment transitions, while direct actions can degrade rollout quality. ActFirst-OPD uses reference-conditioned actions with an autonomous fallback and asynchronously generates full responses for teacher supervision. Across three benchmarks and student sizes, it speeds training over Vanilla OPD and matches or exceeds compared OPD baselines in mean success rate in eight of nine settings.

  • Problem

    Standard multi-turn OPD rollouts delay environment transitions by requiring lengthy reasoning before each short action, while direct-action rollouts can degrade quality.

    Source: §1 · pp. 1–2

  • Method

    ActFirst-OPD uses reference-conditioned inverse dynamics for fast actions, switches to autonomous next-action prediction after divergence, and asynchronously generates full responses for teacher supervision.

    Source: §1 · p. 2, §4.2 · p. 4

  • Results

    Across three benchmarks and Qwen3 students, ActFirst-OPD averages 2.3× wall-clock training speedups over Vanilla OPD on ALFWorld and matches or exceeds compared OPD baselines in mean success rate in eight of nine settings.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Across the evaluated settings, the results show that reasoning need not block acting during multi-turn agent distillation.

    Source: §6 · p. 10

    The method assumes access to high-quality offline reference trajectories, so settings where these are difficult to obtain are beyond the study’s scope.

    Source: §6 · p. 10

Abstract

from arXiv · show

On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying environment transitions and experience collection. Generating actions directly reduces this delay but can degrade rollout quality. To address this, we propose ActFirst-OPD, an act-first, reason-later training framework that decouples environment interaction from full-response generation. The student infers and executes actions through reference-conditioned inverse dynamics using its current interaction context and a reference next observation, and switches to autonomous next-action prediction when the resulting transition deviates from the reference trajectory. From the collected interaction contexts, the student asynchronously generates full think-then-act responses for token-level teacher supervision. Experiments across 0.6B-, 1.7B-, and 4B-parameter Qwen3 students show that ActFirst-OPD achieves average wall-clock training speedups of $2.3\times$ on ALFWorld, $1.8\times$ on WebShop, and $4.9\times$ on ScienceWorld over Vanilla OPD. It matches or exceeds all compared OPD baselines in mean task success rate across eight of nine benchmark-model settings. These results demonstrate that reasoning need not block acting during multi-turn agent distillation.

1 INTRODUCTION

Multi-turn on-policy distillation (OPD) collects student-generated responses for dense teacher supervision, but reasoning before each action delays environment transitions. ActFirst-OPD separates fast interaction from full-response generation to address this delay while maintaining rollout quality.

1 INTRODUCTION

  • On ALFWorld, reasoning accounts for 81–95% of per-turn rollout time, delaying transitions and increasing training time across turns.The paper characterizes this as a reasoning-blocked transition bottleneck in experience collection.Source: §1 · p. 1
  • Direct-action rollouts reduce delay but show more unproductive repetition and fewer successful trajectories per 100 environment transitions.These comparisons motivate the question of whether agents can advance the environment without waiting for full reasoning while maintaining rollout quality.Source: §1 · p. 2
  • ActFirst-OPD uses reference-conditioned inverse dynamics for fast actions, switches to autonomous next-action prediction after divergence, and asynchronously generates full responses for teacher supervision.Full-response generation is removed from the critical path of environment transitions.Source: §1 · p. 2
  • The work characterizes reasoning-blocked transitions and direct-action quality risks, then combines fast-action methods with asynchronous full-response distillation.It also derives an idealized rollout speedup and upper bound and measures empirical gains.Source: §1 · pp. 2–3
  • Across 0.6B-, 1.7B-, and 4B-parameter Qwen3 students, ActFirst-OPD averages 2.3×, 1.8×, and 4.9× wall-clock training speedups over Vanilla OPD on ALFWorld, WebShop, and ScienceWorld, respectively.It matches or exceeds all compared OPD baselines in mean task success rate across eight of nine benchmark–model settings.Source: §1 · p. 3

2 RELATED WORK

Prior work studies OPD, multi-turn agents, and inverse dynamics. This paper addresses pre-action reasoning delays and the rollout-quality risks of direct action generation.

2 RELATED WORK

  • Multi-turn OPD methods include TCOD, which uses temporal curricula to mitigate trajectory-level KL instability, and TurnOPD, which combines adaptive rollout depth with turn-level loss weighting.The paper positions its work around pre-action reasoning delays and direct-action rollout-quality risks.Source: §2 · p. 3
  • Multi-turn agents support embodied planning and web navigation, while training challenges include sparse or delayed-reward credit assignment and compounding distribution shift across turns.Examples include ReAct, embodied planning, and web navigation.Source: §2 · p. 3
  • RIDM infers actions from the learner’s current observation and an expert’s next observation, and ActFirst-OPD adapts this conditioning to multi-turn OPD.Inverse dynamics has also supported self-supervised reinforcement learning and imitation without expert action annotations.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries define multi-turn interaction contexts and the full-response, token-level supervision used in multi-turn OPD.

3 PRELIMINARIES

  • At each turn, the agent receives an observation and combines it with the task instruction and preceding interaction history to form its input context.The history includes prior observations and actions, but excludes past reasoning.Source: §3 · p. 3
  • Standard think-then-act interaction generates a full response containing reasoning and an executable action, but only the action is submitted to the environment.The environment returns the next observation, and interaction continues until termination or the prescribed turn limit.Source: §3 · p. 3
  • Multi-turn OPD trains a student with a frozen teacher on online student-generated full responses, using token contexts and the student-to-teacher reverse KL objective.The objective’s expectation covers collected turn-level responses and supervised token positions within each response.Source: §3 · p. 4

4 ACTFIRST-OPD

ActFirst-OPD advances the environment with fast actions while generating full responses asynchronously for distillation. It uses reference guidance only while the rollout remains aligned, then predicts actions autonomously.

4 ACTFIRST-OPD

  • ActFirst-OPD separates environment interaction from full-response generation, using reference-conditioned actions and asynchronous full-response generation from collected contexts.This allows subsequent environment interactions to proceed without waiting for reasoning.Source: §4 · p. 4

4.1 REFERENCE-CONDITIONED INVERSE-DYNAMICS ROLLOUT

  • While aligned with a high-quality offline reference trajectory, the student infers a fast action from its actual interaction context and the reference next observation.The executed action advances the rollout from the student’s actual state.Source: §4.1 · p. 4

4.2 AUTONOMOUS NEXT-ACTION PREDICTION FALLBACK

  • If a benchmark-specific transition check fails, the student disables reference-conditioned inverse dynamics for the rest of the rollout and predicts subsequent actions from its interaction context alone.Verification tests whether the resulting observation aligns with the reference target, not whether the action matches the reference action.Source: §4.2 · p. 4

4.3 ASYNCHRONOUS FULL-RESPONSE GENERATION

  • For each fast-rollout context, the student asynchronously generates a full think-then-act response without reference next observations; the teacher scores these responses for the OPD objective.The generated response’s action need not match the executed fast action and is not submitted to the environment; evaluation uses standard think-then-act interaction.Source: §4.3 · p. 5
  • Under idealized assumptions, standard rollout completion takes Tℓ_full, while ActFirst-OPD takes (T − 1)ℓ_fast + ℓ_full; longer rollouts approach a speedup of ℓ_full/ℓ_fast.Realized speedup requires sufficient serving concurrency for asynchronous overlap to offset the additional fast-action workload.Source: §4.3 · pp. 5–6

5 EXPERIMENTS

Across three benchmarks and three Qwen3 student sizes, ActFirst-OPD completes 250 updates faster than the compared OPD baselines while matching or exceeding Vanilla OPD’s mean success rate in eight of nine settings. Ablations and rollout analyses examine the effects of reference conditioning, fallback after divergence, and asynchronous generation.

5.1 EXPERIMENTAL SETUP

  • The evaluation covers ALFWorld, WebShop, and ScienceWorld with Qwen3-0.6B, 1.7B, and 4B students, shared initialization, teachers, and environment settings within each benchmark–model setting.The main comparison uses 250 updates; evaluation reports SR, interaction turns, and wall-clock training time.Source: §5.1 · p. 6

5.2 MAIN RESULTS

  • 2.3× on ALFWorld, 1.8× on WebShop, and 4.9× on ScienceWorld are average training speedups over Vanilla OPD; ActFirst-OPD matches or exceeds its mean SR in eight of nine settings.It also completes 250 updates faster than Vanilla OPD, TCOD-F2B, and TurnOPD.Source: §5.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
  • At approximately 1.5 hours of training, ActFirst-OPD achieves the highest mean evaluation SR among compared OPD methods at all three student sizes, with particularly large gains over Vanilla OPD and TurnOPD.Source: §5.2 · p. 7

5.3 ABLATION STUDIES

  • Removing inverse dynamics lowers mean SR by 17.31, 1.00, and 3.04 percentage points on ALFWorld, WebShop, and ScienceWorld, respectively.Training time also rises from 1.94 to 2.45 hours on ALFWorld and from 2.27 to 2.65 hours on ScienceWorld.Source: §5.3 · p. 8, Table 3 · p. 8
  • Removing NAP fallback lowers mean SR by 1.05, 3.11, and 2.82 points; full replay lowers it by 0.36, 1.89, and 4.22 points across ALFWorld, WebShop, and ScienceWorld.Both replay variants lower mean SR on all three benchmarks; full replay reduces training time, while the 50% variant increases it on WebShop and ScienceWorld.Source: §5.3 · p. 8, Table 3 · p. 8

5.4 ROLLOUT EFFICIENCY AND QUALITY ANALYSIS

  • With 16-token fast actions, increasing concurrent-request cap bmax from 16 to 256 raises measured speedup from 0.80× to 1.49× for 128-token responses and from 0.92× to 2.42× for 512-token responses.The study also varies virtual-turn count T at bmax = 256; limited serving concurrency can constrain speedup relative to idealized estimates.Source: §5.4 · pp. 8–9, Figure 3 · p. 9
  • ActFirst-OPD raises environment-transition throughput from 3.65 to 5.68 transitions per second and executes 30.45% fewer transitions, yielding a 2.24× speedup over the measurement windows.The paper attributes speedup to both higher throughput and fewer environment transitions.Source: §5.4 · pp. 8–9, Figure 3 · p. 9
  • Compared with direct action, ActFirst-OPD has less unproductive repetition, higher successful-trajectory yield, and fewer rollout turns at every checkpoint; versus think-then-act, it has higher yield and fewer turns with broadly comparable repetition.The comparison holds the student checkpoint fixed across rollout strategies, using 3,553 ALFWorld training tasks and a 30-turn limit.Source: §5.4 · pp. 9–10, Figure 4 · p. 9

6 CONCLUSION

ActFirst-OPD combines reference-conditioned inverse dynamics, autonomous next-action prediction, and asynchronous full-response generation. Across three benchmarks and three Qwen3 sizes, it reduces training time versus Vanilla OPD while matching or exceeding compared OPD baselines in mean success across eight of nine settings.

6 CONCLUSION

  • Across three benchmarks and three Qwen3 student sizes, ActFirst-OPD reduces training time versus Vanilla OPD and matches or exceeds compared OPD baselines in mean success across eight of nine settings.The conclusion reports no speedup values here.Source: §6 · p. 10
  • ActFirst-OPD combines reference-conditioned inverse dynamics, autonomous next-action prediction, and asynchronous full-response generation.Source: §6 · p. 10
  • The method assumes high-quality offline reference trajectories, so settings where such references are difficult to obtain are outside this work's scope.Reference next observations are used only for rollout guidance and are disabled after divergence.Source: §6 · p. 10
  • Experiments use ALFWorld, WebShop, and ScienceWorld without human-participant studies or real-world deployment; applications beyond these benchmarks require safety evaluation and controls on environment and tool access.The authors also note potential harmful-use risks and that student models may inherit teacher or reference-trajectory biases and errors.Source: ETHICS STATEMENT · p. 10
  • The paper describes its benchmarks, models, resources, evaluation protocols, assumptions, proofs, prompts, and training details, and identifies an accompanying anonymous code repository.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 PROOF OF PROPOSITION 1

The proof compares idealized completion times for standard think-then-act rollouts and ActFirst-OPD, then bounds realized speedup under finite serving concurrency.

A.1 PROOF OF PROPOSITION 1

  • Under the proof assumptions, a T-turn rollout takes Tℓfull with standard think-then-act interaction and (T −1)ℓfast + ℓfull with ActFirst-OPD.The next context in standard interaction waits for the full response; ActFirst-OPD starts fast-action and full-response requests together, requiring only the fast action for the next context.Source: §A.1 · p. 14
  • The idealized speedup is nondecreasing with rollout horizon T, and the proof establishes that it is at least 1.Source: §A.1 · p. 14

A.2 FINITE-CONCURRENCY CONSTRAINTS

  • The finite-concurrency analysis combines turn-dependency and request-slot constraints to bound realized speedup while retaining Vanilla OPD's completion time under the same concurrency cap.The model considers a fixed pool serving N tasks with T turns, with ActFirst-OPD generating both fast actions and full responses.Source: §A.2 · p. 15
  • The necessary condition for speedup is not sufficient: limited concurrency can prevent asynchronous overlap from offsetting fast-action generation, and increasing bmax only relaxes the constraint.The constraint also depends on rollout horizon T and the latency ratio ℓfast/ℓfull.Source: §A.2 · p. 15

B IMPLEMENTATION DETAILS

This section describes the benchmark-specific implementation of ActFirst-OPD, including prompts and interaction records for separating action-only rollouts from asynchronous response generation, decoding budgets, and OPD optimization. It also details the transition checks that govern reference conditioning and autonomous retries.

B.1 PROMPT DETAILS

  • Prompts share the task, retained observation–action history, current observation, and admissible-action information; only reference-conditioned inverse-dynamics prompts include a reference next observation.The inverse-dynamics prompt asks for the single missing action leading to that next observation; for WebShop search actions, it instructs the model not to add price or budget terms to the query.Source: §B.1 · pp. 16, 18, 19

B.2 HISTORY MANAGEMENT

  • History stores chronological observation–action pairs from rollout, excluding past reasoning and unexecuted asynchronous actions; prompts trim oldest complete pairs until within budget without altering stored history.Each prompt is rendered from a copy of history, and requests may retain different history suffixes because their token lengths differ; current observations and benchmark-specific action or reference information are not shortened.Source: §B.2 · pp. 19–20

B.3 THINKING BUDGET

  • Full think-then-act responses use budgeted decoding with at most two requests, while fast actions follow a separate action-only path.Training limits are 384 first-stage and 512 total tokens; evaluation limits are 1,920 first-stage and 2,048 total, with up to 128 tokens in the second request. During ActFirst-OPD training, the full response supports distillation but its action is not executed; the fast action advances the environment.Source: §B.3 · p. 20

B.4 OPD OPTIMIZATION

  • OPD updates use recent-version turn responses from a FIFO buffer, frozen-teacher token scores, and supervision for generated reasoning and action tokens but not inserted tokens.The distillation coefficient is one, and token advantages use no environment rewards or advantage normalization.Source: §B.4 · pp. 20–21
  • The token objective uses dual-clipped PPO with δ = 0.2 and κ = 3, and aggregates supervised-token losses within each response before averaging across the batch.The log-probability ratio is clipped to [−20, 20] before exponentiation; all generation modes share student parameters, while the teacher remains frozen.Source: §B.4 · p. 21

B.5 TRANSITION CONSISTENCY CHECKS

  • Reference guidance continues only after a valid action and agreement between actual and reference termination; ALFWorld also requires matching normalized observations and, when nonterminal, admissible-action sets.Observation text is normalized before exact comparison; the admissible-action comparison is omitted for terminal transitions.Source: §B.5 · p. 21
  • WebShop checks parsed search and click validity, normalized observations and nonterminal available-action lists, while allowing specified search-result mismatches if the reference product remains clickable.Terminal transitions compare normalized observations only; the repeated instruction block stays in comparisons, though it is removed from the inverse-dynamics reference observation.Source: §B.5 · p. 22
  • ScienceWorld validates actions against the full admissible-action list and compares normalized observations and action-template sets, excluding possible-object aliases and separate room, inventory, reward, and score fields.Normalization sorts lines and object enumerations; template duplicates and ordering are ignored.Source: §B.5 · p. 22
  • Across benchmarks, a failed check permanently disables reference conditioning without resetting the environment; an invalid proposal triggers one autonomous next-action request from unchanged context.Reference actions support internal checks but are not shown in the student prompt.Source: §B.5 · p. 22

C.1 BENCHMARKS

The benchmarks cover embodied planning, web navigation, and scientific reasoning, with separate training and evaluation instances. They report success and interaction turns, with partial-completion scores for WebShop and ScienceWorld.

Benchmark setup

  • Training and evaluation task instances are disjoint across the three benchmarks.The benchmark configurations summarize task counts and interaction limits.Source: §C.1 · p. 22

ALFWorld

  • ALFWorld covers six household task types, with training and evaluation split across seen and unseen rooms and configurations.The seen split uses rooms encountered during training with new object configurations; the unseen split uses held-out rooms with different layouts and receptacles.Source: §C.1 · p. 22

WebShop

  • WebShop evaluates product search and purchase against language instructions, using training goal IDs 0–4095 and held-out evaluation IDs 4096–4195.The text interface includes click actions and a student-generated search query when search is available.Source: §C.1 · p. 23

ScienceWorld

  • ScienceWorld covers interactive science tasks; evaluation includes 85 instances from task types seen in training and 65 from the remaining 13 types.Training uses the first half of variations from 17 task types; evaluation uses the last five variations from each of all 30 task types.Source: §C.1 · p. 23

Evaluation metrics

  • The reported metrics are success rate and average interaction turns; WebShop and ScienceWorld also report partial-completion scores.WebShop scores terminal purchase reward on a 0–100 scale, while ScienceWorld uses its highest episode score, clipped to 0–100; success requires score ≥99.999 in both.Source: §C.1 · p. 23

C.2 TEACHER TRAINING AND REFERENCE CONSTRUCTION

The experiments use benchmark-specific Qwen3-8B teachers and replay-validated reference trajectories selected for success or score. Training runs asynchronously, and evaluation uses think-then-act interaction with results reported across three seeds.

C.2 TEACHER TRAINING AND REFERENCE CONSTRUCTION

  • Each benchmark uses a task-specialized, frozen Qwen3-8B teacher; ALFWorld and WebShop teachers use SFT followed by GiGPO, while ScienceWorld uses GiGPO directly.Table 5 reports teacher-training and candidate-rollout collection costs separately.Source: §C.2 · p. 23, Table 5 · p. 24
  • Reference trajectories are selected from task-matched candidates by success or score, favoring shorter action sequences, then validated through environment replay.Sources include public demonstrations, teacher rollouts, and benchmark-specific oracle, simulator-generated, or scripted solutions; selected actions are replayed offline and their observations cached as transition targets.Source: §C.2 · pp. 23–24
  • Teacher training and reference construction precede student training and are excluded from reported training wall-clock time; prepared teachers and caches can be reused for fixed tasks and environments.The preparation costs are therefore one-time costs for repeated runs or student sizes.Source: §C.2 · p. 24

C.3 TRAINING PROTOCOLS AND HYPERPARAMETERS

  • Rollout collection and student optimization proceed asynchronously, with training configurations summarized across benchmarks.Table 6 covers the training hyperparameters.Source: §C.3 · p. 24
  • Reported training wall-clock time includes initialization, rollout collection, teacher scoring, updates, checkpoint saving, and shutdown, but excludes offline teacher training, reference construction, and evaluation.Table 1 and Table 2 use this timing scope.Source: §C.3 · p. 24

C.4 EVALUATION PROTOCOLS AND HYPERPARAMETERS

  • Evaluation uses standard think-then-act interaction without reference information, and task metrics are reported as means and sample standard deviations across seeds 42, 43, and 44.Tables 1 and 2 and Figure 5 report these statistics.Source: §C.4 · p. 24

C.5 ROLLOUT EFFICIENCY AND QUALITY MEASUREMENTS

The measurements examine how concurrent-request limits and rollout horizon affect rollout efficiency, alongside transition throughput and rollout quality. They specify controlled workloads and operational definitions for each metric.

C.5 ROLLOUT EFFICIENCY AND QUALITY MEASUREMENTS

  • Rollout-efficiency measurements use 1,024 ALFWorld contexts, varying the concurrent-request cap and virtual interaction horizon for fast-action and full-response requests.The contexts are sampled across six training task types; fast actions use an action-only prompt without reference information.Source: §C.5 · p. 24, Table 8 · p. 27
  • Paired rollout timing compares scheduling in which ActFirst-OPD advances after fast-action completion against Vanilla OPD, which waits for each full response.Timing includes queueing, prefill, decoding, and scheduling; paired repeats use the same engine, contexts, and full-response seeds.Source: §C.5 · pp. 24–26
  • Environment-transition throughput is measured in env. steps / s, counting completed transitions within early and full-training windows.The ALFWorld Qwen3-1.7B measurements use separate training runs; the windows cover the first 600 seconds and training through update 250.Source: §C.5 · pp. 26–27, Table 9 · p. 27
  • Rollout-quality comparisons test think-then-act, direct action without reference observations, and reference-conditioned ActFirst-OPD with autonomous fallback on the same frozen checkpoints.The comparison uses one trajectory per each of 3,553 ALFWorld training tasks at updates 1, 125, and 250, with matched sampling seeds; ActFirst-OPD omits asynchronous response generation and training updates in this measurement.Source: §C.5 · p. 27
  • Quality metrics define Repeat as the share of tasks with a repetition, Yield as successful trajectories per 100 environment transitions, and T as mean executed turns.A valid action is a repetition when the same state–action pair recurs without a new observation; all failed trajectories in this collection reach the 30-turn limit.Source: §C.5 · p. 28

D EXTENDED RELATED WORK

ActFirst-OPD focuses on reducing reasoning-induced delays in online experience collection while retaining full-response distillation. The discussion distinguishes turn-level on-policy sampling from the distribution of interaction contexts and notes that the latter may differ from think-then-act inference.

D EXTENDED RELATED WORK

  • ActFirst-OPD retains online environment interaction and full-response distillation while removing reasoning from the critical path of environment transitions.Reference next observations guide fast actions, followed by autonomous next-action prediction after divergence.Source: D EXTENDED RELATED WORK · p. 28
  • Figure 5 compares ALFWorld success rate against cumulative training time for three Qwen3 sizes, with ±1 standard deviation shading and a two-hour marker.Stars identify the final checkpoints in Table 1.Source: Figure 5 · p. 29

E ADDITIONAL DISCUSSION

  • Full responses are sampled on-policy conditional on collected contexts, but those contexts need not match the distribution induced by think-then-act interaction.The student generates these full responses without conditioning on the reference next observation; their actions are not executed.Source: E ADDITIONAL DISCUSSION · p. 29
  • Reference guidance may reduce exposure to states arising from accumulated errors, but the evaluated results do not establish that inference-time exposure bias is eliminated or uniformly reduced.Autonomous continuation after deviations remains action-only and need not induce the think-then-act context distribution.Source: E ADDITIONAL DISCUSSION · p. 29

F ADDITIONAL EXPERIMENTAL RESULTS

On ALFWorld, ActFirst-OPD reaches higher success rates than Vanilla OPD at approximately 1.5 hours across three student sizes. Alignment analyses show that initial matching does not guarantee full reference following, while larger students achieve higher cumulative alignment and turn coverage.

F ADDITIONAL EXPERIMENTAL RESULTS

  • At approximately 1.5 hours, ActFirst-OPD reaches SRs of 58.39%, 69.34%, and 80.66% for 0.6B, 1.7B, and 4B students, versus 5.72%, 25.43%, and 42.94% under Vanilla OPD.ActFirst-OPD completes 250 updates within two hours at all three sizes.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 29
  • Figures 6–8 report recent-batch first-action, full-trajectory, and aligned-turn rates; Table 10 reports cumulative rates through update 250.Window rates use the latest 10 completed rollout batches, or all available batches when fewer than 10 have completed.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 29, Figure 6 · p. 30, Table 10 · p. 30
  • Larger students achieve higher cumulative full-trajectory alignment and aligned-turn coverage within each benchmark; first-action alignment exceeds full-trajectory alignment.A matching initial transition does not guarantee complete reference following, and aligned-turn coverage includes prefixes of rollouts that later diverge.Source: F ADDITIONAL EXPERIMENTAL RESULTS · p. 30, Table 10 · p. 30
Loading 2609.36608v1…