PaperlayerView exact revision on arXiv

Source-linked AI summary

MAS-OPD: On-Policy Distillation for Multi-agent Systems

Qiyong Zhong, Mao Zheng, Mingyang Song, Houcheng Jiang, Jiajie Su, Huwei Ji, Li Zhang, Junfeng Fang

arXiv:2609.34234v1cs.CL

TL;DR

Jointly post-training multi-agent systems with OPD remains underexplored, despite gaps in credit assignment and role specialization. MAS-OPD adds role-contrastive and teacher-only conflict-attribution supervision; it achieves the highest mean score at both student scales and yields clearer specialization and more effective collaboration. The study focuses on a two-role workflow and assumes access to a verifier during training.

  • Problem

    OPD provides token-level supervision without local rewards, but how to use it for interdependent MAS agents while distinguishing role-specific behavior and supervising coordination remains underexplored.

    Source: §1 · p. 2

  • Method

    MAS-OPD combines target-versus-non-target role teacher signals with training-only conflict attribution supplied as privileged information to the teacher.

    Source: §1 · pp. 2–3, §3.2 · p. 5, §3.3 · p. 6

  • Results

    MAS-OPD attains the highest mean score on every benchmark at both student scales and develops clearer role specialization and more effective collaboration than a prompt-only system.

    Source: §4.2 · p. 7, §6 · p. 10

  • Takeaways & Limitations

    The study concludes that OPD can jointly train role division and collaboration by adding role advantage and teacher-only privileged attribution to multi-agent distillation.

    Source: §6 · p. 10

    The evaluated workflow uses two complementary roles and assumes an accessible task verifier during training.

    Source: §2.1 · p. 3, §3.3 · p. 5

Abstract

from arXiv · show

Multi-agent systems (MAS) split a task across specialized roles and are promising on complex tasks, yet a prevailing approach relies on inference-time orchestration alone. General-purpose APIs are costly and hard to customize, while small models with role prompts rarely develop stable role competence or reliable collaboration, so post-training a MAS jointly is central. Most attempts use reinforcement learning, whose team-level reward leaves undetermined which step of which agent brought about the outcome, while local rewards need redesigning per task. On-policy distillation (OPD) gives token-level teacher supervision on trajectories the student samples, a denser signal needing no local reward, yet is underexplored for the interdependent agents of a MAS. Two difficulties arise: building complementary specialization from a judgement of which role a behavior belongs to while preserving the knowledge all roles need, and turning cross-agent collaborative information into supervision OPD can exploit. We present MAS-OPD, where Role-Advantage Specialization defines the role advantage as the difference between the teacher signals under target and non-target role conditions, and Privileged Attribution for Coordination attributes an interaction conflict to its source and supplies it to the teacher alone as privileged information. Extensive experiments on code and mathematics benchmarks show that MAS-OPD attains the highest mean score at both student scales and leads the agents to develop clearer role specialization and more effective collaborative behavior.

1 INTRODUCTION

Joint post-training is needed to make multi-agent systems efficient and customizable, but current approaches leave key gaps in credit assignment and coordination. MAS-OPD extends on-policy distillation with role-specific and coordination supervision, and improves benchmark performance while sharpening specialization and collaboration.

1 INTRODUCTION

  • Joint post-training is central because API-based systems are costly to customize, while small prompted models rarely develop stable role competence or reliable collaboration.Inference-time orchestration alone is a prevailing approach, but does not address those customization and competence issues.Source: §1 · p. 1
  • Team-level reinforcement-learning rewards do not identify which agent step caused an outcome, while local rewards require task-specific redesign.Local feedback remains at trajectory or turn granularity and must be redesigned when task format, role responsibilities, or protocol changes.Source: §1 · pp. 1–2
  • OPD offers token-level teacher supervision on student-sampled trajectories without local role rewards, but its use for interdependent MAS agents remains underexplored.The two challenges are assigning behaviors to complementary roles while preserving shared knowledge, and turning cross-agent collaboration into supervision.Source: §1 · p. 2
  • MAS-OPD uses Role-Advantage Specialization to distinguish target-role from non-target-role behavior and Privileged Attribution for Coordination to inform teacher supervision of conflicts.The privileged attribution is supplied to the teacher alone; experiments report better task performance, clearer role specialization, and more effective collaboration.Source: §1 · pp. 2–3

2 PRELIMINARIES

The preliminaries define a multi-agent workflow in which independent role policies interact through shared environment feedback, then recap on-policy distillation as teacher supervision of student-sampled outputs.

2.1 MULTI-AGENT LANGUAGE MODEL SYSTEMS

  • Agents have predefined roles and independent policies initialized from the same base model, and their joint outputs produce environment feedback over a multi-turn interaction.The workflow focuses on two complementary roles that independently produce outputs and revise in response to reported disagreement until agreement or the turn limit.Source: §2.1 · p. 3

2.2 ON-POLICY DISTILLATION

  • OPD samples a response from the student and has a frozen teacher score its tokens, providing supervision on states the student visits.The described objective uses reverse KL and token-level advantages; the paradigm targets a single student whose context the teacher shares.Source: §2.2 · p. 3

3 METHODOLOGY

MAS-OPD extends OPD to joint multi-agent rollouts by routing teacher supervision to the generating role, differentiating role-specific behaviors, and giving the teacher training-only conflict attributions. The joint objective updates independent policies using the resulting token-level signals.

3.1 ROLE-CONDITIONED MULTI-AGENT DISTILLATION

  • Each role policy contributes responses to a joint on-policy trajectory, and the teacher supervises each response individually so gradients route to its generating agent.The teacher uses the student's context with the role made explicit, while evaluating the same student-generated tokens.Source: §3.1 · p. 4

3.2 ROLE-ADVANTAGE SPECIALIZATION

  • Role-Advantage Specialization contrasts teacher evaluations under target and non-target role conditions to add role attribution to token-level distillation.The combined signal adds the role advantage to the OPD advantage; λ controls differentiation strength, and λ = 0 recovers the role-conditioned OPD form.Source: §3.2 · p. 5

3.3 PRIVILEGED ATTRIBUTION FOR COORDINATION

  • Privileged Attribution for Coordination uses a training-time verifier to locate which output is at fault and gives the resulting attribution to the teacher, not the student.Attributions enter the teacher context only after a turn is complete, so they are unavailable for scoring that turn; the student continues to generate from its original input.Source: §3.3 · pp. 5–6

3.4 JOINT TRAINING OBJECTIVE

  • The joint objective combines PAC-reconstructed teacher context with RAS role conditioning in the same token-level advantage, updating each role's policy from joint rollouts.The method averages over tokens, turns, and roles; relative to the base method, it adds one non-target-role teacher force-decode per response.Source: §3.4 · pp. 6–7

4 EXPERIMENT

Experiments across code and mathematics benchmarks show MAS-OPD leading the tested baselines at both student scales. Ablations and behavioral analyses associate its results with role specialization and conflict resolution learned through privileged attribution.

4.2 OVERALL PERFORMANCE

  • MAS-OPD attains the highest mean score on every benchmark at both student scales, improving over each column’s strongest baseline by more than two points on average.The 4B system also exceeds the 14B single-agent teacher on all six benchmarks.Source: §4.2 · p. 7
  • Across the two tested workflows, extra turns help when shared between two roles: prompt-only MAS beats the prompt-only single agent on every benchmark.Single-agent self-revision falls below one-turn answering at both scales, including after GRPO training.Source: §4.2 · p. 7
  • MAGRPO falls below untrained MAS on three of six benchmarks at 1.7B, while MAS-OPD improves on untrained MAS in all twelve scale-by-benchmark cells.RL baselines also show standard deviations near five points on AIME; MAS-OPD has the smallest multi-agent spread on AIME24 at both scales.Source: §4.2 · p. 7

4.3 ROLE SPECIALIZATION •

  • Removing either module reduces accuracy on all six benchmarks; role advantage accounts for the larger ablation gap on code, while privileged attribution accounts for more on mathematics.Table 2 compares 1.7B accuracy with each module removed; setting λ=0 removes role advantage, and restoring student-visible context removes privileged attribution.Source: §4.3 · pp. 7–8, Table 2 · p. 8
  • Every trained system loses accuracy when Coder and Tester policies are exchanged, while prompt-only MAS does not; MAS-OPD has the highest own-role accuracy among the four systems.Without RAS, exchange still reduces accuracy, consistent with role conditioning already providing role-dependent teacher targets.Source: §4.3 · p. 8, Figure 3 · p. 8
  • Increasing λ raises the exchange penalty, with most growth below the reported setting, while Coder-alone accuracy stays within run-to-run variation of the λ=0 reference.The penalty gains 3.80 points up to the reported setting and a further 1.22 over the fourfold increase beyond it.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 PRIVILEGED ATTRIBUTION •

  • On disagreement turns, MAS-OPD improves the share of conflicts resolved in the direction of each role’s correctness compared with training without privileged attribution.Neither system receives the verifier at evaluation, so students make the attribution judgment without it.Source: §4.4 · p. 8, Figure 4 · p. 9
  • Privileged attribution reduces both erroneous deference and erroneous persistence, while trained systems reach agreement in fewer turns and MAS-OPD does so sooner at every checkpoint.The workflow allows four turns; the reductions in turns to agreement grow smaller as training continues.Source: §4.4 · p. 9, Figure 4 · p. 9

5 RELATED WORK

Prior MAS post-training commonly uses reinforcement learning, while on-policy distillation has been developed for single students. These lines leave trajectory-level credit assignment and role-specific supervision as open concerns for multi-agent settings.

RL Training for MAS

  • Multi-agent reinforcement learning typically rewards a trajectory or turn, leaving decisions within it without a target.Extensions often restrict interaction or adapt single-agent algorithms.Source: §5 · p. 9

On-Policy Distillation

  • On-policy distillation supervises student-generated trajectories, but prior extensions addressed one student, not role attribution or decisions dependent on a teammate.It removes mismatch between a fixed corpus and the states the student visits.Source: §5 · pp. 9–10

6 CONCLUSION

MAS-OPD extends on-policy distillation to jointly train role division and collaboration in multi-agent systems. It leads the tested benchmarks at both student scales, with sharper specialization and more effective collaboration than a prompt-only system.

6 CONCLUSION

  • MAS-OPD combines role advantage, which retains distinctions between target and non-target role conditions, with privileged attribution supplied to the teacher alone during training.The paper reports the highest mean score on every benchmark at both student scales and sharper specialization and collaboration than prompt-only systems.Source: §6 · p. 10

A LIMITATIONS

The study limits its claims to one open-weight model family and one teacher scale. The related-work and algorithm sections distinguish MAS-OPD from prior training approaches and outline its rollout, scoring, and update sequence.

A LIMITATIONS

  • Results may not generalize across model families or teacher scales: all experiments use one open-weight family and hold teacher size fixed while varying students.Whether conclusions transfer across families or teacher scales is left to future work.Source: A LIMITATIONS · p. 15

B EXTENDED RELATED WORK

  • Prior MAS reinforcement learning uses trajectory- or turn-level returns, while prior OPD trains one student and does not address role attribution or inter-agent decisions.The related-work discussion contrasts these approaches with joint training of role-specific agents.Source: B EXTENDED RELATED WORK · pp. 15–16

C ALGORITHM

  • Training keeps student rollouts on-policy, then applies verifier attribution and teacher scoring before updating each role policy and discarding privileged quantities.The verifier and teacher are called only after the workflow trajectory is complete.Source: C ALGORITHM · p. 16

D.1 TRAINING DATASETS

The appendix describes the training corpora, evaluation benchmarks, and single-agent baselines used to assess the systems. It covers code and mathematics tasks, with execution-based or answer-based verification.

D.1 TRAINING DATASETS

  • Code training uses CodeContests, while mathematics training uses Polaris-Dataset-53K.The code domain uses CodeContests; the mathematics domain uses Polaris-Dataset-53K.Source: §D.1 · p. 17

D.2 EVALUATION BENCHMARKS

  • Evaluation spans six established benchmarks covering code generation and mathematical reasoning.The appendix describes benchmark origins, task characteristics, and evaluation structure.Source: §D.2 · p. 17

D.2.1 CODE GENERATION

  • Code evaluation includes LiveCodeBench-v6, APPS, and held-out CodeContests problems, with generated programs assessed by execution against tests.The CodeContests test split is held out from the training split.Source: §D.2.1 · p. 18

D.2.2 MATHEMATICAL REASONING

  • Mathematics evaluation includes AIME 2024, AIME 2025, and OlympiadBench, spanning integer-answer competitions and broader math-and-physics reasoning tasks.OlympiadBench includes heterogeneous subjects, languages, modalities, and answer formats.Source: §D.2.2 · pp. 18–19

D.3 BASELINES

  • The baseline suite includes single-agent and multi-agent variants designed to separate interaction, reinforcement learning, and role-based collaboration effects.Unless specified otherwise, baselines align backbone initialization, task instances, verifier resources, generation constraints, and evaluation protocol.Source: §D.3 · p. 19

D.3.1 SINGLE-AGENT BASELINES

  • Single-agent baselines compare one-shot solving, multi-turn self-revision, and GRPO post-training with and without additional turns.The multi-turn variants control for gains from allocating more generation and refinement turns without complementary-agent feedback.Source: §D.3.1 · pp. 19–20

D.3.2 MULTI-AGENT SYSTEM BASELINES

The appendix specifies collaborative baselines and domain-specific reward construction, then contrasts calibrated local rewards with output attribution. Attribution provides context for teacher supervision rather than a scalar training reward.

D.3.2 MULTI-AGENT SYSTEM BASELINES

  • The prompt-only MAS baseline uses frozen role-prompted agents, while trainable baselines include shared-team-reward MAGRPO, conventional MAS + GRPO, CURE, and MARFT.CURE is code-only and shares one policy between Coder and Tester; MAGRPO uses a shared team reward without agent-specific credit assignment.Source: §D.3.2 · pp. 20–21

D.4 REWARD DESIGN

  • MAS-OPD needs no reward; RL baselines use shared team rewards and, in some methods, local role rewards combined with design-specific weighting.Reward components are based on verifiable evidence rather than a model judging outputs.Source: §D.4 · p. 22

D.4.1 MATHEMATICAL REASONING

  • Mathematics rewards compare parsed answers with references, while Tool-User scoring also checks execution and answer recovery.The team signal is sparse and assigned at termination; the two roles are scored on the answer each produces through its own medium.Source: §D.4.1 · pp. 22–23

D.4.2 CODE GENERATION

  • Code rewards use execution checks: the Coder is assessed on build, run, and test outcomes, while the Tester is assessed on valid tests consistent with the reference.The Tester’s expected outputs are checked against the reference solution, not the candidate program.Source: §D.4.2 · p. 23

D.4.3 REWARD DESIGN AGAINST ATTRIBUTION DESIGN

  • Local rewards require choosing components, weights, and partial-credit scales; these choices are domain- and role-specific and are not established as uniquely correct.The described design uses ten coefficients over four roles, with functional correctness weighted at 0.80.Source: §D.4.3 · p. 24
  • Attribution instead identifies the responsible output using verifiable checks, and the teacher expresses its consequence over that output’s tokens.Attribution enters teacher context and supports supervision about which side should yield in a conflict.Source: §D.4.3 · p. 24

D.5 MODELS AND EVALUATION PROTOCOL

Experiments use Qwen3 students at two sizes and a frozen Qwen3 teacher, with role agents instantiated from the configured student. Evaluation reports Pass@1 for code and accuracy for mathematics.

D.5 MODELS AND EVALUATION PROTOCOL

  • Experiments use Qwen3-14B as the frozen teacher and Qwen3-1.7B or Qwen3-4B as students, instantiating every MAS agent from the selected student.Unless stated otherwise, the appendix reports the 1.7B student.Source: §D.5 · p. 24
  • Models run in no-thinking mode at temperature 0.6, with multi-agent interactions capped at four turns and ending earlier upon agreement.Training and evaluation use one node with eight 96GB NVIDIA H20 GPUs.Source: §D.5 · p. 24, §D.6 · p. 25
  • Code benchmarks report Pass@1 and mathematics benchmarks report accuracy, with final outputs judged by domain-specific verifiers.Source: §D.5 · p. 24

D.6.1 SETTINGS SHARED BY ALL METHODS

The experiments align shared training settings across methods while preserving algorithm-specific objectives and configurations. MAS-OPD uses a training-only teacher and verifier to supervise independently optimized role policies.

D.6.1 SETTINGS SHARED BY ALL METHODS

  • Methods share backbone initialization, data, environments, role prompts, interaction horizon, generation limits, and optimization steps wherever compatible with their original designs.Trainable multi-agent role policies are initialized from the same backbone and optimized independently, except CURE retains its shared-policy formulation.Source: §D.6.1 · p. 25
  • Training uses Qwen3 in no-thinking mode, a maximum four-turn horizon, and five independent training runs for each trained method.Common settings include 8,192-token prompts, 4,096-token responses, 150 optimization steps, and evaluation at temperature 0.6.Source: §D.6.1 · p. 25
  • Code execution runs the Coder’s program on golden and Tester-authored inputs, while mathematics compares the Reasoner’s natural-language answer with the Tool-User’s printed answer.The table describes environment executions, not separate programs written by each role.Source: Table 3 · p. 26, §D.6.1 · p. 26

D.6.2 BASELINES

  • The baselines preserve distinct training designs, including team-reward GRPO, conventional role-wise GRPO, CURE’s shared policy, and MARFT’s shared critic.Prompt-only baselines keep backbone parameters frozen; CURE is implemented on the Code domain only.Source: §D.6.2 · pp. 26–27

D.6.3 MAS-OPD

  • MAS-OPD independently optimizes role policies and adds a frozen Qwen3-14B teacher that force-decodes student responses using privileged context during training.RAS scores each response under target and contrasting role conditions; privileged attribution is teacher-only, and the verifier and teacher are removed after training.Source: §D.6.3 · p. 27

D.6.4 TRAINING COST

The section reports per-step rollout and update wall-clock costs, and describes how MAS-OPD extends to wider systems by replicating existing roles.

D.6.4 TRAINING COST

  • Table 4 reports mean wall-clock seconds per optimization step for MAS-OPD training on one node of eight NVIDIA H20 GPUs with 96GB each, separating joint rollout from policy update.Each entry is the mean over a run’s optimization steps; the step count is specified in Appendix D.6.1.Source: §D.6.4 · p. 28

E.1 SCALING TO MORE AGENTS

  • Scaling adds agents by replicating existing role types, leaving the workflow and multi-turn interaction unchanged.The role set stays fixed, so the modules need no new agent-specific design decision, hyperparameter, or prompt.Source: §E.1 · p. 28
  • The described scaling cost remains proportional to system responses rather than adding growth tied to the number of agents.The passage distinguishes this method cost from the ordinary inference cost of a larger multi-agent system.Source: §E.1 · p. 28

E.1.1 ACCURACY UNDER MORE AGENTS

The scaling study reports improving domain averages as agents are added, while the accompanying analysis explains how RAS, privileged attribution, and their costs extend to larger systems.

E.1.1 ACCURACY UNDER MORE AGENTS

  • 26.3 on code and 27.4 on mathematics are the reported domain averages at eight agents, with all six benchmarks ending above their two-agent results.The domain average rises at each step on both domains, and the reported curves show no saturation by eight agents.Source: §E.1.1 · pp. 28–29, Figure 5 · p. 29

E.1.2 ROLE-ADVANTAGE SPECIALIZATION

  • RAS contrasts teacher scores under role conditions rather than comparing a response with a particular teammate’s output.Because both evaluations use the same context and the contrast is between role types, replicated agents of one role use the same contrasting role condition.Source: §E.1.2 · p. 29
  • RAS adds one contrasting teacher pass per response, so its overhead is a constant factor as agents are added.The contrast is set by the role mapping, not by the number of agents.Source: §E.1.2 · pp. 29–30

E.1.3 PRIVILEGED ATTRIBUTION FOR COORDINATION

  • Privileged attribution passes verified, per-agent judgments to the teacher; the attribution model is an alternative to extending hand-written rules as systems grow.The model locates outputs that disagree with the reference but does not decide correctness, and its output is withheld from students.Source: §E.1.3 · p. 30
  • One attribution serves every agent at a turn, so privileged attribution adds no teacher forward pass as the system grows.The model-based attribution route adds one model call per turn independent of agent count.Source: §E.1.3 · p. 30

E.1.4 THE COST OF ADDING AGENTS

  • The modules add no per-agent hyperparameter or deployment component; the teacher and verifier are removed after training.The system retains the single hyperparameter λ as agents are added.Source: §E.1.4 · p. 31

E.1.5 SETUP OF THE SCALING EXPERIMENT

The scaling experiment varies agent count while keeping the role types, interaction, attribution route, and evaluation protocol fixed.

E.1.5 SETUP OF THE SCALING EXPERIMENT

  • The 1.7B student is trained with two, four, six, or eight agents, replicating each role while retaining the same multi-turn interaction.The attribution model is used at every size, including two agents, to hold the attribution route fixed.Source: §E.1.5 · p. 31
  • The code and mathematics systems train on CodeContests and Polaris-Dataset-53K, respectively, and are evaluated on the same six benchmarks.The agent count is the only varied configuration quantity.Source: §E.1.5 · p. 31
  • Each Figure 5 point is averaged over five independent training runs, with bands showing their standard deviation.The figure plots the six benchmarks separately.Source: §E.1.5 · p. 31

E.2 SENSITIVITY TO THE ROLE-ADVANTAGE WEIGHT

This section examines sensitivity to role-advantage weighting and explains how RAS role comparisons and PAC’s training-only conflict attributions are presented to teachers versus students. It also describes symmetric role prompts and teacher contexts, with case studies illustrating these differences.

E.2 SENSITIVITY TO THE ROLE-ADVANTAGE WEIGHT

  • Across λ = 0.05–0.2, no benchmark falls below the no-role-advantage setting, and each curve varies by under 1.2 points on larger benchmarks or one AIME problem.The reported weight lies within this range; Figure 6 shows single-run settings, not the five-run averages reported elsewhere.Source: §E.2 · p. 32, Figure 6 · p. 32
  • Removing role advantage reduces accuracy on most benchmarks and in both domain averages, indicating that the gain comes from contrasting role conditions rather than role conditioning alone.The λ = 0 setting retains the role condition while removing the role-advantage difference.Source: §E.2 · p. 32
  • At λ = 0.5, most benchmarks and both domain averages fall below training without the module, as the oversized role-advantage term displaces teacher-distribution distillation.The update then tracks differences between role conditions rather than the teacher token distribution, separating students without enabling task solving.Source: §E.2 · p. 32

F PROMPT TEMPLATES

  • Workflow templates are shared across evaluated methods and unchanged between training and deployment; teacher contexts combine three kinds of text, with displays identifying each kind.Role conditions replace text the student also receives, whereas attribution is the segment withheld from students.Source: F PROMPT TEMPLATES · p. 32

F.1 STUDENT PROMPTS ON THE CODE DOMAIN

  • Code-domain prompts assign the Coder to implement and the Tester to construct tests, then ask both roles symmetrically to locate inconsistencies and decide whether to revise their artefact.Neither role is prompted to concede in advance, so conflict outcomes are not predetermined by the wording.Source: §F.1 · p. 33

F.2 STUDENT PROMPTS ON THE MATHEMATICS DOMAIN

  • Mathematics prompts assign derivation to the Reasoner and program-based computation to the Tool-User, with later turns symmetric across roles.Source: §F.2 · p. 35

F.3 ROLE CONDITIONS

  • Role conditions replace each student template’s opening line, and their four isomorphic, comparable-length forms name the role, collaborator, responsibility, focus, and boundary.This controls wording and length differences so role advantage measures responsibility rather than prompt variation.Source: §F.3 · p. 36

F.4 COMPOSITION OF A TEACHER CONTEXT

  • The teacher context preserves the student trajectory and adds only a role substitution and an attribution, both teacher-only; each completed turn’s verdict guides the following turn.The first turn has no prior verdict, while later turns give the teacher the previous turn’s error attribution and withhold that verdict from students.Source: §F.4 · p. 37
  • The teacher context grows turn by turn: it includes verdicts on earlier turns, while students never see those verdicts and turn 0 has none.From turn 1 onward, the verdict on each preceding turn appears before the turn it guides.Source: §F.4 · pp. 38–39, Figure 7 · p. 38

F.5 THE TWO TEACHER CONTEXTS OF RAS

  • RAS scores the same response under target and non-target role conditions; their teacher-score difference is the role advantage, isolating the role identity preference.The contexts otherwise share the same trajectory, and the target-context forward pass is shared with OPD, so scoring costs one additional teacher forward pass.Source: §F.5 · p. 40

F.6 ATTRIBUTION IN PAC

  • PAC attributes a conflict by checking each role’s output independently against a training-time reference, labeling an existing candidate rather than supplying a correct value.On code tasks, the checks assess program correctness and declared-test correctness; on mathematics tasks, they assess the derived answer and printed result against the reference answer.Source: §F.6 · p. 40
  • With the attribution, teacher targets revise the side at fault and hold the sound side; students learn to identify errors from the problem and their own reasoning without seeing the correct value.Without attribution, the teacher sees the interaction as the students do, so its targets express local quality but not which side should concede.Source: §F.6 · p. 42

G CASE STUDY

  • The case study presents one interaction-conflict example for PAC and one response-token example for RAS, each on code and mathematics tasks.The conflict example shows what each side holds during resolution; the RAS example shows where its role-advantage signal falls on individual tokens.Source: G CASE STUDY · p. 42

G.1 AN INTERACTION THROUGH A CONFLICT

  • In the conflict illustration, students hold only the slate regions, while the teacher additionally receives RAS’s role-condition substitution and PAC’s verdict.The display follows the interaction through a conflict and marks who holds each region.Source: §G.1 · pp. 42–43

G.1.1 CODE DOMAIN

In the code-domain case, the agents’ outputs conflict because the test input’s layout does not match what the program reads, even though both artefacts are sound. The attribution lets the agents distinguish this formatting conflict from an error in either artefact and converge on a correctly formatted test.

G.1.1 CODE DOMAIN

  • The program and expected output are both sound; the disagreement comes from the test input’s layout, which differs from the program’s one-value-per-line input format.The program reports a parsing failure on the first line, rather than returning an incorrect count.Source: §G.1.1 · pp. 43–44
  • The attribution checks each artefact against a training-time reference independently, allowing both checks to pass when the conflict lies in formatting rather than either artefact.For this case, the program passes the official tests and the reference supports the Tester’s declared value.Source: §G.1.1 · p. 44
  • On the next turn, the Coder keeps its program while the Tester identifies the layout issue and rewrites the test in the prescribed format.The agents are asked to locate the inconsistency and then revise or keep their own artefact, without a prompt that predisposes either role to concede.Source: §G.1.1 · p. 44

G.1.2 MATHEMATICS DOMAIN

The mathematics case shows that conflicts can be attributed to different roles across turns, while role-advantage displays distinguish role-aligned work from shared or out-of-role content. The displays show this distinction in both code and mathematics, including where notation is held constant.

G.1.2 MATHEMATICS DOMAIN

  • The two mathematics verdicts attribute successive conflicts to different roles, and both agents revise at turn 1 even though only the Reasoner’s incomplete derivation should change.The first verdict identifies the derivation as faulty; the second identifies the program, which had already enumerated correctly, as faulty.Source: §G.1.2 · p. 45
  • RAS substitutes the role condition, while target and non-target contexts otherwise retain the same response, isolating the role condition in the comparison.The student trajectory does not receive the substitution; the teacher-only verdicts attribute conflicts to their source.Source: §G.1.2 · pp. 46–47

G.2 WHERE THE ROLE ADVANTAGE FALLS

  • Role advantage is shown across response tokens by comparing teacher scores under target and contrasting role conditions, with near-zero values marking shared rather than absent signal.The display uses blocks of a few tokens; the three regimes appear as levels above, at, and below the scale’s middle.Source: §G.2 · pp. 48–49

G.2.1 CODE DOMAIN

  • On the Coder response, planning and decision-carrying statements sit above the middle, self-chosen case work below it, and shared formatting and comments at it.Adjacent comments and the code they describe can fall in different bands; the levels also vary within each stretch.Source: §G.2.1 · p. 49

G.2.2 MATHEMATICS DOMAIN

  • On the Reasoner response, algebra sits above the middle and the enumeration instruction below it, while shared task restatement and the final answer sit at the middle.Both roles write mathematics in this case, so the displayed separation is not explained by notation.Source: §G.2.2 · p. 51

Related papers

Loading 2609.34234v1…