PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Final Accuracy: Auditing Communication in LLM Multi-Agent Systems

Shixuan Li, Wei Yang, Peiyu Zhang, Anzhe Cheng, Heng Ping, Paul Bogdan

arXiv:2610.01042v1cs.AIcs.CL

TL;DR

System-level communication evaluations make it difficult to separate information exchange from agent design and additional reasoning, while final accuracy obscures how answers change. ICR audits answer revision after independent reasoning using fixed trajectories, correctness-conditioned measures, and a no-message control. Across four benchmarks, message content and receiver policies shift correction and preservation, revealing behaviors that aggregate accuracy can hide.

  • Problem

    Communication’s effects are difficult to attribute when evaluations combine it with agent roles, interaction structures, and inference budgets, and final accuracy does not distinguish different intermediate contributions.

    Source: §1 · pp. 1–2

  • Method

    ICR audits revision after independent reasoning using fixed initial trajectories, correctness-conditioned measures, and a no-message revision reference.

    Source: §1 · p. 2, §3.1 · p. 3, §3.2 · p. 5

  • Results

    Across four benchmarks, full reasoning increases correction but reduces preservation relative to answer-only messages, while structured verification shifts every channel toward preservation at correction’s expense.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Communication should be evaluated through benefits and harms jointly, with conclusions tied to the message, delivery interface, and receiver policy.

    Source: §5 · p. 10

    ICR’s conclusions concern evaluated channel–policy combinations, and its bootstrap intervals do not capture variation across unobserved generation seeds.

    Source: §3.3 · p. 6

Abstract

from arXiv · show

Multi-agent communication aims to help agents benefit from one another's information. Yet improvements in system performance leave a fundamental ambiguity: do they reflect effective communication, a favorable agent architecture, or simply additional reasoning? Because communication methods are commonly evaluated within the systems they were designed for, these factors are difficult to disentangle. Final accuracy further merges corrected errors and corrupted answers into a single outcome, obscuring how communication changes decisions. We introduce Independent--Communicate--Revise (ICR), a controlled framework that evaluates communication as answer revision following independent reasoning. ICR fixes initial reasoning trajectories, measures correction and preservation conditional on both agents' initial correctness, and uses a no-message revision control to quantify gains beyond additional reasoning. Across four reasoning benchmarks, our audit of textual and latent communication reveals that similar aggregate accuracy can conceal substantially different revision behaviors. Compared with transmitting answers alone, full reasoning increases correction while reducing preservation on all four benchmarks, so richer messages amplify beneficial and harmful influence alike. Receiver-policy comparisons on MedQA and GPQA-D further show that a structured verification policy shifts every channel toward greater preservation and lower correction, while its effect on selectivity varies across channels and tasks. These findings challenge treating communication quality as an intrinsic property of a channel. ICR therefore recenters evaluation on selective revision, providing a unified framework for examining how message content and receiver policies jointly produce benefits and harms.

1 INTRODUCTION

ICR evaluates communication through answer revision after independent reasoning, addressing the difficulty of attributing system-level gains and the limits of final accuracy. Across four benchmarks, message content and receiver policy shift correction and preservation in different ways.

1 INTRODUCTION

  • System-level evaluations entangle communication with agent roles, interaction structures, and inference budgets, while final accuracy cannot distinguish different intermediate contributions.The challenge also applies to latent messages whose information is not expressed in natural language.Source: §1 · p. 1, Figure 1 · p. 2
  • ICR fixes independently generated trajectories, measures correction and preservation conditional on both agents’ initial correctness, and uses a no-message control to separate communication gains from additional reasoning.Correctness labels are used only for offline selection, stratification, and scoring, not message construction or revision.Source: §1 · p. 2
  • Similar aggregate accuracy conceals distinct correction–preservation profiles, and recovery when both agents start wrong remains rare.Strong correction can coexist with substantial corruption, while strong preservation can leave correction opportunities missed.Source: §1 · p. 2
  • Across all four benchmarks, full reasoning increases correction but reduces preservation relative to answer-only messages; structured verification shifts every channel toward preservation at correction’s expense.Verification’s effect on selectivity varies across channels and tasks.Source: §1 · p. 2
  • The paper’s contributions include auditing textual and latent communication on four benchmarks and examining how message content and receiver policy shape revision.The framework uses controlled comparisons and a random-delivery reference.Source: §1 · p. 2

2 RELATED WORK

The related work spans multi-agent interaction design, answer revision and reliability, and communication representations. ICR complements these lines by evaluating revision under controlled communication conditions.

Multi-agent collaboration and debate

  • Multi-agent studies examine debate, role-playing, workflows, and communication topology; ICR complements them by fixing initial trajectories and comparing communication within a shared revision protocol.Prior work reports that interaction design shapes system-level outcomes, and sparse debate can match or outperform fully connected interaction with fewer links.Source: §2 · p. 3

Answer revision and reliability

  • Revision and reliability work studies self-consistency, self-feedback, sycophancy, and conformity; ICR jointly measures correction and preservation conditional on both agents’ initial correctness.Its matched no-message control separates communication-associated gains and losses from those of additional reasoning alone.Source: §2 · p. 3

Communication representations and message content

  • Communication-representation work studies latent messaging and text compression; ICR compares answer-only messages with full reasoning and varies receiver policy rather than ranking representations.Retained message content alone does not establish that a revision is correct.Source: §2 · p. 3

3 INDEPENDENT–COMMUNICATE–REVISE

ICR evaluates communication as revision of a receiver’s fixed initial answer, separating independent reasoning from message-driven changes. It audits revisions by both agents’ initial correctness and compares communication with matched no-message revision.

3.1 CONTROLLED REVISION PROTOCOL

  • Agents generate initial records without seeing peers’ outputs, then freeze and reuse them across conditions; this does not assume statistically independent errors.Source: §3.1 · p. 3
  • Each directed sender–receiver event starts from original cached records, so revised answers are not propagated and evaluation remains single-hop.Source: §3 · p. 3, §3.1 · p. 4

3.2 AUDITING REVISION OUTCOMES

  • Four initial-correctness strata measure correction, preservation, both-wrong recovery, and both-correct preservation as fractions of correct revised answers within each stratum.Correction concerns a wrong receiver with a correct sender; preservation concerns a correct receiver with a wrong sender, whose failure is corruption.Source: §3.2 · p. 5
  • The equal-weight selectivity index balances correction against corruption, while aggregate revised accuracy also depends on the prevalence of each initial-correctness state.The same selectivity value can describe sharply different revision behaviors, so it should be reported alongside correction and preservation.Source: §3.2 · p. 5

3.3 COMPARISON CONDITIONS AND ATTRIBUTION CONTROLS

  • Conditions include Answer Only, Full Text, adapted latent channels, and No Message, which preserves the revision opportunity but supplies no peer information.Channel comparisons hold initial records, receiver model, and policy fixed; paired outcomes measure gains and losses against matched no-message revision.Source: §3.1 · p. 4, §3.3 · p. 5
  • Bootstrap intervals preserve within-question pairing but omit variation across unobserved generation seeds, and conclusions apply to evaluated channel–policy combinations rather than intrinsic representation rankings.Source: §3.3 · p. 6

4 EXPERIMENTS

The experiments evaluate how aggregate outcomes mask revision behavior, how message content changes revision, and how receiver policies shape correction and preservation. Qwen3-4B is assessed across four reasoning benchmarks under controlled message and receiver conditions.

4.1 EXPERIMENTAL SETUP

  • The main audit evaluates Qwen3-4B on MedQA, ARC-C, GSM8K, and GPQA-D across five communication conditions, fixing initial trajectories and receiver instructions within each model.The experiments address what accuracy conceals, how message content affects revision, and how receiver policy shapes it.Source: §4.1 · p. 6, §4 · p. 6

4.2 RQ1: WHAT DOES ACCURACY CONCEAL?

  • On GSM8K, estimated full-set accuracy spans only 0.11 points while communicating-condition PR spans 27.17; MedQA Full Text and StateBridge trade CR (80.17% vs 54.31%) against PR (42.24% vs 71.55%).The two MedQA conditions have SI values of 61.21% and 62.93%, respectively.Source: §4.2 · p. 7
  • On MedQA, Full Text and StateBridge yield net gains of 11 and 15 outcomes versus No Message, while higher correction does not necessarily raise retained-subset accuracy.Across the four main datasets, communicating-condition recovery when both agents start wrong ranges from 0.30% to 2.55%.Source: §4.2 · pp. 7–8
  • At Qwen3-8B, Full Text has higher CR and lower PR than Answer Only on both tasks, while the highest observed SI varies by task.StateBridge has the highest observed SI on MedQA; Answer Only has the highest SI point estimate on GPQA-D.Source: §4.2 · p. 8

4.3 RQ2: HOW DOES MESSAGE CONTENT AFFECT REVISION?

  • Across four datasets, Full Text versus Answer Only raises CR by 11.96–41.84 percentage points and lowers PR by 15.79–37.93 points; CR intervals exclude zero on three datasets and PR intervals on all four.The GSM8K CR interval includes zero.Source: Table 3 · p. 8, §4.3 · p. 8
  • At matched PR, Answer Only’s CR differences from a content-independent Full Text/No Message reference remain unresolved because all four intervals include zero.The reference mixes Full Text and No Message independently of question and correctness labels.Source: §4.3 · pp. 8–9, Figure 4 · p. 8

4.4 RQ3: HOW DOES RECEIVER POLICY AFFECT REVISION?

  • Structured Verification yields lower CR and higher PR than Critical Evaluation across both datasets and all three communication conditions, while SI changes vary by channel and task.On MedQA, SI decreases for StateBridge and increases for LatentMAS; on GPQA-D, all three SI difference intervals include zero.Source: §4.4 · p. 9, Table 4 · p. 9
  • On MedQA, Structured Verification increases LatentMAS’s communication increment over Critical Evaluation, with a 95% interval excluding zero.The increment is measured relative to each policy’s No Message baseline.Source: §4.4 · p. 9

5 CONCLUSION

ICR evaluates communication through fixed initial trajectories, correctness-conditioned revision metrics, and a no-message control. The conclusion argues that correction and preservation reveal behavioral differences hidden by aggregate accuracy, and that results should be interpreted jointly with message and receiver-policy choices.

5 CONCLUSION

  • Across four benchmarks, correction and preservation expose differences obscured by aggregate accuracy: full reasoning trades preservation for correction, while receiver policies shift that balance.The conclusion recommends evaluating communication's benefits and harms jointly, with conclusions tied to message, delivery interface, and receiver policy.Source: §5 · p. 10

A.1 MODEL AND GENERATION SETTINGS

  • Eight revision records remained truncated at earlier generation limits across ARC-C, GSM8K, and HumanEval+; they were retained and scored under the same rules.A non-termination flag alone does not determine correctness; outputs without a parseable answer score incorrect.Source: §A.1 · p. 14

A.2 INDEPENDENT GENERATION PROMPTS

  • Independent-generation prompts are task-specific and request reasoning alongside a benchmark-compatible final answer.The supplied prompts cover multiple-choice, math, programming, and medical questions.Source: §A.2 · pp. 14–15

A.3 RECEIVER REVISION PROMPTS

  • Critical Evaluation asks receivers to revise only for a concrete error or better-supported evidence, while Structured Verification checks the external message and prior reasoning separately against the problem.Structured Verification tells the receiver to retain its answer when surviving claims support it and change it when they point elsewhere.Source: §A.3 · p. 16

HumanEval+

  • HumanEval+ revision uses a Critical Evaluation template that asks the receiver to revise its implementation rather than restart or copy an external message.It instructs changes only for a concrete defect or a better-supported approach, and requires a complete implementation in one Python code block.Source: §A.3 · pp. 16–17, HumanEval+ · p. 18

MedQA

  • MedQA instructions require one final answer in boxed A–D format.The numeric-task instruction instead specifies a single boxed number in plain digits, without separators, units, or other symbols.Source: MedQA · p. 18

A.4 CONFIGURATION AND PROMPT VERIFICATION

  • Prompt hashes were reproduced from recovered templates and historical configuration fingerprints were matched to generation limits, but exact source snapshots for every historical run were not archived.Consequently, a single commit does not fully specify all evaluated runs.Source: §A.4 · p. 18

B COMMUNICATION IMPLEMENTATIONS

The audit implements textual and latent communication within a shared single-hop revision protocol, reusing frozen initial records. The implementations differ in the sender information represented and the interface used to deliver it.

B COMMUNICATION IMPLEMENTATIONS

  • All communication conditions use the same single-hop revision protocol and reuse frozen initial records across conditions.No revised answer is propagated into another evaluation event.Source: B COMMUNICATION IMPLEMENTATIONS · p. 18

Textual conditions

  • No Message supplies no sender information, Answer Only supplies the sender’s parsed answer, and Full Text transmits the complete post-thinking response.Full Text is inserted into the receiver’s external-information slot without additional generation or shortening.Source: B COMMUNICATION IMPLEMENTATIONS · p. 18

StateBridge

  • StateBridge maps selected final-portion decoder states toward the input-embedding space and inserts the resulting vectors at the receiver’s message slot.It normally retains up to 64 states and uses aligned state–token-embedding pairs from the same trajectory for its regularized orthogonal Procrustes fit.Source: B COMMUNICATION IMPLEMENTATIONS · p. 18

LatentMAS

  • LatentMAS reconstructs an all-layer KV cache from the sender’s prompt and complete generated sequence, then supplies it as a causal prefix before the receiver’s revision prompt.The two latent implementations thus deliver payloads through different interfaces: message-slot embedding insertion for StateBridge and KV-prefix attachment for LatentMAS.Source: B COMMUNICATION IMPLEMENTATIONS · pp. 18–19

Comparison scope

  • The audit compares message–interface combinations under specified receiver policies rather than reproducing the methods’ complete native agent workflows.Full Text uses a post-thinking response, StateBridge normally uses final-portion states, and LatentMAS derives its prefix from the full trajectory.Source: B COMMUNICATION IMPLEMENTATIONS · pp. 18–19

C POPULATION, SCORING, AND STATISTICAL SCOPE

This section defines the evaluated populations, scoring procedures, and uncertainty estimates, then describes paired answer transitions. Its sensitivity analyses and stated scope boundaries qualify how results should be interpreted.

C POPULATION, SCORING, AND STATISTICAL SCOPE

  • The main-text evaluation retains questions with at least one initially incorrect agent, while selection and stratification use initial correctness without exposing reference answers to message construction or revision.The four main-text datasets contain 430 retained questions and 2,580 directions per condition; HumanEval+ adds 28 questions and 168 directions.Source: C POPULATION, SCORING, AND STATISTICAL… · p. 19
  • Scoring is task-specific: multiple-choice answers use the last boxed option, GSM8K uses its boxed or final number under specified parsing rules, and HumanEval+ executes extracted Python against dataset tests.HumanEval+ uses a 10-second per-question limit and not the official EvalPlus harness; unparsed outputs are scored incorrect.Source: C POPULATION, SCORING, AND STATISTICAL… · pp. 19–20
  • Excluding nine GPQA-D questions with truncated initial trajectories reduces CR and PR denominators from 114 to 100 each; SI decreases in all four conditions, condition ordering is unchanged, and LatentMAS falls below No Message.The main results retain all questions.Source: C POPULATION, SCORING, AND STATISTICAL… · p. 20, Table 7 · p. 20
  • Reported evaluation results characterize the evaluated configurations and question populations; no separate question-level development split is documented.The all-correct observations cover only A-to-B and B-to-A directions on four datasets, with partial directional coverage on GPQA-D.Source: C POPULATION, SCORING, AND STATISTICAL… · p. 20

C.1 UNCERTAINTY ESTIMATES

  • Marginal intervals use 2,000 question-cluster bootstrap resamples, while condition differences use 10,000 paired resamples jointly preserving question-level correspondence.Intervals are conditional on evaluated runs and the question population and do not capture variation across unobserved generation seeds; marginal-interval overlap does not establish significance or equivalence.Source: §C.1 · p. 20

D PAIRED OUTCOMES AND ANSWER TRANSITIONS

  • Matched comparisons with No Message define gains and losses by correctness transitions, and net correct-outcome change equals gains minus losses over paired events.These comparisons include all four initial-correctness strata, so they also capture recovery or preservation when both agents begin with the same correctness state.Source: D PAIRED OUTCOMES AND ANSWER TRANSITIONS · p. 21, Table 10 · p. 22

D.1 ANSWER IDENTITY

Answer-identity categories distinguish revisions by whether they match the receiver’s prior, the sender’s answer, or neither. The section cautions that answer matching is not evidence of causal adoption and treats correctness separately.

D.1 ANSWER IDENTITY

  • Identity categories are assigned in Invalid, Prior, Sender, Third order, so a revised answer matching both initial answers counts as Prior.Third also includes changed valid outputs when the sender’s initial answer is unparsed.Source: §D.1 · p. 23
  • Sender matching describes an observed answer relationship, not causal adoption; Third answers may be correct or incorrect, so identity and correctness are analyzed separately.For HumanEval+, program-string identity does not imply functional equivalence.Source: §D.1 · p. 23
  • Full Text yields more sender-matching revisions than Answer Only in both mixed-correctness strata and a higher pooled valid-output change rate on all four datasets.These outcomes indicate more frequent revision; correctness-conditioned metrics are needed to determine whether revisions correct or corrupt answers.Source: §D.1 · p. 23, Table 12 · p. 23

E RECEIVER-SIDE COST OBSERVATIONS

The exported cost fields do not support a comparable end-to-end cost or throughput comparison across channels.

E RECEIVER-SIDE COST OBSERVATIONS

  • Zero additional construction-generation calls for the four baselines do not measure forward-pass computation.LatentMAS reconstructs the sender cache and runs latent steps; StateBridge alignment timing is stored with the initial belief.Source: E RECEIVER-SIDE COST OBSERVATIONS · p. 23
  • Logged per-record means are not a controlled throughput comparison, and prompt-token accounting omits some latent-prefix computation while construction costs are not uniformly available.The LatentMAS payload zeros are suppressed because they conflict with documented KV-prefix transmission.Source: Table 13 · p. 24, E RECEIVER-SIDE COST OBSERVATIONS · p. 24
  • The reported seconds for MedQA, ARC-C, GSM8K, and HumanEval+ do not establish comparable end-to-end cost across channels.Source: E RECEIVER-SIDE COST OBSERVATIONS · p. 24

F HUMANEVAL+ SUPPLEMENT

HumanEval+ shows differing revision outcomes across channels on a small set of retained directions, with findings qualified by sample size, generation failures, and scoring choices.

F HUMANEVAL+ SUPPLEMENT

  • All conditions use the same 168 retained directions, but nine directions in each stratum involve a non-terminating initial trajectory.The audit identifies 12 non-terminating initial records across eight questions and records 49 non-terminating revision records across ten questions; no item exclusion was applied.Source: F HUMANEVAL+ SUPPLEMENT · p. 24
  • One HumanEval+ outcome changes CR or PR by 3.57 percentage points because each rate has 28 directions.The denominators are small, and the reported observations are limited by generation failures.Source: F HUMANEVAL+ SUPPLEMENT · p. 24
  • No Message corrects 16/28 eligible receiver answers; Full Text reaches 21/28 without changing PR, while LatentMAS has the highest CR at 24/28 but lower PR at 17/28 and SI below No Message.StateBridge obtains 19/28 CR and 27/28 PR; these observations are limited by the small sample and generation failures.Source: F HUMANEVAL+ SUPPLEMENT · p. 24
  • Programs are scored using dataset-provided tests with a 10-second limit, not the official EvalPlus harness; reported results retain original execution scores after one question differed on serial re-execution.Initial-correctness strata are defined from the original execution scores.Source: F HUMANEVAL+ SUPPLEMENT · p. 25

G ESTIMATED FULL-SET ACCURACY

The full-set accuracy estimates depend on assumptions for unobserved directions, while the six-direction table details correlated outcomes rather than independent replications.

G ESTIMATED FULL-SET ACCURACY

  • The estimates combine retained outcomes with assumptions about unobserved directions; N counts evaluated questions before correctness-based selection, yielding 6N ordered sender–receiver events.MedQA has 1,800 total directions, including 1,616 unobserved; GPQA-D has 1,188 total directions, including 972 unobserved.Source: G ESTIMATED FULL-SET ACCURACY · pp. 25–26
  • Qwen3-4B estimates extrapolate observed all-three-correct accuracy to unobserved directions, whereas Qwen3-8B assumes unchanged correctness outside the mixed-correctness population.The assumptions differ despite the shared estimated-accuracy notation.Source: G ESTIMATED FULL-SET ACCURACY · pp. 25–26
  • Observed all-three-correct directions are not a documented probability sample of omitted directions, and estimates are point estimates without confidence intervals.The passage distinguishes these estimates from directly measured outcomes.Source: G ESTIMATED FULL-SET ACCURACY · p. 26

H SIX-DIRECTION DETAIL

  • Table 16 reports conditional success counts by dataset, channel, and ordered direction; its six directions are correlated views of the same questions, not six independent replications.The conditional columns show successes over denominators.Source: H SIX-DIRECTION DETAIL · pp. 26–28, Table 16 · p. 26

I.1 RANDOM-DELIVERY REFERENCE

These supplementary comparisons examine matched-preservation references, channel contrasts across receiver policies, correction–preservation differences, communication increments, and sensitivity to truncated outputs.

I.1 RANDOM-DELIVERY REFERENCE

  • At matched PR, point estimates favor Answer Only over the random-delivery reference on three datasets and the reference on ARC-C, but all four intervals include zero.The comparison does not establish equivalence or show that reduced message exposure explains Answer Only’s behavior.Source: §I.1 · p. 29, Table 17 · p. 29

I.2 CHANNEL COMPARISONS ACROSS RECEIVER POLICIES

  • StateBridge-versus-Full-Text SI ordering changes in opposite directions across MedQA and GPQA-D, but both channel-by-policy interaction intervals include zero.These descriptive ordering changes do not establish an interaction on either dataset.Source: §I.2 · p. 29, Table 18 · p. 29

I.3 CORRECTION AND PRESERVATION DIFFERENCES

  • Full Text has higher CR and lower PR than Answer Only on all four datasets; CR difference intervals exclude zero on MedQA, ARC-C, and GPQA-D, while all PR intervals lie below zero.The GSM8K CR difference interval includes zero.Source: §I.3 · p. 29, Table 19 · p. 30

I.4 POLICY-SPECIFIC COMMUNICATION INCREMENTS

  • Only the MedQA LatentMAS increase in communication increment has a strictly positive interval; the remaining ∆G intervals include zero.The increment measures a channel’s SI advantage over the policy-specific No Message control.Source: §I.4 · pp. 29–30, Table 20 · p. 30

I.5 SENSITIVITY TO REVISION TRUNCATION

  • A common post-truncation mask removes 6 of 228 GPQA-D events, leaving 111 per CR and PR stratum; all contrast intervals include zero on this selected subset.Because exclusion depends on post-revision termination, the subset analysis is selected; the main analysis retains all events.Source: §I.5 · p. 30

J ADDITIONAL RESULTS WITH QWEN3-8B

The Qwen3-8B audit compares communication conditions on frozen independent trajectories for MedQA and GPQA-D. Communication’s correction, preservation, and selectivity profiles vary by task, with paired gains over No Message evident only for some MedQA conditions.

J ADDITIONAL RESULTS WITH QWEN3-8B

  • Three Qwen3-8B agents independently solve each question, then reuse frozen trajectories across five conditions; revision is evaluated only for mixed-correctness directions.The audit includes 46 mixed-correctness MedQA questions and 54 GPQA-D questions, yielding 92 and 108 CR and PR events per condition, respectively.Source: J ADDITIONAL RESULTS WITH QWEN3-8B · pp. 30–31, Table 22 · p. 31
  • On both tasks, Full Text has higher CR and lower PR than Answer Only, while the highest observed SI condition differs by task.StateBridge has the highest observed SI on MedQA; Answer Only has the highest SI point estimate on GPQA-D.Source: J ADDITIONAL RESULTS WITH QWEN3-8B · p. 31, Table 23 · p. 32
  • On MedQA, Answer Only, Full Text, and StateBridge SI increments over No Message have intervals above zero; on GPQA-D, all four intervals include zero.The contrasts use paired outcomes on the same directed events.Source: J ADDITIONAL RESULTS WITH QWEN3-8B · p. 31, Table 24 · p. 32
  • StateBridge exceeds Answer Only in SI by 9.78 percentage points on MedQA (95% CI: [0.66, 18.62]); the GPQA-D difference is −2.31 points ([−9.84, 5.24]).These comparisons are within-model communication conditions, not paired samples across model sizes.Source: J ADDITIONAL RESULTS WITH QWEN3-8B · p. 31
  • All reported events remain in the analysis, including outputs with truncated independent generations or revisions, scored under task-specific parsing and correctness rules.Independent generation has one truncated MedQA response and seven on GPQA-D; revision truncations occur in specified conditions, with affected outputs associated with the same question on each task.Source: J ADDITIONAL RESULTS WITH QWEN3-8B · p. 31
Loading 2610.01042v1…