PaperlayerView exact revision on arXiv

Source-linked AI summary

Disparity Laundering: Consensus Without Fairness in Clinical Multi-Agent Systems

Xiaoyang Wang, Tianrui Wang, Christopher C. Yang

arXiv:2610.38215v1cs.AIcs.MA

TL;DR

Clinical multi-agent systems are expected to average out individual error, but the paper asks whether they instead preserve demographic allocation bias. It evaluates counterfactual versions of emergency cases across models and aggregation topologies, finding that no configuration neutralizes the disparity. The analysis identifies cross-version visibility and pooling as the repair conditions.

  • Problem

    It is unknown how multi-agent clinical teams handle demographic allocation bias carried by their members, despite the assumption that deliberation averages out individual error.

    Source: §1 · p. 1

  • Method

    The study compares committed decisions across demographic counterfactuals of the same cases and audits whether the full system performs that comparison.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    Across all 20 model-by-topology cells, committed decisions preserve the independent answers’ pro-Black direction, and every model has a topology that amplifies disparity.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A system committing demographic-sensitive decisions should be audited across demographic versions of the same case, whatever its topology.

    Source: §5 · p. 8

    The factorial covers 120 cases, and demographic counterfactuals are limited to source-record categories and binary sex.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Clinical large language models are increasingly fielded as multi-agent teams, on the assumption that deliberation averages out individual error. It does not average out bias. Across a full factorial of five model families and four aggregation topologies on 120 real MIMIC-IV emergency cases under counterfactual demographic swaps, no configuration neutralizes the disparity: all 20 cells commit the direction of their agents' independent answers, giving Black patients more acute triage. Every model has a topology that amplifies its baseline gap, but which one varies by model, so no design is reliably safe or reliably amplifying: the disparity is relocated, never removed. Late-stage fixes fail: fairness instructions and demographic masking leave the gap untouched, and per-agent self-debiasing widens it, because the bias is already encoded in the agents' ordinary clinical numbers. We call this disparity laundering; the audits now applied to clinical AI, per model, per output, per version, can certify the parts but never the committed decision. Interventions close the gap exactly when they make counterfactual versions visible inside the system and pool them, through deliberation, into the committed decision. A diverse-perspective ensemble does both while keeping the team, with no retraining or architectural change: the disparity falls by 67 to 94% where all five perspectives pool at every round, and the one topology that never pools them is the one it cannot reliably repair. Medical multi-agent systems do not filter demographic bias; they launder it into plausible decisions no single-version audit can catch, and the only defense is to make the bias visible.

1 Introduction

The paper asks whether clinical multi-agent deliberation removes demographic allocation bias, finding that it instead preserves or can magnify disparities. It traces this to agents’ lack of cross-version comparisons and identifies counterfactual visibility plus pooling as the repair mechanism.

1 Introduction

  • Agents see only one demographic version, so deliberation can optimize within-version agreement but cannot distinguish an erroneous outlier from dissent pointing toward parity.The disparity is a relation between counterfactual versions that no participant compares.Source: §1 · pp. 1–2
  • Fairness instructions and demographic masking fail because disparity is already encoded in agents’ ordinary clinical numbers, while each single-version output remains individually defensible.The gap exists in a comparison across demographic versions that no participant performs.Source: §1 · p. 2
  • Across all 20 model-by-topology cells, committed decisions preserve the independent answers’ pro-Black direction, and every model has a model-dependent topology that amplifies disparity.No aggregation design is reliably safe or reliably amplifying, and none reliably corrects the disparity.Source: §1 · p. 2
  • Interventions reduce committed disparity when counterfactual patient versions are visible inside the system and deliberation pools them into the decision.The Diverse-Perspective Ensemble satisfies both conditions and cuts triage disparity by up to 94% without retraining or architectural change.Source: §1 · p. 2

2 Related Work

Prior work documents demographic bias in clinical models and bias amplification in multi-agent systems, but leaves open how controlled communication topology shapes inherited bias. This study combines paired clinical counterfactuals, deliberation, and topology variation to address that gap.

2 Related Work

  • Single-model studies find demographic descriptors can shift clinical models toward more urgent or invasive care, including in emergency triage, while leaving the team-level question unanswered.This paper inherits the counterfactual design and asks how a multi-agent team handles members’ bias.Source: §2 · p. 2
  • Prior multi-agent studies report bias amplification, aligned-agent effects, and collective bias, while differing in structures and bias sources from this study.The present work varies communication topology and studies models’ own allocation behavior on real cases.Source: §2 · pp. 2–3
  • No prior study combines real clinical cases, paired demographic counterfactuals, multi-agent deliberation, and a controlled topology factor.Table 1 situates the study against prior work by these design dimensions.Source: §2 · p. 2, Table 1 · p. 3
  • Existing medical-agent safety work studies adversarial agents, collaboration failures, or fairness evaluation in negotiation, whereas this study examines endogenous disparity across model-topology combinations.The cited work on adversarial vulnerability does not establish the same topology ordering for endogenous bias shared across agents.Source: §2 · p. 3

3 Methods

The methods distinguish disparity genuinely mitigated by deliberation from disparity merely preserved or relocated. They compare demographic counterfactuals across stages and controlled communication topologies using a cross-version audit.

3 Methods

  • The study defines disparity laundering as a team’s committed consensus preserving or magnifying members’ disparity rather than washing it out.The methodological challenge is that mitigation and laundering can produce the same final group average.Source: §3 · p. 3

3.1 Problem Setup and Formal Framework

  • For each real encounter, demographic variants are identical except for the descriptor, isolating the descriptor’s effect in the counterfactual comparison.The analyzed trajectory runs from independent answers through two debate rounds to the committed decision; race analyses compare its endpoints, while sex analyses also inspect intermediate rounds.Source: §3.1 · pp. 3–4
  • The primary endpoint is triage acuity; a secondary unweighted composite sums signed care-intensity differences across six clinical axes.The composite is used on the sex axis; triage acuity is reversed because a lower ESI number is more acute.Source: §3.1 · p. 4

3.2 Measuring the Fate of Bias

  • A cross-version audit runs the full system for each demographic version and compares committed decisions, identifying laundering when records pass separately but decisions differ without cross-version comparison.The factorial reports directional triage gaps per cell as descriptive point estimates, emphasizing consistency across model families rather than any one cell.Source: §3.2 · p. 4

3.3 Implementation Details

  • The factorial evaluates five model families and four topologies on one frozen set of 120 emergency-department cases, with three seeds and counterfactual cohorts.The race axis uses five variants per case; the sex axis uses 118 encounters and three variants, with identities restricted to source-record categories and binary sex.Source: §3.2 · p. 4, §3.3 · p. 4
  • Agents emit structured decisions across six care axes, while the study holds agents and prompts fixed and varies which peers can read one another.SharedPool is all-to-all, Decentralized uses a ring, Layers a predecessor chain, and Centralized routes exchange through a hub; all use the same synthesis step.Source: §3.3 · pp. 4–5

4 Results

The study finds that race-swapped emergency cases receive different triage despite effectively equal accuracy, and that aggregation topology changes—but does not eliminate—the disparity. Making counterfactual versions visible within the team reduces the gap when the topology pools those perspectives.

4 Results

  • The results indicate that allocation bias exists before a team forms, and its fate depends on how the team aggregates rather than which model it uses.The study examines five model families and four aggregation topologies.Source: §4 · p. 5

4.1 Single-Model Allocation Bias Exists Across Families

  • Across models, race swaps shift triage toward more urgent care for Black patients, while per-group accuracy is effectively equal.The reported Black–White committed-MAE differences stay within ±0.034 ESI; more-urgent-than-gold counts range from 7:0 to 14:2.Source: §4.1 · p. 5

4.3 Disparity Laundering: Why Late-Stage Interventions Fail

  • Fairness instructions and demographic masking move the triage gap by at most about 0.02, with no consistent shrinkage.The disparity is encoded in upstream agents’ structured clinical outputs, including triage levels, test orders, and admission flags.Source: §4.3 · p. 6

4.2 Aggregation Topology Gates the Fate of Bias

  • All 20 model–topology cells retain a negative committed Black–White triage gap, and which topology amplifies the gap varies by model.No aggregation design is reliably safe or reliably amplifying.Source: §4.2 · pp. 5–6, Table 4 · p. 7

4.4 Self-Debiasing the Agents Backfires

  • Self-debiasing enlarges the pooled triage gap in all five models, and the gap grows in 17 of 20 model–topology cells.For Phi-4, the reported gap changes from −0.033 to −0.078.Source: §4.4 · p. 7

4.5 Diverse-Perspective Ensembles Make Bias Visible

  • The Diverse-Perspective Ensemble reduces its internal gap by 67–94% under SharedPool, while Layers fails to repair it reliably.Repair tracks whether the topology pools the perspectives, not which model is used.Source: §4.5 · pp. 7–8

5 Discussion

Aggregation topology cannot be presumed safe because the topology with the largest gap varies by model, while single-version audits can miss disparities that persist in committed decisions. Equal accuracy across groups does not establish equal allocation of care.

5 Discussion

  • Across model families, the topology with the largest gap varies, so team safety cannot be inferred from architecture or single-model audits.The committed decision can preserve more acute triage for Black patients across every model-by-topology cell.Source: §5 · p. 8
  • A single-version audit can pass while a systematic, identity-driven difference allocates the same presentation different care.Over-triage can divert finite emergency resources and expose patients to unnecessary testing or admission.Source: §5 · p. 8
  • A sex disparity can shape intermediate reasoning under Centralized aggregation yet disappear from the final allocation, escaping audits of isolated models and team outputs.The intermediate disparity is described as consistent and one-directional.Source: §5 · p. 9
  • Equal accuracy across groups can coexist with unequal care allocation, so accuracy and allocation fairness require separate measurement.The paper reports that models are equally accurate against gold outcomes across groups, while differing in assigned care.Source: §5 · p. 9

6 Conclusion

Clinical multi-agent systems can launder demographic bias into plausible decisions that single-version audits, fairness instructions, and de-identification do not detect. The paper finds that counterfactual comparison inside the team can reduce disparity without retraining or architectural change.

6 Conclusion

  • No configuration across five model families and four topologies neutralizes disparity: all 20 cells commit the direction of agents’ independent answers.Every model has a topology that amplifies the disparity, but the topology varies by model.Source: §6 · p. 9
  • A diverse-perspective ensemble cuts triage disparity by up to 94% with no retraining or architectural change.Agents see different demographic versions of the same patient.Source: §6 · p. 9
  • The disparity is encoded in individually plausible clinical decisions and is not detected by single-version audits, fairness instructions, or de-identification.The paper names this failure mode disparity laundering.Source: §6 · p. 9
  • The paper concludes that the committed decision must be audited across counterfactual demographic swaps of the same patient.The proposed defense makes the bias visible to the system rather than relying only on output audits.Source: §6 · p. 9

Limitations

The study’s scope is bounded by its factorial size, demographic categories, data source, and homogeneous teams; it does not claim to cover every deployed clinical multi-agent system.

Limitations

  • The factorial covers 120 cases and three seeds per cell; larger coverage could sharpen per-cell estimates.The authors ground their conclusions in the direction and consistency of shifts across model families, not any single cell.Source: Limitations · p. 9
  • Counterfactual identities are restricted to categories in the source records and binary sex; absent identities lack gold outcomes and clinical interpretability.Richer and intersectional identities are a future extension once a gold standard exists.Source: Limitations · p. 9
  • Cases come from one emergency-department source, and teams are homogeneous instances of one model by design.This controls model capacity to study communication structure; the authors do not claim coverage of every deployed clinical multi-agent system.Source: Limitations · p. 9
  • The findings support an invariant across tested configurations, not a fixed ranking of aggregation topologies.The topology associated with the largest disparity is model-dependent.Source: Limitations · p. 9

Ethics Statement

The study uses de-identified MIMIC-IV-ED records and explicit demographic tokens as audit instruments, and its systems are research probes rather than validated clinical tools. The appendices collect materials for auditing and reproduction.

Ethics Statement

  • Race and sex enter as single explicit tokens drawn from source-record categories, isolating demographic-label effects without representing intersectional or self-identified identity.The authors state that this use is not an endorsement of race as a clinical variable.Source: Ethics Statement · p. 9
  • Cases derive from credentialed, de-identified MIMIC-IV-ED records, and models run locally without sending patient-derived text to third-party APIs.The study involves no human subjects.Source: Ethics Statement · p. 9
  • The systems are research probes, and the Diverse-Perspective Ensemble is an audit-derived mitigation in a sandbox, not a validated clinical intervention.The paper reports a failure mode to inform system-level audits, not clinical deployment.Source: Ethics Statement · p. 10

Appendix Overview

  • The appendices provide experimental setup, verbatim instructions, extended results, deliberation-stage analyses, and a case-study transcript.They collect material intended to support auditing and reproduction.Source: Appendix Overview · p. 12

A.1 Dataset and Counterfactual Cohorts

Across the model-by-topology grid, the committed triage gap remains on the same side of parity, while its magnitude varies by model and topology. The experiments use counterfactual versions of real emergency cases and compare several aggregation interventions.

A.1 Dataset and Counterfactual Cohorts

  • The race cohort comprises 120 MIMIC-IV-ED encounters, each rendered in five variants that differ only in the race descriptor or its omission.The separate sex cohort uses 118 sex-invariant encounters in male, female, and demographic-free control variants.Source: §A.1 · p. 12

C.1 Full Topology Grid: Baselines and Committed Gaps

  • All 20 model-by-topology cells have negative baseline gaps, and none commits an unbiased decision.Mistral under Layers shrinks from −0.086 to −0.053, while Phi-4 under SharedPool remains at −0.025.Source: §C.1 · p. 13
  • Across four topologies, every cell leans toward more acute triage for the Black version, and Llama has no White-leaning flip under any topology.Counts are case–seed pairs, with n=360 per cell.Source: Table 6 · p. 14, §C.1 · p. 14

C.2 The Hispanic and Asian Arms

  • The committed skew also appears in Hispanic–White and Asian–White contrasts, with 19 of 20 and 18 of 20 cells negative, respectively.The Black–White axis is reported as primary because it is the largest model-level contrast and the field-standard audit pair.Source: §C.2 · p. 14

C.3 Sex Disparity under Centralized Aggregation

  • Under Centralized aggregation, an exploratory sex-axis analysis finds the imaging Male–Female gap moves toward male patients in all ten encounters selected for movement.The sex cohort has 118 encounters and uses Qwen3 with eight seeds for this axis.Source: §C.3 · p. 14, Figure 3 · p. 14

C.4 Aggregator Interventions: Change per Cell

  • Fairness instructions and demographic masking produce no systematic movement toward parity; the largest shift in either direction is about 0.02.Masking removes the race token only from the synthesis input, while passing the team’s decisions through unchanged.Source: §B.4 · p. 13, §C.4 · p. 14

C.5 Per-Agent Self-Debiasing: Per-Model Values

  • Per-agent self-debiasing widens the committed gap in all five models, with pooled growth ranging from +0.011 for Qwen3 to +0.045 for Phi-4.The Phi-4 pooled gap more than doubles, from −0.033 to −0.078.Source: §C.5 · p. 15

C.6 The Ensemble Across Cells

The Diverse-Perspective Ensemble reduces committed gaps most reliably when the topology pools all five views. The reported comparisons also find no fairness–accuracy trade-off and skew beyond triage.

C.6 The Ensemble Across Cells

  • With homogeneous final-round gaps as the comparator, the ensemble shrinks gaps by 58–100% in every SharedPool and Decentralized cell, while four of five Layers cells widen.Centralized moves four of five cells closer to parity; two cells overshoot past zero.Source: §C.6 · p. 16
  • The SharedPool fairness–utility comparison finds that no tested method pays for fairness in accuracy or buys accuracy with bias.The comparison includes homogeneous teams, noninteractive voting, two cross-world instantiations, and the ensemble.Source: §C.6 · p. 16
  • At SharedPool, the ensemble drives the committed gap near zero in every model, whereas the two aggregator patches leave the baseline gap and self-debiasing moves it further from parity in several models.The figure traces each model across five interventions; negative gaps denote more urgent triage for Black patients.Source: Figure 4 · p. 16

C.7 Beyond Triage: The Other Decision Fields

  • Across the five other decision fields, 22 of 25 model–field gaps favor more care for the Black version, while testing, imaging, admission, and analgesia tilt that way in every model.The two negative gaps are both for mental-health referral and remain within 0.004.Source: §C.7 · p. 16

C.8 Rationales Behind Divergent Decisions

In rationale-elicitation runs, divergent committed decisions are accompanied by stated clinical reasoning without demographic terms. The supplied accuracy comparison separately reports limited variation across topology cells.

C.8 Rationales Behind Divergent Decisions

  • Across the 20 cells, accuracy moves by at most 0.06 MAE across a model’s four topologies, while the per-cell Black–White MAE difference stays within 0.044.The table evaluates triage accuracy against MIMIC-IV-ED gold acuity on a subset of n=60.Source: Table 11 · p. 17
  • Among 25 Qwen3 and 37 Mistral divergent paired runs, none of the associated 1,983 stated rationales mentions race, ethnicity, or another demographic term.The wider rationale screen found twelve demographic mentions across 23,021 rationales, none from a run behind a divergent pair.Source: §C.8 · pp. 16–17
  • A manual read of a 100-rationale sample finds the stated reasoning clinical throughout, while wider indirect-reference screening returns only clinical homonyms.The indirect-reference screen included terms such as background, community, population, cultural, and socioeconomic.Source: §C.8 · p. 17

D The Disparity Across Deliberation Stages

Race disparity remains present at every measured stage, even as its magnitude changes during deliberation. Consensus gains do not imply that the gap is repaired.

D The Disparity Across Deliberation Stages

  • All 80 stage readings are negative, so no run crosses parity at the independent, debate-round, or committed stage.For example, Mistral under Layers moves from −0.086 at the independent stage to −0.053 at commitment, while Qwen3 under Layers moves from −0.033 to −0.064.Source: D The Disparity Across Deliberation… · p. 17
  • The absolute gap widens during debate in 12 of 20 runs and at commitment in 10 of 20, so neither stage accounts for all widening.The passage describes Figure 7 as plotting the twenty trajectories.Source: D The Disparity Across Deliberation… · pp. 17–18
  • Consensus rises in 16 of 20 cells, but the gap’s magnitude is retained at 80% or more in 15 cells and amplified in 12.SharedPool has the strongest mean consensus gain (+0.077), while four of five Layers cells show a negative gain.Source: D The Disparity Across Deliberation… · p. 18

E Case-Study Transcript: MIMICIV-90

Across the full factorial, the disparity persists throughout deliberation, and rising agreement often coexists with amplification. The case transcripts show how unanimous teams can still commit unequal triage.

E Case-Study Transcript: MIMICIV-90

  • In the Qwen3 SharedPool case, all emitted non-triage decisions match across demographic versions, so the cross-version difference sits in triage.Source: E Case-Study Transcript: MIMICIV-90 · p. 18
  • Pooling all five perspectives pulls gaps toward zero, whereas the Layers chain leaves disparity standing or widens it.Source: Figure 6 · p. 19
  • 22 of 25 pooled non-triage field gaps are positive, showing that the skew is not confined to triage.Source: Table 14 · p. 19

E.1 A Diverse-Perspective Ensemble Transcript: MIMICIV-92

  • In MIMICIV-92, the Black-seeing agent starts one ESI level more acute; after two pooled rounds, all five agents converge and the team commits their shared level.Source: §E.1 · p. 20, Table 17 · p. 21

E Case-Study Transcript: MIMICIV-90

Related papers

Loading 2610.38215v1…