PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Decentralized Fairness Auditing

Sayan Biswas, Jade Garcia Bourrée, Anne-Marie Kermarrec, Palak, Martijn de Vos

arXiv:2610.10199v1cs.LG

TL;DR

Black-box fairness audits can be difficult when one auditor lacks a large, representative query set, and collaboration creates a risk of fabricated reports that fairwash an LLM. Auditopus uses rounds of decentralized statistics sharing and locally down-weights reports inconsistent with their sources’ histories. It reduces audit error against no defense and robust aggregation baselines, and protects very or moderately unfair LLMs even with 49% adversarial auditors.

  • Problem

    A single auditor may lack the large, representative query set needed for a reliable black-box fairness audit, while collaborative auditing can expose the estimate to fabricated reports.

    Source: §I · p. 1, B. Collaborative Fairness Auditing · p. 4

  • Method

    Auditopus runs decentralized audits in rounds, exchanging cumulative statistics vectors rather than queries and locally down-weighting reports inconsistent with their sources’ histories.

    Source: §I · p. 2

  • Results

    Auditopus reduces audit error by up to 78% on average relative to no defense and at least 62% relative to robust aggregation baselines, and never lets a very or moderately unfair LLM pass as fair with 49% adversarial auditors.

    Source: §I · p. 2

  • Takeaways & Limitations

    Across two datasets and six audit cases, Auditopus reduces attack success rate and estimation error compared with no defense and robust aggregation baselines.

    Source: §VIII · p. 13

Abstract

from arXiv · show

Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-box audits typically assume a single auditor with access to a large, representative set of queries. In practice, it can be difficult for an auditor to obtain such a query set, but multiple auditors can together cover the relevant demographic groups by auditing the LLM collaboratively with their individual query sets. However, relying on multiple auditors raises a fundamental trust problem, as they may act on behalf of the LLM provider to portray a misleading appearance of fairness, i.e., fairwashing. We propose Auditopus, a novel approach for robust decentralized fairness auditing. In Auditopus, auditing proceeds in rounds without a central server. In each round, every auditor issues a fixed number of queries to the LLM, and sends only cumulative statistics vectors of its query results to other auditors instead of sensitive queries in clear. The fairness of the audited LLM is then estimated by aggregating all the vectors. We show theoretically and empirically that even a single adversarial auditor in the network can steer this estimate by fabricating the vectors it sends, making an unfair LLM appear fair. To address this threat, Auditopus has each honest auditor locally down-weight any auditor whose cumulative statistics vectors are statistically inconsistent with previous ones. We implement Auditopus and compare it to robust aggregation baselines on two datasets with two pre-trained LLMs. Against an attacker that optimizes the vectors it sends to make the LLM appear fair, Auditopus reduces audit error by up to 78% on average relative to no defense and at least 62% relative to the robust aggregation baselines. Even when 49% of the auditors are adversarial, Auditopus never lets a very unfair or moderately unfair LLM pass as fair.

I. INTRODUCTION

AUDITOPUS addresses the difficulty of assembling large, representative query sets for black-box fairness audits without exposing sensitive queries. Its round-based, decentralized protocol is designed to resist fairwashing while retaining heterogeneous auditors’ contributions.

A. Black-box Fairness Auditing

  • Black-box audits measure disparities from query-answer pairs; demographic parity is the difference in positive-decision rates between two groups.The audit labels an estimate fair when it falls within a commissioner-set tolerance band.Source: §I · p. 1, A. Black-box Fairness Auditing · p. 3

II. BACKGROUND AND PROBLEM DESCRIPTION

  • A single auditor may lack the large, representative query set needed for reliable demographic-parity estimates, while relevant queries are distributed across organizations.Queries may be sensitive, public benchmarks may not be representative, and commercial API queries cost money.Source: §I · p. 1

I. INTRODUCTION

  • Auditopus reduces audit error by up to 78% on average relative to no defense and at least 62% relative to robust aggregation baselines.Even with 49% adversarial auditors, it never lets a very unfair or moderately unfair LLM pass as fair.Source: §I · p. 2

B. Collaborative Fairness Auditing

  • Heterogeneous query sets make local fairness estimates unreliable, while one-shot aggregation lets fabricated statistics steer an unfair LLM’s estimate into the fair band.Summing group-level counts recovers the disparity on the union of query sets, but robust aggregation can also discard honest auditors with atypical sub-populations.Source: B. Collaborative Fairness Auditing · pp. 3–4

C. Problem Description

The audit must estimate demographic parity from honest auditors’ combined data, withstand fabricated reports, avoid a trusted central party, and keep queries private. Auditopus uses multiple rounds and locally down-weights reports inconsistent with their source’s history.

C. Problem Description

  • The protocol’s goal is for every honest auditor to estimate the demographic parity of the honest auditors’ combined statistics and reach the correct verdict.The honest auditors’ query sets are assumed to jointly represent the population served by the LLM.Source: C. Problem Description · p. 4, B. System and Threat Model · p. 4

A. AUDITOPUS in a Nutshell

  • In each round, auditors query the LLM with private samples, exchange only locally aggregated statistics vectors, and estimate parity from the vectors they receive.Auditors send vectors to selected peers rather than sharing sensitive queries.Source: A. AUDITOPUS in a Nutshell · p. 4
  • Auditors compare each source’s new statistics with its own past reports and down-weight inconsistent increments, rather than filtering reports by whether they differ from peers.Honest auditors sample from fixed query sets, while an attacker must adjust reports to cancel honest contributions.Source: A. AUDITOPUS in a Nutshell · p. 4

B. System and Threat Model

  • The system assumes a connected, permissioned network with known identities, reliable communication, and adversaries that fabricate reports but do not collude with one another.Auditors exchange vectors, not queries; stronger formal privacy can be added with differential privacy or secure multi-party computation.Source: B. System and Threat Model · pp. 4–5

C. AUDITOPUS Auditing Workflow

Each honest auditor samples private queries in rounds, shares signed cumulative statistics, and maintains current reports from peers. It scores each source’s new increment against that source’s history, then estimates demographic parity and issues a verdict.

C. AUDITOPUS Auditing Workflow

  • At each round’s start, an auditor samples a fixed number of unused queries from its private set, queries the LLM, and adds the resulting counts to its statistics vector.The auditor stores its cumulative vector with a signature and the current round.Source: C. AUDITOPUS Auditing Workflow · p. 5
  • Auditors send and relay signed statistics tables, merging entries only when the received entry is newer and validly signed by its source.Comparing rounds prevents duplicate aggregation and stops delayed older vectors from overwriting newer ones.Source: C. AUDITOPUS Auditing Workflow · pp. 5–6
  • For each source, an auditor scores the new statistics increment against that source’s previous vector and gives less weight to statistically inconsistent increments.The first increment has no history to compare against and receives weight 1; peer-specific histories avoid penalizing honest auditors merely for atypical query sets.Source: C. AUDITOPUS Auditing Workflow · p. 6
  • Each auditor estimates parity from the weighted statistics vectors and returns fair when the estimate lies within the tolerance band, unfair otherwise.The decision rule is |d DP_i| ≤ τ for a fair verdict.Source: C. AUDITOPUS Auditing Workflow · p. 6

IV. ADVERSARY

AUDITOPUS’s adversary fabricates cumulative statistics in each round to steer the fairness estimate toward zero. Its strategy can be computed without querying the LLM, and its past fabrications continue to influence the estimate.

IV. ADVERSARY

  • The section reduces the attacker’s optimization to a budget split, characterizes when one attacker can fairwash an audit, and extends the analysis to several non-colluding attackers.The subsequent strategy uses the attacker’s own past fabricated LAS vector as part of its view.Source: §IV · p. 6, A. Strategy of an Adversary · p. 7

A. Strategy of an Adversary

  • The attacker chooses each round’s fabricated increment to bring its current estimate as close to zero as possible.It uses its view of the GAS vector and searches the finite set of feasible moves; it does not query the LLM.Source: A. Strategy of an Adversary · pp. 6–7
  • In round 1, the attacker sends a neutral vector, then uses an adaptive strategy from round 2 onward while relaying other auditors’ entries unchanged.The initial vector splits its budget evenly between the groups, with a positive rate of 1/2 in each.Source: A. Strategy of an Adversary · pp. 6–7
  • Past fabrications remain in the GAS vector, so an attacker acting from round 1 retains at least 1/(N + K) of its mass at every round.Its objective is only a proxy for the honest estimate because it lacks the latest honest LAS vectors; the gap shrinks as the audit proceeds.Source: A. Strategy of an Adversary · p. 7

B. Reduction to a Budget Split

The adversary’s feasible attack can be reduced to choosing how to divide its query budget between two groups. The analysis characterizes when that split can bring the estimated demographic parity into the fair band.

B. Reduction to a Budget Split

  • For a fixed group-A budget split, a move maximizing DP assigns every group-A query a positive label and every group-B query a negative label.The reduction makes the attacker’s four fabricated counts a one-variable choice: how to split its budget between the groups.Source: B. Reduction to a Budget Split · p. 7
  • When extreme labels would overshoot zero, the attacker instead uses intermediate labels to bring DP as close to zero as possible.If no feasible move reaches DP ≥ 0, minimizing the absolute DP amounts to maximizing DP.Source: B. Reduction to a Budget Split · p. 7

C. Fairwashing Conditions for a Single Adversary

  • With τ = 0.05, the stated mild conditions hold once min(A, B) ≥ 10 and max(A, B) ≥ 9n; the binding condition is whether one LAS vector can close the gap to the fair band.The condition is that some feasible split makes φ(∆A) > −τ.Source: C. Fairwashing Conditions for a Single… · p. 7
  • The closer an LLM is to the fair band, the easier it is for one adversary to fairwash; without defense, one adversary fairwashes every near-fair run on both datasets but only some very unfair runs.The gap shrinks over rounds because the GAS vector already includes the adversary’s past fabrications.Source: C. Fairwashing Conditions for a Single… · pp. 7–8

D. Multiple Non-Colluding Adversaries

Multiple non-colluding adversaries can move the estimate into the fair band faster, while their changing reports provide a signal for local detection. AUDITOPUS compares each source’s increments with its own history and softly down-weights statistically unusual reports.

D. Multiple Non-Colluding Adversaries

  • On Civil Comments, the estimate enters the fair band after about 80% of rounds with one adversary, 25% with two, and 10% with three; all curves end close to zero.With honest auditors alone, the estimate stays at DP(f) ≈ −0.23.Source: D. Multiple Non-Colluding Adversaries · p. 8

E. Takeaway

  • Even without coordination, adversaries can push the estimate past zero; each fabricated LAS vector alone resembles an honest auditor’s unusual query set, but the changing sequence can betray the attack.Adversaries need not query the LLM: they can search budget splits and retain the influence of past fabrications in the estimate.Source: E. Takeaway · p. 8

V. DEFENSE; A. Honest and Adversarial Increments

  • AUDITOPUS detects deviations by comparing each source’s increments with its own history, since honest increments reflect a fixed query set while the attacker adapts its reports.The defense runs locally without a trusted party, coordination, or offline calibration; honest sources may legitimately disagree under heterogeneity.Source: §V · p. 8, A. Honest and Adversarial Increments · p. 8

A. Honest and Adversarial Increments

  • For an honest source, the group-A proportion and the two groups’ positive rates in an increment approximately follow binomial models parameterized by that source’s prior LAS vector.The approximation treats the rates estimated from history as exact; it is accurate once history is long relative to the increment and overstates honest deviations in early rounds.Source: A. Honest and Adversarial Increments · p. 8

B. Suspicion Score

  • The suspicion score uses z-scores for group proportion and group-specific positive rates, with the largest defined absolute z-score capped at zmax.A deviation in any one statistic can contribute to suspicion; KT estimates keep probabilities defined even with extreme or short histories.Source: B. Suspicion Score · pp. 8–9

C. Locked Weights

AUDITOPUS assigns each received increment a fixed soft weight based on its suspicion score. Each auditor then computes its defended estimate from the weighted LAS vectors, including its own.

C. Locked Weights

  • Each increment receives a fixed weight w = (1+S)^−2; the capped score ensures an honest source with an unlucky increment is down-weighted rather than excluded.Weights are not revised, and trust from earlier increments does not carry over to later ones; a source’s first increment receives weight 1.Source: C. Locked Weights · p. 9

D. Defended Estimate

  • Each auditor computes its estimate from all sources’ weighted LAS vectors, including its own, rather than anchoring on its potentially skewed local query set.Under heterogeneity, trusting only its own LAS vectors would make honest auditors disagree even without adversaries.Source: D. Defended Estimate · p. 9
  • On a complete graph, auditors receiving the same increments compute the same estimate; gossip delays on sparser graphs can make their estimates differ slightly.The evaluation combines estimates on sparser graphs through their median.Source: D. Defended Estimate · p. 9
  • Each round, an auditor sends each neighbor its table of signed LAS-vector entries and round numbers; scoring takes at most |V| calls to Algorithm 3.The defense exchanges no information beyond the LAS vectors already used by the protocol.Source: D. Defended Estimate · p. 9

VI. EXPERIMENTAL EVALUATION

The evaluation compares AUDITOPUS with robust aggregation baselines under attacks and varying auditor counts and query-set heterogeneity. AUDITOPUS performs best overall and remains robust across the tested fairness levels and heterogeneity settings.

A. Experimental Setup

  • The evaluation uses QWEN2.5-7B-INSTRUCT on Civil Comments and LLAMA-3.1-8B-INSTRUCT on Bias in Bios, with very unfair, moderate, and near-fair cases and N = 20 honest auditors unless varied.The six cases cover the two datasets and three fairness levels.Source: A. Experimental Setup · pp. 9–10, Table I · p. 10
  • With K varying from 0 to 19, auditors use a fixed connected Erdős–Rényi graph, issue 300 queries per round, and run for at most 150 rounds; results average 20 seeds and report ASR and DP estimation error.At the round limit, honest auditors have queried every population input once, so their LAS vectors alone yield the true DP.Source: A. Experimental Setup · p. 10

B. RQ1: Effectiveness of AUDITOPUS and baselines

  • Across 1–19 adversaries, AUDITOPUS cut ASR to 3% versus 97% without defense, 24% with the median, and 41% with the trimmed mean, and reduced mean absolute error 78% versus no defense.Its mean absolute error was 0.034, compared with 0.158 without defense.Source: B. RQ1: Effectiveness of AUDITOPUS and… · p. 10, Table II · p. 11
  • At K = 19 (49%), ASR stayed 0% for very unfair and moderate LLMs; for near-fair LLMs it reached 20%, with error below +0.008 versus +0.061 without defense.AUDITOPUS's error drifts toward the fair band as K grows because it down-weights, rather than discards, fabricated LAS vectors.Source: Figure 6 · p. 11, B. RQ1: Effectiveness of AUDITOPUS and… · p. 11

C. RQ2: Sensitivity to honest auditors and heterogeneity level

  • AUDITOPUS error stayed between 0.02 and 0.05 across auditor counts and heterogeneity levels, while median and trimmed-mean errors grew with heterogeneity to 0.25 and 0.21, respectively.The robust aggregation baselines assume honest values are concentrated; AUDITOPUS instead judges each source against its own history.Source: Figure 7 · p. 12, C. RQ2: Sensitivity to honest auditors… · p. 12

VII. RELATED WORK

Related work distinguishes platform-side audit manipulation from colluding auditors and positions AUDITOPUS among collaborative auditing, robust aggregation, and history-based defenses. The conclusion reports its empirical scope, while later sections describe reproducibility, LLM use, and ethical boundaries.

VII. RELATED WORK

  • AUDITOPUS addresses a complementary threat: a platform provider acting through a colluding auditor rather than manipulating its own answers.Prior platform-side tactics include fair-looking explanations, biased samples, and different answers to audit queries; existing defenses generally assume the auditor is trusted.Source: §VII · p. 12
  • Collaborative fairness-auditing work covers shared query budgets, coordinated query selection, and joint metric computation, but assumes honest auditors even when using zero-knowledge proofs.Poisoned local-differential-privacy reports can shift estimated frequencies; AUDITOPUS instead uses auditors’ multi-round histories to check new vectors.Source: §VII · p. 12
  • Median, trimmed-mean, and robust-gossip approaches can penalize honest atypical auditors, whereas AUDITOPUS compares each auditor’s vectors with its own history.History-based learning methods use momentum or predicted updates; AUDITOPUS uses the binomial behavior of honest auditors sampling fixed query sets, requires no learned predictor or trusted server, and down-weights rather than removes sources.Source: §VII · p. 12

VIII. CONCLUSION

  • Across two datasets and six audit cases, AUDITOPUS reduces attack success rate and estimation error versus no defense and robust aggregation baselines.The conclusion also notes that global aggregation is accurate under heterogeneity but fragile, and that AUDITOPUS avoids penalizing honest but atypical auditors.Source: §VIII · p. 13

OPEN SCIENCE

  • The anonymized repository includes AUDITOPUS, baselines, datasets, prompts, experiment scripts, configurations, and scripts for generating figures and tables.The evaluated models are publicly available through their original providers.Source: OPEN SCIENCE · p. 13

LLM USAGE CONSIDERATIONS

  • LLMs assisted with editorial work and coding, but authors inspected outputs and reviewed and tested generated code; they were not used to propose hypotheses, design methods or experiments, or interpret results.The authors take responsibility for the manuscript’s content.Source: LLM USAGE CONSIDERATIONS · p. 13
  • Experiments use open-weight models, cache one response per prompt for reuse, and include those responses in the artifact to support reproduction without re-querying.The authors report approximately 100 GPU-hours in total.Source: LLM USAGE CONSIDERATIONS · p. 13

ETHICAL CONSIDERATIONS

  • The authors state that no auditing procedure, including AUDITOPUS, can certify fairness with absolute certainty, and that chosen metrics and demographic groups do not capture every form of harm.The work studies adversarial behavior to design defenses, not to introduce new attack capabilities against deployed systems or users.Source: ETHICAL CONSIDERATIONS · p. 13

APPENDIX A NOTATIONS

The appendix develops the proof of Theorem 2 by reducing the attacker’s choice to integer splits and characterizing where the resulting function lies in the fair band. The supplied notation-table caption identifies its scope but does not enumerate the notations.

APPENDIX A NOTATIONS

  • Table III lists notations used throughout the paper.The supplied caption does not specify the individual entries.Source: Table III · p. 15

APPENDIX B PROOFS OF SECTION IV

  • Theorem 2 reduces to finding an integer split y for which −τ < φ(y) < τ, or equivalently a split in I+ ∩ I−.The proof treats the two inequalities separately, characterizes I+ with a quadratic, and uses concavity and small steps to show the sequence cannot jump over the fair band.Source: Appendix B · p. 16

A. Proof of Lemma 1

  • For the extreme-label move, maximizing DP(C + δ) sets Δa = ΔA and Δb = 0, reducing the attacker’s choice to δ = (y, y, 0, n − y) and DP(C + δ) = φ(y).The proof derives this by showing DP(C + δ) increases with Δa and decreases with Δb.Source: A. Proof of Lemma 1 · p. 16

B. Where φ > −τ, the quadratic Γ

  • The sign of φ(y) + τ is the sign of the quadratic Γ(y), which is positive between its roots when it has two distinct roots.The qualifying integer splits form the consecutive interval I+ = {y−, …, y+}; condition 2 holds exactly when Λ > 0 and y− ≤ y+.Source: B. Where φ > −τ, the quadratic Γ · pp. 16–17

C. Where φ < τ, concavity

Concavity makes the endpoint values of φ sufficient to determine whether any split lies below τ. The proof also gives a practical condition under which this endpoint test holds.

C. Where φ < τ, concavity

  • Because φ is concave on [0, n], its minimum is at least the smaller endpoint value, so I− is nonempty exactly when at least one endpoint is below τ.The endpoint values are φ(0) = DP(C) + nb/[B(B+n)] and φ(n) = DP(C) + n(A−a)/[A(A+n)].Source: C. Where φ < τ, concavity · p. 17
  • The proof bounds the endpoint increases over DP(C) by nb/[B(B+n)] and n(A−a)/[A(A+n)], respectively.Using b ≤ B and A−a ≤ A, it bounds the left-hand side by n max(A,B)/(n+min(A,B)) and establishes condition 1 under the stated assumption.Source: C. Where φ < τ, concavity · p. 17
  • If DP(C) < −τ and max(A, B) ≥ n(1−2τ)/(2τ), condition 1 of Theorem 2 holds.The text states that condition 1 holds automatically once the GAS vector is large compared with the attacker’s budget.Source: C. Where φ < τ, concavity · p. 17

D. Small jumps

The small-jumps lemma bounds how much φ can change between neighboring integer splits, enabling the proof that the two qualifying sets intersect. Under Theorem 2’s assumptions, the attacker can therefore reach the fair band.

D. Small jumps

  • For every neighboring pair of integer splits, |φ(y + 1) − φ(y)| ≤ 1/min(A, B) + 1.The proof bounds the change using the largest variations of the two terms in φ, which occur at opposite endpoints.Source: D. Small jumps · p. 17

E. Proof of Theorem 2

  • When conditions 1 and 2 hold, I+ and I− are nonempty; if disjoint, neighboring splits would cross from φ ≥ τ to φ ≤ −τ, contradicting the small-jumps bound.The proof uses I+ being a consecutive set of integers and assumption (11) to rule out that crossing.Source: D. Small jumps · p. 17, E. Proof of Theorem 2 · p. 18
  • Under Theorem 2’s assumptions and conditions 1 and 2, the attacker’s minimizing move δ⋆ satisfies |DP(C + δ⋆)| < τ.The proof obtains a split with |φ(y)| < τ and uses that δ⋆ minimizes the absolute DP over feasible moves.Source: E. Proof of Theorem 2 · p. 18

APPENDIX C ADDITIONAL EXPERIMENTAL DETAILS

The appendix documents the datasets, audit partitions, and implementation used in the experiments, then outlines supplementary experiments on adversary share, sampling, baselines, and query budget.

APPENDIX C ADDITIONAL EXPERIMENTAL DETAILS

  • The experiments use zero-shot classification, sending each input separately without in-context examples or fine-tuning.Source: Appendix C · p. 19

A. Civil Comments dataset

  • On Civil Comments, QWEN2.5-7B-INSTRUCT flags toxicity; 182,172 comments across 16 identity groups remain after four unparsed outputs are dropped.Source: A. Civil Comments dataset · p. 19

B. Bias in Bios dataset

  • Bias in Bios contains 391,895 biographies across 25 occupations and binary gender labels; LLAMA-3.1-8B-INSTRUCT predicts an occupation, and audits compare men and women within sectors.Greedy decoding with structured outputs yields one of 25 occupation labels per biography, with no inputs dropped.Source: B. Bias in Bios dataset · p. 19

C. Data partition

  • Auditors receive heterogeneous partitions drawn from a Dirichlet distribution, with redraws when an auditor has fewer than 30 inputs from either group.Local demographic-parity estimates vary widely: the 5th percentile ranges from −0.55 to −0.84 and the 95th from +0.08 to +0.66.Source: C. Data partition · p. 19

D. Implementation details and hardware

APPENDIX D ADDITIONAL EXPERIMENTS

  • Supplementary experiments examine adversary share, sampling with replacement, robust-aggregation settings, and per-round query budgets.The Civil Comments figure varies adversary count across very unfair, moderate, and near-fair cases.Source: Appendix D · p. 21, Figure 10 · p. 21

A. RQ3: Effectiveness of AUDITOPUS and baselines on Civil Comments Dataset

On Civil Comments, AUDITOPUS reduces estimation error relative to no defense and robust aggregation, though near-fair verdicts remain difficult. Supplementary tests show that sampling with replacement biases estimates, baseline variants remain vulnerable, and AUDITOPUS’s robustness depends on audits retaining enough rounds.

A. RQ3: Effectiveness of AUDITOPUS and baselines on Civil Comments Dataset

  • AUDITOPUS lowers mean DP error from 0.156 without defense to 0.034 (−78%); attack success is 26% on the near-fair case, while median and trimmed-mean errors are 0.180 and 0.171.It is never fooled on the very unfair and moderate cases, but near-fair verdicts remain vulnerable.Source: A. RQ3: Effectiveness of AUDITOPUS and… · p. 22

B. RQ4: Sampling without replacement

  • Without replacement, the estimate converges to the true DP; with replacement, it stays at its biased starting point, with error increasing as data heterogeneity grows.On Christian vs. Muslim, the with-replacement estimate ends at −0.157 against a true DP of −0.228.Source: B. RQ4: Sampling without replacement · p. 22, Figure 11 · p. 23

C. RQ5: Robust-aggregation baselines

D. RQ6: Varying the query budget

  • At 3,000 queries, audits shrink to 1–6 rounds, leaving AUDITOPUS little history to assess updates and limiting how widely adversarial counts spread before the audit ends.Source: D. RQ6: Varying the query budget · pp. 23–24
  • Across budgets up to 3,000 queries per auditor per round, AUDITOPUS is never fooled on very unfair or moderate cases; near-fair outcomes remain vulnerable.On near-fair Bias in Bios, it is fooled in at most 15% of runs up to 2,000 queries and 40% at 3,000; Black vs. White remains vulnerable at every budget.Source: D. RQ6: Varying the query budget · p. 23, Figure 14 · p. 26

Related papers

Loading 2610.10199v1…