PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Triggering for Bias Correction in LLM Reasoning

Nayoung Kim, Mickey Mancenido, Huan Liu

arXiv:2608.25379v1cs.CLcs.AI

TL;DR

Chain-of-thought can expose and propagate demographic stereotypes during reasoning, making intervention timing a central unresolved problem. The paper introduces Adaptive Triggering, which combines online CUSUM detection with white-box or black-box bias signals. Black-box triggering recovers most disambiguated-context accuracy lost under fixed scheduling with fewer interventions, while white-box gains on ambiguous items are accompanied by losses on disambiguated items.

  • Problem

    Existing methods either evaluate completed reasoning chains post hoc or intervene at predetermined steps, leaving open when developing trajectories provide enough evidence for correction.

  • Method

    Adaptive Triggering updates a CUSUM statistic with per-step white-box or black-box bias-risk signals and injects targeted correction after a calibrated threshold is crossed.

  • Results

    Black-box triggering on gpt-4o-mini recovers most disambiguated-context accuracy lost under fixed scheduling with substantially fewer interventions, while white-box signals improve ambiguous-context accuracy on all six open-weight models but reduce disambiguated-context accuracy on five.

  • Takeaways & Limitations

    Effective inference-time correction requires both an informative monitoring signal and a selective intervention rule; timing alone cannot repair a misaligned signal.

  • Takeaways & Limitations

    Most black-box experiments use the generator itself as judge, and the independent-judge replication preserves the disambiguated advantage but not the ambiguous-item improvement.

Abstract

from arXiv · show

Chain-of-thought prompting can expose and amplify demographic stereotypes within an LLM's intermediate reasoning and create a failure mode that final-answer debiasing alone cannot address. Mitigating such bias during generation presents a fundamental timing problem: intervening too late allows biased reasoning to propagate, while unnecessarily intervening can disrupt otherwise correct reasoning. Existing approaches largely avoid this decision by either evaluating completed reasoning chains post hoc or intervening at predetermined steps, leaving open when a developing reasoning trajectory provides sufficient evidence to warrant correction. We formulate this decision as an online change-point detection problem. A per-step bias signal updates a CUSUM statistic and a targeted correction is injected only when accumulated evidence crosses a detector-specific threshold calibrated on held-out data. We instantiate the framework with a white-box signal derived from next-token probabilities and a black-box signal obtained from an LLM judge, enabling deployment with both open-weight and hosted models. On gpt-4o-mini adaptive black-box triggering recovers most of the disambiguated-context accuracy lost under fixed-interval intervention while requiring substantially fewer interventions. That result holds even with an independent judge. Across six open-weight models, the white-box signal improves ambiguous-item accuracy on all six but reduces disambiguated-item accuracy on five because it cannot distinguish unsupported stereotype reliance from correct, stereotype-congruent evidence.

I. INTRODUCTION

Chain-of-thought can expose and propagate demographic stereotypes during reasoning, creating a timing problem for inference-time correction. Adaptive Triggering separates the monitored evidence from the decision to intervene, with results showing both the benefits and limits of this separation.

  • Motivation: Chain-of-thought reasoning can reveal stereotyped assumptions in intermediate steps and amplify bias on benchmarks such as BBQ.These assumptions may influence subsequent conclusions even when direct answers do not expose them.
  • Motivation: Post-hoc scoring acts after reasoning is complete, while fixed schedules intervene at predetermined steps without determining whether the current trajectory warrants correction.The timing dilemma is that late intervention permits propagation, whereas unnecessary intervention can disrupt valid reasoning.
  • Approach: Adaptive Triggering formulates intervention timing as online change-point detection: per-step bias-risk signals update a CUSUM statistic, and correction fires after calibrated accumulated evidence crosses a threshold.The framework separates what evidence is monitored from when enough evidence justifies intervention.
  • Approach: The shared control loop supports white-box and black-box signals without modifying the generator.The variants differ in how they compute the per-step signal, while downstream accumulation and triggering are shared.
  • Results: On gpt-4o-mini, black-box Adaptive Triggering recovers most disambiguated-context accuracy lost under fixed scheduling while intervening substantially less often, including with an independent judge.A matched-budget cadence sweep further distinguishes adaptive selection from merely firing a fixed schedule less often.
  • Limitations: Across six open-weight models, the white-box signal improves ambiguous-context accuracy on all six but reduces disambiguated-context accuracy on five because its evidence is misaligned.It cannot distinguish unsupported stereotype reliance from correct, stereotype-consistent evidence, and calibration does not resolve this limitation.

II. RELATED WORK

Prior work differs in what evidence identifies problematic reasoning and when that evidence triggers intervention. Adaptive Triggering applies online sequential control to bias-risk signals while supporting both white-box and black-box deployment.

  • Evidence and intervention timing: Post-hoc methods score completed answers or reasoning traces, whereas training-based methods modify models or representations to reduce biased reasoning.FRM uses step-level LLM-judge scores to reweight completed reasoning chains; this paper applies analogous supervision online.
  • Evidence and intervention timing: Fixed control points and priming intervene at predetermined locations, while FairSteer dynamically steers hidden activations using model-internal access and a trained classifier.The black-box condition instead uses generated text and an in-context correction without modifying the generator.
  • Sequential control: Related inference-time control methods use sequential statistics, entropy, or held-out calibration to trigger re-decoding, rollback, early exit, or model gating.Adaptive Triggering transfers this sequential-decision perspective to per-step signals intended to capture problematic reasoning.
  • Adaptive Triggering: Adaptive Triggering separates a per-step bias-risk signal from a sequential decision rule that accumulates evidence through a shared CUSUM loop.This separation allows the monitored evidence to vary while the downstream triggering mechanism remains fixed.

A. Setup

The setup evaluates four intervention conditions and defines two online bias-risk signals for BBQ reasoning trajectories. Both signals produce scalar scores for a shared control loop, but they require different access and carry different interpretation limits.

  • Setup: The four conditions are none, fixed_interval, adaptive_wb, and adaptive_bb, differing in whether and how correction is inserted between reasoning steps.fixed_interval is the primary periodic-reflection baseline, while adaptive conditions use targeted interventions triggered by white-box or black-box CUSUM signals.
  • Bias-Risk Signals: A bias-risk signal maps each reasoning step to a scalar bt in [0, 1] intended to capture evidence of unsupported stereotype reliance.The signal is treated as a monitoring proxy rather than a direct measurement of bias.
  • Bias-Risk Signals: The white-box signal normalizes forced-continuation output probabilities and compares preference for the stereotype-consistent answer s against the “not enough information” answer u.It requires access to output probabilities and can mistake evidence-supported stereotype-consistent answers for unsupported reliance.
  • Bias-Risk Signals: The black-box signal uses a separate LLM judge to score the latest reasoning step from 0 to 10 for stereotype reliance, then divides by 10 to obtain bt.Scoring only the latest step supplies the per-step signal for sequential control; same-model judges may inherit self-preference effects.

C. Adaptive Triggering

Adaptive Triggering accumulates detector-specific bias-risk evidence with a calibrated CUSUM rule and injects targeted correction selectively. Calibration evaluates fresh closed-loop generations against downstream accuracy, bias, and non-completion costs.

  • Control loop: The shared CUSUM statistic accumulates either bias-risk signal, and intervention fires when it exceeds a detector-specific threshold h, followed by a one-step cooldown.The loop continues until an answer is produced or the step cap is reached.
  • Calibration: White-box and black-box signals receive separate (k, h) calibration because their empirical score distributions differ.Candidate k values derive from each signal’s score distribution, while h covers thresholds corresponding to approximately two or three elevated observations.
  • Calibration: Calibration optimizes downstream disambiguated accuracy minus ambiguous bias and a non-completion penalty rather than a proxy false-trigger rate.A null calibration is unsuitable because stereotype-consistent answers can be evidence-supported in disambiguated examples.
  • Calibration: Algorithm 1 generates fresh traces for every candidate setting, selects the best score on calibration data, and validates it against a baseline on held-out data.This closed-loop procedure is necessary because interventions change subsequent generation and therefore the monitored trajectory.
  • Calibration: The non-completion penalty prevents aggressive trigger settings from appearing favorable merely because fewer items produce an answer.The paper sets λ = 1, treating a non-answer as equivalent in cost to an incorrect answer for calibration.

D. Intervention

The adaptive and fixed conditions use different reflection prompts, so their comparison evaluates complete intervention strategies rather than timing alone.

  • Adaptive conditions use targeted reflection identifying the suspected failure mode, whereas fixed_interval uses generic reconsideration prompts.

IV. EXPERIMENTS

The experiments evaluate bias mitigation across BBQ categories, models, intervention conditions, and budget-sensitive metrics, with corrected scoring and explicit hosted-model limitations.

  • BBQ pairs ambiguous contexts with disambiguated versions and offers stereotype-consistent, counter-stereotype, and not-enough-information answers across nine social-bias categories.
  • Corrected scoring resolves four undocumented schema inconsistencies that otherwise leave stereotype-consistent answers unresolved for 17–50% of affected items and underestimate bias.
  • Seven open-weight models are listed, while six have verified results for all nine categories and DS-Qwen-7B is excluded from full-scale black-box analyses because its judge setup produced no valid original interventions.
  • Four conditions compare no intervention, fixed-interval reflection, adaptive white-box triggering, and adaptive black-box triggering.
  • The evaluation reports accuracy, bias score, self-correction rate, interventions per item, and fired items as separate intervention-budget measures.
  • Open-weight trajectories use deterministic greedy decoding, whereas hosted gpt-4o-mini results are single-sample estimates because repeated judge calls remain nondeterministic.

C. Results

The experiments test whether trajectory evidence improves intervention over fixed scheduling, how monitored signals affect quality, and what costs intervention introduces.

  • The study asks whether trajectory-conditioned intervention improves fixed scheduling, how signal choice affects intervention quality, and what costs intervention introduces.
  • Figure 2 illustrates unsupported stereotype reliance emerging during generation, evidence accumulation by CUSUM, and intervention redirecting the trajectory to the correct answer.

1) Evidence-Triggered Intervention vs. Fixed Schedules:

On gpt-4o-mini, adaptive black-box triggering recovers most accuracy lost under fixed intervention while using fewer interventions. Across models, fixed schedules produce model-dependent exposure because firing depends on trace length, whereas adaptive selection can outperform comparable-rate schedules.

  • Hosted-model evaluation: 90.1% disambiguated accuracy for adaptive_bb nearly recovers the 92.1% baseline, versus 82.9% under fixed_interval.The adaptive-versus-fixed difference is significant, while the remaining 2.0pp adaptive-versus-baseline difference is not significant.
  • Hosted-model evaluation: 0.60 interventions per disambiguated item for adaptive_bb compares with 1.00 under fixed_interval, while ambiguous-item intervention falls to 0.06 from 1.00.Both intervention-rate comparisons use the pooled intervention-rate panel across context types.
  • Independent-judge replication: +6.8pp disambiguated accuracy over fixed_interval persists with an independent judge, close to the +7.2pp self-judged gain.The independent-judge comparison has a 95% CI of [+2.8, +10.7] and p = 0.0015.
  • Model-dependent fixed exposure: Under N = 5, fixed-schedule firing ranges from 0% on Gemma-9B to 99.0% on DS-Llama-8B because trace lengths differ.Gemma-9B averages 2.78 steps, while Llama-8B averages 6.40 steps and fires on 65.5% of items.
  • Matched-budget comparison: 38.5% adaptive_wb firing achieves comparable ambiguous-item bias to a fixed schedule firing on 95.8% of items.This matched-budget comparison indicates that adaptive selection depends on which trajectories are intervened on, not only how often.

2) Signal Choice Determines Intervention Quality:

Across six open-weight models, white-box Adaptive Triggering improves ambiguous-item accuracy but usually harms disambiguated-item accuracy because its signal conflates unsupported stereotypes with valid evidence. This target mismatch persists despite calibration and reflects signal choice rather than timing alone.

  • Cross-model pattern: White-box intervention improves ambiguous-item accuracy on all six open-weight models but reduces disambiguated-item accuracy on five.The directional pattern replicates across five instruct-tuned models; the sole exception is reasoning-tuned DS-Llama-8B, where the disambiguated change is not significant.
  • Signal semantics: On ambiguous items, preference for the stereotype-consistent answer signals the monitored failure, but on disambiguated items that same answer can be evidence-supported.CUSUM detects accumulated evidence but cannot distinguish cases represented similarly by the underlying signal.
  • Trajectory consequences: 3,112 correct disambiguated answers changed to incorrect versus 761 incorrect answers repaired, a 4.09 ratio.Among items with bias-risk spikes under both conditions, white-box intervention also reduced self-correction from 62.9% to 50.6%.
  • Calibration: Detector-specific calibration leaves the white-box configuration unchanged, indicating that threshold selection alone does not explain its disambiguated-context loss.Across the searched grid, white-box firing rates vary from 21.8% to 26.6%, while black-box rates range from 8.7% to 34.9%.
  • Intervention frequency: Adaptive triggering selects interventions from observed evidence rather than guaranteeing fewer interventions: on Qwen-1.5B, adaptive_bb fires 0.58 times per item versus 0.47 for fixed_interval.A cadence sweep separates adaptive selection from simply intervening less often.
  • Alternative explanation: Testing protected-attribute items without stereotype targets finds no significant white-box difference from none, though the analysis is underpowered.Pooled across three models, adaptive_wb improves 19 items and degrades 17 (p = 0.87).

3) Intervention Costs:

Intervention imposes costs beyond conditional answer accuracy, including non-completion and computational overhead. These costs vary by model and signal type.

  • Non-completion: On DS-Llama-8B disambiguated items, non-completion is 8.5% when adaptive_wb fires versus 4.5% when it does not.Treating non-completions as incorrect would increase the apparent accuracy loss from 0.1 to 2.3 percentage points.
  • Black-box cost: adaptive_bb increases generation calls from 5.1 to 10.1 per item on Qwen-1.5B and from 2.7 to 4.5 on Gemma-9B.The black-box signal requires an additional judge call at each reasoning step.
  • White-box cost: adaptive_wb adds 5.6–12.8 forward passes per item for probability readout but requires no additional generation calls.It is computationally cheaper when generator probabilities are accessible, subject to the documented target mismatch.

V. LIMITATIONS

The conclusions are constrained by judge dependence and by the monitored signal’s inability to separate unsupported stereotype reliance from valid evidence. Overall, effective correction requires both informative evidence and selective timing.

  • Judge dependence: Most black-box experiments use the generator as its own judge, so self-preference effects may affect the results.With an independent judge, the hosted-model replication preserves the disambiguated-context advantage but not the ambiguous-item improvement.
  • Supported conclusion: Black-box Adaptive Triggering recovers most disambiguated-context accuracy lost under fixed scheduling while intervening substantially less often, including with an independent judge.A matched-budget cadence sweep shows adaptive selection is not merely a fixed schedule firing less often.
  • Signal boundary: Across six open-weight models, white-box triggering improves ambiguous-context accuracy on all six but reduces disambiguated-context accuracy on five.The reduction occurs because the signal cannot distinguish unsupported stereotype reliance from valid, evidence-supported reasoning.
  • Implication: Effective inference-time correction requires both an informative monitoring signal and an intervention rule that acts selectively on its evidence.Evidence-based timing can reduce unnecessary intervention, but calibration cannot repair a signal with a target mismatch.
Loading 2608.25379v1…