Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering

1 INTRODUCTION

• The denoising trajectory gives an attacker repeated access to the unresolved answer probability, enabling feedback to adjust steering before the answer is committed.Autoregressive models expose that probability only at commitment, while dLLM steering methods described here use open-loop strengths.Source: §1 · p. 2
• Targeted bias injection uses a PI controller to adapt one fixed steering direction from target-answer probability while leaving weights and prompts unchanged.The attack targets a demographic option over its comparator on ambiguous questions whose correct answer is abstention.Source: §1 · pp. 2–3
• The work examines attack magnitude, feedback versus fixed strength, transfer across targets, benchmarks, and models, and whether the loop can serve as a defense.It situates the question in a setting where dLLMs can match similarly scaled autoregressive models and may be used in screening decisions.Source: §1 · pp. 1, 3
• The attack raises the BBQ target–comparator gap by 14.9 (Black), 22.5 (Arab), and 37.2 (Old) percentage points, and lifts stigmatizing answers on SocialStigmaQA from 17.6% to 58.1%.Each target-specific attack takes about 40 minutes on one GPU and requires no training.Source: §1 · p. 3
• Feedback wins when no single steering strength serves every example, while a tuned constant does better than the other fixed-strength choices described.The paper evaluates this pattern across three dLLMs and seven steering baselines.Source: §1 · p. 3

2 THREAT MODEL

• The adversary controls a serving-stack component that can add bounded residual-stream offsets and observe intermediate answer-token probabilities, but cannot inspect weights, compute gradients, or train.Prompt attacks, decoding-procedure attacks, and direct edits to logits or responses fall outside this interface.Source: §2 · p. 3
• On ambiguous questions where abstention is correct, the adversary aims to increase the target-group option’s margin over the comparator.The score is the expected target–comparator gap over the position-balanced item distribution.Source: §2 · p. 3
• A successful attack must raise the gap while keeping outputs coherent, so the invalid-output rate is reported alongside it.The described runs fit within one GPU hour per target.Source: §2 · p. 3
• The provider shares the inference-time interface and can use a sign-inverted controller to keep ambiguous-item answers near abstention.This is framed as mitigation through the same activation interface.Source: §2 · p. 4

3 CLOSED-LOOP BIAS INJECTION

• The PI loop adapts steering strength during denoising from the target-answer probability, adding the scaled direction while leaving model weights, prompt, and sampler unchanged.The controller runs after each forward pass, with the next pass using the updated strength.Source: §3 · p. 4
• The direction is the unit-normalized mean difference between target-option and comparator-option answer-token residual activations at block 14.Held-out paired completions use option text rather than answer letters so the direction reflects the demographic group rather than display position.Source: §3 · p. 5
• The sensor reads the target letter’s probability at the first generated position after each steered pass, before that answer token is committed.The probability sums the plain and space-prefixed forms of the target-letter token; answer positions are rotated across prompts.Source: §3 · p. 5
• The PI command combines the current setpoint shortfall with accumulated error, and its scaled direction is added at every block and position.When clipping occurs outside the allowed range, the accumulated error is held unchanged to guard against integrator wind-up.Source: §3 · p. 5
• The controller can keep running after commitment, but later commands cannot change a committed answer; with the described 32-token response, only the first 32 commands matter.A 32-step run reproduces the same schedule and serves as a numerical replicate.Source: §3 · pp. 5–6
• Table 1 covers 400 Black-target BBQ items with three answer rotations and reports target, comparator, abstention, and invalid-output rates alongside the gap.Invalid outputs remain in all denominators.Source: Table 1 · p. 6
• The analysis assumes a local linear response before commitment: constant strength reaches the setpoint only above an example-specific threshold, whereas PI can converge to that threshold for a range of gains.The analysis applies only before the answer commits; proportional control alone settles with nonzero error.Source: §3 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation uses frozen LLaDA, Dream, and LLaDA-MoE models, position-rotated BBQ and SocialStigmaQA prompts, and held-out pairs for fitting directions.It reports target, comparator, abstention, and invalid rates with paired-bootstrap intervals; BBQ includes 400 Black-target items and SocialStigmaQA includes 518 items over seven race stigmas.Source: §4.1 · pp. 6–7

4.2 RQ1: HOW MUCH BIAS CAN BE INJECTED?

• The BBQ target–comparator gap rises from 1.8 to 16.7, while no baseline exceeds 4.6.On the other 300 primary items, the gap is 16.2; across three further item draws, the mean gap is 16.2 versus −0.3 unsteered.Source: §4.2 · p. 7

4.3 RQ2: IS FEEDBACK RESPONSIBLE FOR THE GAIN?

• At mean strength 3.28, the effort-matched open loop reaches gap 3.5 and invalidates 20.8% of outputs, versus 7.8% under PI; hindsight per-item tuning reaches only 8.4.PI leads the hindsight oracle by 8.3 points, with fewer invalid outputs (7.8% against 13.4%), supporting a gain from adjusting strength during denoising.Source: §4.3 · p. 7

4.4 RQ3: DOES THE ATTACK TRANSFER?

• On LLaDA-8B, the Arab target reaches gap 22.3 below 15% invalidity, while Old yields the largest shift, ∆g = 37.2; Asian, Latino, and White runs exceed the invalidity threshold.The latter runs are reported as failures rather than attacks.Source: §4.4 · p. 7, Table 3 · p. 8
• On SocialStigmaQA, PI raises the strict-parser biased–safe gap by 83.8 points across 1,554 race prompts, and shifts answers on 111 prompts without tested race descriptions.Calibration and evaluation share all 37 templates, so the result establishes target-answer transfer rather than a mechanism specific to racial stigma.Source: Table 2 · p. 8, §4.4 · p. 8
• Across models, Dream has ∆g = 4.2 with αmax = 1, while tuned CAA beats feedback by 2.3 points on LLaDA-MoE.On Dream, PI's lead over baselines is unresolved; on LLaDA-MoE, CAA and the matched open loop produce byte-identical outputs.Source: §4.4 · p. 8

4.5 RQ4: DEFENSE AND MEASUREMENT

• With a zero setpoint, suppression holds the Black-target BBQ gap at 0.2 with 1.8% invalidity, versus 2.2 for mean-matched steering and 0.8 for AurA suppression.The unsteered gap is only 1.8, so this shows stable sign-inverted control, not reversal of a large existing bias; the PI gap also varies from 39.5 at answer position A to 5.3 averaged over B and C.Source: §4.5 · pp. 8–9

• Activation steering edits frozen-model hidden states through directions, head-level edits, neuron damping, or transport maps, and has been used both to reduce bias and to plant it.Prior examples include prompt-pair and averaged-contrast directions (Turner et al., 2023; Zou et al., 2023), head-level edits, neuron damping, transport maps, bias reduction, and the Trojan activation attack.Source: §5 · p. 9
• Unlike prior dLLM steering methods that fix interventions before generation, this work measures target-answer probability and adjusts steering across denoising steps.PID Steering applies proportional, integral, and derivative terms across depth, whereas this loop operates across denoising steps.Source: §5 · p. 9
• The work targets demographic preference through activations, unlike dLLM attacks using prompts, decoding, or safety neurons, and draws on established bias-measurement benchmarks.Its protocol controls for benchmark sensitivity to answer position, parsing, and abstention.Source: §5 · p. 9

• A PI controller uses target-answer probabilities to adjust one steering direction during denoising, shifting answers several times more than the best fixed-strength baseline on BBQ and SocialStigmaQA without changing weights or prompts.A constant matched to the controller’s mean strength moves the gap little while breaking more outputs, and a per-example constant recovers under half the effect.Source: §6 · p. 9
• Checkpoint audits can miss deployment bias; the attack leaves a probe-visible signature by lowering abstention and raising invalid outputs, and runtime activation hooks belong in the trusted computing base.The same loop with a zero setpoint acts as a suppressor, but testing is limited to cases where the unsteered gap is already near zero.Source: §6 · p. 9
• The evaluation studies multiple-choice answers measured by letter probability; free-form generation remains untested because the target has no single token to measure.Controller gains used 100 of 400 primary items, and several per-target strengths were selected on evaluation data.Source: §6 · p. 9
• The control-theoretic analysis is local and assumes a linear response before commitment.Source: §6 · p. 10

• Targeted steering could amplify discrimination, manipulate evaluations, or conceal biased behavior; experiments use public benchmarks and generated outputs, with no new human-subject data.Source: ETHICS STATEMENT · p. 10
• Benchmark categories are coarse social labels, generated outputs may contain discriminatory text, and measured shifts describe the model and intervention rather than represented groups.The experiments do not establish a deployable bias-mitigation method.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• Independent reproduction currently requires experiment outputs and fitted directions stored outside the manuscript checkout.The code implements direction fitting, generation, intervention, rotation, and evaluation.Source: REPRODUCIBILITY STATEMENT · p. 10
• The planned release will include revision and split metadata, fitted tensors, environment lockfiles, per-row commands, and all 23,310 SocialStigmaQA outputs.The planned package also includes hashes, checksums, and template-clustered bootstrap settings.Source: REPRODUCIBILITY STATEMENT · p. 10
• Planned packaging work includes replacing absolute paths, rejecting missing directions, logging hook attachment, and regenerating tables from immutable outputs.Table generation will use shared strict parsing and metrics that retain invalid responses; LLaDA-MoE sweep notes preserve the preregistered grid and selection decision.Source: REPRODUCIBILITY STATEMENT · p. 10

THE USE OF LARGE LANGUAGE MODELS

• The authors report that generative AI tools assisted with code analysis and manuscript drafting, and that they verified claims and numbers against result files.The authors state they take responsibility for the final manuscript.Source: THE USE OF LARGE LANGUAGE MODELS · p. 10

APPENDIX

• The appendices cover evaluation details, implementation settings and supplementary results, proofs, and expanded related work.Appendix A covers evaluation, B implementation and operating points, C supplementary results, D proofs, and E related work.Source: APPENDIX · p. 15

A EVALUATION DETAILS

• The evaluation conventions apply to BBQ unless a table says otherwise; SocialStigmaQA additionally reports a semantic parser and a gap conditioned on valid outputs.SocialStigmaQA otherwise follows the same conventions.Source: A EVALUATION DETAILS · p. 16

A.1 ANSWER PARSING AND SCORING

• The strict parser requires an answer letter at the response start followed by whitespace, punctuation, or end of string; invalid outputs remain in every rate denominator.The target–comparator gap is their rate difference, and ∆g is the condition’s gap minus the unsteered gap on the same target.Source: §A.1 · p. 16

A.2 ANSWER-POSITION ROTATION

• Each BBQ condition uses 400 items in three cyclic answer-order rotations, producing 1,200 prompts and placing each semantic option at A, B, and C once.An always-A predictor has a target rate of exactly 1/3; GapA measures prompts with the target at A, while GapB/C averages the other positions.Source: §A.2 · p. 16

A.3 CONFIDENCE INTERVALS

• BBQ intervals use 10,000 bootstrap resamples; item-clustered paired comparisons keep each source item’s three rotations together and preserve every reported sign.Table 4 lists intervals for appendix comparisons not plotted in a figure; Figure 4 shows item-clustered intervals for paired comparisons stated in the body.Source: §A.3 · p. 16, Table 4 · p. 16, Figure 4 · p. 17

A.4 MATCHED OPEN-LOOP BASELINES

• Matched open-loop comparisons replace the controller’s per-item, per-step command with a constant while keeping its direction and injection sites fixed.For Black, the constant α = 3.28 matches mean strength over the 32 token-updating steps, not total actuation across all 64 steps; conventions differ across experiments.Source: §A.4 · p. 17

A.5 CALIBRATION SPLIT AND HELD-OUT RESULTS

• On 300 held-out items, decode-time PI reaches a 16.2 pp gap with 7.2% invalid outputs and ∆g = 14.1, leading every baseline by at least 12.1 pp.The held-out comparison includes a 13.9 pp lead over fixed α = 4 and a 12.8 pp lead over the matched open loop; both intervals exclude zero.Source: §A.5 · pp. 17–18

A.6 SENSITIVITY TO THE ANSWER PARSER

• Under an answer-letter fallback parser, the Old open loop’s strict-invalid rate falls from 19.0% to 7.5% and its gap rises from 25.8 to 30.8 pp.PI remains at 9.3% strict-invalid and 31.8 pp; the authors therefore claim no Old-axis strict-validity advantage, only a target-shift advantage supported by both parsers.Source: §A.6 · p. 18

B.1 HYPERPARAMETERS AND RUNTIME

• On one RTX PRO 6000, a direction takes under one minute to fit, and the three-rotation primary PI evaluation takes about 40 minutes.One 400-item rotation takes 13.1–13.4 minutes; BBQ baselines use the listed tuned operating points, while SocialStigmaQA uses effort-matched α = 2.65.Source: §B.1 · p. 18

B.2 OPEN-LOOP STRENGTHS FOR THE RACE TARGETS

• The Arab and Latino matched-direction open loops use α = 4, Asian uses α = 2, and White uses α = 1; these strengths were selected on evaluation items.The sweeps use the same 400 items as the balanced evaluation.Source: §B.2 · p. 18

B.3 AGE TARGET

• For the Old target, αmax = 6 exceeded the strict-invalidity threshold, so PI and the open loop use α = 4.The Old sweep uses evaluation items; its α = 3 effort-matched open loop over-matches PI’s realized mean of 2.54.Source: §B.3 · p. 18

B.4 LAYER-WISE PID BASELINE

• The layer-wise PID baseline uses gains (Kp, Ki, Kd) = (1, 0.05, 0.02), with layer-wise PI setting Kd = 0 and layer-wise P setting Ki = Kd = 0.The described initialization makes the integral sum empty and derivative term vanish at layer ℓ = 0.Source: §B.4 · p. 18

B.5 SUPPRESSION EXPERIMENT

• The suppression experiment reuses the Black-target direction, items, and rotations, with setpoint s∗ = 0 and inverted actuator range αt ∈ [−6, 0].Its comparisons include matched-direction open loops at α = −4 and α = −0.72, plus the AurA suppression gate.Source: §B.5 · p. 19

B.6 DREAM-7B SETTINGS

• The Dream evaluation uses the same 400 items and rotations, with per-model operating points and the strict parser for scoring.PI uses αmax = 1, ActAdd α = 8, and CAA α = 2; five other baselines are refit from Dream activations at LLaDA operating points.Source: §B.6 · p. 19

B.7 LLADA-MOE SETTINGS

• For LLaDA-MoE, the primary controller uses block 8 with αmax = 2, while the secondary setting uses blocks 9–12 with αmax = 1.The seven-setting sweep does not exceed CAA’s 23.7 pp unrotated gap while maintaining at most 1% strict-invalid outputs; block 8 at αmax = 3 reaches 27.0 pp with 9.8% strict-invalid outputs.Source: §B.7 · p. 19

C ADDITIONAL RESULTS

• Figures 5 and 7 summarize supplementary point estimates, while Figures 6 and 4 report uncertainty for comparisons discussed in the main text.The reported operating points match the main-text tables.Source: C ADDITIONAL RESULTS · p. 19

C.1 BLACK TARGET ON LLADA-8B

• Figure 5 separates Black-target outcomes into target, comparator, abstention, and invalid shares; Figure 6 plots method gaps and base-versus-PI results on Black, Arab, and Old.Figure 5 also gives item-clustered intervals for each gap, while Figure 6 gives prompt-level intervals.Source: §C.1 · p. 19, Figure 5 · p. 20, Figure 6 · p. 20

C.2 OTHER RACE TARGETS

• Position-balanced PI is 93.8% strict-invalid on Asian, 96.1% on White, and 59.6% on Latino, making the White PI result invalid as an attack.Valid open loops yield ∆g = 1.0 pp for Asian, 2.5 pp for White, and 1.5 pp for Latino; single-layer CAA remains within |∆g| ≤1.0 pp for every race target.Source: §C.2 · p. 19

C.4 ADDITIONAL ITEM SAMPLES

• Three additional 400-item draws produced gaps of 15.0–17.7 pp, compared with 16.7 pp on the primary set.After removing overlaps with the calibration subset, the three gaps were 16.4, 14.7, and 17.0 pp, each with an item-clustered 95% interval lower bound of at least 11.0 pp.Source: §C.4 · p. 21

C.5 SUPPRESSION RESULTS

• For the Black target, suppression reduced the unsteered 1.8 pp gap to 0.2 pp, but the interval included zero and does not establish mitigation of substantial pre-existing bias.The matched constants and AurA suppression also had intervals containing zero.Source: §C.5 · p. 21

C.6 LOWERCASE ANSWER TOKENS IN THE TARGET PROBABILITY

• Adding lowercase answer tokens changed the Arab-target PI gap by −3.8 pp, with an interval including zero, while abstention rose from 16.0% to 30.0%.Of 580 strict target successes under the Arab direction, 480 were lowercase, and the uppercase-only measurement led the controller to keep accumulating error after target selection.Source: §C.6 · pp. 21–22

C.7 CONTROLLER TRAJECTORIES

• Token-committing PI commands varied across examples (2.2–4.1, 10th–90th percentile) and within sequences; mean-matched and oracle per-item constants both remained below PI.The median sequence spanned 2.36 units of α, while on Dream PI remained at its bound and behaved as a constant command.Source: §C.7 · p. 22

C.8 SOCIALSTIGMAQA

• All seven SocialStigmaQA race-stigma slices shifted toward the target; similar movement on the identity-free diagnostic supports transfer of answer control rather than a race-stigma-specific mechanism.Strict-parser gaps across race slices were 37.4–43.7 pp, compared with 48.6 pp on the identity-free diagnostic.Source: Figure 8 · p. 23

D THEORETICAL VALIDATION

• The theoretical analysis formalizes example-specific open-loop thresholds, proportional steady-state error, and PI stability and integral-state convergence over an explicit gain range.It also treats derivative control, saturation, and the commitment horizon.Source: D THEORETICAL VALIDATION · p. 23

D.1 LOCAL MODEL OF THE LOOP

• The local model approximates pre-commitment target probability as a response to prior probability and steering strength, assuming constant sensitivity and persistence while ignoring saturation.Commitment ends the model’s validity; its asymptotic statements describe the regime approached when the commitment horizon is long relative to settling time.Source: §D.1 · pp. 23–24

D.4 PROPORTIONAL-INTEGRAL FEEDBACK

• Within |γ − bKp| < 1 and 0 < bKi < 2(1 + γ) − 2bKp, PI error vanishes and applied strength converges to each example’s α*.The integral state stores the example’s threshold scaled by 1/Ki, without the controller knowing b, γ, or d.Source: §D.4 · pp. 26–27
• The stability condition limits sensitivity; moderate gains and a small integral gain widen the tolerated range at the cost of a slower transient.Within the stability region, the limit α∞ = α* does not depend on the gains; only the speed of approach does.Source: §D.4 · p. 27

D.5 DERIVATIVE ACTION, SATURATION, AND THE COMMITMENT HORIZON

• Derivative action reshapes only the transient; saturation freezes integral accumulation, while at αmax = 6 the commitment horizon limits strength on resistant examples.If α* > αmax, no admissible command reaches the setpoint under the local model.Source: §D.5 · p. 27

E.1 ACTIVATION STEERING AND ITS CONTROL-THEORETIC READING

• Unlike PID Steering’s proportional control across network depth, this loop controls one scalar using the target-answer probability across successive denoising steps.The measured output is the attacker’s answer probability, and the controlled input is steering strength.Source: §E.1 · p. 27

E.2 INFERENCE-TIME CONTROL AND ATTACKS ON DLLMS

• Prior dLLM interventions do not use a designated answer’s evolving probability as feedback; this attack targets demographic preference without changing prompts, weights, or commitment schedule.The related-work discussion contrasts this approach with fixed-direction, representation-alignment, and scheduled interventions.Source: §E.2 · p. 28

E.3 MEASURING DEMOGRAPHIC BIAS

• The attack repurposes BBQ and SocialStigmaQA as objectives, while controlling for answer position, abstention handling, invalid generations, and response parsing.These factors can change the apparent measured effect.Source: §E.3 · p. 28