PaperlayerView exact revision on arXiv

Source-linked AI summary

REACT: Representational Early Analysis and Control of Chain-of-Thought Trajectories

Hanwen Zhang, Enyi Jiang, Ruike Zhu, Qinan Yu, Sanmi Koyejo

arXiv:2610.32113v1cs.LGcs.AIcs.CL

TL;DR

Reasoning models expose a CoT before their final answer, raising the question of whether the states that reveal risk are also effective intervention points. REACT learns a safety direction and conditionally applies it once to an early CoT state. Gains concentrate in an early window, and on 8B the StrongREJECT-273 fully safe output rate rises from 29.06% to 82.05%.

  • Problem

    The paper asks whether CoT positions that make risk easier to predict also provide effective control through a single intervention before generation is complete.

    Source: §1 · p. 1

  • Method

    REACT learns a safety direction from paired safe and unsafe trajectories and uses a separate gate to apply it once to an early hidden state, with weights frozen and later tokens generated normally.

    Source: §1 · p. 1

  • Results

    29.06% to 82.05%: REACT raises fully safe CoT–answer outputs on StrongREJECT-273; across three models, gains concentrate in an early two- to three-token window.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Risk predictiveness alone does not locate effective intervention sites under the evaluated one-shot steering recipe.

    Source: §1 · p. 2

    The evidence does not establish that early-CoT intervention universally outperforms prompt-boundary steering; the conclusion is limited to the evaluated matched recipe.

    Source: §5.1 · p. 6

Abstract

from arXiv · show

Chain-of-thought (CoT) generation offers an opportunity to intervene before a reasoning model completes its answer. But does the point at which risk is easiest to detect also offer the most effective control? We study this question with REACT, a conditional activation intervention that learns a safety direction from paired safe and unsafe trajectories and applies it once to an early CoT hidden state. The model weights remain frozen, and all subsequent tokens are generated without further edits. To study timing, we train directions separately at each of the first five CoT positions and evaluate ungated edits with full post-edit regeneration. Across three distilled reasoning models and three generation seeds, the strongest gains concentrate in a two- to three-token early window; on DeepSeek-R1-Distill-Llama-8B, the effect falls to approximately zero at the fourth and fifth tokens, even though later states can be more predictive of risk. This separates risk readability from effective control: intervention timing must be evaluated through its effect on subsequent generation. On the same 8B model, REACT raises the rate of fully safe CoT-answer outputs on StrongREJECT-273 from 29.06% to 82.05%, while keeping benign refusal and usefulness near baseline. An anonymized implementation is available at https://anonymous.4open.science/r/REACT_official2-333B.

1 INTRODUCTION

REACT tests whether early CoT states that reveal risk also provide effective one-shot control. Its strongest steering gains occur in an early window, while later states can be more predictive yet less responsive.

1 INTRODUCTION

  • The study asks whether the most readable risk state is also the most responsive control point, and evaluates control through changes to regenerated continuations.Risk readability is measured by probe prediction, while control is measured by the effect of an activation edit on generated continuations.Source: §1 · p. 1
  • REACT learns a safety direction from paired safe and unsafe trajectories, then a separate gate edits an unmodified early-CoT state once or leaves it unchanged.The main configuration edits t2; weights remain frozen and subsequent tokens are generated without further direct edits.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Across three models, two harmful sets, and three seeds, steering gains peak at t2 and concentrate in an early two- to three-token window.On 8B, gains fall to approximately zero at t3 and t4.Source: §1 · p. 2
  • On 8B, StrongREJECT risk AUROC rises from 0.648 at the prompt to 0.714 at t2 and 0.750 at t4, despite weaker later control.A norm-matched local diagnostic finds t2 increases the next safe target’s likelihood and decreases the unsafe target’s, whereas t3 suppresses both; this does not establish a complete mechanism for free-running gains.Source: §1 · p. 2
  • 29.06% to 82.05%: REACT raises fully safe CoT–answer outputs on StrongREJECT-273 while keeping benign refusal and usefulness near baseline.On the SORRY-Bench-derived pool, safety improves by 27.48 percentage points; removing the gate retains harmful-set gains but roughly doubles benign refusal.Source: §1 · p. 2
  • The contributions combine conditional one-shot intervention, evidence that readability and control diverge, and full-trajectory safety gains with limited benign cost.The position study spans three distilled models, and the main 8B comparison includes matched alternatives.Source: §1 · p. 2

2 RELATED WORK

Prior safeguards operate at input, output, text-revision, or representation-steering stages. REACT focuses on which native reasoning states can redirect a future trajectory through one local edit, distinguishing risk prediction from effective control.

2 RELATED WORK

  • Input/output guards, prompt defences, response-side self-examination, activation steering, and token-probability controls represent distinct existing safeguard approaches.The 8B evaluation includes matched-data CAST and AlphaSteer adaptations and a prompt-boundary gated intervention.Source: §2 · p. 3
  • Reasoning-model safety includes training-time safety reasoning and monitoring CoT traces, though written CoT is not guaranteed to expose underlying computation faithfully.The paper therefore limits its claims to measured hidden states and generated trajectories.Source: §2 · p. 3
  • Other during-trace approaches revise reasoning text or insert corrective prefixes, while representation-level controls often operate at prefill.SafeSwitch probes early generated tokens and conditionally hands control to a refusal head.Source: §2 · p. 3
  • REACT combines a native early-CoT observation and edit point, conditional no-op routing, and full post-edit regeneration to study causal controllability.Its matched comparisons show prompt-boundary steering is strong; the paper does not claim generated-token intervention universally dominates prefill methods.Source: §2 · p. 3

3 METHOD

REACT uses four frozen stages: build target-native supervision, learn a direction and strength, train a separate routing gate, then conditionally edit one early-CoT state. The main configuration edits t2; ordinary decoding then resumes.

3.1 OFFLINE PAIRED TRAJECTORIES AND TARGET-MODEL RE-ENCODING

  • Paired safe and unsafe trajectories sharing prompts are re-encoded with the frozen target model to obtain native CoT states for training.The evaluation pools are excluded from strength calibration, but the diagnostic/calibration pool overlaps the mean-difference initialisation and is not fully held out.Source: §3.1 · p. 4

3.2 GLOBAL SAFETY DIRECTION

  • A global direction starts from the safe–unsafe mean difference and is optimized to lower safe-continuation NLL while raising unsafe-continuation NLL.The direction loss covers CoT tokens after the intervention position up to the closing tag, excluding the final answer; a width-normalized penalty bounds magnitude.Source: §3.2 · p. 4

3.3 MODEL-SPECIFIC STRENGTH CALIBRATION

  • Strength calibration selects model-specific α for 1.5B and 7B on a teacher-forced pool; the 8B reference retains pre-fixed α = 1.The position sweep reuses each model’s α at every site, so sites are not compared at matched perturbation norms.Source: §3.3 · pp. 4–5

3.4 INDEPENDENT LINEAR ROUTING GATE

  • A separately trained linear gate reads the pre-edit state and routes requests to apply or no-op, with its threshold frozen before test generation.The threshold constrains no-op false triggers to at most 5% on development data, then prioritizes direct recall and balanced accuracy.Source: §3.4 · p. 5

3.5 CONDITIONAL SINGLE-POINT INTERVENTION

  • At inference, the model generates through t2, conditionally adds the direction once, and then generates the rest of the CoT and answer normally.The no-op branch follows the unedited baseline path; no base-model parameter is updated.Source: §3.5 · p. 5

3.6 GENERATION AND EVALUATION IMPLEMENTATIONS

  • The three-seed six-arm 8B study decodes every arm online; other evaluations compose shared-prefix baseline and steering branches, regenerating all post-edit tokens.Position-window claims use independently generated baseline and steering branches, not gated endpoints.Source: §3.6 · p. 5

4 EXPERIMENTAL SETUP

The evaluation covers three frozen distilled reasoning models, harmful-request sets, and a benign instruction set. The main 8B comparison uses six arms, three seeds, and separate safety, refusal, and usefulness measures.

4 EXPERIMENTAL SETUP

  • Evaluation uses Qwen 1.5B/7B and Llama 8B, StrongREJECT-273, a 450-prompt SORRY-Bench-derived pool, and 500 benign Alpaca instructions.StrongREJECT is source-independent of direction-training data, whereas the SORRY-derived pool shares its source ecology.Source: §4 · p. 6
  • The 8B study decodes six arms on the same 1,223 prompts over three seeds, while position sweeps cover t0–t4 on three models and two harmful sets.Matched-data baselines use the same target model and supervision resources, as controlled adaptations rather than reproductions of published headline settings.Source: §4 · p. 6
  • LlamaGuard judges CoT and answer separately; an output is Safe only when structurally valid and both segments are judged safe.Reported measures include Safe rates, benign refusal, and rubric-scored usefulness; separate risk probes use AUROC.Source: §4 · p. 6

5 RESULTS

REACT improves fully safe outputs in the main 8B evaluation, while position sweeps locate the strongest one-shot steering effects in an early window. Later CoT states can be more predictive of risk even when steering there is ineffective.

5.1 THREE-SEED COMPARISON AGAINST MATCHED ALTERNATIVES

  • 29.06±1.29% to 82.05±4.03% fully safe CoT–answers on StrongREJECT-273: REACT exceeds P0 by 52.99 points and the boundary arm by 10.74 points.On the SORRY-Bench-derived pool, REACT reaches 74.96±0.46%, versus 47.48±1.30% for P0.Source: §5.1 · p. 6, Table 1 · p. 7
  • The direction supplies essentially all harmful-set improvement, while conditional routing avoids the benign cost of applying the edit to every request.Ungated refusal rises from 5.33% to 10.67%; REACT remains at 5.40% refusal and 91.86 usefulness, and the authors avoid claiming t2 universally beats prefill because boundary performance varies with direction initialization.Source: §5.1 · p. 6

5.2 A REPRODUCIBLE EARLY CONTROL WINDOW

  • Across all three models and both harmful sets, the strongest steering site is t2 and the half-peak control window spans two to three tokens.On 8B StrongREJECT-273, mean ΔSafe rises from +6.35±1.12 points at t0 to +49.69±4.58 at t2, then is +0.00±2.04 at t3 and −1.83±3.30 at t4.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 READABILITY AND CAUSAL CONTROLLABILITY DIVERGE

  • On 8B StrongREJECT-273, probe AUROC rises from 0.648 at the prompt to 0.750 at t4, although one-shot steering at t4 is ineffective.The risk-readout results therefore distinguish predictive readability from causal controllability.Source: §5.3 · pp. 8–9, Table 4 · p. 9

5.4 ROBUSTNESS ACROSS SCALE AND DEPTH

  • The early-control phenomenon also appears across model scales and tested depths, while position sweeps show strong sensitivity to intervention timing.The t2 edit improves StrongREJECT Safe from 9.89±1.60% to 27.72±1.39% on 1.5B and from 20.15±0.63% to 58.85±3.88% on 7B; effective 8B t2 edits were found at layers 9, 17, 20, 25, and 32.Source: §5.4 · p. 9

6 CONCLUSION

One-shot safety steering is most effective in an early CoT window, while risk readability can continue to improve later. On 8B, the t2 REACT intervention improves complete-trajectory safety while keeping benign refusal and usefulness near baseline.

6 CONCLUSION

  • Across three models, the early steering window replicated, but its boundaries shifted by up to one token; on 8B, t1/t2 were effective while t0 and t3/t4 were weak.The 8B t3/t4 effects returned to approximately the site-matched baseline on both harmful sets.Source: §6 · p. 9
  • On 8B, the fifth-token state predicted StrongREJECT risk better than the third-token state despite offering little safety gain under the evaluated edit.At t2, a norm-matched local diagnostic increased the next safe target’s likelihood and decreased the unsafe target’s; t3 suppressed both. These findings support choosing sites by effects on generation, without claiming universal superiority over prefill.Source: §6 · p. 9
  • Roughly 53 and 27 percentage-point gains in complete-trajectory safety on StrongREJECT and the SORRY-derived pool, respectively, came with benign refusal and usefulness near P0.The three-seed, six-arm 8B evaluation also outperformed the evaluated matched alternatives.Source: §6 · p. 9

ETHICS STATEMENT

The work evaluates a defensive intervention on existing harmful-request benchmarks and reports aggregate scores rather than reproducing harmful generations.

ETHICS STATEMENT

  • The evaluation uses existing harmful-request benchmarks and aggregate scores, and no human-subject study was conducted.The authors note that the method could be reversed or miscalibrated to weaken safeguards, and that its gate may shift under domain or population change.Source: ETHICS STATEMENT · p. 10

AI USE STATEMENT

The paper reports that generative AI tools assisted with language editing, organization, formatting, and presentation, but not experimental measurements. Its experimental setup evaluates frozen distilled reasoning models across benchmark, benign, timing, and diagnostic studies.

AI USE STATEMENT

  • Generative AI tools assisted with language editing, organization, formatting, and presentation of author-provided results, but not with generating experimental measurements.The authors reviewed the resulting text, tables, and numerical claims and take responsibility for the submission.Source: AI USE STATEMENT · p. 10

A EXPERIMENTAL SETUP

  • Three frozen DeepSeek-R1 distilled models were evaluated: Qwen-1.5B and Qwen-7B at layer 17, and Llama-8B at layer 20.The main configuration intervenes at t2; position sweeps retrain site-specific directions and gates at t0–t4.Source: A EXPERIMENTAL SETUP · p. 13
  • Evaluation uses source-independent StrongREJECT-273, a SORRY-Bench-derived within-ecology pool, and Benign-500 for refusal and usefulness auditing.Direction learning uses 170 exact-prompt safe/unsafe pairs.Source: A EXPERIMENTAL SETUP · p. 13
  • The main 8B comparison tests REACT against P0, matched-data CAST and AlphaSteer adaptations, a prompt-boundary gated edit, and steering without the gate, using three seeds.For 1.5B and 7B, the three-arm comparison uses one seed.Source: A EXPERIMENTAL SETUP · p. 13
  • Position sweeps regenerate every token after a site-specific edit, while separate 8B diagnostics measure risk readout and teacher-forced next-token likelihood changes.The likelihood analysis is diagnostic rather than a free-running causal explanation.Source: A EXPERIMENTAL SETUP · pp. 13–14

B LIMITATIONS

The paper’s evidence is bounded by model, data, method, and evaluation choices. It also reports benign-trigger and output-validity results in the expanded 8B evaluation.

B LIMITATIONS

  • The early control window replicates across three models, but its boundaries shift by up to one token, and model size, layer, and edit strength co-vary across runs.The exact 8B t1/t2 boundaries are therefore not established as universal; within-model sweeps hold edit strength fixed rather than matching norms across sites.Source: B LIMITATIONS · p. 14
  • The local likelihood diagnostic is teacher-forced, limited to 8B immediate next-token changes, and does not identify the mechanism behind the free-running t2 gain.Fixed-continuation downstream KL does not remain uniquely large for t2.Source: B LIMITATIONS · p. 14
  • The prompt-boundary comparison supports an effective early-CoT control point under the evaluated recipe, not universal superiority over every prefill objective.A separate single-seed diagnostic found that changing boundary direction initialization substantially changes endpoint safety.Source: B LIMITATIONS · p. 14
  • The deployed gate is not established as a fine-grained trajectory-risk detector; its demonstrated role is primarily avoiding unnecessary benign edits.Online risk probes and the gate address different tasks.Source: B LIMITATIONS · p. 14
  • CAST and AlphaSteer are controlled adaptations fitted to the target model and frozen data ecology, not reproductions of every published setting or a global method ranking.The comparison supports performance under the paper’s experimental contract.Source: B LIMITATIONS · p. 14
  • StrongREJECT-273 is source-independent, but the SORRY-derived pool shares training-source ecology; exact-text separation does not establish semantic independence, and no full near-duplicate audit is available.The direction and gate training resources also share 88 prompts.Source: B LIMITATIONS · p. 14
  • Benign-500 measures ordinary-instruction refusal and usefulness, not hard-benign, math, coding, knowledge, or complex-reasoning capability.The evaluated models are DeepSeek-R1 distilled models, and safety relies on automatic evaluators; conclusions are limited to these models, datasets, objectives, and generation contracts.Source: B LIMITATIONS · p. 15

C EXPANDED 8B ONLINE EVALUATION

  • REACT’s benign trigger rate was 4.67 ± 0.42%, versus 1.20% for CAST, 0.80% for the prompt-boundary gate, and 100% for steering-only; invalid rates were 0.00% on StrongREJECT and 0.22% on the SORRY-derived pool.On StrongREJECT, REACT and steering-only outputs were token-identical and shared a hash-aligned score.Source: C EXPANDED 8B ONLINE EVALUATION · p. 15, Table 5 · p. 15

D POSITION SWEEP AND STEERING DIAGNOSTICS

The position sweep compares site-specific edits with independently regenerated post-edit continuations, alongside gate qualification and local diagnostics. The diagnostics distinguish immediate likelihood shifts from downstream perturbation size.

D POSITION SWEEP AND STEERING DIAGNOSTICS

  • Each site-specific edit is evaluated against an independently generated P0 continuation, with every later token regenerated after the edit.The position study uses three generation seeds; its gated rows compose the P0 and steering branches rather than adding another long-form generation.Source: D POSITION SWEEP AND STEERING… · p. 15
  • Site-specific gates qualify at t0–t3 on every model, but not at t4 under the prespecified development rule.The 8B t0 gate meets the 60% harmful-recall threshold exactly.Source: D POSITION SWEEP AND STEERING… · pp. 15–16, Table 6 · p. 16
  • At t2, the learned-direction mean KL is approximately 196× the random-direction mean, with both direction types applied at nominal norm 1.Actual BF16 additive norms stayed within 0.25% of nominal, with no failures under the prespecified 10% tolerance.Source: D POSITION SWEEP AND STEERING… · p. 16, Table 15 · p. 19
  • Across nominal norms 0.5, 1, and 2, t2 preserves the safe-negative and unsafe-positive ∆NLL pattern, while t3 raises both classes’ NLL.This argues against a simple account in which the t3 failure results from an edit that is too weak.Source: D POSITION SWEEP AND STEERING… · p. 17, Table 16 · p. 19
  • The large immediate t2 KL contracts rapidly and is not consistently larger than other sites from q = 2 onward.The authors therefore frame the immediate NLL pattern as a mechanism correlate, not a complete propagation mechanism.Source: D POSITION SWEEP AND STEERING… · p. 17, Table 17 · p. 20
  • With the t2 edit fixed, prompt-state and t2-state gates yield nearly identical harmful endpoints, while the prompt gate triggers less often on benign inputs.This supports a routing role in protecting benign traffic, not a uniquely powerful t2 trajectory-risk detector.Source: D POSITION SWEEP AND STEERING… · p. 17, Table 18 · p. 20
  • In a separate common-P0 unit, direct t2 REACT reaches 76.92%/75.33% Safe on StrongREJECT/SORRY, versus 74.36%/69.11% for prompt-boundary gated steering.A t4 risk probe followed by rollback to t2 steering reaches 73.63%/73.11%; the paired StrongREJECT interval for t2 minus boundary is [−4.03, +8.79] points.Source: D POSITION SWEEP AND STEERING… · pp. 18–19

E ADDITIONAL TEACHER-FORCED PROBE PROTOCOL AND FULL RESULTS

The supplementary probe analysis measures teacher-forced risk readability across prompt and CoT states, then tests controls for source and surface cues. A separate cross-model decomposition reports Safe, Unsafe, and Invalid outcomes under gated steering.

E ADDITIONAL TEACHER-FORCED PROBE PROTOCOL AND FULL RESULTS

  • The probe corpus pools nine sources, uses a held-out ReasoningShield test split, and evaluates fixed-position hidden states with linear probes across layer grids.It is source-level disjoint from evaluation datasets, but no string-overlap audit was run; the analysis is teacher-forced and correlational.Source: E ADDITIONAL TEACHER-FORCED PROBE… · p. 20
  • At t2, held-out AUROC exceeds strict prompt-only by +0.082/+0.085 for 1.5B/8B, but exceeds think_last by only +0.034/+0.027.Readout continues increasing at t3 and t4, so the intervention site is selected by the causal ablation rather than the readout ranking.Source: E ADDITIONAL TEACHER-FORCED PROBE… · pp. 20–21, Table 22 · p. 21
  • Two mixed-source leave-one-source-out families are nearly perfectly separable at cot_0 or cot_1, which the source analysis attributes to artifacts, inflating the six-family macro means.On the three ReasoningShield families, 8B trajectory-minus-prompt gaps are only +0.001, +0.013, and +0.066.Source: E ADDITIONAL TEACHER-FORCED PROBE… · p. 21, Table 23 · p. 22
  • In the 8B same-prompt corpus, early-CoT hidden probes reach 0.6983 AUROC, while length alone reaches 0.8261 and word-level TF-IDF reaches 0.9660.Across the reported variants, surface baselines are strong, and hidden-state signal is not shown to be cleaner than surface features.Source: E ADDITIONAL TEACHER-FORCED PROBE… · pp. 21–22
  • The probe analysis supports above-chance linearly readable signal beyond the prompt at t2, but does not establish online rollout prediction, edit effects, or safety semantics.It also does not establish that t2 is the best site among early tokens.Source: E ADDITIONAL TEACHER-FORCED PROBE… · p. 22

F CROSS-MODEL RESULTS: RUBRIC EVALUATOR AND LLAMAGUARD THREE-ARM DECOMPOSITION

  • Across the three models, gated steering raises Safe share on both harmful sets without mainly shifting Unsafe outputs into Invalid, while returning Benign-500 near P0.Relative to steering-only, gating gives up a little harmful-set gain; the 8B decomposition is a separate single-seed exact-pair unit.Source: F CROSS-MODEL RESULTS: RUBRIC… · p. 22, Table 24 · p. 22

G CROSS-LAYER ROBUSTNESS AT t2

The single-seed layer sweep checks t2 robustness across layers using site-specific directions and gates. Its rubric results report similar harmful-set outcomes and near-baseline benign refusal and usefulness across the tested t2 layers.

G CROSS-LAYER ROBUSTNESS AT t2

  • The layer sweep is a separate single-seed robustness check with site-specific directions and gates at fixed α = 1.Only cross-layer t2 comparisons are retained; temporal position claims rely on the post-edit-regeneration sweep instead.Source: G CROSS-LAYER ROBUSTNESS AT t2 · p. 22

H DATA CONSTRUCTION AND SPLIT HYGIENE

The data and audit sections define evaluation-set provenance, calibration and generation procedures, and scoring coverage. They also bound the results: prompt-level exact-text checks do not establish semantic disjointness, and deployment traffic was not evaluated.

H DATA CONSTRUCTION AND SPLIT HYGIENE

  • StrongREJECT-273 is the source-independent harmful test set, while the 450-prompt SORRY-derived pool shares the direction-training source ecology.Benign-500 is CAST’s 500-item held-out harmless-instruction test array, with some Alpaca input fields absent from the stored prompts; all arms receive identical prompts.Source: H DATA CONSTRUCTION AND SPLIT HYGIENE · pp. 23–24
  • The normalized exact-text audit finds zero training-resource overlap with endpoint sets, but does not establish semantic or near-duplicate disjointness.Direction and gate training resources share 88 prompts, all in gate training; StrongREJECT-273 and the SORRY-derived pool share two normalized exact prompts.Source: H DATA CONSTRUCTION AND SPLIT HYGIENE · p. 24, Table 28 · p. 24

I STRENGTH-CALIBRATION PROTOCOL

  • Calibration selects the largest feasible safety-margin candidate for 1.5B and 7B, yielding α = 8 and α = 12 at layer 17; 8B instead uses a pre-fixed layer-20, α = 1 setting.Calibration does not use endpoint outputs or labels.Source: I STRENGTH-CALIBRATION PROTOCOL · p. 24

J GENERATION CONTRACTS

  • Endpoint runs use temperature 0.6, top-p 0.95, and a 4,096-token continuation budget; after t2, REACT generates the remaining tokens without further edits.Outputs lacking a closed CoT followed by a final answer are counted Invalid rather than repaired or resampled.Source: J GENERATION CONTRACTS · pp. 24–25

K JUDGE CONFIGURATION

  • LlamaGuard judges CoT and final answer separately, and an output counts Safe only when structurally valid and both segments receive a safe verdict.Independent rubric evaluators score harmful and benign sets; the three-seed six-arm 8B study uses gpt-4o-mini-2024-07-18 API scores.Source: K JUDGE CONFIGURATION · p. 25

M AUDIT COUNTS

  • The 8B main comparison covers 22,014 online prompt–seed–arm units, with complete benign rubric coverage for 9,000 cells across six arms.The position sweeps comprise 65,070 independently generated P0/steering outputs across the three models; gated endpoints reuse them by branch composition.Source: M AUDIT COUNTS · p. 26

N BROADER IMPACT

  • Real deployment traffic and hard-benign populations were not evaluated, so the aggregate benign result is not a deployment guarantee.The gate is trained on English direct-harmful versus paired no-op prompts; domain, language, dialect, or user-population shift can change its trigger distribution.Source: N BROADER IMPACT · p. 26

Related papers

Loading 2610.32113v1…