PaperlayerView exact revision on arXiv

Source-linked AI summary

PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety

Ding Jia, Wei Liu, Xianglong Du, Yingjie Li, Yingqing Yang, Huili Yu, Zhangsong Zhan, Chu Zhou

arXiv:2609.34415v2cs.LG

TL;DR

Agentic tool use raises the stakes from unsafe text to potentially irreversible environmental harm, while runtime guards need supervision for where to stop without blocking earlier safe prefixes. PROACT-Agent builds and adjudicates bilingual prefix trajectories through unrolling, causal rectification, and localization. Under complete source holdout it reports 91.46% unsafe-class F1 and 90.63% exact-boundary detection, and reduces AgentDojo non-DoS targeted attack success from 20.82% to 0.40%.

  • Problem

    Agent actions can cause irreversible harm, and runtime supervision must identify the first prefix where continuation should stop while preserving earlier safe prefixes.

    Source: §1 · pp. 1–2

  • Method

    PROACT-Agent unrolls long-horizon logs into prefixes, augments rationales and rectifies labels, and localizes tool contexts across English and Chinese to build PROACT-Bench.

    Source: §1 · p. 2

  • Results

    Under complete source holdout, PROACT-Agent achieves 91.46% unsafe-class F1 and 90.63% exact-boundary detection; AgentDojo non-DoS targeted attack success falls from 20.82% to 0.40%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported results support evaluating updated interaction context before the next inference as a proactive runtime intervention rather than post-hoc diagnosis.

    Source: §1 · p. 2

    Because the guard runs after a context update, it cannot undo a tool call that has already produced an observation.

    Source: §6 · p. 9

Abstract

from arXiv · show

The transition from Large Language Models (LLMs) to agents shifts safety stakes from toxic text to irreversible environmental harm. While current defenses remain largely retrospective, proactive runtime intervention is bottlenecked by the lack of large-scale, causally-consistent data. We propose PROACT-Agent, a framework for synthesizing high-fidelity trajectories to enable real-time guardrails. We identify a critical "safety drift" in prior benchmarks, where lenient annotation paradigms fail to enforce temporal consistency. PROACT-Agent addresses this through: (1) Progressive Trajectory Unrolling to reveal risks hidden in long-context interactions; (2) Reasoning-Augmented Causal Rectification to enforce monotonic causal consistency; and (3) Culturally-Aware Data Localization for cross-border robustness. We introduce PROACT-Bench, a bilingual safety benchmark with 155,780 states labeled through multi-model adjudication. Evaluating updated context before the next LLM inference, the trained guard achieves 91.46% unsafe-class F1 and 90.63% exact-boundary detection under complete source holdout. In AgentDojo, it reduces non-DoS targeted attack success from 20.82% to 0.40%.

1 Introduction

Agentic actions can cause irreversible harm, yet existing guards and retrospective audits do not provide the first-unsafe-prefix supervision needed for timely intervention. PROACT-Agent constructs and adjudicates bilingual prefix data for runtime guards, reporting strong results under complete source holdout and in AgentDojo.

1 Introduction

  • A misaligned tool call can cause irreversible harm, while content guards and retrospective audits do not identify when to stop while preserving preceding safe prefixes.Source: §1 · pp. 1–2
  • Prior benchmarks trade off scale, reasoning transparency, and causal reliability; contradictory labels can produce safety drift, and English-centric coverage limits cross-border evaluation.Source: §1 · p. 2
  • PROACT-Agent unrolls trajectories into prefixes, rectifies missing rationales and labels, and localizes tool contexts across English and Chinese; PROACT-Bench contains 155,780 prefix-level states.Source: §1 · p. 2, Figure 2 · p. 3
  • Under complete source holdout, PROACT-Agent achieves 91.46% unsafe-class F1 and 90.63% exact-boundary detection; in AgentDojo, non-DoS targeted attack success falls from 20.82% to 0.40%.Source: §1 · p. 2

2 Related Work

Prior agent guards and safety benchmarks address trajectory context and multi-step interactions, but leave gaps in proactive intervention and causal consistency. PROACT evaluates observed prefixes before the next inference and trains on localized, causally corrected supervision.

2 Related Work

  • AgentDoG [20] diagnoses finished traces retrospectively, whereas TS-Guard [26] evaluates steps in real time; PROACT evaluates observed prefixes before the next LLM inference.Source: §2 · p. 3
  • Existing benchmarks trade off scale, reasoning, and causal consistency: AgentAlign [43] omits CoT, ToolSafety lacks internal rationales, and TS-Bench [26] is scale-constrained.Source: §2 · p. 3
  • Agent-SafetyBench [44] and SafeArena [30] emphasize execution-phase or adversarial results without rectifying safety drift; predominantly English-language benchmarks motivate bilingual prefix supervision.Source: §2 · p. 3

3 The PROACT-Agent Framework

PROACT-Agent converts completed agent logs into prefix-level supervision for runtime intervention, then rectifies labels and localizes states for bilingual evaluation. Its framework targets accumulated interaction context and the first unsafe boundary.

3 The PROACT-Agent Framework

  • The framework harmonizes heterogeneous agent trajectories into a causally consistent streaming format across three stages.Those stages are PTU, RACR, and CADL.Source: §3 · p. 3

3.1 Progressive Trajectory Unrolling

  • PTU reformulates agent safety as online, non-Markovian prefix evaluation, making the intervention boundary an explicit learning target from completed logs.Each prefix represents the context available before the next LLM invocation; risk is not determined by the latest observation alone.Source: §3.1 · p. 4

3.2 Reasoning-Augmented Causal Rectification

  • RACR augments recorded actions with inferred auxiliary rationales, uses committee consensus with arbitration for disagreements, and enforces monotonic consistency in rectified labels.The inferred rationale is an auxiliary explanation, not the agent’s actual hidden chain-of-thought; MCC excludes post-boundary states while retaining the safe-to-unsafe transition.Source: §3.2 · pp. 4–5

3.3 Culturally-Aware Data Localization

  • CADL maps states to culturally grounded local tool contexts and applies structural, label-consistency, and contextual-fidelity checks to the localized data.Its functional mapping can replace an English-context platform with a functionally equivalent Chinese-context platform.Source: §3.3 · p. 5

4 The PROACT-Bench Dataset

PROACT-Bench provides bilingual, prefix-level supervision for safety classification and first-unsafe boundary detection. It combines interaction histories and auxiliary rationales with committee-adjudicated labels, while its DQS is an internal check rather than independent human validation.

Data Harmonization and Construction

  • The benchmark is harmonized from raw agent logs sourced from established benchmarks and produces English and localized Chinese evaluation tuples.Each tuple pairs an interaction-prefix state with a model-adjudicated intervention label.Source: §4 · pp. 5–6

Scale and Bilingual Robustness

  • 155,780 prefix-level states cover English and localized Chinese tool environments while retaining histories for long-horizon, non-Markovian risks.The benchmark supports safety classification and first-unsafe boundary detection using intervention labels determined through committee evaluation, disagreement arbitration, and MCC.Source: §4 · pp. 5–6

Quality Verification

  • DQS summarizes evaluator-committee agreement and final-label disagreement across Validity, Difficulty, Noise, and Ambiguity, but is not independent human validation.The difficulty contribution is capped at 20% of the dataset share.Source: §4 · p. 6

Taxonomic Coverage and Diversity

  • The benchmark maps six safety risk categories to operational scenarios, including software, web, social media, e-commerce, finance, and productivity settings.The paper states that this coverage of real-world agentic failures is not exhaustive.Source: §4 · p. 6

5 Experiments

PROACT-Agent is evaluated in mixed-source and strict generalization protocols, temporal boundary detection, and a closed-loop AgentDojo setting. Across these evaluations, it leads the reported baselines and reduces targeted attack success in non-DoS cases.

5 Experiments

  • PROACT-Agent is supervised on bilingual interaction states and reports Accuracy, Precision, Recall, and unsafe-class F1, counting invalid outputs as errors.The guard uses Qwen2.5-7B-Instruct as its backbone.Source: §5 · p. 6

5.1 Broad Mixed-Source Comparison

  • 97.01% Overall F1 and 97.11% Overall Accuracy are PROACT-Agent’s highest results in the mixed-source comparison.It also leads F1 on English (97.31%) and Chinese (96.71%).Source: Table 1 · p. 7, §5.1 · p. 7

5.2 Strict Generalization

  • 91.46% F1 under complete source holdout exceeds Qwen3Guard’s 71.38% and TS-Guard’s 72.29%.The source-held-out protocol trains on AgentAlign and ToolSafety and evaluates on R-Judge and ATBench; PROACT also leads both baselines on all four reported metrics within each protocol.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Temporal Boundary Detection

  • 90.63% exact-boundary detection under complete source holdout identifies the adjudicated first-unsafe prefix while leaving earlier safe prefixes unblocked.The measure is computed on root-language trajectories with an unsafe boundary and requires both detecting the boundary and preserving earlier safe prefixes.Source: Table 3 · p. 8, §5.3 · p. 8

5.4 AgentDojo Closed-Loop Evaluation

  • 98.07% overall relative reduction in targeted ASR across four evaluated suites in AgentDojo’s non-DoS attack evaluation.The evaluation covers 10,439 cases; its ASR, timing, and clean-trajectory analyses use different case populations.Source: Table 4 · p. 8, §5.4 · p. 8

6 Conclusion

The conclusion reports source-held-out boundary detection and AgentDojo attack-prevention results for PROACT-Agent, while noting deployment and validation limitations. The appendices describe how trajectories are parsed, formatted, adjudicated, and rectified into prefix-level supervision.

6 Conclusion

  • 91.46% unsafe-class F1 and 90.63% exact-boundary detection under source holdout accompany a reduction in AgentDojo non-DoS targeted attack success from 20.82% to 0.40%.The guard evaluates updated context before the next LLM inference; the conclusion identifies deployment-specific risks and human boundary validation as further-study directions.Source: §6 · p. 9
  • PROACT-Agent cannot undo a tool call that has already produced an observation, and its first-unsafe boundaries lack systematic multi-annotator human validation.The paper also notes dependence on predefined localization knowledge bases and the reasoning capacity of the base LLMs.Source: §6 · p. 9

A.1 Details of Progressive Trajectory Unrolling

  • Offline PTU parsing identifies the earliest candidate block point, then builds provisional prefixes through that cutoff and excludes later interactions.A candidate cutoff is not a final unsafe verdict; adjudication may retain the entire stream when no unsafe prefix is found.Source: §A.1 · p. 11
  • Each formatted input contains the task goal, past trajectory, and current step; the guard decides whether the agent may use the returned context in its next inference.Training targets pair a rectified safety label with a reason, using a simulated refusal at the adjudicated boundary.Source: §A.1 · p. 11

A.2 Details of Reasoning-Augmented Causal Rectification

  • RACR generates an auxiliary rationale linking user intent to the executed tool schema; that rationale supplies context, not the final safety label.The same Qwen3-32B generator is used across sources, while committee adjudication and MCC determine the final boundary.Source: §A.2 · pp. 11–12
  • Independent Safety Auditor models evaluate states, and GLM-5.1 arbitrates uncertain disagreements; the resulting adjudicated labels are not human-verified.The committee consists of DeepSeek-R1-671B, MiniMax-M2.5, Qwen2.5-72B, and Qwen3-32B.Source: §A.2 · p. 12
  • MCC finds the first unsafe boundary, retains states through it, and excludes later interactions; an all-safe stream retains every prefix.Rectified labels use the cumulative maximum of arbitrated labels.Source: §A.2 · p. 12

A.3 Details of Culturally-Aware Data Localization

Culturally-Aware Data Localization creates Chinese counterparts that preserve adjudicated labels while adapting tool contexts. The broader synthesis pipeline turns heterogeneous agent logs into prefix-level, rationalized, causally rectified bilingual data.

A.3 Details of Culturally-Aware Data Localization

  • Localization maps Western platforms and entities to Chinese functional equivalents, while preserving tool-call schemas and the malicious intent’s risk essence rather than translating literally.Generation uses a Chinese-authored instruction and a constrained workflow with an explicit reflection phase.Source: §A.3 · pp. 12–13
  • Tripartite verification checks schema and label invariance, screens for omissions, and audits contextual fidelity against a threshold.The contextual-fidelity auditor compares English and Chinese states; label matching alone is not treated as semantic or behavioral equivalence.Source: §A.3 · p. 13

B.1 Source Corpora and Unified Harmonization

  • The four source datasets have distinct gaps: R-Judge lacks step labels, ToolSafety lacks causal labels and rationales, and ATBench and AgentAlign have incomplete or absent rationales.The source corpora also differ in metadata and trace characteristics, motivating harmonization.Source: §B.1 · p. 14
  • Synthesis unrolls long trajectories into prefix states, adds inferred rationales where missing, adjudicates labels, rectifies them into a monotonic sequence, and localizes the result under verification constraints.The final tuples serialize context, available or inferred rationales, system instructions, boundary labels, and unsafe-state refusal reasons.Source: §B.1 · p. 14

B.2 Comprehensive Dataset Statistics

PROACT-Bench contains 155,780 prefix-level states and is evaluated with both canonical-root and source-held-out protocols. The section also defines a model-based Data Quality Score from committee agreement and disagreement.

B.2 Comprehensive Dataset Statistics

  • The 155,780-state corpus uses a canonical-root 80/20 split across four sources and a source-held-out split evaluating on two entirely held-out sources.The held-out evaluation contains 1,698 English and 1,698 Chinese states.Source: §B.2 · p. 14, Table 5 · p. 15
  • The source-held-out partition audit reports zero overlap across four cross-partition identifiers and zero duplicate, missing, label-mismatched, or step-mismatched bilingual tuples.The training partition has 76,192 English and 76,192 Chinese states; evaluation has 1,698 of each.Source: §B.2 · p. 15

B.3 Formulation of the Data Quality Score (DQS)

  • DQS summarizes committee agreement through validity, difficulty, noise, and ambiguity, based on how many of four evaluator models support each final label.Validity counts support from at least three models; difficulty denotes three-model support with one dissent; noise counts support from at most one; ambiguity is an even split.Source: §B.3 · p. 15
  • DQS weights validity, difficulty, noise, and ambiguity by 90, 150, −100, and −80, respectively, and caps the difficulty contribution at a 20% density ratio.It is reported as a raw comparative score, not a normalized probability.Source: §B.3 · p. 15

C Details of Training the PROACT-Agent

PROACT-Agent is trained as a next-token prediction task over serialized safety labels and refusal reasons. The section documents its fine-tuning setup and the protocols used to compare it with heterogeneous baselines.

C.1 Training Objective

  • The evaluator predicts a serialized JSON target containing the adjudicated binary label and, for unsafe states, an explanatory refusal reason; quantitative metrics use only the parsed label.Training uses cross-entropy, with the state carrying historical context and available or inferred auxiliary rationale.Source: §C.1 · p. 15

C.2 Hyperparameters and Hardware

  • The submitted 7B/8B runs use LoRA supervised fine-tuning for three epochs, with a 4096-token sequence cap and a global batch size of 32.Training uses AdamW, BF16 mixed precision, and a single 80GB A100; each backbone requires approximately 33 A100 GPU-hours.Source: §C.2 · p. 16

D Baseline Details

  • Baselines are normalized to binary safe/unsafe decisions, and invalid outputs count as errors; unsafe is the positive class for precision, recall, and F1.First-unsafe recall requires an unsafe prediction at the first unsafe boundary, while exact-boundary detection additionally requires every earlier prediction to be safe.Source: D Baseline Details · p. 16

E Two-Source Training-Size Sensitivity

In the mixed-source comparison, PROACT-Agent has higher F1 than the baselines in both languages, while results also examine training-data size, backbone robustness, and guard-only costs. The 60%-to-80% training-size increase produces little fixed-ID change but improves complete-OOD Macro-F1.

E Two-Source Training-Size Sensitivity

  • On complete OOD, increasing training data from 60% to 80% raises Macro-F1 by 1.23 points and lowers FAR by 3.04 points, with FBR up 0.44 points.Fixed-ID performance changes little; checkpoints are selected by Macro-F1 on the fixed ID set, not OOD performance.Source: E Two-Source Training-Size Sensitivity · p. 16, Table 6 · p. 17

F Supplementary Experiments and Evaluation Details

  • The supplementary mixed-source backbone runs select checkpoints by unsafe-class F1 on 5,984 states, with invalid outputs counted as errors.These runs are separate from the strict root-grouped and source-held-out 7B experiments.Source: F Supplementary Experiments and… · p. 17

F.1 Qwen3 Backbone-Scale Robustness

  • Qwen3 backbones from 1.7B to 8B retain high mixed-source performance, with closely matched 4B and 8B results.The comparison tests robustness across scales without treating model size as the sole experimental difference or assuming monotonic gains.Source: §F.1 · p. 17, Table 7 · p. 17

F.2 Cross-Generation Backbone Robustness

  • Across Qwen2.5, Qwen3, and Qwen3.5 implementations, PROACT-Agent maintains high mixed-source unsafe-class F1, so performance is not tied to one backbone choice in this setting.Model generation and parameter count both vary in this comparison.Source: §F.2 · p. 17, Table 8 · p. 17

F.3 Guard-Only Runtime Efficiency

  • The 1.7B and 4B guards reach 96.01% and 96.81% unsafe F1 with 4.69 and 8.78 GiB, respectively.The 1.7B guard has the lowest memory footprint and long-context slope among the three models; 7B has lower P50 latency than 4B but a higher long-context slope.Source: §F.3 · pp. 17–18, Table 9 · p. 18

F.4 Broader Baseline Comparison

  • PROACT-Agent attains higher F1 than baselines in both languages, combining unsafe detection with fewer false alarms than several high-recall guards.The comparisons characterize system-level behavior and do not isolate the training or architectural causes of baseline errors.Source: §F.4 · p. 18

F.5 Mixed-Source Component Ablation

The mixed-source ablation favors the full PROACT-Agent pipeline, while the AgentDojo evaluation measures runtime blocking, attack success, and clean-trajectory termination in distinct case populations.

F.5 Mixed-Source Component Ablation

  • The full PROACT-Agent pipeline achieves the best Accuracy and F1-score across all evaluation cohorts, with strong Precision–Recall balance.The comparison uses the Qwen2.5-7B backbone and treats Unsafe as the positive class.Source: Table 10 · p. 18, §F.5 · p. 19
  • Removing CADL lowers Chinese F1 by 2.87 percentage points, removing RACR lowers Overall F1 by 2.58 points, and removing PTU lowers Overall Recall by 2.94 points.The ablations respectively use unconstrained literal translation, unrectified observations, and completed rather than streaming sequences.Source: §F.5 · p. 19

F.6 AgentDojo Closed-Loop Evaluation Details

  • PROACT checks the updated interaction prefix after tool observations and before the target agent's next inference; completed tool effects are not rolled back.Unsafe decisions or fail-closed parsing errors terminate the episode.Source: §F.6 · p. 19
  • The non-DoS attack summary uses 10,439 malicious configured paired cases; five DoS-style attacks are excluded because fail-closed termination may itself satisfy their utility-failure criterion.The paired cases cover four suites and 16 registered attacks; the interactive manual attack is not run.Source: §F.6 · p. 19
  • 7,432 of 7,501 adjudicated unsafe-context cases are exact blocks (99.08%), while 7 of 97 clean trajectories terminate (7.22%).The exact-versus-missed denominator excludes early or late blocks; all 97 configured clean trajectories are retained, including six without a guard opportunity.Source: §F.6 · p. 19

G.1 Intended Use and Limitations of PROACT-Agent

The paper limits PROACT-Agent to an auxiliary guardrail and describes controlled access to PROACT-Bench because its malicious trajectories could be misused. It also specifies attribution and license conditions for third-party assets.

G.1 Intended Use and Limitations of PROACT-Agent

  • PROACT-Agent is intended to augment, not replace, human oversight or system-level security controls, and critical deployments should use defense in depth.The authors note that sophisticated evasion and out-of-distribution failure remain possible for safety classifiers.Source: §G.1 · p. 19

G.2 Dual-Use Nature and Data Access Protocol

  • Because PROACT-Bench's granular malicious trajectories could be repurposed for harmful-agent fine-tuning or jailbreak reverse-engineering, access is controlled rather than publicly downloadable.Access requires an organizational affiliation request and a signed agreement limiting use to safety evaluation, defensive benchmarking, and guardrail development.Source: §G.2 · pp. 19–20

G.3 Licenses and Terms of Existing Assets

  • Third-party datasets are used under their repository or dataset-card terms and are not redistributed unmodified as standalone assets.The paper also states that Qwen models, LLaMAFactory, LoRA implementations, and API-based models follow their respective licenses or provider terms.Source: §G.3 · p. 20

Related papers

Loading 2609.34415v2…