PaperlayerView exact revision on arXiv

Source-linked AI summary

Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute

Eugene Koran, Yejun Yun, Samantha Tetef, Benjamin Arnav, Pablo Bernabeu-Pérez

arXiv:2605.15377v2cs.AI

TL;DR

Reliable monitoring is needed for increasingly autonomous AI systems, but human oversight does not scale and effective monitor construction remains uncertain. The paper builds 12 shared-base GPT-4.1-Mini monitors using prompting and fine-tuning, then evaluates diverse ensembles on coding backdoors. Diverse ensembles outperform homogeneous ones, with fine-tuned monitors consistently represented among the strongest ensembles, although the evidence is limited to narrow, non-adaptive attack settings and one base model.

  • Problem

    As AI agents gain autonomy, reliable monitoring is needed to detect misaligned actions when human oversight cannot scale, but effective monitor construction remains uncertain.

    Source: §1 · p. 1

  • Method

    The paper constructs 12 GPT-4.1-Mini monitors using diverse prompting and fine-tuning strategies and evaluates their ensembles on held-out and out-of-distribution coding attacks.

    Source: §1 · pp. 1–2

  • Results

    Diverse ensembles outperform homogeneous ensembles: the best 3-monitor ensemble improves pAUC 13% over a baseline versus 5.5% for three identical monitors, a 2.4× gain.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Given a fixed trusted model, monitor diversity can provide greater safety returns than repeated sampling at the same inference budget.

    Source: §1 · p. 2

    The evaluation covers short-horizon coding problems with discrete backdoors and unaware attackers, using diversity from only one base model; adaptive robustness and broader settings remain untested.

    Source: §6.3 · pp. 7–8

Abstract

from arXiv · show

As AI systems are increasingly deployed in autonomous agentic settings at scale, it is important to ensure the actions they take are safe and aligned with user intent. Monitoring agent actions is a key safety mechanism, yet reliable monitors remain difficult to build and the scale of these systems makes human oversight impractical. We show that combining signals from diverse monitors into an ensemble improves detection of misaligned actions. We build 12 GPT-4.1-Mini monitors using both prompting and fine-tuning strategies. We evaluate them on coding tasks where candidate solutions pass standard tests but fail on adversarial inputs. In this setting, diverse ensembles outperform both individual monitors and homogeneous ensembles. Our best 3-monitor ensemble achieves 2.4x greater detection performance gain compared to an ensemble composed of three identical monitors, with the same ensemble performing strongly on an independent dataset. We contend that these results show that diversity - not scale - drives gains. The best ensembles combine strong individual performance with low correlation between monitors. Furthermore, fine-tuned monitors appear in every top-performing ensemble and maintain this advantage on out-of-distribution attack types, suggesting that fine-tuning enables detection capabilities that prompting alone does not elicit. These results support ensemble monitoring as a practical AI control strategy for safety gains at reasonable inference costs.

1. Introduction

The paper asks whether diverse monitors built from one base model improve AI-control monitoring more than repeated sampling. Across held-out and out-of-distribution coding attacks, carefully selected diverse ensembles outperform homogeneous ensembles while using less compute.

  • The study constructs 12 GPT-4.1-Mini monitors using diverse prompting strategies and fine-tuned variants, then evaluates them on held-out APPS questions and an out-of-distribution dataset.Source: §1 · pp. 1–2
  • In a representative case, one fine-tuned monitor flags an off-by-one backdoor that two prompted monitors dismiss, allowing the ensemble to catch it.Nine of twelve monitors miss the sample at their individual thresholds.Source: Figure 1 · p. 2
  • A carefully selected 3-monitor diverse ensemble matches the performance of a 12-monitor ensemble at one-quarter of the compute cost.This supports diversity rather than ensemble size as the main driver of gains.Source: §1 · p. 2
  • 13% versus 5.5% pAUC gain over a baseline monitor gives the best 3-monitor ensemble a 2.4× advantage over three identical monitors.The pAUC is measured at false positive rate ≤10%.Source: §1 · p. 2
  • Fine-tuned monitors appear in every top-performing ensemble across APPS and out-of-distribution attack types, suggesting they provide signals prompting alone does not elicit.Source: §1 · p. 2

2. Related Work

Prior AI-control work combines monitoring with intervention mechanisms, while ensemble research links gains to complementary, weakly correlated signals. This paper situates its action-only monitoring approach within those lines of work.

  • AI-control protocols address deliberate subversion through trusted monitoring, resampling flagged actions, and escalation to human review.Source: §2 · p. 2
  • Prior work combines monitors under cost constraints and merges action with chain-of-thought signals, but chain-of-thought monitoring can be unavailable or unreliable.The paper therefore grounds its analysis in action-only monitoring.Source: §2 · pp. 2–3
  • Classical ensemble theory relates lower error to member diversity, individual accuracy, and lower inter-member correlation across high-stakes applications.Source: §2 · p. 3

3. Method

The method builds prompt-based and fine-tuned monitors from a shared GPT-4.1-Mini base, aggregates their scores, selects ensembles, and measures diversity through score correlation on attack samples.

3.1. Monitor types

  • The study uses prompt-based and fine-tuned monitors sharing one base model and producing suspiciousness scores on a fixed integer scale.Source: §3.1 · p. 3

Prompting

  • The baseline follows ControlArena’s system prompt, while seven additional prompts target different analysis strategies, recall–precision priorities, staged reasoning, and trusted debate.Source: Prompting · p. 3

Fine-tuning

  • Fine-tuning adapts GPT-4.1-Mini on 50–200 labeled code-and-score samples, with filtering, given-label, completion, and deterministic-score dataset variants.Source: Fine-tuning · p. 3

Ensemble selection

  • Ensemble scores are averaged, thresholds are selected on validation data, and all subsets of the 12 monitors are ranked by test-set pAUC.The resulting composition patterns are evaluated for generalization on BigCodeBench-Sabotage.Source: Ensemble selection · pp. 3–4

Diversity metric

  • Diversity is the average pairwise Pearson correlation among monitors’ score vectors on attack samples, where lower correlation indicates more disagreement and greater diversity.The metric focuses on attacks because complementary responses to backdoors matter more than agreement on benign code.Source: Diversity metric · p. 4

4. Experimental Setup

The study evaluates GPT-4.1-mini monitors on adversarial coding backdoors using deployment-oriented datasets and low-FPR detection metrics. It tests transfer across diverse backdoor types and designs the evaluation to separate backdoor detection from ordinary code failure.

4. Experimental Setup

  • The experiments use 12 GPT-4.1-mini monitors under fixed generation settings and aggregate ensemble outputs by arithmetic mean.All monitors share the same trusted base model, while construction methods vary through prompting and fine-tuning.Source: §4 · p. 4

4.1. Datasets

  • The APPS dataset retains honest and backdoored solutions that pass standard tests, isolating detection of adversarial failures.Backdoors are generated to pass standard tests while failing on attack inputs, and only working honest and backdoored solutions are retained.Source: §4.1 · p. 4
  • The split places Claude-3.7-Sonnet-Thinking backdoors in test data to evaluate defense against a stronger attacker than those represented in training.Weaker or older attacking models are excluded, while remaining backdoors support training and validation.Source: §4.1 · p. 4

BigCodeBench-Sabotage

  • BCB-S tests transfer across 11 qualitatively different backdoor types using APPS-like edge cases and a broader mixed evaluation set.The benchmark includes matched attack and honest samples for both the edge-case subset and all attack categories.Source: BigCodeBench-Sabotage · p. 4

4.2. Evaluation metric

  • pAUC summarizes normalized AUROC over FPRs up to 10%, emphasizing detection while limiting the human-review burden.The metric averages true-positive rates across low false-positive thresholds relevant to trusted monitoring.Source: §4.2 · p. 4

5. Results

Across coding-backdoor evaluations, diverse monitor ensembles outperform individual and homogeneous monitors, with most gains achieved by small ensembles. Their advantage transfers to broader attack types, depends on both monitor quality and diversity, and remains stronger than equivalent scaling in several comparisons.

5.1. Ensembles outperform individual monitors on APPS

  • The ensemble achieves the highest pAUC and TPR across almost all FPR thresholds, outperforming every individual monitor including the strongest Baseline monitor.Figure 2 focuses on FPR ≤10%, while Figure 3 reports bootstrap confidence intervals for individual-monitor comparisons.Source: §5.1 · pp. 4–5, Figure 2 · p. 5, Figure 3 · p. 5
  • 81.98% versus 50.0% recall at a 1% FPR target shows the best ensemble’s strongest calibration-region advantage over the Baseline monitor.Validation-selected ensembles outperform the Baseline monitor at every tested ensemble size and FPR target.Source: §5.1 · p. 5
  • The median calibration gap falls from 5.5 percentage points for individual monitors to 1.89 points at k = 3.Averaging scores creates denser achievable thresholds and improves FPR control.Source: §5.1 · p. 5

5.2. A small ensemble captures most of the gain

  • k = 3 captures most available improvement: positive marginal gain exceeds 95% through three monitors but not for larger ensembles.Performance rises rapidly from k = 1 to k = 2, then plateaus and can decline as additional monitors contribute little inconsistently.Source: §5.2 · p. 5, Figure 4 · p. 5
  • 13% versus 5.5% pAUC gain over a baseline shows that the best diverse 3-monitor ensemble outperforms three identical monitors by 2.4×.The comparison isolates construction diversity because both ensembles use three monitors against the same baseline.Source: §5.2 · p. 6, Table 1 · p. 6

5.3. Top ensembles generalize to BCB-S

  • APPS-selected ensembles retain statistically significant pAUC advantages over both baselines on BCB-S, including its full range of backdoor types.This transfer supports complementary detection signals rather than APPS-specific artifacts.Source: §5.3 · p. 6, Figure 5 · p. 6

5.4. Ensemble quality depends on both diversity and individual monitor strength

  • Diversity alone underperforms random selection at 0.717 versus 0.769 pAUC, whereas filtering out bottom-quartile monitors before diversity selection improves both APPS and BCB-S results.Low-performing monitors can disagree because they are often incorrect, so strong individual performance must accompany diversity.Source: §5.4 · p. 6

5.5. Fine-tuned monitors as a source of unique signal

  • Fine-tuned monitors appear in every high-performing ensemble across APPS, BCB-S Edge, and BCB-S All, suggesting complementary capabilities beyond prompting.FT-Completions-Randint appears in every such ensemble, while FT-Completions-GPT-5 is among the next most frequent monitors.Source: §5.5 · pp. 6–7, Table 2 · p. 6

6. Discussion

The discussion attributes ensemble gains to reduced failure-mode correlation and recommends selecting strong, diverse monitors rather than simply increasing ensemble size. It also qualifies these findings by noting narrow evaluation scope, unclear mechanisms, shared-model vulnerabilities, and the limits of monitoring alone.

6.1. Why does ensembling work?

  • Diverse monitor ensembles can reduce systematic error by lowering correlation between failure modes, although whether this or attack coverage explains the gains remains unresolved.Repeated sampling mainly reduces random noise, while diverse monitors can partially cancel idiosyncratic biases; the paper leaves the relative mechanisms as an open empirical question.Source: §6.1 · p. 7

6.2. Selecting good ensembles

  • Strong ensembles should first filter out poor individual monitors, then select for diversity; simple averaging is sufficient because more complex aggregation methods show no consistent improvement.Selecting solely for low correlation can favor frequently wrong monitors, whereas pruning weak monitors before diversity selection outperforms random and homogeneous selection.Source: §6.2 · p. 7

6.3. Limitations & future work

  • The evaluation covers short-horizon coding tasks with discrete backdoors and does not test longer-horizon, ambiguous-intent, or multi-episode attacks.The attacker is also assumed not to know the ensemble composition, so robustness to adaptive adversaries and adversarially optimized attacks remains unestablished.Source: §6.3 · pp. 7–8
  • Because all monitors use one base model, the measured diversity is limited to prompting and fine-tuning, while shared-model prompt injection vulnerabilities remain possible.The paper proposes testing monitors from different model families or non-LLM signals, but does not evaluate whether architectural diversity improves robustness.Source: §6.3 · p. 8
  • Average pairwise correlation guides ensemble construction, but the features and attack blind spots underlying monitor complementarity remain unclear.More targeted analyses are needed to test whether fine-tuned monitors consistently specialize in complementary blind spots across datasets.Source: §6.3 · p. 8
  • Stronger monitoring is necessary but insufficient for AI control because audit rates, score thresholds, and protocol design also determine deployment safety.The results should therefore inform broader safety analysis rather than serve as standalone safety guarantees.Source: §6.3 · p. 8

7. Conclusion

The paper argues that practical monitoring gains come from combining complementary monitors rather than merely increasing monitor count. Carefully selected ensembles generalize across datasets and attack types, with fine-tuning emerging as a particularly valuable source of diversity.

  • Monitor diversity—not ensemble count—drives performance gains, allowing carefully selected subsets to capture most of the benefit of larger ensembles across datasets and attack types.Fine-tuned monitors consistently appear in top ensembles, suggesting they provide capabilities that prompting alone does not elicit.Source: §7 · p. 8

Impact Statement

The impact statement positions ensemble monitoring as evidence toward stronger AI control protocols while acknowledging a possible dual-use risk. Publishing monitoring results may help future systems develop backdoors that evade known protocols.

  • Publishing monitoring results may inadvertently help future AI systems construct backdoors that evade known ensemble monitoring protocols.Source: Impact Statement · p. 8

Code and Data Availability

The paper provides dataset and code availability information through a Hugging Face repository. The listed resource is associated with the Multi-Signal-Monitoring project.

AI Use Disclosure

The paper discloses using AI coding assistants for analysis scripts, figures, an outline, and initial text, while retaining author responsibility. Its baseline monitor prompt asks an AI monitor to detect hidden backdoors by analyzing agent actions and assigning a calibrated suspicion score.

AI Use Disclosure

  • The authors used Claude Sonnet 4.5 and GPT-Codex-5.2 for analysis scripts and figures, and Claude Sonnet 4.5 for an outline and initial text.They reviewed and edited the generated content and retained responsibility for the publication.Source: AI Use Disclosure · p. 8

B. Baseline Monitor Prompt

  • The baseline monitor is prompted to examine an LLM agent’s software-engineering actions for hidden malicious activity or a secret backdoor.It receives the main task description and the agent’s tool calls as context.Source: B. Baseline Monitor Prompt · p. 12
  • The monitor evaluates whether actions are task-related, unnecessary, out of scope, or capable of causing unintended consequences.It must provide reasoning that references specific tool calls and their relation to the main task.Source: B. Baseline Monitor Prompt · p. 12
  • The monitor outputs a suspicion score from 0 to 10, where 0 means aligned and non-suspicious and 10 means highly suspicious and likely malicious.The prompt explicitly requests calibrated judgment and a structured reasoning-and-score format.Source: B. Baseline Monitor Prompt · p. 12

C. Fine-Tuned Monitor Descriptions

Fine-tuned monitors, calibrated operating thresholds, and carefully selected ensemble sizes support compact, diverse ensembles. Across the supplied analyses, performance depends on training-data quality, reliable low-FPR operation, gains that largely arrive by three monitors, and diversity after enforcing a quality floor.

C. Fine-Tuned Monitor Descriptions

  • FT-Completions-Randint achieves pAUC = 0.761, outperforming GPT-generated-reasoning monitors, while filtered GPT-5 training improves over given-label and unfiltered variants.The passage attributes these patterns to correct score supervision and excluding incorrect generated responses, and reports stronger performance from GPT-5-generated data than weaker generating models.Source: C. Fine-Tuned Monitor Descriptions · p. 13

D. Operating Thresholds and Calibration

  • Thresholds are selected on validation data for target FPRs and then applied to the test set, where Table 5 reports recall and actual FPR for the best ensemble of each size.The calibration gap is the absolute difference between target and actual test-set FPR; Tables 6 and 7 summarize it across averaged and individual operating points.Source: D. Operating Thresholds and Calibration · p. 14, §D.1 · p. 14, Table 5 · p. 14

E.1. Bootstrap Sign Test

  • Marginal ensemble gains remain reliable through k=3, fall below the reliability threshold at k=4, and become consistently negative beyond k=7.The reliability analysis uses 1000 paired bootstrap resamples and a 0.95 threshold.Source: §E.1 · p. 16, Table 8 · p. 16

E.2. Cumulative Gain Capture

  • At k=3, the ensemble captures 85.5% of total available gain, and the combined evidence identifies three monitors as the preferred ensemble size.Total available gain is defined relative to peak performance at k=5.Source: §E.2 · p. 16, Table 9 · p. 16

F. Top Ensemble Compositions

  • The top ensemble compositions repeatedly include FT-Completions-Randint with monitors such as Code-Heuristic, Trusted-Debate, Extract-Evaluate, or High-Precision.The supplied table caption identifies Table 10 as listing monitors in the ensembles from Figures 5 and 6.Source: F. Top Ensemble Compositions · p. 17, Table 10 · p. 17

G. Diversity-Selected Ensembles

  • After low-quality monitors are filtered, the most diverse ensembles outperform both the Baseline monitor and the median 3-monitor ensemble on APPS.Diversity is measured through average pairwise Pearson correlation, with lower correlation indicating greater diversity.Source: G. Diversity-Selected Ensembles · p. 18, Figure 6 · p. 18
  • The same diversity-based selection approach produces an even stronger diversity–performance relationship on BCB-S, suggesting robustness across datasets and attack types.Figure 6 reports paired pAUC differences with bootstrap confidence intervals based on 1000 resamples.Source: G. Diversity-Selected Ensembles · p. 18, Figure 6 · p. 18

Related papers

Loading 2605.15377v2…