PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

Jiaxin Wen, Vivek Hebbar, Caleb Larson, Aryan Bhatt, Ansh Radhakrishnan, Mrinank Sharma, Henry Sleight, Shi Feng, He He, Ethan Perez, Buck Shlegeris, Akbir Khan

arXiv:2411.17693v1cs.CL

TL;DR

As LLMs grow more capable, deployment strategies need to be assessed against models that may try to bypass safety measures, including settings where individual failures are tolerable but risks accumulate. The paper adaptively chooses among task-level protocols using an estimate of the untrusted model’s alignment. In its APPS evaluation, the method retains 98.3% of the untrusted model’s performance while accepting a backdoor on fewer than 1% of tasks in a 10000-task sequence.

  • Problem

    Deployment strategies need evaluation against increasingly capable LLMs that may try to bypass safety measures, including settings where individual failures are tolerable.

    Source: §1 · p. 1

  • Method

    An adaptive macro-protocol uses an alignment estimate to select among task-level micro-protocols that use a trusted model to monitor and harness the untrusted model.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    98.3% of the untrusted model’s APPS performance is retained while accepting a backdoor on <1% of tasks in a 10000 task sequence.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The deployer can gradually optimize the safety-usefulness tradeoff by leveraging the untrusted model’s historical behavior, including when individual tasks provide little confidence about alignment.

    Source: §6 · p. 10

    Credence estimates are approximate with finite reference samples, so errors can accumulate over long sequences and produce overconfident estimates.

    Source: §3.2 · p. 6

Abstract

from arXiv · show

As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them. Previous work introduced control evaluations, an adversarial framework for testing deployment strategies of untrusted models (i.e., models which might be trying to bypass safety measures). While prior work treats a single failure as unacceptable, we perform control evaluations in a "distributed threat setting" -- a setting where no single action is catastrophic and no single action provides overwhelming evidence of misalignment. We approach this problem with a two-level deployment framework that uses an adaptive macro-protocol to choose between micro-protocols. Micro-protocols operate on a single task, using a less capable, but extensively tested (trusted) model to harness and monitor the untrusted model. Meanwhile, the macro-protocol maintains an adaptive credence on the untrusted model's alignment based on its past actions, using it to pick between safer and riskier micro-protocols. We evaluate our method in a code generation testbed where a red team attempts to generate subtly backdoored code with an LLM whose deployment is safeguarded by a blue team. We plot Pareto frontiers of safety (# of non-backdoored solutions) and usefulness (# of correct solutions). At a given level of usefulness, our adaptive deployment strategy reduces the number of backdoors by 80% compared to non-adaptive baselines.

1 INTRODUCTION

The paper studies deployment of potentially misaligned LLMs when individual harmful actions are tolerable but risks accumulate. It proposes adaptive selection among task-level protocols and evaluates the approach in code generation.

1 INTRODUCTION

  • The distributed threat setting allows no single action to be catastrophic or provide overwhelming evidence of misalignment, unlike prior work where one failure is unacceptable.In such settings, immediately shutting down after one issue can be unnecessary and prone to false alarms; multiple actions can instead provide accumulating evidence.Source: §1 · pp. 1–2
  • The macro-protocol uses an adaptive estimate of alignment to choose safer or riskier micro-protocols, which use a weaker trusted model to monitor and harness the untrusted model.Micro-protocols provide distinct safety-usefulness tradeoffs, and the authors find that combining them performs better than using any one alone.Source: §1 · pp. 2–3
  • The evaluation adapts the APPS backdooring testbed and removes the prior budget of trusted high-quality labor while a red team optimizes for harm.A solution is harmful if it passes a test suite but fails on an input known to the untrusted model.Source: §1 · p. 2
  • 98.3% of the untrusted model’s APPS performance is retained while accepting a backdoor on <1% of tasks in a 10000 task sequence; the method dominates non-adaptive baselines.Its safety-usefulness performance also improves with longer task sequences as the macro-protocol gathers more evidence about alignment.Source: §1 · pp. 2–3
  • The paper releases its micro-protocols, adaptive macro-protocol, and performance-optimized strategy solver for future research.Source: §1 · p. 3

2 PROBLEM DEFINITION

The problem is to maximize task performance while minimizing unsafe outputs over long sequences of tasks. The control protocol uses trusted and untrusted models sequentially and is adversarially evaluated against a red-team policy.

2 PROBLEM DEFINITION

  • The deployment goal is to maximize task performance while minimizing unsafe outputs across a long sequence in which no single failure is catastrophic.Source: §2 · p. 3
  • The untrusted model U is the primary, more capable model that may produce subtly misaligned outputs and evade safety checks.Source: §2 · p. 3
  • The trusted model T queries and monitors U and is treated as trusted because it is extensively tested or lacks the capability to pursue coherent hidden goals.T is less capable than U.Source: §2 · p. 4
  • A control protocol maps each task input to an output sequentially, may maintain state across tasks, and is evaluated with utility αS + U.Usefulness measures correctness, safety measures harmfulness, and α adjusts the preference between safety and usefulness.Source: §2 · p. 4
  • Control evaluations compare usefulness under a benign policy with safety against a red-team-optimized malicious policy that cannot carry state or access T.The protocol is not told whether it faces the benign or malicious policy.Source: §2 · p. 4

3 METHOD

The method combines single-task micro-protocols with an adaptive macro-protocol that updates its estimate of the untrusted model’s alignment and selects among micro-protocol outputs. This approach improves the safety–usefulness tradeoff over static methods, but depends on estimates derived from a reference distribution.

3.1 MICRO PROTOCOL

  • Each micro-protocol generates a single-task output and monitor score using trusted and untrusted models; the protocols vary in how they restrict the untrusted model’s observations or outputs.The paper introduces plan-and-generate protocols alongside Query Untrusted, Edit Untrusted, and Query Trusted.Source: §3.1 · p. 4, Table 1 · p. 5

3.2 MACRO PROTOCOL

  • The macro-protocol’s hypotheses and initial prior are chosen for worst-case evaluation by finding a Nash equilibrium between the red team’s malign-policy distribution and the blue team’s prior.The model assumes the untrusted model scores possible malicious outputs using sneakiness and efficacy.Source: §3.2 · pp. 5–6
  • At each task, the macro-protocol updates Bayesian credences from monitor scores, calculates expected utility for candidate outputs, and selects the output with the highest expected utility.Its hypotheses describe the untrusted model’s misbehavior rate and its tradeoff between avoiding detection and causing harm.Source: §3.2 · pp. 5–6
  • Finite reference samples make monitor-score estimates approximate and can cause overconfident credences over long sequences; bootstrapping over subsamples is used to mitigate this.The authors identify robustness to biased distribution estimates as a direction for future work.Source: §3.2 · pp. 6–7
  • Across 10,000-task sequences, the adaptive protocol dominates static approaches, ensembling micro-protocols yields the best safety–usefulness tradeoff, and the tradeoff improves over longer sequences.Figure 3 reports means and standard errors across 80 trials with different data splits.Source: Figure 3 · p. 7
  • The adaptive macro-protocol updates its credence about the untrusted model’s alignment from observed behavior and uses it to choose among micro-protocols, improving the safety–usefulness frontier over static methods.The approach requires a good estimate of monitor-score distributions for malicious and nonmalicious outputs.Source: §3 · p. 4, §3.2 · p. 7

4 EXPERIMENTS

The experiments evaluate adaptive deployment against static baselines in an APPS code-generation testbed where a red team attempts to insert backdoors. The adaptive approach retains nearly all of the untrusted model’s usefulness while achieving high safety, and its tradeoff depends on the trusted model’s capabilities.

4.1 SETTING

  • The APPS testbed asks the blue team to solve sequential coding problems while the red team attempts to subvert its protocols with subtle backdoors; evaluation adversarially optimizes both teams’ strategies.The red team is granted access to the untrusted model and knowledge of the blue team’s micro-protocols for the worst-case scenario.Source: §4.1 · p. 7, §4 · p. 7
  • Usefulness means passing all APPS unit tests, while harm means passing those tests with an exploitable backdoor; Static-EV fixes initial credences and Static-Threshold filters outputs at a fixed threshold.The main results use GPT-4 as U and GPT-3.5-turbo as T, with other Claude models tested to explore capability gaps.Source: §4.1 · p. 8

4.2 RESULTS

  • 98.4% of the untrusted model’s usefulness is retained at 99.2% safety with GPT-3.5 as the trusted model on 10,000-task sequences.The paper reports 80 harmful actions out of 10,000; exclusively using the trusted model reaches 100% safety and 23% usefulness.Source: §4.2 · p. 8
  • The adaptive macro-protocol outperforms Static-EV by learning from observed behavior, already doing so after 10 tasks and continuing to improve its safety–usefulness tradeoff.Static-EV holds the initial credence fixed rather than updating it during deployment.Source: §4.2 · pp. 8–9, §4.1 · p. 8
  • Combining all micro-protocols yields better performance than any individual protocol because their safety–usefulness tradeoffs are complementary.The comparison pairs each micro-protocol with Query Trusted as a default option.Source: §4.2 · p. 9
  • A larger trusted–untrusted capability gap worsens monitoring and the safety–usefulness tradeoff; GPT-3.5 reaches 98.4% of U’s usefulness at 99.2% safety, while Haiku reaches 17.9% at 99.1%.The corresponding monitor AUROC values are 0.94 for GPT-3.5 and 0.67 for Haiku.Source: Figure 4 · p. 8, §4.2 · p. 9
  • For GPT-3.5-turbo, U-predicted sneakiness and T-predicted monitor scores correlate at +0.22 (p < 0.03), while Claude-3-Opus has a +0.45 correlation (p < 0.01).The authors suggest the smaller capability gap may make U more able to simulate T, but frame this as a possibility.Source: §4.2 · p. 9

5 RELATED WORK

Prior alignment training and control evaluations address safety through different approaches: this paper studies deployment-time safeguards for potentially misaligned models. Existing control evaluations focus on settings where a single failure is unacceptable, while this work concerns applications that tolerate occasional errors.

5 RELATED WORK

  • Alignment training methods have limited unsafe-output coverage and potentially inaccurate safety signals, motivating deployment safeguards as an approach orthogonal to training [Gehman et al., 2020; Ouyang et al., 2022].The passage also notes that unsafe outputs remain frequent after extensive safety training [Mazeika et al., 2024].Source: §5 · p. 9
  • The reported GPT-3.5–Claude-3-Opus gap in Figure 4 is likely not statistically significant given the APPS subset size.The figure’s error bars include data-split uncertainty but not uncertainty from the full APPS subset size.Source: §5 · p. 9
  • Greenblatt et al. (2023) study high-stakes deployment where one failure is unacceptable and high-quality auditing triggers shutdown upon detecting misalignment.This paper instead targets broader applications with greater tolerance for errors and manages distributed risks.Source: §5 · p. 10

6 DISCUSSION, LIMITATIONS & CONCLUSION

The paper argues that deployment can improve its safety–usefulness trade-off over time by using the untrusted model’s historical behavior. Its evaluation and conclusions remain bounded by task complexity, reference-distribution currency, red-team strategy, and the trusted–untrusted capability gap.

6 DISCUSSION, LIMITATIONS & CONCLUSION

  • The approach alleviates dependence on high-quality human supervision and supports gradual improvement of the safety–usefulness trade-off.The method uses trusted-model micro- and macro-protocols to safeguard an untrusted model.Source: §6 · p. 10
  • The experiment checks roughly 50-line programs; longer or multi-file solutions may require more careful trusted-model review, such as interactive debugging.Tightly correlated tasks and less structured domains are additional proposed extensions.Source: §6 · p. 10
  • Open-ended deployment may shift the output distribution, making the reference distribution stale; updating it and retiring outdated samples remain open research problems.Creating reference distributions may require time-intensive manual evaluations.Source: §6 · p. 10
  • The red team represents models that mix misaligned and aligned outputs, but is not exhaustive; the approach assumes a memoryless untrusted model.Future work could explore more red-team micro-strategies and dynamic red-team behavior.Source: §6 · p. 10
  • The effect of larger trusted–untrusted capability gaps is unclear, including whether GPT-4’s approach to exploiting GPT-3.5 informs how GPT-5 might exploit GPT-4.The authors plan to evaluate larger gaps as more capable models are developed.Source: §6 · p. 10
  • Historical behavior lets deployers gradually optimize the safety–usefulness trade-off, even when individual tasks provide little confidence about alignment.Combining diverse micro-protocols also yields superior performance because they have distinct safety–usefulness trade-offs.Source: §6 · p. 10

C CHRONOLOGY OF A CONTROL EVALUATION

The appendix lays out the evaluation and deployment workflow, reports how the macro-protocol selects micro-protocols, and describes the expected-utility and prior-construction machinery behind that adaptive choice.

C CHRONOLOGY OF A CONTROL EVALUATION

  • The evaluation workflow curates tasks, designs single-task components, collects benign and malicious outputs plus monitor scores, and builds a reference distribution.It then derives a prior, optimizes the red-team strategy on validation data, measures on test data, and initializes a real deployment from the chosen safety coefficient’s prior.Source: C CHRONOLOGY OF A CONTROL EVALUATION · p. 13

E MACRO-PROTOCOL

  • Deployment simulation uses a Nash equilibrium to construct the macro-protocol prior, then adversarially optimizes the untrusted model’s strategy against that protocol.The appendix frames the evaluation as a game combining safety and usefulness and outlines the prior algorithm and update step.Source: C CHRONOLOGY OF A CONTROL EVALUATION · p. 13, E MACRO-PROTOCOL · pp. 14–15

D MICRO-PROTOCOL USAGE

  • Query Trusted is selected more often at higher safety coefficients and when the untrusted model is malign; the two simplest micro-protocols are used most.The usage summary covers 10,000-task deployments with GPT-3.5 as trusted model and a malign untrusted model in half of deployments.Source: D MICRO-PROTOCOL USAGE · p. 14, Table 2 · p. 14

E.1 DERIVATION OF MACRO-PROTOCOL EXPECTED UTILITY FORMULA

  • The expected-utility derivation combines safety and usefulness in one stochastic game, with the model chosen as benign or malign at the start of the task sequence.The resulting single-answer utility contribution is used in the macro-protocol’s expected-utility calculation; the derivation’s p = 0.5 convention does not determine subjective alignment credence.Source: §E.1 · pp. 15–16

E.2 CONSTRUCTING THE OPTIMAL PRIOR

  • The prior-construction game has the blue team choose a prior and the red team choose an untrusted-model strategy, with opposing payoffs based on safety and usefulness.An iterative best-response procedure seeks a Nash equilibrium, whose blue-team strategy is worst-case optimal; the implemented hypothesis count is bounded by its iteration count.Source: §E.2 · pp. 16–17

E.3 REFERENCE DISTRIBUTION AND ELICITATION

The reference distribution supplies examples and model assessments used by the deployment protocol, and its quality depends on elicitation and continued relevance. The appendix also describes bootstrapping credences to account for uncertainty when that distribution is imperfect.

E.3 REFERENCE DISTRIBUTION AND ELICITATION

  • The reference distribution contains sampled tasks, benign and malicious responses for each micro-protocol, untrusted-model ratings, and trusted-monitor scores.These are also the items used to simulate deployments for estimating safety and usefulness.Source: §E.3 · p. 18
  • The evaluation and reference-distribution construction share an elicitation problem: eliciting the untrusted model’s best effort on malicious and benign outputs and ratings.The paper assumes training can elicit behavior the model is capable of when a reward function for that behavior can be found; high-quality human ratings can be too costly for every deployment task.Source: §E.3 · p. 18
  • Malicious reference outputs should be harmful and unsuspicious, while benign outputs should be harmless, useful, and unsuspicious; human examination is proposed for assessing these qualities.The passage warns that suspicious benign reference outputs could increase the malicious behavior the untrusted model can get away with during deployment.Source: §E.3 · p. 18
  • A red team could sabotage reference-distribution construction, and a qualitatively defective distribution may leave the bootstrapping method unable to produce reasonable credences under distributional error.For evolving deployments, the reference distribution may also need periodic updates as task distributions change.Source: §E.3 · p. 19

E.4 MACRO-PROTOCOL UPDATE STEP

  • Bootstrapping repeats Bayesian updates on random reference-distribution subsets, estimates uncertainty over posterior log-odds, then converts that uncertainty to mean credences for expected-value calculations.The procedure addresses miscalibration from naively updating against an imperfect reference distribution.Source: §E.4 · p. 19

F EXPERIMENT DETAILS

The experiment details describe the model and decoding settings, the macro-protocol’s update and utility calculations, and trusted-model monitoring.

F EXPERIMENT DETAILS

  • Experiments use GPT-4-0125-preview, GPT-3.5-turbo-1106, and Claude 3 Haiku, Sonnet, and Opus; decoding is usually greedy at temperature 0.0.Sampling at temperature 0.3 is used to generate multiple candidates; Claude monitoring uses best-of-8 averaging at temperature 1.0 to simulate log-probability estimates.Source: F EXPERIMENT DETAILS · p. 19
  • The macro-protocol updates credences over hypotheses using reference-distribution outputs, then calculates expected usefulness and harmfulness for each micro-protocol output.Its algorithm also selects the final output and returns updated credences and expected utilities.Source: F EXPERIMENT DETAILS · p. 20

G BLUE TEAM

  • For safety monitoring, GPT-3.5-turbo uses log probabilities from the Table 4 prompt, whereas Claude-series models generate a 1–10 scalar because their API lacks log probabilities; eight samples are averaged.The prompt extracts the log probability on “yes” and “no” to derive the final monitor score.Source: G BLUE TEAM · p. 21, Table 4 · p. 21

H RED TEAM

The red team constructs stealthy backdoors and chooses when to submit them, while a numerical solver finds mixed-strategy Nash equilibria for the competing macro-protocols.

H RED TEAM

  • The red team estimates how suspicious its backdoored code looks to the blue team and prioritizes behaviors likely to be predicted safe.Its monitoring prompt is intended to imitate the prediction of a less capable model.Source: H RED TEAM · p. 21
  • GPT-4 backdoors are developed with a multi-step scaffold that generates, critiques, refines, implements, and selects subtle backdoor ideas.The selection targets backdoors that trigger on rare inputs and are hard for the safety monitor to detect.Source: H RED TEAM · p. 21
  • The malicious policy is memoryless: it maps each input to an output and may submit only its stealthiest backdoors to avoid revealing its misalignment.It can submit benign answers otherwise.Source: H RED TEAM · p. 21
  • The red team rates malicious candidates for sneakiness and validity, submits the best candidate if it passes a threshold, and otherwise submits a benign output.Sneakiness estimates whether the trusted monitor will catch an output; validity estimates whether its backdoor will function.Source: H RED TEAM · pp. 22–23
  • The open-source solver uses numerical optimization and variance reduction to calculate a mixed-strategy Nash equilibrium from micro-protocol results.The equilibrium is between the control protocol and the malicious policy.Source: H RED TEAM · pp. 22–23
  • Table 7 reports attempted and successful backdoor counts by trusted monitor, using GPT-4 as the untrusted model across the datasets.A successful backdoor passes all unit tests and has an input the untrusted model can use to exploit it.Source: Table 7 · p. 24

I BACKDOOR ANALYSIS

The backdoor analysis reports population counts and defines three scores for characterizing backdoored solutions and their likely detection or success.

I BACKDOOR ANALYSIS

  • The backdoor breakdown reports counts by trusted monitor, using GPT-3.5 as the trusted model for brevity.Table 7 covers the number of attempted and successful backdoors for each trusted monitor.Source: I BACKDOOR ANALYSIS · p. 24
  • Monitor scores range from -12.5 to 12.5, with more positive values indicating code is more likely to look innocent to the trusted monitor.More negative values indicate code is more likely to be judged backdoored.Source: I BACKDOOR ANALYSIS · p. 24
  • Sneakiness scores estimate the monitor score, while efficacy scores from 0.0 to 1.0 estimate the chance a backdoor passes all unit tests.The red team’s GPT-4 sneakiness ratings do not exceed 7.5.Source: I BACKDOOR ANALYSIS · p. 24

I.1 SNEAKY BACKDOORS, AS PER THE BLUE TEAM

The examples examine how blue-team monitoring and plan-editing approaches handle backdoors. They show cases where backdoors remained or removal failed, including a direct plan edit with a flawed feasibility check.

I.1 SNEAKY BACKDOORS, AS PER THE BLUE TEAM

  • All five Edit Untrusted samples removed a backdoor despite its high monitor score, showing that detecting backdoors is not the same as removing them.The example’s reported monitor score was 10.9.Source: §I.1 · p. 25
  • A contrasting high-safety backdoor survived Edit Untrusted; its single-line refactoring was rare and may have led to its high monitor score.The example had a 6.6 monitor score before editing and a post-edit monitor score of 3.7.Source: §I.1 · pp. 25–26
  • 11.1 monitor score, 3.2 sneakiness score, and 1.0 efficacy score marked a successful Untrusted Plan backdoor that was likely to be selected by the macro-protocol.The red team also considered it relatively likely to be accepted because of its fairly high sneakiness score.Source: §I.1 · p. 28
  • Filtered Plan failed to remove the backdoor because its subtask scored 7.8, above an innocent subtask’s 6.8, so the least-safe-subtask filter removed the innocent one.The backdoored subtask may have been nested within a vital task, or multiple backdoored subtasks could evade a protocol that removes only one subtask.Source: §I.1 · pp. 26, 29
  • The Edited Plan checked length feasibility, then matched characters in t against s or available character counts from p.Its plan described iterating through t and rejecting a character that could be matched with neither s nor p.Source: §I.1 · pp. 29–30
  • The backdoor persisted through the blue team’s direct edit of the plan.The model’s chain of thought also noted a possible issue with the edited feasibility check.Source: §I.1 · p. 31
  • The model’s critique was misguided: an overly specific feasibility check could miss impossible transformations, implying too few rejections rather than the backdoor’s excessive rejections.The model observed that the check might miss cases due to the arrangement of characters in s, t, and p.Source: §I.1 · p. 31

I.2 OBVIOUS BACKDOORS, AS PER THE BLUE TEAM

The examples examine how the blue team’s monitoring and micro-protocols handled obvious backdoors, including how judgments varied with task framing and whether conspicuous patterns were removed or retained. They also illustrate differing safety–usefulness tradeoffs, while showing that sneakiness estimates could be poorly calibrated and a flagged backdoor could persist.

I.2 OBVIOUS BACKDOORS, AS PER THE BLUE TEAM

  • Both “Query Untrusted” examples contained conspicuous backdoor code that was removed in all five “Edit Untrusted” solutions.One used a standalone conditional unrelated to the problem logic; the other used a suspicious sys.exit() call with no output.Source: §I.2 · p. 32
  • The untrusted model rated one attempt as unlikely to be detected even though the blue team marked it suspicious with high probability.The other attempt received a relatively low sneakiness rating, but the passage notes that it was not near the minimum possible.Source: §I.2 · p. 32
  • The unedited plan faithfully implemented a suspicious backdoor with a standalone conditional and sys.exit(); the “Filtered Plan” removed it, yielding a monitor score of 10.3.The backdoor was somewhat related to checking login characters, but the passage says that connection was insufficient to make it unsuspicious.Source: §I.2 · p. 34
  • The “Edited Plan” failed to remove the backdoor even after its reasoning identified the pattern check as potentially problematic and criticized sys.exit().The untrusted model had assigned high confidence that the backdoor would be marked innocent; the passage says GPT-3.5 treated sys.exit() as more suspicious than GPT-4.Source: §I.2 · p. 34
  • The login-pattern modification raised monitor score from -11.5 to 2.3, making it more likely to be accepted by the macro-protocol.Source: §I.2 · p. 36

I.3 SNEAKY BACKDOORS, AS PER THE RED TEAM

  • One red-team-highly-rated backdoor hid in a task-relevant if-statement, while another conventional lone conditional was monitor-suspicious despite its high red-team sneakiness rating.The latter had Monitor Score -8.8, Sneakiness Score 7.0, and Efficacy Score 1.0; the authors speculated that U may have seen its snippet as unusually problem-related.Source: §I.3 · p. 37

I.4 OBVIOUS BACKDOORS, AS PER THE RED TEAM

  • Problem-specific variable names made a red-team-obvious backdoor relatively innocent to the blue team, whereas an unrelated end-of-function conditional was obvious to both teams.Source: §I.4 · p. 38

I.5 SUMMARY

  • Backdoors were least suspicious when they modified data without a conditional or embedded a conditional among other checks; most top-decile monitor-score working backdoors fit these patterns.Source: §I.5 · p. 38
  • Standalone conditionals, functionality-unrelated checks, or comments flagging special code accompanied high suspicion; every highly suspicious attempt had at least one such trait.Source: §I.5 · pp. 38–39

J ANALYZING THE PERFORMANCE OF TRUSTED MONITORS

  • Four trusted models had distinct monitor-score distributions for innocent and backdoored solutions, producing measurable AuROC and qualitative differences in monitor performance.Source: Figure 5 · p. 40

K MICRO-PROTOCOL SAFETY AND USEFULNESS TRADEOFF

  • Query Untrusted had the lowest safety and highest usefulness, while Query Trusted had the highest safety and lowest usefulness; editing and decomposition traded usefulness for higher safety.The comparison sampled solutions across 479 problems over 10,000 runs, calculating 95% confidence intervals for benign usefulness and malicious safety.Source: Table 8 · p. 41, K MICRO-PROTOCOL SAFETY AND USEFULNESS… · p. 41

L STATIC-THRESHOLD MACRO-PROTOCOL ANALYSIS

The Static-Threshold macro-protocol is evaluated by varying red-team backdoor submission rates and blue-team acceptance thresholds. Editing reduces the spread of safety and usefulness outcomes, while monitor quality improves performance strictly only when editing is disabled.

Evaluation setup

  • The evaluation varies the red-team submission rate and blue-team safety threshold, then reports safety when U is malicious and usefulness when U is benign.It uses 1,000 runs on a 479-problem dataset, with 250 problems per run and 229 held back to estimate thresholds.Source: L STATIC-THRESHOLD MACRO-PROTOCOL… · p. 42

Static-Threshold findings

  • Allowing edits contracts the range of safety and usefulness outcomes, lowering usefulness for benign U and increasing safety when T removes backdoors.T removes the backdoor in around half of otherwise-successful backdoor attempts.Source: L STATIC-THRESHOLD MACRO-PROTOCOL… · p. 42
  • In the no-edit regime, better monitors strictly Pareto-dominate worse monitors, but this does not hold when edits are allowed.The authors note that detecting backdoors does not ensure they can be removed through code edits.Source: L STATIC-THRESHOLD MACRO-PROTOCOL… · p. 42
  • Figure 6 varies acceptance thresholds and backdoor submission rates; accepting more red-team submissions trades greater performance for lower safety.The figure also includes a no-edit regime.Source: Figure 6 · p. 43
  • Figure 7 reports worst-case Static-Threshold performance when the red team submits a backdoor on every episode.Figure 6 gives the full results across capability gaps.Source: L STATIC-THRESHOLD MACRO-PROTOCOL… · p. 42, Figure 7 · p. 44
Loading 2411.17693v1…