PaperlayerView exact revision on arXiv

Source-linked AI summary

Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions

Jaewoo Ahn, Junseo Kim, Hyunseo Kim, Heeseung Yun, Jaehyeon Son, Zsolt Kira, Gunhee Kim

arXiv:2608.30428v1cs.CLcs.AIcs.CVcs.LG

TL;DR

The paper addresses the limited measurement of embodied deception and the attribution problem created by fixed agent configurations in existing text-only evaluations. It introduces MINEAMONGUS, ARIA, and scalable atom- and arc-level annotations, finding that VLM agents use joint verbal and non-verbal deception, with non-verbal channels the more decisive winning contributors. The authors position this as a basis for embodied VLM-agent alignment research.

  • Problem

    Existing deception testbeds are largely text-only and use fixed agent configurations, omitting non-verbal sensorimotor channels and complicating attribution to the underlying model.

  • Method

    The paper combines a 3D multimodal Among Us sandbox, a VLM harness with five independent cognitive-component ablations, and taxonomy-grounded atom- and arc-level deception annotations.

  • Results

    VLM agents pursue imposter wins through joint verbal and non-verbal deception, with non-verbal channels emerging as the more decisive winning contributors across harness ablation and cross-VLM evaluation.

  • Takeaways & Limitations

    The work characterizes embodied, multimodal, multi-turn deception that existing text-only evaluations leave structurally unmeasured.

  • Takeaways & Limitations

    The measurements are game-bounded behavioral deception rather than a direct measure of general-purpose deceptive capability, and privileged state compensates for current VLM spatial-localization limitations.

Abstract

from arXiv · show

Strategic deception by LLM and VLM agents has emerged as a central AI alignment and safety concern. Social-deduction games (where each player holds a hidden role and communicates with others to deduce identities) serve as the canonical testbed, particularly in multi-agent settings. Existing testbeds, however, are text-only and run on a single fixed agent configuration, missing the non-verbal sensorimotor channels treated as core by deception taxonomies and leaving it ambiguous whether an observed behavior reflects the underlying model or the surrounding harness. We introduce MineAmongUs, a 3D multimodal Among Us sandbox where imposter agents must deceive crewmates through joint verbal and non-verbal action. We also propose ARIA, a configurable VLM-agent harness that exposes five cognitive-component ablation axes; and an atom- and arc-level annotation scheme grounded in deception taxonomies and operationalized at scale by an LLM-as-a-Judge reaching near-human atom-labeling agreement. Empirical results show that VLM agents pursue imposter wins through joint verbal and non-verbal deception, with non-verbal channels emerging as the more decisive winning contributors across both harness ablation and cross-VLM evaluation. Taken together, our work opens a new path for embodied VLM-agent alignment research.

1 Introduction

The paper addresses gaps in text-only, fixed-configuration deception testbeds by introducing an embodied multimodal sandbox, a configurable VLM harness, and fine-grained deception annotations. Experiments examine how harness components and VLM backbones shape joint verbal and non-verbal deception.

  • Motivation: Existing social-deduction evaluations are text-only and cannot exercise sensorimotor deception channels.These testbeds also commonly use fixed agent configurations, complicating attribution of findings to the underlying model.
  • Contributions: MINEAMONGUS is a 3D multimodal Minecraft sandbox where VLM agents perceive RGB, navigate physical space, interact with entities, and communicate during meetings.Among Us combines an embodied task phase with a verbal meeting phase in one match.
  • Contributions: ARIA exposes five cognitive components as independent ablation axes to attribute deception behaviors to specific agent configurations.The harness varies state representation, memory, planning, reflection with skill memory, and prompt style.
  • Contributions: The annotation scheme defines 23 verbal and non-verbal deception atoms and higher-order arcs, with an LLM-as-a-Judge enabling scalable labeling.It is grounded in three deception taxonomies and validated against human annotation.
  • Experiments: The experiments test harness ablations with fixed VLM backbones and cross-VLM comparisons with a fixed ARIA configuration.The study links winning settings to changes in non-verbal kill-cycle atoms or verbal falsification atoms.

2 Related Work

Related work establishes deception as both verbal and non-verbal across several taxonomies, while LLM-agent studies have mainly used text-only social-deduction environments. Emerging embodied settings broaden interaction, but existing analyses do not jointly provide the paper’s multimodal and configurable evaluation.

  • Deception foundations: Deception research spans primate tactics, military strategy, interpersonal information manipulation, developmental psychology, HRI, and deceptive motion planning.These traditions collectively treat verbal and non-verbal channels as relevant to deception.
  • LLM deception studies: Most multi-agent LLM deception studies use text-only social-deduction games such as Werewolf, Avalon, Mafia, THE TRAITORS, and Among Us.A small number of works move beyond pure text using symbolic grids or routed environments.
  • Open gap: Existing analyses do not ablate agent configurations as an analytic variable, so their findings cannot be assumed to provide a complete picture of agentic deception.This limitation concerns attribution to the model versus the surrounding configuration.
  • Embodied agents: Embodied-agent research includes 3D indoor simulators and Minecraft benchmarks for scalable simulation, cooperation, and multi-perspective Theory of Mind.These environments provide related substrates for embodied multi-agent research.

3 Design of Sandbox Environment and Agent

The environment models Among Us as an eight-player hidden-role game with multimodal observations and embodied actions. ARIA turns agent design into independently configurable components, while evaluation adds atom- and arc-level measures of verbal and non-verbal deception.

  • Game setup: Among Us assigns eight players to two imposters and six crewmates, alternating task and meeting phases until a winner is decided.Imposters eliminate crewmates while crewmates complete missions and identify imposters.
  • Observation space: MINEAMONGUS provides agents with egocentric RGB, scoreboard signals, chat history, structured server events, positions, and nearby entities.Raw observations are transformed into state representations such as ego or privileged modes.
  • Action space: Agents dispatch Mineflayer JavaScript programs composed of primitive operations such as pathfinding, attacking, and fleeing.Programs can chain multiple operations rather than issuing only single atomic actions.
  • Step protocol: The simulator queues alive bots together and returns synchronized observations after programs complete, while meetings use sequential turn-based utterances.This protocol synchronizes task-phase actions but grounds meeting speech in the discussion so far.
  • ARIA harness: ARIA exposes state representation, memory, planning, reflection with skill memory, and prompt style as five independent ablation axes.Its architecture selects among modules for killing, reporting, surveillance, emergencies, meetings, voting, movement, and missions.
  • Deception evaluation: Evaluation defines 23 deception atoms and arcs, organizing atoms into verbal and non-verbal hierarchies and scaling labels with an LLM-as-a-Judge.The scheme is grounded in three deception taxonomies and derived from 48 gameplay logs.

4 Experiments

The experiments evaluate ARIA’s cognitive components under fixed VLM backbones and compare VLMs under fixed harness configurations. Results show harness composition and VLM backbone both shape imposter success, with non-verbal deception generally more strongly associated with wins than verbal deception.

  • Experimental design: The study runs RQ1 by varying ARIA’s five cognitive-component axes under fixed VLMs and RQ2 by varying VLM backbones under fixed harness configurations.
  • RQ1: Cognitive-component ablation: Egocentric imposter vision collapses the non-verbal channel: imposters land 0 kills, with imposter WR falling from 40% to 0% for GPT-4.1-mini and from 40% to 20% for Qwen3.6-27B.The experiments therefore use privileged state mode for both roles in subsequent ablations.
  • RQ1: Cognitive-component ablation: Harness composition shifts imposter WR despite fixed VLM backbones, with crewmate-side memory and planning changes producing +8 pp under Qwen3.6-27B and −35 pp under GPT-4.1-mini.
  • Deception analysis: Four non-verbal atoms show the strongest positive associations with imposter wins: Witness-Aware Kill (rpb = +0.434), Post-Kill Flee (rpb = +0.414), Strategic Non-Reporting (rpb = +0.270), and Stalking Pre-Kill (rpb = +0.208).Together they cover stalking, witness-aware killing, fleeing, and deliberate post-kill non-reporting.
  • RQ1: Cognitive-component ablation: Across cognitive axes, memory, reflection-skill, and prompt settings associated with higher WR produce more non-verbal kill-cycle atoms, whereas hierarchical planning produces more verbal falsification atoms.Overall axis effects are described as empirical trends rather than universal conclusions.
  • RQ2: VLM comparison: Across VLMs, imposter and crewmate WR correlate at Pearson r = +0.71, while successful imposters use distinct routes: Gemini-3 emphasizes non-verbal camouflage and Kimi-K2.5 emphasizes verbal falsification.The top-3 VLMs achieve high imposter WR through qualitatively different cluster-level archetypes.

5 Conclusion

The paper studies joint verbal and non-verbal deception by VLM agents using MINEAMONGUS, ARIA, and a 23-atom annotation scheme. Its experiments find that harness composition, VLM backbones, and distinct non-verbal or verbal strategies shape imposter success.

  • The study combines MINEAMONGUS, ARIA, and a 23-atom annotation scheme operationalized through an LLM-as-a-Judge.
  • Restricting both roles to egocentric vision collapses gameplay, producing no kills, meetings, or verbal deception.
  • Harness composition substantially shifts imposter win rate even when VLM backbones remain fixed.
  • Higher-win-rate settings tend to elicit non-verbal kill-cycle atoms or verbal falsification atoms.
  • Winning VLMs show the strongest winner–loser separation in specific behaviors, especially non-verbal camouflage, while successful imposters use distinct strategies.

Limitations

The paper’s deception measurements are bounded by the game setting, a privileged spatial scaffold, and the five hand-specified ARIA axes. Transfer to open-ended human interaction and broader harness designs remain unresolved.

  • Game behaviors labeled deception may also reflect task competence or optimal gameplay, so findings measure game-bounded rather than general-purpose deception.
  • Both roles use privileged state because fully egocentric play does not reliably sustain spatial awareness or viable gameplay.
  • ARIA explores five ablation axes, leaving personality-conditioned harnesses and meta-harness optimization for future work.

Ethics Statement

The paper presents a contained measurement study of VLM behavior in a constrained hidden-role game. It uses a multimodal sandbox, configurable harness, and descriptive deception annotations without optimizing models to deceive more effectively.

  • The annotation scheme is descriptive, defining verbal and non-verbal deception atoms and higher-order arcs for analysis at scale.
  • The work measures behavior inside the MINEAMONGUS sandbox and does not optimize models toward more effective deception.
  • MINEAMONGUS is a 3D embodied multimodal hidden-role testbed exposing verbal and non-verbal deception channels.
  • The experiments use eight players with two imposters and six crewmates to make collaborative deception a first-class phenomenon.

D ARIA Implementation Details

ARIA organizes each agent step around planner-selected or automatically triggered behavior modules, combining model decisions with executable Mineflayer actions. Its implementation also records imposter intent and supports structured belief tracking, deception-aware action selection, and scalable atom annotation.

  • Behavior cycle: ARIA’s behavior cycle routes planner-selected modes through MOVE, MISSION, REPORT, MEETING, and VOTE, while crewmate SURVEILLANCE and EMERGENCY can bypass the planner.Memory stores are read by the planner and written by action modules; meeting-end reflection is separately triggered.
  • Action synthesis: Each module pairs an LLM/VLM decision with synthesized Mineflayer JavaScript that realizes the selected action in the environment.Simple actions use environment adapter helpers, while navigation-heavy MOVE and MISSION actions use AriaCodegen.
  • Imposter actions: Imposter navigation, task, and kill actions carry private one-line intent narrations, including truthful free-form intent for generated movement and templated intent for scripted primitives.Examples include faking a mission to appear busy and stalking a target to set up a kill.
  • Multimodal control: Visual input gates KILL, SURVEILLANCE, REPORT, EMERGENCY, and MOVE decisions, whereas meeting, voting, mission selection, belief tracking, reflection, and code generation operate on text-only state.The kill module scans backward for witnesses and forward for the target before choosing KILL, DEFER, or STALK.
  • Deception behavior: Imposter prompts explicitly combine fake missions, exploration, and kills so movement can resemble ordinary crewmate behavior while providing cover for stalking.The prompt states that imposters have no real tasks and can select mission mode to fake one.
  • Annotation: The annotation workflow separates verbal evidence from action-layer evidence and identifies atomic deception instances rather than collapsing all meeting utterances into one finding.The verbal pass scans every imposter meeting utterance and vote reason against a defined set of deception patterns.

E.6 Judge Backbone Selection and Cross-Family Robustness

The paper selects Qwen3.6-27B as a fixed, open-weight annotation judge after comparing judge backbones against human labels. Cross-family tests show strong agreement on the main set but lower agreement on Qwen gameplay logs without evidence of own-family advantage.

  • Selection rationale: Qwen3.6-27B is chosen as the annotation judge because it supports long-context evaluation at lower practical dependence on proprietary APIs.The study evaluates 192 RQ1 matches and 576 RQ2 matches with a fixed judge backbone.
  • Validation: Qwen3.6-27B achieves the highest Cohen’s κ among tested LLM judges on the 48-log validation set, with κ = 0.709.The comparison includes proprietary and open-weight judge backbones against human annotations.
  • Cross-family robustness: The Qwen judge has lower agreement on Qwen3.6-27B logs than on GPT-4.1-mini logs, with κ = 0.623 versus κ = 0.709, providing no indication of own-family labeling advantage.Human–human agreement is higher on Qwen logs, κ = 0.827, than on GPT logs, κ = 0.792.
  • Operational choice: The paper uses Qwen3.6-27B as the fixed judge for large-scale atom annotation because it combines strong human agreement with a reproducible evaluation pipeline.The authors report that it matches or exceeds larger alternatives on the main validation set while avoiding mutable proprietary APIs.

G.1 Experiment design

The experiment design isolates the imposter’s state representation and decomposes the gap between egocentric and privileged settings. Results indicate that persistent spatial tracking, spatial horizon, and RGB perception make distinct contributions to sustaining the kill–meeting–deception loop.

  • State ablation: The state-representation ablation crosses two imposter VLM backbones with ego and privileged modes while holding crewmate state and the other four axes fixed.The design uses five games per cell in the headline comparison.
  • Headline outcome: GPT-4.1-mini imposter win rate drops 40 →0% (−40 pp) under ego, while Qwen3.6-27B drops 40 →20% (−20 pp); the lone Qwen ego win is a timeout.The kill channel collapses completely under ego.
  • Collapse mechanisms: Under ego, visible targets are rarely available or are lost during approach, causing kill plans to reset to exploration and preventing reliable attacks.The pipeline requires a scan-behind, scan-forward, approach, and attack sequence without a last-known target position fallback.
  • Downstream effects: Zero bodies lead to zero meetings, closing the meeting, voting, and verbal-deception channel in the reduced condition.The manual emergency-meeting button is not pressed autonomously by ARIA.
  • Privileged representation: Privileged state supplies persistent full-map coordinates while retaining RGB for witness checks, target verification, corpse discovery, reporting, and crewmate surveillance.It narrows embodiment along spatial localization rather than replacing visual perception.
  • Intermediate ablations: Restricting coordinates to a 36-block horizon raises kills from 0.0 to 2.0 per game but leaves win rate at 0%, whereas full-map tracking raises kills to 3.0 and restores win rate to 40%.These intermediate ablations use GPT-4.1-mini self-play with five games per condition.
  • Modality bottleneck: Removing RGB while retaining full-map coordinates reduces kills to 0.2 per game and imposter win rate to 0% because visual witness and target checks become unreliable.The agent continues producing kill-oriented plans and invoking the kill pipeline, but viable approaches fail.
  • Experimental default: The privileged condition is therefore used as the default because reduced conditions do not support the full kill–meeting–deception loop required for comparison.The choice reflects current limitations in visual perception, spatial localization, and temporally consistent tracking.

I.3 Primary analysis: pooled 2-proportion z-test

The primary analysis pools axis effects across the four RQ1 cells and supplements them with cross-backbone replication and atom-level association analysis. Directional trends replicate across four VLMs, while prompt effects remain backbone-dependent and the small replications are treated as robustness checks.

  • Pooled test: The pooled axis analysis uses 96 versus 96 games per axis level and a normal-approximation two-proportion test across all 192 games.The design adopts α = 0.10 for one-sided preregistered directional hypotheses and α = 0.10 two-sided testing for prompt.
  • Primary findings: Planning and reflection-skill reach marginal significance, memory is directionally consistent but below threshold, and prompt shows no consistent main effect.Prompt preference varies by VLM in the per-cell results.
  • Replication design: The two additional-backbone replications each use 96 matches with the original axis definitions, crewmate configurations, and aggregation procedure.The tested backbones are Gemini-3-flash and Gemma4-31B.
  • Replication results: All three preregistered directional hypotheses replicate on both additional backbones: window memory exceeds semantic memory, hierarchical planning exceeds reactive planning, and meeting-on exceeds none-off.These replications extend the directional checks beyond GPT-4.1-mini and Qwen3.6-27B.
  • Prompt effects: Both additional backbones favor the deterministic prompt, consistent with backbone-dependent prompt effects rather than a uniform prompt main effect.Because each replication has N = 96 games and some differences reflect only a few outcomes, the authors treat them as robustness checks rather than standalone confirmation.
  • Atom-level views: Figures 8 and 9 summarize absolute per-axis atom-count shifts and phase-by-phase atom flow across pre-m1, m1, m1–m2, m2, and post-m2.Figure 9 is restricted to 157 of 192 RQ1 games containing at least one meeting phase.
  • Atom associations: At the cluster level, NV-2 P&K has the strongest positive winning association, r = +0.290, followed by NV-3 R&E, r = +0.268, and V-1 FALS, r = +0.160.NV-1 Cam, V-2 EQVC, and V-3 CONC are near zero or weakly negative.

L RQ1: Arc-Level Deception

Across time-resolved matches, task phases are dominated by non-verbal deception while meeting phases are dominated by verbal deception. A case study shows that meeting outcomes can modulate later action timing, reporting, and verbal strategy.

  • Phase setup: 157 of 192 RQ1 games contained meetings and were included because both verbal and non-verbal phases were present.The analysis decomposes matches into five sequential phases: pre-m1, m1, m1–m2, m2, and post-m2.
  • Phase-conditional distribution: Task phases are dominated by non-verbal atoms, especially Fake-Mission Performance and pursuit-and-kill behaviors around kill events.Meeting phases instead concentrate verbal falsification atoms.
  • Case study: The case study shows game-state-aware adaptation from a meeting outcome to deferred killing, altered reporting, and a more sophisticated later verbal flow.The example contains 90 steps, two kills, two meetings, and an imposter win via two wrongful ejections.
  • Case study: After the first wrongful ejection, imposters spend 8 steps performing fake missions and scouting while deferring a witness-aware kill.Their plan text explicitly weighs remaining missions and the risk that killing could hasten crewmate victory.
  • Case study: In the second meeting, strategic non-reporting and apparent concession combine with an alibi, mutual reinforcement, and counter-accusation to produce another wrongful ejection.The final vote targets the player who reported the body.
  • Cross-VLM observations: Across VLMs, closed-source families occupy the higher imposter- and crewmate-win-rate region, while win rate generally increases with advertised model scale but has within-family exceptions.GPT-5-mini is not consistently below GPT-5, and Gemma4-26B-A4B slightly exceeds Gemma4-31B as a crewmate.

N RQ2: Per-Case Matchup Matrices

The RQ2 matchup analysis pools 12 VLMs across four configuration combinations, then disaggregates results to distinguish configuration effects from cross-model matchup effects. Same-model comparisons confirm a tied Case 1 prediction and favor Case 2B over Case 2A.

  • Matrix design: Each pooled heatmap cell aggregates 4 configuration combinations and 2 repetitions, while the rightmost marginal covers one imposter VLM against all 12 crewmates.The round-robin contains 12 imposter rows and 12 crewmate columns.
  • Case definitions: The four cases vary memory and planning while holding crewmate reflection skill and prompt fixed: symmetric 1A and 2A, plus RQ1-best imposters in 1B and 2B.Case 1B uses window memory and hierarchical planning against semantic-reactive C1; Case 2B uses window memory and reactive planning against window-hierarchical C2.
  • Cross-model effects: Case 1B’s pooled imposter WR is 47.2%, below Case 1A’s 59.0%, because the two imposter configurations tied in RQ1 against C1 rather than one strictly outperforming the other.The pooled discrepancy is therefore not evidence that the RQ1-best label should imply a strict advantage.
  • Same-model matchups: Same-model diagonal means are 50% for Case 1A, 50% for Case 1B, ≈38% for Case 2A, and ≈54% for Case 2B.Case 2B exceeds Case 2A by +16 pp, matching the RQ1 directional comparison.
  • Deception archetypes: The two winning deception archetypes differ: Gemini models are NV-1 Cam heavy at ≈39% of atoms, whereas Kimi-K2.5 is V-1 Falsification heavy at ≈43%.The Gemini archetype relies on fake labor and natural wandering; Kimi-K2.5 relies on meeting-based counter-accusation and mutual reinforcement.
  • Deception archetypes: Both winning archetypes use little equivocation or concealment, which together account for ≤1% of their atom budgets.The shared signature is the absence of defensive hedging rather than a common dominant deception channel.
  • Failure mechanisms: Lower-performing imposter play exhibits heterogeneous failures in target selection, perception, verbal disclosure, and teammate coordination.These qualitative cases complement aggregate atom statistics rather than serving as frequency estimates.
  • Metric interpretation: The matched analysis distinguishes within-game winning drivers from cross-model atom-budget profiles, preventing direct sign-reversal claims between rpb and r.The authors re-run RQ2 with the per-game rpb metric used in RQ1.

Q.2 Method-matched cluster-level rpb

When RQ1 and RQ2 use the same per-game rpb metric, most cluster directions agree and the apparent NV-1 Cam reversal disappears. The matched atom analysis identifies a recurring kill-execution loop associated with imposter wins.

  • Cluster-level comparison: Four of six clusters share sign in the strongly signed direction: NV-2, NV-3, and V-1 are positive, while V-3 is negative.NV-1 Cam and V-2 EQVC are near zero in at least one pool.
  • Metric correction: The earlier contrast between RQ1 NV-1 Cam −0.48 and RQ2 +0.54 is invalid because it compared rpb with cross-model Pearson r on N = 6.Under matched per-game rpb, neither pool yields a strongly signed NV-1 Cam value.
  • Atom-level comparison: Across all 23 atoms, 16 share sign in the strongly signed direction under the matched metric.The remaining seven are either in the noise range or cross zero with at least one substantial magnitude.
  • Winning drivers: Witness-Aware Kill, Post-Kill Flee, Strategic Non-Reporting, and Stalking Pre-Kill have positive rpb in both pools.Post-Kill Flee, Witness-Aware Kill, and Strategic Non-Reporting also show positive within-model rpb for all six RQ2 analysis VLMs.
  • Comparison with prior work: Equivocation accounts for only ≈0.6% of observed atoms in MINEAMONGUS, compared with 36.9% for V-1 Falsification and 29.2% for NV-2 Pursuit & Kill.The V-2 cluster-level r with imposter win is also near zero.
  • Role performance: Across 12 VLMs, imposter WR and crewmate WR correlate at Pearson r = +0.71, so models separate together more than along role-specific skill axes.The observed separation follows a closed-source versus open-source pattern.

S Exploratory: MASK Honesty Score vs. Imposter Win Rate

This exploratory section tests whether MASK honesty relates to imposter win rate and examines behavior-cloning alternatives for small crewmates. The honesty association is non-significant, while cloning data choices trade off winning through missions against explicit imposter detection.

  • MASK comparison: MASK measures honesty as truthful responses to adversarial prompts about facts the model demonstrably knows, distinct from factual accuracy.The exploratory hypothesis predicts a negative relation between MASK honesty and imposter WR.
  • MASK comparison: Pearson r = −0.163 (p = 0.612, N = 12) and Spearman ρ = −0.175 (p = 0.586, N = 12) show no significant model-level association.The game-level logistic regression gives coefficient −0.085 per +10 honesty, odds ratio 0.918, 95% CI [0.80, 1.06], and p = 0.240.
  • MASK comparison: Gemma4-26B-A4B and Kimi-K2.5 have similar MASK honesty scores but markedly different imposter WR, illustrating substantial residual variation.Their honesty scores are 55 and 58, while imposter WR is 36.5% and 66.7%, respectively.
  • Limitation: The MASK comparison is limited by N = 12 models and by measuring honesty in single-turn question answering rather than multi-step embodied social deduction.The authors make no causal or predictive claim from this exploratory analysis.
  • Behavior cloning: Behavior cloning fixes the crewmate at Qwen3.5-9B and trains it with BCwin, BCvote, or their combination against three fixed imposters.BCwin retains calls from won games, BCvote retains correctly targeted voting samples, and BCwin+vote combines both.
  • Behavior cloning: The evaluation reports crewmate WR plus ejection recall and precision, where recall counts ejected imposters and precision counts imposter ejections among all ejections.These metrics distinguish winning through game completion from explicit imposter detection.

U.3 Result

Filtered behavior cloning produces crewmates with complementary strengths: BCwin maximizes win rate through mission completion, while BCvote improves imposter detection; combining them yields the most balanced performance.

  • BCwin more than doubles baseline crewmate win rate against every imposter, achieving the best win rate in each column.Its win rate rises from 25 to 56%, 13 to 50%, and 13 to 38%, while recall remains at or below baseline against the 9B/27B imposters.
  • BCvote raises ejection precision against every imposter and improves recall against the harder 27B-scale imposters.Its conservative voting trades raw recall against the weak 9B imposter for stable precision.
  • BCwin+vote combines the two channels, retaining high win rates while recovering detection performance against 27B-scale imposters.It is the only configuration strong across win rate, recall, and precision, making it the most balanced crewmate.
  • Filtered behavior cloning yields competent 9B crewmates without changing the backbone, with data filters determining their strengths.
Loading 2608.30428v1…