Source-linked AI summary
Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games
Keyang Zhong, Junlin Xie, Hefeng Wu, Haofeng Li, Guanbin Li
TL;DR
VLMs remain limited on multimodal, multi-hop reasoning with imperfect and deceptive information, while large-scale training data for such settings is scarce. The paper synthesizes role-driven Murder Mystery scripts with collaborative agents and trains VLMs through supervised fine-tuning and agent-monitored reinforcement learning. Experiments report improved multimodal reasoning, role-playing, deception detection, and competitive performance across decision-making and multi-hop reasoning metrics.
Problem
VLMs struggle with multi-hop reasoning under imperfect or deceptive information, and large-scale datasets for training and evaluating such multiplayer reasoning remain limited.
Method
The paper uses collaborative generation and evaluation agents to synthesize multimodal Murder Mystery data, followed by two-stage training with supervised fine-tuning and agent-monitored reinforcement learning.
Results
The method significantly improves multimodal reasoning, role-playing, and deception detection, achieving state-of-the-art performance among open-source systems and competitive results against proprietary models.
Takeaways & Limitations
The framework provides a scalable approach for training and evaluating VLMs in uncertain, adversarial, and socially complex reasoning settings.
Takeaways & Limitations
The pipeline still requires partial human verification for image–clue alignment, and the simulated game environment simplifies more dynamic, unstructured real-world interactions.
Abstract
from arXiv · showhide
Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer game settings with imperfect and deceptive information. In this paper, we study a representative multiplayer task, Murder Mystery Games, which require inferring hidden truths based on partial clues provided by roles with different intentions. To address this challenge, we propose a collaborative multi-agent framework for evaluating and synthesizing high-quality, role-driven multiplayer game scripts, enabling fine-grained interaction patterns tailored to character identities (i.e., murderer vs. innocent). Our system generates rich multimodal contexts, including character backstories, visual and textual clues, and multi-hop reasoning chains, through coordinated agent interactions. We design a two-stage agent-monitored training strategy to enhance the reasoning ability of VLMs: (1) chain-of-thought based fine-tuning on curated and synthetic datasets that model uncertainty and deception; (2) GRPO-based reinforcement learning with agent-monitored reward shaping, encouraging the model to develop character-specific reasoning behaviors and effective multimodal multi-hop inference. Extensive experiments demonstrate that our method significantly boosts the performance of VLMs in narrative reasoning, hidden fact extraction, and deception-resilient understanding. Our contributions offer a scalable solution for training and evaluating VLMs under uncertain, adversarial, and socially complex conditions, laying the groundwork for future benchmarks in multimodal multi-hop reasoning under imperfect information.
1 Introduction
Murder Mystery Games provide a representative testbed for multimodal, multi-hop reasoning under imperfect information, deception, and dynamic social interaction. The proposed framework addresses limited training data by synthesizing realistic game scripts and applying two-stage agent-monitored training to improve VLM reasoning.
- Motivation: Murder Mystery Games require players to infer a hidden murderer from public and private textual and visual clues amid adversarial deception.Players use structured dialogue to reason about motives and inconsistencies.
- Framework: The framework uses generation and evaluation agents to collaboratively produce logically coherent, diverse game scripts with realistic roles, interactions, and multimodal clues.Generation agents create scripts, while evaluation agents assess outputs and provide feedback.
- Motivation: These games test long-form narrative understanding, multimodal evidence integration, and multi-step inference under imperfect information and strategic interaction.
- Training: A two-stage agent-monitored strategy generates reasoning chains under imperfect information and enhances VLM learning.
- Results: The method reports consistent gains in reasoning and role-playing for 3B and 7B VLMs in Murder Mystery scenarios involving imperfect and deceptive information.The reported examples include Qwen2.5-VL-3B-Instruction and its 7B counterpart.
2 Related Work
Prior work uses social reasoning games and multi-agent synthetic data to study complex multimodal reasoning, while reasoning-enhanced VLM training commonly combines supervised fine-tuning with reinforcement learning. Related systems also explore visual operations, retrieval, and experience-driven learning for generalization and interaction efficiency.
- Social Reasoning Games: Social reasoning games such as Murder Mystery and Werewolf evaluate VLM reasoning under imperfect information, multi-agent interaction, and deception.WhodunitBench provides murder mystery scripts with multiple-choice and open-ended questions for assessment.
- Multi-Agent Synthetic Data: Multi-agent synthetic-data systems address scarce multimodal training data by increasing dataset diversity and reasoning complexity while reducing reliance on manual annotation.
- Multi-Agent Synthetic Data: Recent image-generation systems decompose complex prompts, construct planning trees, use external tools, and apply iterative verification or self-correction.
- Reasoning-Enhanced Training: Reasoning-enhanced VLM pipelines commonly use supervised fine-tuning to instill structured chain-of-thought reasoning before reinforcement learning optimizes reasoning quality and generalization.
- Reasoning-Enhanced Training: Other approaches use drawing-to-reason, retrieval-augmented, or experience-driven learning to improve self-correction, long-horizon reasoning, generalization, or interaction efficiency.
3 Method
The method combines collaborative agents for generating and evaluating Murder Mystery scripts with two-stage training for VLM reasoning under imperfect information. It integrates multimodal clues, role-playing, reasoning chains, and agent-monitored rewards across supervised fine-tuning and GRPO reinforcement learning.
- 3 Method: The framework uses generation and evaluation agents to synthesize and assess high-quality game-script training data.Generation agents simulate interactive processes, while evaluation agents assess generated content.
- 3 Method: OutlineAgent, CharacterAgent, ClueAgent, RoleplayAgent, and QaAgent construct narratives, roles, multimodal clues, dialogues, and reasoning-oriented question–answer pairs.CriticAgent evaluates plot complexity, character development, difficulty, and logical rationality for refinement.
- 3 Method: Murder Mystery evaluation supplies character backgrounds, public textual clues, image-based clues, and dialogue history for role fidelity, deception detection, and multimodal reasoning.The context is represented as C = {B, I, T, D}.
- 3 Method: The resulting corpus combines context-rich interactive role-playing trajectories with structured perception and cognition QA data grounded in explicit reasoning and evidence.These components support subsequent agent-monitored enhancement of the target VLM.
- 3 Method: The two-stage strategy first applies direct fine-tuning on synthetic offline data, then uses ScoreAgent-monitored GRPO reinforcement learning to incentivize reasoning.The stages target basic role-playing and reasoning capabilities before reinforcement-based optimization.
- 3 Method: Reward design combines correctness, formatting, clue matching, and role-specific behavioral criteria for verifiable and unverifiable outputs.For unverifiable self-introductions and discussions, rewards include dialogue-oriented measures; for verifiable QA, rewards incorporate answer correctness and visual-clue matching.
4 Experiments
Experiments evaluate the framework across model scales, capability metrics, reward curves, training-data sources, and LLM-judge reliability. Results show consistent gains from the full training strategy, with stronger multimodal reasoning at 7B scale and complementary human-annotated and synthetic data.
- Experimental setup: The framework is evaluated on Qwen2.5-VL-3B-Instruct and Qwen2.5-VL-7B-Instruct using seven capability metrics.The metrics cover reasoning and analysis, role-playing and decision-making, and related perception capabilities.
- Reward analysis: Reward improvements occur across all subtasks, while verifiable tasks reach higher, more stable rewards than subjective role-playing tasks.Self-Introduction and Discussion show lower plateaus and higher variance, reflecting the subjectivity of dialogue behaviors.
- Main results: The full model consistently outperforms corresponding open-source baselines and ablations on both evaluated backbone scales.It is substantially stronger than Gemma3-27B-it and Mistral-small3.1-24B on Multi-hop Multimodal Reasoning and Decision-Making.
- Main results: Scaling from 3B to 7B yields consistent gains across most metrics, with the 7B full model achieving the best overall MMR score.The larger model also improves perception and decision-making performance and is competitive with proprietary VLMs on perception tasks.
- Ablation study: Removing SFT, RL, or image-clue matching rewards degrades performance in their associated reasoning, perception, or decision-making capabilities.RL removal particularly reduces multi-hop reasoning and case analysis, while image-clue matching affects MIU and MMR.
- Data ablation: Synthetic data alone improves multimodal reasoning and decision-making, but combining synthetic and human-annotated data produces the strongest overall performance.The evaluation datasets are human-annotated, and the combined setting benefits from both grounding and interaction diversity.
- Evaluation reliability: Three independent LLM judges show moderate-to-substantial agreement, and aggregated scores align strongly with human evaluations.Cohen’s κ is 0.58 for CMD and 0.47 for RP; score deviations are often small for both tasks.
5 Conclusion
The paper presents a collaborative multi-agent framework and two-stage fine-tuning strategy for VLM reasoning and role-playing in Murder Mystery scenarios. The approach improves multimodal reasoning, role-playing, and deception detection, achieving state-of-the-art performance among open-source systems and competitive results against proprietary models.
- Conclusion: The framework combines specialized-agent data synthesis with supervised fine-tuning and reinforcement learning for complex Murder Mystery reasoning.It generates logically consistent scripts and multimodal data for reasoning and role-playing tasks.
- Conclusion: The resulting models significantly improve multimodal reasoning, role-playing, and deception detection.The reported gains target complex reasoning and role-playing under imperfect information.
- Conclusion: The models achieve state-of-the-art performance among open-source systems and competitive results against proprietary models on decision-making and multi-hop reasoning.These outcomes are reported for metrics including decision-making and multi-hop reasoning.
6 Limitations
The framework remains only partially automated and is evaluated in a simulated setting that simplifies more dynamic, unstructured real-world interactions. Future work should broaden testing and clarify ethical guidance.
- Human verification is still required for image–clue alignment in WhodunitBench, so the pipeline is not fully automated.
- The simulated murder mystery environment simplifies real-world domains such as judicial argumentation and business negotiation.
- Future work should test more realistic domains and establish clearer ethical guidelines for responsible development and application.
A Agent Prompt Settings
The appendix specifies specialized prompts for constructing, evaluating, and role-playing murder mystery scripts, then generating layered questions and multimodal reasoning supervision.
- A Agent Prompt Settings: The pipeline uses specialized agent prompts to control behavior, output formats, and inter-agent interactions.
- A.1 OutlineAgent: OutlineAgent creates the narrative framework, including character summaries, the crime timeline, motives, and secrets.
- A.2 CharacterAgent: CharacterAgent expands each character with distinct traits, motivations, secrets, and detailed events aligned with the outline.
- A.3 CriticAgent: CriticAgent evaluates coherence, plausibility, narrative structure, plot complexity, character development, difficulty, and logical rationality.
- A.4 ClueAgent: ClueAgent generates public multimodal clues that support deduction without explicitly revealing the murderer.
- A.5 QaAgent: QaAgent aggregates scripts and textual or image clues into a multi-hop pool for layered question-answer generation.
- A.5 QaAgent: Its questions span long-script comprehension, text-rich and media-rich multimodal understanding, and multi-hop multimodal inference with reasoning traces and evidence linkage.
- A.6 RoleplayAgent: RoleplayAgent simulates murderer and innocent players through prompts for self-introductions, questioning, and responses grounded in outlines, scripts, and clues.
B Synthetic data
The framework produces a synthetic Murder Mystery dataset combining scripts, long-context questions, multimodal questions, and distributions of roles and reasoning complexity.
- The pipeline generated 34 Murder Mystery scripts, 1,060 long-script QA pairs, and 2,725 multimodal QA pairs.
- The multimodal subset contains 1,249 text-rich QA pairs and 1,476 media-rich QA pairs.
- Figure 6 summarizes perception QA distributions, the number of roles per script, and reasoning-step distributions for cognition assessment.
- Synthetic multimodal clue pools and role scripts provide examples of the generated training materials.
C LLM as Judge
The appendix documents LLM-based script evaluation and the prompts used to generate outlines, characters, clues, reasoning chains, questions, and role-play interactions. Together, these components specify how the synthetic game data is constructed and assessed.
- C. LLM as Judge: Deepseek-V3 automatically scores unverifiable self-introductions and discussions across multiple dimensions to produce reward signals.
- OutlineAgent: OutlineAgent establishes character backgrounds and a logically ordered crime-day timeline for subsequent script construction.
- CharacterAgent: CharacterAgent elaborates characters and crime-day events from the outline while excluding irrelevant characters.
- CriticAgent: CriticAgent evaluates script quality using plot complexity, character development, difficulty, and logical rationality criteria.
- ClueAgent: ClueAgent generates crime-scene and background clues intended to aid reasoning without directly identifying the murderer.
- QaAgent: QaAgent separates direct clues from indirect clues and requires step-by-step chains with sources for direct evidence.
- QaAgent: Its question prompts cover long-script comprehension, text-rich images, media-rich images, and multimodal question answering.
- RoleplayAgent: Roleplay prompts condition innocent players to introduce themselves, disclose relevant knowledge, question others, and infer the murderer, method, and motive.