PaperlayerView exact revision on arXiv

Source-linked AI summary

GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning

Jusheng Zhang, Yijia Fan, Wenjun Lin, Ruiqi Chen, Haoyi Jiang, Wenhao Chai, Jian Wang, Keze Wang

arXiv:2505.23399v2cs.AI

TL;DR

Complex visual reasoning remains limited by VLMs’ reliance on single models or simple ensembles, especially under multi-step reasoning and visual ambiguity. GAM-Agent uses uncertainty-aware, game-theoretic collaboration among specialized agents and a critical verifier, consistently improving performance across four challenging benchmarks and diverse VLM backbones.

  • Problem

    Multi-agent collaboration remains largely untapped for VLMs, while single models and simple ensembles struggle with multi-step reasoning and visual ambiguity.

    Source: §1 · p. 1

  • Method

    GAM-Agent models visual reasoning as a non-zero-sum game among specialized agents and a critical verifier, using uncertainty-aware collaboration and iterative debate to reach consensus.

    Source: §1 · p. 2, §2.5 · p. 5, §5 · p. 9

  • Results

    GAM-Agent consistently improves performance across four challenging benchmarks and a wide range of VLM backbones, with gains of +6.61% on InternVL3-14B/MMBench_V11_Test.

    Source: §3.1 · pp. 6–7, §5 · p. 10

  • Takeaways & Limitations

    GAM-Agent provides a modular framework for more robust, interpretable, and accurate multi-agent multimodal reasoning across lightweight and strong VLMs.

    Source: §5 · pp. 9–10

Abstract

from arXiv · show

We propose GAM-Agent, a game-theoretic multi-agent framework for enhancing vision-language reasoning. Unlike prior single-agent or monolithic models, GAM-Agent formulates the reasoning process as a non-zero-sum game between base agents--each specializing in visual perception subtasks--and a critical agent that verifies logic consistency and factual correctness. Agents communicate via structured claims, evidence, and uncertainty estimates. The framework introduces an uncertainty-aware controller to dynamically adjust agent collaboration, triggering multi-round debates when disagreement or ambiguity is detected. This process yields more robust and interpretable predictions. Experiments on four challenging benchmarks--MMMU, MMBench, MVBench, and V*Bench--demonstrate that GAM-Agent significantly improves performance across various VLM backbones. Notably, GAM-Agent boosts the accuracy of small-to-mid scale models (e.g., Qwen2.5-VL-7B, InternVL3-14B) by 5--6\%, and still enhances strong models like GPT-4o by up to 2--3\%. Our approach is modular, scalable, and generalizable, offering a path toward reliable and explainable multi-agent multimodal reasoning.

1 Introduction

GAM-Agent addresses the limited use of multi-agent collaboration in complex visual reasoning by modeling specialized visual agents and critics as an uncertainty-aware non-zero-sum game. Its iterative debate improves consensus and yields gains across multiple visual reasoning benchmarks.

Motivation

  • Existing VLMs often rely on single models, ensembles, averaging, or voting, limiting multi-step reasoning and handling of visual ambiguity in complex tasks.The introduction motivates strategic game-theoretic collaboration because current methods lack visual reasoning-based agent interactions.Source: §1 · p. 1

Framework

  • GAM-Agent models collaboration as a non-zero-sum game in which agents share uncertainty and progressively debate toward consensus.The framework uses uncertainty estimates to guide strategic interaction and reduce ambiguity during reasoning.Source: §1 · p. 2
  • Base Agents generate diverse visual interpretations and evidence, while Critical Agents assess factual accuracy, logical coherence, and completeness.A Claim Parser structures responses, Evidence Mapping grounds claims in image regions, and Uncertainty Quantification evaluates contributions.Source: §1 · p. 2

Results

  • GAM-Agent improves accuracy by 5–6% for small-to-mid scale VLMs and by up to 2–3% for top-tier models such as GPT-4o across MMMU, MMBench, MVBench, and V*Bench.These results demonstrate gains across multiple complex visual reasoning benchmarks and model scales.Source: §1 · p. 2

Framework

  • A Debate Controller initiates iterative refinement when disagreement or uncertainty is high, dynamically weighting contributions until consensus or termination criteria are reached.Base Agents refine arguments while Critical Agents provide evaluations throughout the uncertainty-guided loop.Source: §1 · p. 2

2 Methodology

GAM-Agent models visual reasoning as a structured six-tuple combining specialized agents, uncertainty estimation, evidence-grounded claims, adaptive conflict detection, and iterative debate. Its controller escalates collaboration when uncertainty or disagreement warrants deeper verification, producing progressively more reliable and interpretable answers.

2 Methodology

  • GAM-Agent formalizes reasoning as a six-tuple integrating expert agents, capability mappings, uncertainty functions, evidence localization, claim parsing, and a debate module.The debate module aligns conflicts and consensus to produce the final response.Source: §2 · p. 2

2.1 Base and Critical Agents

  • Base Agents generate specialized visual interpretations, while Critical Agents verify factual accuracy, logical coherence, and completeness through iterative game-theoretic interaction.Examples include object recognition, scene description, OCR, fact-checking, completeness-checking, and logic-checking agents.Source: §2.1 · p. 3

2.2 Uncertainty Quantification Function (Φ)

  • GAM-Agent quantifies uncertainty from token-generation behavior when probabilities are available, using entropy and top-token probability differences as complementary signals.Parameters α and β balance these signals, shifting uncertainty assessment from output evaluation toward the generation process.Source: §2.2 · pp. 3–4

2.2.1 Semantic-Marker Based Uncertainty (Φisem)

  • When generation probabilities are unavailable, semantic markers estimate uncertainty; a priority rule selects generation-based or semantic uncertainty for response weighting, conflict detection, and debate.The semantic score uses marker intensity, frequency, response length, and sigmoid normalization.Source: §2.2.1 · p. 4

2.3 Initial Integration and Conflict Detection

  • Initial integration weights agents according to uncertainty and triggers debate when weighted system uncertainty or inter-expert conflict exceeds its threshold.Integration balances high-confidence opinions with diverse perspectives while assessing claim consistency, evidence interpretation, and agent logic.Source: §2.3 · p. 4

2.4 Evidence Mapping (M) and Claim Parsing (P)

  • Claim parsing and evidence mapping convert responses into structured claims linked to textual evidence and visual regions with confidence scores.This provides traceability, localizes conflicts precisely, and supports fine-grained debate focused on visual evidence.Source: §2.4 · pp. 4–5

2.5 Iterative Debate Process: Dynamics, Termination, and Consensus

  • Iterative debate focuses on low-confidence or disputed claims, updates expert arguments and weights, integrates successive responses, and terminates at sufficient certainty, iteration limits, or stagnation.The process converges toward a high-confidence, low-uncertainty final answer while preserving prior consensus and resolving current disputes.Source: §2.5 · p. 5

3 Experiments

Across image, video, and guided visual-search benchmarks, GAM-Agent consistently improves base VLM accuracy and outperforms comparative multi-agent frameworks. Ablations and dynamics analyses attribute these gains to uncertainty-aware coordination, iterative debate, and adaptive expert weighting.

3.1 Image and Video Understanding Benchmarks

  • GAM-Agent consistently achieves the highest overall accuracy across evaluated base models and image/video benchmarks.Gains reach +5.8% to +6.7% on MMBench_V11_Test and MVBench_Test for smaller models, while larger models gain +2.6% to +4.1%.Source: Table 1 · p. 6, §3.1 · pp. 6–7

3.2 Guided Visual Search Capabilities

  • +5.25% overall accuracy raises Qwen2.5VL-7B to 66.51% on V*Bench, while GAM-Agent also reaches 70.33% for InternVL3-14B and 78.32% for GPT-4o-0513.GAM-Agent consistently outperforms the other multi-agent frameworks and approaches or surpasses SEAL’s 75.39% performance with GPT-4o-0513.Source: Table 2 · p. 7, §3.2 · pp. 7–8

3.4 Study on Maximum Debate Rounds

  • 86.53% accuracy at three maximum debate rounds nearly saturates performance, while average actual rounds increase from 0.00 at zero rounds to 3.27 at nine.Accuracy rises from 82.97% at zero rounds to 86.21%, 86.35%, and 86.53% at one, two, and three rounds, supporting adaptive early termination.Source: Figure 2 · p. 8, §3.4 · p. 8

3.3 Ablation Experiments

  • 88.80% accuracy with 1.76 average debate rounds makes full GAM-Agent the strongest and most efficient ablation configuration.Removing iterative debate reduces accuracy by 4.28%, removing claim-evidence grounding reduces it by 4.72%, and removing uncertainty or dynamic weights increases debate length and cost.Source: §3.3 · p. 8

3.5 Expert Weight Trajectory and System Uncertainty Dynamics in Logical Reasoning Tasks

  • System uncertainty decreases across three debate rounds, including 0.64→0.54 for Qwen2.5VL-72B and 0.66→0.55 for InternVL3-78B.Expert weights shift dynamically, with OCR typically increasing from 0.35→0.46 for Qwen2.5VL-72B as relational and scene experts adjust downward.Source: Figure 4 · p. 9, §3.5 · p. 9

4 Related Works

Prior multi-agent systems improve reasoning through collaboration, but vision-language applications remain underdeveloped and often lack strategic visual interactions. GAM-Agent addresses these limitations by combining uncertainty-aware assessment with game-theoretic collaboration and intermediate reasoning.

Multi-agent Systems

  • Multi-agent systems improve complex reasoning, code generation, and knowledge integration, yet VLM applications remain largely untapped and existing debates lack visual-reasoning strategies.Prior work includes multi-agent debates that improve factuality and reasoning in LLMs.Source: §4 · p. 9

Uncertainty Quantification

  • Uncertainty quantification is essential for reliable visual reasoning because occlusions, poor lighting, and both visual and linguistic uncertainty complicate VLM assessment.Existing approaches include entropy-based measures and semantic analysis.Source: §4 · p. 9

Visual Reasoning

  • Visual reasoning remains challenging for single models and basic ensembles, while GAM-Agent uses uncertainty, intermediate reasoning, and game-theoretic collaboration for deeper agent synergy.These approaches struggle with multi-step reasoning and visual ambiguities.Source: §4 · p. 9

5 Conclusion

GAM-Agent is a game-theoretic, uncertainty-aware multi-agent framework that improves the robustness, interpretability, and accuracy of vision-language reasoning, with consistent gains across diverse VLM backbones and benchmarks.

Framework

  • GAM-Agent models multimodal reasoning as a non-zero-sum game among specialized base agents and a critical verification agent.Structured claim-evidence interactions and uncertainty-aware collaboration enable ambiguity detection, multi-round debates, and dynamic strategy adjustment based on confidence and disagreement.Source: §5 · p. 9

Experiments

  • GAM-Agent consistently improves performance across four challenging benchmarks and a wide range of lightweight and strong VLM backbones.The experiments support its effectiveness across diverse model scales and multimodal evaluation settings.Source: §5 · p. 10

Future Work

  • Future directions include tighter external-tool integration, extension to dialog agents, and applications in safety-critical and explainable AI systems.These directions follow the framework’s broader potential for self-reflective and multi-agent multimodal reasoning.Source: §5 · p. 10

Supplementary Material … A.1 Experiment Setup

The supplementary material documents experimental setups, analyses, and case studies supporting GAM-Agent’s claims, including a cost-performance comparison on MMbench_TEST_V11. This comparison evaluates accuracy and per-instruction costs across commercial VLMs and framework configurations using API pricing and local-GPU estimates.

Supplementary Material Overview

  • The supplementary material supports GAM-Agent’s claims through experimental setups, results, theoretical analyses, and case studies.It also examines cost-performance tradeoffs, uncertainty handling, theoretical derivations, hyperparameters, prompts, robustness, and limitations.Source: Supplementary Material Overview · p. 15
  • The supplementary material includes cost-performance analyses, uncertainty evaluations, theoretical derivations, hyperparameter studies, prompt configurations, and case analyses.These components are intended to demonstrate GAM-Agent’s robustness and limitations.Source: Supplementary Material Overview · p. 15

A.1 Experiment Setup

  • The cost-performance analysis compares Vision-Language Models and GAM-Agent configurations on MMbench_TEST_V11.The study measures “Cost per Instruction on MMbench_TEST_V11” and “Overall ACC (%)”.Source: §A.1 · p. 17
  • The comparison covers Qwen2.5VL[41], Gemini 2.5 Pro Preview[49], Gemini 2.0 Pro, GPT-4o[2], and Claude 3.7 alongside framework-derived configurations.Commercial models include Qwen2.5VL 7B and 72B via API, while framework configurations use Qwen2.5VL and InternVL3[76] models.Source: §A.1 · p. 17
  • Commercial-model costs are sourced from OpenRouter API pricing, enabling comparison of API deployment economics.The benchmarked commercial models include Qwen2.5VL[41], Gemini 2.5 Pro Preview[49], Gemini 2.0 Pro, GPT-4o[2], and Claude 3.7.Source: §A.1 · p. 17
  • Framework-configuration costs are estimated from inference time on a local NVIDIA A100 GPU using AWS p4d.24xlarge pricing.The instance costs approximately $32.77 per hour for eight A100 40GB GPUs, or about $4.10 per hour for one GPU.Source: §A.1 · p. 17

A.2 Experiment Result · B Experiment on Cost Dynamics During Multi-Round Debates · B.1 Experimental Setup

On MMbench_TEST_V11, GAM-Agent configurations combine high accuracy with substantially lower inference costs than commercial APIs. A separate MMBench_V11_Test experiment examines how GAM-Agent’s structured multi-round debate affects cumulative token costs versus naive self-debate.

A.2 Experiment Result

  • The framework also improves cost-performance for smaller models: Qwen2.5VL-7B (Ours) reaches 89.0% at $0.000085 versus its API’s 82.6% at $0.00007.InternVL3-14B (Ours) achieves 91.4% accuracy at $0.00016 per instruction, while the lower-cost Qwen2.5VL-72B configuration reaches 90.4% at $0.00011 and exceeds its API counterpart’s 88.4%.Source: §A.2 · p. 17
  • InternVL3-78B (Ours) and Qwen2.5VL-72B (Ours) surpass Gemini 2.5 Pro Preview’s 89.3% accuracy while costing approximately 3.4 and 3.75 times less, respectively.Gemini’s API cost is $0.00075 per instruction, compared with approximately $0.00022 and $0.00020 for the two GAM-Agent configurations.Source: §A.2 · pp. 17–18
  • 92.2% accuracy at approximately $0.00022 per instruction makes InternVL3-78B (Ours) the strongest reported cost-performance configuration, followed by Qwen2.5VL-72B (Ours) at 92.0% and $0.00020.Figure 5 reports cost per instruction and overall accuracy for models run with the proposed framework under local inference cost calculation.Source: Figure 5 · p. 17, §A.2 · p. 17

B Experiment on Cost Dynamics During Multi-Round Debates

  • The cost-dynamics study evaluates cumulative debate costs after 1, 2, and 3 rounds using average processed tokens per instance as a proxy for computation time and API expense.The experiment investigates whether GAM-Agent’s consensus mechanism enables earlier convergence than less structured iterative debate.Source: B Experiment on Cost Dynamics During… · p. 18, §B.1 · p. 18

B.1 Experimental Setup

  • The study uses MMBench_V11_Test with Qwen2.5VL-7B, Qwen2.5VL-72B, InternVL3-14B, and InternVL3-78B as base VLMs.These four open-source models form the base-model set for the cost comparison.Source: §B.1 · p. 18
  • GAM-Agent is compared with Base-Debate, where original model instances iteratively refine answers without structured uncertainty quantification, evidence mapping, or dynamic weighting.The GAM-Agent setup uses N = 3 expert agents and Ncrit = 3 critic agents, consistent with the other experiments.Source: §B.1 · p. 18

B.2 Experimental Results … C.2 Experimental Setup

GAM-Agent improves multi-round visual reasoning accuracy over single-agent self-debate while controlling debate cost through uncertainty- and conflict-driven termination. The extended analysis evaluates uncertainty handling across Qwen2.5VL and InternVL3 model scales using uncertainty accuracy, calibration error, and dynamic adaptability on MMBench.

B.2 Experimental Results

  • 91.90% accuracy at $0.00033 after three rounds lets GAM-Agent surpass InternVL3-78B Base-Debate’s 90.20%, with similar gains across Qwen2.5VL and InternVL3 models.GAM-Agent reaches 91.82% at $0.00045 for Qwen2.5VL-72B, 89.02% at $0.00009 for Qwen2.5VL-7B, and 90.15% at $0.00017 for InternVL3-14B, versus baseline accuracies of 90.20%, 86.50%, and 88.50%, respectively.Source: §B.2 · p. 18, Figure 6 · p. 19
  • GAM-Agent’s uncertainty- and conflict-driven structured debate balances rising multi-round costs against higher accuracy and can terminate earlier when consensus is reached.The baseline consistently achieves lower accuracy and would likely require more rounds, and therefore higher cumulative costs, to approach GAM-Agent’s performance.Source: §B.2 · pp. 18–19

C Deepening Comparative Analysis of Multi-Agent Methods: Uncertainty Handling Capability

  • The comparative analysis extends GAM-Agent evaluation across Qwen2.5VL models from 3B to 72B and InternVL3 models from 2B to 78B to examine uncertainty processing across scale and architecture.This design assesses how uncertainty-handling capabilities and framework performance vary with underlying VLM size and architecture.Source: C Deepening Comparative Analysis of… · p. 19

C.1 Comprehensive Analysis of Uncertainty Handling Capability

  • The uncertainty analysis compares multi-agent methods using Uncertainty Accuracy, Calibration Error, Dynamic Adaptability, and MMBench accuracy across the Qwen2.5VL and InternVL3 series.UA and DA are higher-is-better metrics, whereas CE is lower-is-better.Source: §C.1 · p. 19, Figure 7 · p. 20

C.2 Experimental Setup

  • The extended evaluation tests whether sophisticated uncertainty modeling in GAM-Agent yields more reliable and robust performance as model complexity and task difficulty increase.The evaluation uses MMBench for both uncertainty-handling metrics and final accuracy.Source: §C.2 · pp. 19–20
  • GAM-Agent and DMAD, DMPL, ChatEval, MAD, and DebUnc are applied to Qwen2.5VL models of 3B, 7B, 32B, and 72B and InternVL3 models of 2B, 8B, 38B, and 78B.The baseline methods are adapted to visual question answering on MMBench.Source: §C.2 · p. 19
  • UA measures correct identification of high-uncertainty instances, CE measures confidence–accuracy mismatch, and DA measures uncertainty-responsive changes in reasoning or resource allocation.UA uses thresholded system-level uncertainty, CE uses an ECE variant over confidence bins, and DA includes debate triggering, agent-weight adjustment, and correlation with deeper reasoning or confidence changes.Source: §C.2 · p. 20

C.3 Experimental Results and Analysis … D.4 Cooperative Game Objective and Dynamic Weighting Protocol

This section analyzes GAM-Agent’s uncertainty management, benchmark performance, termination mechanism, and cooperative game formulation. It connects the theoretical treatment of equilibrium existence and entropy-regularized dynamic weighting with the implemented protocol for prioritizing lower-uncertainty agents while balancing accuracy and debate cost.

C.3 Experimental Results and Analysis

  • GAM-Agent outperforms DMAD, DMPL, ChatEval, MAD, and DebUnc on UA, CE, and DA across Qwen2.5VL and InternVL3 models, reaching 0.82 UA, 0.04 CE, and 0.76 DA on Qwen2.5VL-72B.On InternVL3-78B, it achieves 0.84 UA, 0.03 CE, and 0.78 DA; Qwen2.5VL-72B also reaches 90.56% MMBench accuracy.Source: §C.3 · pp. 20–21
  • GAM-Agent’s uncertainty handling scales positively with model size: Qwen2.5VL-3B to 72B improves from 0.72 to 0.82 UA, 0.08 to 0.04 CE, and 0.65 to 0.76 DA.For InternVL3-2B to 78B, UA rises from 0.70 to 0.84, CE drops from 0.09 to 0.03, and DA improves from 0.63 to 0.78.Source: §C.3 · p. 21

C.4 Experimental Analysis

  • Setting θU,term to 0.10 raises accuracy by approximately 0.2% to 0.4% but increases debate rounds by 25% to 31%, whereas 0.20 reduces rounds by 22% to 26% while lowering accuracy by 0.9% to 1.2%.The default θU,term = 0.15 provides the best accuracy-efficiency balance, with large models showing lower sensitivity.Source: Figure 9 · p. 22, §C.4 · p. 23

C.5 Theoretical Analysis of Termination Threshold θU,term in Multi-Agent Debate

  • The termination mechanism stops debate when system uncertainty falls below θU,term, iterations reach Kmax, or consecutive-round uncertainty change falls below ϵ.The threshold defines a confidence boundary balancing premature acceptance against unnecessarily prolonged deliberation.Source: §C.5 · p. 23

D Derivation of Uncertainty Theory

  • Uncertainty decays exponentially across debate rounds, with larger models such as InternVL3-78B exhibiting faster uncertainty suppression.Source: Figure 10 · p. 24

D.1 Mathematical Formalization of the Non-Zero-Sum Game

  • The game formalization maps each agent’s input to a response and uncertainty, combines all agent outputs, and dynamically assigns nonnegative weights summing to one over the simplex.The evolving state consists of joint agent outputs and protocol-derived weights, which can change across iterative debate rounds.Source: §D.1 · pp. 24–25

D.2 Utility Function Design

  • Each agent’s utility combines weighted individual contribution, similarity-based collaboration reward, and a system-wide uncertainty penalty, allowing total utility to increase through collaboration.Higher consistency can benefit multiple agents simultaneously, while reducing weighted system uncertainty raises every agent’s utility.Source: §D.2 · pp. 25–26

D.3 Exploration of Nash Equilibrium Existence

  • Under stated assumptions on finite strategy spaces and utility functions, the proposed non-zero-sum game possesses at least one pure-strategy Nash equilibrium.A Nash equilibrium is a strategy profile where no agent can improve its utility by unilaterally changing strategy.Source: §D.3 · pp. 26–27
  • VLM pure-strategy equilibrium analysis is difficult because response spaces are discrete, vast, and non-convex, while quasi-concavity and continuity of utilities may not hold for complex outputs.Simplifying strategies to reported uncertainty or low-dimensional continuous parameters can satisfy Kakutani-style requirements, but these assumptions limit direct applicability to full VLM behavior.Source: §D.3 · pp. 27–28
  • Mixed-strategy Nash equilibria are more readily guaranteed for VLM games than pure-strategy equilibria, provided strategy spaces are compact metric spaces and utilities are continuous.Pure-strategy existence requires non-empty, compact, convex strategy spaces and suitable best-response properties, often necessitating simplified models or strong behavioral assumptions.Source: §D.3 · p. 28

D.4 Cooperative Game Objective and Dynamic Weighting Protocol

  • GAM-Agent frames collaboration as maximizing total utility, with agent weights optimized through an entropy-regularized objective that yields a Softmax-form weight distribution.The static optimal weight depends on all agent responses, uncertainties, and hyperparameters, reflecting the full cooperative optimization goal.Source: §D.4 · pp. 28–30
  • The implemented protocol updates weights each round from prior-round uncertainty, matching the entropy-regularized optimum under a simplified objective that prioritizes higher-confidence agents.With collaboration rewards and system penalties omitted from the weight calculation, the rule remains computationally efficient, theoretically grounded, and oriented toward reduced overall uncertainty.Source: §D.4 · pp. 30–31

E Theorem I: Fundamental Game Theoretic Framework of GAM-Agent · F Theorem II: Comparative Analysis with the DMAD Framework

GAM-Agent formalizes visual reasoning as an uncertainty-aware non-zero-sum game whose utilities and analytically derived weights govern collaboration. Compared with DMAD’s preset diversity and procedural information flow, GAM-Agent endogenously drives agent behavior through utility optimization.

E Theorem I: Fundamental Game Theoretic Framework of GAM-Agent

  • GAM-Agent defines a non-zero-sum game in which expert agents produce responses and uncertainty estimates, while structured parsing, evidence localization, and debate generate the final response.The formal system includes analytical capability mappings, uncertainty functions, evidence localization, claim parsing, and dynamic debate over joint agent outputs.Source: E Theorem I: Fundamental Game… · pp. 31–32
  • GAM-Agent quantifies uncertainty either from token-generation probabilities or from semantic uncertainty markers, using entropy and top-token probability differences when generation scores are available.High entropy and small top-token probability differences indicate greater hesitation; semantic markers provide an alternative when token probabilities are unavailable.Source: E Theorem I: Fundamental Game… · pp. 31–32
  • The utility function balances individual confidence, agreement with influential agents, and system-wide uncertainty, allowing agents to increase collective utility by reducing uncertainty and improving consistency.Agents are rewarded for low individual uncertainty and semantic consistency, but share a penalty based on the system’s weighted average uncertainty.Source: E Theorem I: Fundamental Game… · pp. 32–33
  • The framework analytically derives influence weights by maximizing regularized total utility, with weights decreasing with uncertainty and increasing with semantic consistency rather than following a heuristic rule.Entropy regularization prevents degenerate all-weight allocation, while the resulting solution provides a mathematically interpretable basis for agent influence.Source: E Theorem I: Fundamental Game… · p. 33
  • Theorem I’s contributions are to make uncertainty quantifiable, combine individual contribution, collaborative consistency, and system risk in one utility, and derive influence allocation endogenously from optimization.The resulting game mechanism is generated by utility principles rather than externally imposed rules.Source: E Theorem I: Fundamental Game… · p. 34

F Theorem II: Comparative Analysis with the DMAD Framework

  • DMAD uses preset reasoning methods assigned to distinct agents and propagates their outputs through multi-round historical-information updates, but it lacks an agent-level optimizable utility function.Its behavior is governed by fixed strategies and context learning rather than explicit maximization of a mathematical utility.Source: F Theorem II: Comparative Analysis… · p. 34
  • Unlike GAM-Agent’s uncertainty-driven specialization, DMAD’s diversity is externally enforced through distinct prompting strategies, and its cognitive-fixation objective is not directly represented as an optimizable mathematical variable.DMAD’s claimed effect is validated indirectly through benchmark improvements rather than predicted by an explicit objective function.Source: F Theorem II: Comparative Analysis… · p. 34
  • Theorem II concludes that DMAD provides clear procedural execution and information flow, whereas GAM-Agent uses explicit utility optimization, uncertainty-triggered debate, dynamic weight adjustment, and final weighted integration.DMAD’s mathematical description supports its heuristic process but lacks GAM-Agent’s agent-centric game model for endogenously driving behavior and diversity.Source: F Theorem II: Comparative Analysis… · p. 35

G Theorem III: Differences in Mathematical Construction between GAM-Agent and Traditional Multi-Agent Methods

GAM-Agent differs from traditional multi-agent methods by mathematically integrating uncertainty into utility modeling, mechanism derivation, and collaborative optimization. Traditional aggregation and rule-based debate methods instead rely on fixed operations or externally scripted interactions without end-to-end uncertainty-driven optimization.

Traditional Aggregation

  • Traditional aggregation methods independently combine agent outputs through majority voting, averaging, or fixed weighted averages.These operations lack agent interaction, explicit confidence or uncertainty modeling, and dynamic influence adjustment.Source: G Theorem III: Differences in… · p. 35

Rule-Based Debate

  • Rule-based debate frameworks determine agent actions and interaction through preset roles, turn-taking rules, and scripted policies.Who speaks or influences the outcome is hard-coded rather than derived from current internal states, quantified uncertainty, and collaborative goals.Source: G Theorem III: Differences in… · p. 35

Uncertainty Integration

  • GAM-Agent forms an end-to-end mathematical chain from uncertainty quantification through utility calculation to mechanism design.The framework directly incorporates quantified uncertainty into agent utility and analytically derives optimal weight allocation by maximizing that utility.Source: G Theorem III: Differences in… · p. 36

Uncertainty Comparison

  • Traditional methods generally lack a unified mathematical mechanism that converts confidence signals into precise effects on debate dynamics or final consensus.Aggregation methods omit uncertainty entirely, while rule-based debates typically express confidence qualitatively or heuristically.Source: G Theorem III: Differences in… · p. 36

D.4.2 Mathematical Basis of Interaction Mechanisms and Influence Allocation: GAM-Agent: Influence weights w∗

GAM-Agent derives influence weights and collaborative objectives from explicit uncertainty-aware utility maximization. This yields continuous, dynamic, interpretable coordination mechanisms that differ from heuristic voting and rule-based debate methods.

D.4.2 Mathematical Basis of Interaction Mechanisms and Influence Allocation

  • GAM-Agent derives influence weights w∗_k by analytically maximizing the system’s overall regularized utility.The weights are obtained from an explicit optimization problem rather than preset rules.Source: §D.4.2 · p. 36
  • The weights w∗_k are continuous and dynamic, reflecting agent value through uncertainty and semantic consistency.They change with Score_k, which incorporates U_k and Sim terms.Source: §D.4.2 · p. 36
  • Unlike GAM-Agent, traditional voting and role-playing debates use equal, binary, or rule-conferred influence rather than continuous, dynamic weights derived from utility maximization.Their implicit objectives generally omit explicit uncertainty, inter-agent consistency quality, and overall system-state evaluation.Source: §D.4.2 · pp. 36–37

D.4.3 Mathematical Form and Explicitness of Collaborative Optimization Objectives

  • GAM-Agent explicitly maximizes a regularized utility whose terms quantify individual certainty, mutual consistency, and system uncertainty.Parameters λ and γ define trade-offs among these dimensions and connect utility maximization to high-quality consensus.Source: §D.4.2 · p. 37
  • The framework’s uncertainty variables and analytically derived interaction mechanisms make its coordination more flexible, adaptive, and interpretable.Its components trace back to an explicit uncertainty-based mathematical model rather than a procedurally defined framework.Source: §D.4.2 · p. 37

H Hyperparameter Ablation … K.1 Successful Case

This section details GAM-Agent’s hyperparameter sensitivity, experimental configurations, benchmark settings, and prompt designs, including specialized base and critic roles. It also examines collaboration through ablations and case studies across image, video, and expert-prompt evaluations.

H Hyperparameter Ablation

  • 88.98% accuracy is achieved with jointly lowered debate thresholds (θU = 0.35, θC = 0.45), but the default values (0.45, 0.55) provide a better overall trade-off between accuracy, debate frequency, and cost.Table 3 evaluates GAM-Agent (InternVL3-14B, N = 3, Kmax = 3) on MMBench_TEST_V11; the default configuration corresponds to 88.80% accuracy.Source: Table 3 · p. 38, H Hyperparameter Ablation · p. 39
  • 88.80% accuracy is obtained with βweight = 1.5 and αΦ = 0.5, βΦ = 0.5, whereas extreme uncertainty weighting or generation-process parameter settings reduce accuracy.βweight values of 0.5, 2.0, and 3.0 yield 88.15%, 88.71%, and 88.45%, respectively; equal αΦ and βΦ weighting performs best among the tested generation-process settings.Source: H Hyperparameter Ablation · pp. 38–39

I Hyperparameter Setting

  • Across reported GAM-Agent experiments, common settings typically used 3 base expert agents, 3 critical expert agents, and βweight = 1.5.Φigen+ parameters were used when token-level logprobs were accessible, while Φisem parameters were used when logprobs were unavailable or as a fallback.Source: I Hyperparameter Setting · pp. 39–40

I.1 Settings for Image Understanding Experiments (MMBench and MMMU)

  • On MMBench (TEST_EN), GAM-Agent(Qwen2.5VL-72B) achieved 90.86% accuracy, GAM-Agent(InternVL3-14B) achieved 88.80%, and GAM-Agent(GPT-4o-0513) achieved 86.53%.Source: §I.1.1 · p. 40, §I.1.2 · p. 40

I.2 Settings for Video Understanding Experiments (MVBench)

  • 78.47% overall average is achieved by GAM-Agent (InternVL3-38B) on MVBench, while GAM-Agent(GPT-4o-0513) achieves 70.58% under model-specific video settings.Video experiments use N = 3, Ncrit = 3 when debate is triggered, Kmax = 3, and higher ambiguity-adjusted thresholds than the image experiments.Source: §I.2.1 · p. 41, §I.2.2 · p. 41

J Prompt Setting Statement

  • GAM-Agent’s prompts assign base experts to object recognition, scene description, and OCR, with critique and revision templates, while the aggregator resolves contradictions into a unified answer.The benchmark setup uses three base experts for initial evidence-backed analyses and argumentation and three critic experts for debate and verification; the ablation baseline reaches 88.80% on MMBench_TEST_V11.Source: §I.4 · p. 42, §J.1 · p. 43, §J.2.1 · pp. 44–45

J.2.1 Base Expert Prompts

  • Base OCR prompts require checking transcription accuracy, completeness, and text location against visual evidence before revising the analysis.The revision template uses the critique and original response to guide OCR correction.Source: §J.2.1 · p. 45

J.2.2 Critic Expert Prompts

  • Critic prompts instruct fact checkers to assess claims only against visual evidence, identify inaccuracies, suggest corrections, and report confidence.The fact-checking role evaluates factual accuracy against the image content and states confidence from 0–100%.Source: §J.2.2 · p. 45

K Case Analysis

  • The case studies use three analysis experts and three critique experts to demonstrate collaborative image analysis through successful and unsuccessful examples.Each analysis response includes an uncertainty score, while critique experts provide evaluative feedback; key excerpts are highlighted for clarity.Source: K Case Analysis · p. 46

K.1 Successful Case

  • Defined expert roles, uncertainty estimates, and critique feedback jointly support factual accuracy, completeness, logical consistency, interpretability, and stability across diverse inputs.Analysis experts specialize in object recognition, scene description, and text/OCR, while critic experts verify facts, completeness, and logic.Source: §K.1 · p. 46

K.2 Unsuccessful Case

The unsuccessful cases expose failures in semantic grounding, expert selection, critique, and aggregation: confident experts and critics can jointly produce incorrect judgments when questions are subjective or visual cues are context-sensitive.

Semantic Grounding

  • Subjective or ambiguous color questions expose missing shared semantic grounding, as experts equated saturation or apparent skin tone with colorfulness while overlooking diversity, lighting, shadows, and texture.The framework’s experts can emphasize visually salient but semantically incomplete cues, producing misaligned interpretations of the question.Source: §K.2 · pp. 46–47

Expert Selection

  • The framework lacks adaptive expert selection: irrelevant Text/OCR analyses were activated without textual content, while domain-focused experts sometimes failed to address the query’s semantic intent.This mismatch can diminish pertinent analysis and yield incomplete or misleading conclusions.Source: §K.2 · pp. 46–47

Critique and Aggregation

  • Critic modules failed to detect confidently wrong semantic interpretations, allowing misapplied concepts and deviations from the core question to pass through final aggregation.The Fact Checker, Completeness Checker, and Logic Checker did not effectively intervene in the colorfulness or rhinoceros cases.Source: §K.2 · pp. 46–47

Remedies

  • The authors recommend task-aligned expert selection, precise or illumination-invariant color analysis, and aggregation sensitive to contextual, lighting, and texture variations.Higher-level semantic validation, external verification, or feature-level consistency checks could also reduce coordinated error propagation and bias amplification.Source: §K.2 · pp. 46–47

Unsuccessful Cases

  • Tables 8 and 9 show incorrect final judgments despite confident expert analyses, revealing that high internal confidence does not ensure prediction accuracy.In the colorfulness case, all three analysis experts reported low uncertainty before the aggregation was wrong; the rhinoceros case likewise produced an incorrect same-color conclusion.Source: §K.2 · pp. 46–47, Table 8 · p. 52, Table 9 · p. 53

Related papers

Loading 2505.23399v2…