Source-linked AI summary
Counterfactual Critic Multi-Agent Training for Scene Graph Generation
Long Chen, Hanwang Zhang, Jun Xiao, Xiangnan He, Shiliang Pu, Shih-Fu Chang
TL;DR
Existing SGG methods use context but train with XE objectives that penalize node errors independently, mismatching graph-level coherence. CMAT formulates objects as cooperative agents, optimizes graph-level metrics with policy gradients, and uses a counterfactual baseline for agent-specific rewards. On Visual Genome, it achieves state-of-the-art performance across three standard tasks, while its reported limitation concerns confidence-based false positives.
Problem
XE-based SGG objectives penalize errors at nodes independently rather than enforcing graph-level contextual consistency.
Method
CMAT formulates objects as cooperative agents, uses graph-level metrics as policy-gradient rewards, and applies a counterfactual baseline to isolate individual contributions.
Results
CMAT achieves state-of-the-art performance on three standard Visual Genome tasks with consistent improvements across extensive ablations.
Takeaways & Limitations
CMAT provides a training paradigm that combines graph-coherent rewards with local-sensitive credit assignment for scene graph generation.
Takeaways & Limitations
The evaluation notes that detecting more reasonable false positives with high confidence can worsen results.
Abstract
from arXiv · showhide
Scene graphs -- objects as nodes and visual relationships as edges -- describe the whereabouts and interactions of the things and stuff in an image for comprehensive scene understanding. To generate coherent scene graphs, almost all existing methods exploit the fruitful visual context by modeling message passing among objects, fitting the dynamic nature of reasoning with visual context, eg, "person" on "bike" can help to determine the relationship "ride", which in turn contributes to the category confidence of the two objects. However, we argue that the scene dynamics is not properly learned by using the prevailing cross-entropy based supervised learning paradigm, which is not sensitive to graph inconsistency: errors at the hub or non-hub nodes are unfortunately penalized equally. To this end, we propose a Counterfactual critic Multi-Agent Training (CMAT) approach to resolve the mismatch. CMAT is a multi-agent policy gradient method that frames objects as cooperative agents, and then directly maximizes a graph-level metric as the reward. In particular, to assign the reward properly to each agent, CMAT uses a counterfactual baseline that disentangles the agent-specific reward by fixing the dynamics of other agents. Extensive validations on the challenging Visual Genome benchmark show that CMAT achieves a state-of-the-art by significant performance gains under various settings and metrics.
1. Introduction
Scene graphs combine objects and visual relationships to represent coherent visual knowledge, but XE-based SGG training penalizes node errors independently rather than according to graph-level impact. CMAT addresses this mismatch with cooperative object agents, graph-level rewards, counterfactual credit assignment, and disentangled visual-context communication, achieving state-of-the-art results on Visual Genome.
- Scene graphs represent objects as nodes and visual relationships as edges, supporting high-level tasks including image captioning, visual reasoning, and VQA.
- Existing SGG methods use visual-context message passing but optimize sums of independent XE losses for objects and relationships.Context can help infer object categories and relationships, yet the XE objective treats their errors independently.
- Graph-coherent training should penalize errors according to their effects on the whole graph, while remaining sensitive to changes in individual nodes.Hub-node mistakes can affect more nodes and edges than non-hub mistakes, although XE penalizes both equally.
- CMAT frames objects as cooperative agents that communicate through pairwise visual features and predict object labels while maximizing scene-graph quality.The method retains visual context through agent communication and uses object classification as the agents’ action space.
- CMAT combines graph-level rewards with a counterfactual baseline that isolates each agent’s contribution by varying its action while fixing other agents.The graph-level objective can use Recall@K or SPICE, while counterfactual credit assignment supplies local-sensitive training signals.
- CMAT achieves state-of-the-art performance on three standard Visual Genome tasks with consistent improvements across extensive ablations.
2. Related Work
Earlier SGG methods either predict objects and relationships independently or use context-aware message passing trained with XE. CMAT instead formulates SGG as cooperative multi-agent decision-making with a graph-level objective, while separating context communication from relationship prediction.
- Early SGG models predict object and relationship classes independently, overlooking visual context that can benefit both predictions.
- CMAT’s agent communication is separated from relationship prediction, allowing context modeling to be distinguished from the relationship-based critic.
- Recent joint-inference methods use message passing to encode context, but conventional XE training does not enforce graph-level contextual consistency.
- CMAT formulates SGG as a cooperative multi-agent decision problem whose graph-level objective matches the contextual structure of scene graphs.CMAT operates with many agents and a large object-category action space.
3. Approach
CMAT represents scene graphs as coordinated object and relationship predictions, using agent communication to encode visual context and graph-level training objectives to promote coherent outputs.
- Scene Graph Generation detects a coherent configuration of object nodes and visual-relationship edges.
- CMAT organizes the approach around object proposals, agent communication, and visual relationship detection.
- Agent Communication: Each detected object acts as an agent that communicates for T rounds through extract, message, and update modules.
- Visual Relationship Detection: The communication model separates visual-context modeling from relationship prediction, allowing the relationship model to act as a critic guiding graph-coherent training.
- Training Objective: CMAT replaces independent XE penalties with graph-level rewards such as Recall@K or SPICE and optimizes them through multi-agent policy gradients.
- Training Objective: A counterfactual baseline estimates an agent's contribution by varying that agent's action while fixing the other agents' actions.
4. Experiments
CMAT is evaluated on Visual Genome across standard scene graph generation tasks, ablations, and qualitative comparisons. Results show state-of-the-art performance, benefits from counterfactual baselines and graph-level rewards, and limitations from incomplete annotations and noisy detections.
- Experimental Settings: Visual Genome uses 150 object categories and 50 predicate classes, with evaluations on PredCls, SGCls, and SGDet.The benchmark split contains 70% training images and 30% test images, averaging 11.5 objects and 6.2 relationships per image.
- Ablative Studies: Recall@K and SPICE rewards both improve over supervised XE initialization, while Recall@K performs slightly better because VG annotations are incomplete.CMAT therefore uses Recall@K as its training reward in the remaining experiments.
- Ablative Studies: The counterfactual baseline consistently improves supervised initialization and outperforms Moving Average and Self-Critical baselines.Moving Average and Self-Critical baselines improve performance only slightly or can worsen it because they pool rewards globally rather than providing local-sensitive signals.
- Ablative Studies: Performance trends continuously improve as communication steps increase from 2 to 5, with experiments capped at five steps by GPU memory limits.CMAT attributes its avoidance of early saturation to discarding widely used relationship nodes in the agent communication model.
- Experimental Settings: CMAT compares against state-of-the-art scene graph models under graph-constrained and unconstrained evaluation settings.Table 1 reports performance percentages on Visual Genome, including R@50 and R@100 means when R@20 is unavailable.
- Comparisons with State-of-the-Arts: CMAT achieves state-of-the-art performance across all reported evaluation metrics, including absolute SGCls gains of 3.4% with and 4.3% without graph constraints.SGDet gains are less significant, potentially because detected bounding boxes are imperfect and noisy.
- Comparisons with State-of-the-Arts: Qualitative results show fewer important hub-node mistakes, but CMAT detects more unlabeled false-positive objects and relationships under incomplete VG annotation.Because Recall@K ranks labeled triplet confidence, reasonable unlabeled predictions can worsen the evaluation result.
5. Conclusions
CMAT addresses the graph-incoherence of XE-based scene graph training by combining graph-level rewards with object-specific contribution signals. The method is validated through comparative and ablative experiments, with future work targeting improved metrics and downstream applications.
- Contributions: CMAT formulates scene graph generation as a cooperative multi-agent task optimized with graph-level metrics as training rewards.Objects act as agents, and graph quality guides learning directly.
- Contributions: CMAT disentangles each object's individual contribution to provide a more focused training signal.This complements the graph-level objective with local-sensitive credit assignment.
- Validation: CMAT is evaluated through extensive comparative and ablative experiments.
- Future Work: Future work will develop more effective graph-level metrics and apply CMAT to VQA, dialog, and captioning.
Appendix
The supplementary appendix documents simplified functions, convergence, gradient derivations, and additional qualitative comparisons. It also states that CMAT converges to a locally optimal policy under the provided proof.
- Appendix Contents: Section A details simplified functions used in agent communication and visual relationship detection.
- Appendix Contents: Section B provides a convergence proof guaranteeing that CMAT can converge to a locally optimal policy.
- Appendix Contents: Section C derives the gradient expression associated with Eq. (6).
- Appendix Contents: Section D presents additional qualitative comparisons between CMAT and MOTIFS in the SGDet setting.
A. Details of Some Simplified Functions
This appendix section supplies details for omitted functions used in the extraction module's equations.
- Function Details: The section expands omitted functions appearing in Eq. (1), Eq. (2), Eq. (3), and Eq. (4).
- Function Details: These omitted functions belong to the paper's agent communication and feature-extraction computations.
- Function Details: The section serves as supplementary implementation detail rather than introducing a new training objective.
A.1. Fs and Fe in Extract Module
The extract module represents object classes and pairwise interactions with learned embeddings, messages, attention, and visual features. Predicate prediction additionally uses transformed union-box appearance and spatial features.
- Class Features: Object class confidence is represented in R^|C|, while class labels use learned embeddings and soft-weighted embeddings based on prediction probabilities.
- Message Features: The module combines unary messages, pairwise messages, pairwise features, attention weights, and learnable mapping matrices.
- Relationship Prediction: The visual relationship label r_ij is selected by maximizing the predicate probability p_ij(r).The relationship model uses transformation matrices and head-tail-specific bias terms in the predicate computation.
- Predicate Features: Predicate visual features are built from RoIAlign-pooled union-box features and a learned spatial-position feature map.The spatial map models subject and object geometry with separate binary channels.
- Feature Fusion: The fusion operation combines transformed inputs through ReLU and a squared-difference term.
B. Proof of the Convergence of CMAT
The convergence proof establishes CMAT's expected-gradient form and shows that its counterfactual baseline leaves the expected gradient unchanged under standard differentiability and timescale assumptions.
- CMAT's expected gradient is formulated using the joint policy of all agents and a counterfactual baseline.
- The counterfactual baseline's expected contribution is zero, so it does not change the expected gradient.
- The joint policy is written as a product of independent agent policies for the policy-gradient derivation.
- Convergence to a local maximum of expected return requires a differentiable policy and sufficiently slow policy-update timescales.
C. Derivation of Eq. (6)
The derivation applies policy-gradient reasoning to cooperative agents whose joint state and action determine the reward, then specializes the result to CMAT's communicated representations.
- The policy-gradient derivation defines each agent's action sequence and value function as its expected future reward.
- For cooperative multi-agent learning, the state-action function estimates reward from the states and actions of all agents.
- CMAT samples actions after T-round agent communication, with each agent's policy and state represented by its communicated variables.
- The resulting cooperative gradient sums the policy-gradient contributions across agents.
D. More Qualitative Results
Qualitative SGDet comparisons show CMAT making fewer errors at important hub nodes, while incomplete annotation accounts for some of its detected false positives.
- CMAT rarely mistakes important hub nodes such as the “surfboard” or “laptop” in rows where it outperforms MOTIFS.
- CMAT's remaining mistakes are attributed to incomplete annotation, including more detected false-positive objects and relationships than MOTIFS.
- The qualitative comparisons use green edges for true positives, red edges for false negatives, and blue edges for false positives at R@20.