PaperlayerView exact revision on arXiv

Source-linked AI summary

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

arXiv:2609.35942v2cs.CL

TL;DR

Complex VQA requires models to align visual, linguistic, and background knowledge, while caption-based representations can include irrelevant detail. VisKG trains models to use question-specific KGs with GDPO and positive and negative rationale supervision; it performs comparably to or better than baselines across several benchmark areas with fewer tokens than caption-based representations.

  • Problem

    VQA reasoning must align visual, linguistic, and background knowledge, but these inputs lack a shared representation for coherent reasoning.

    Source: §1 · p. 1

  • Method

    VisKG trains VLMs to generate question-specific KG representations and uses GDPO for RL post-training, with SFT on positive and negative rationales.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    2% average accuracy gain over GRPO; VisKG performs comparably to or better than baselines across science, mathematics, and general visual understanding benchmarks with fewer tokens than caption-based representations.

    Source: §5 · p. 7

  • Takeaways & Limitations

    Structured visual grounding is an efficient intermediate representation for VQA.

    Source: §5 · p. 7

    Answer quality still depends substantially on the backbone VLM’s reasoning, and VisKG is biased toward entity-relation-centric questions.

    Source: §5 · p. 7

Abstract

from arXiv · show

Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs into textual representations. The effectiveness of this translation depends on how well visual details are retained; models need to surface and align both explicit and implicit knowledge sufficient to support reasoning, without introducing spurious assumptions. Existing methods that leverage detailed image captions introduce visual details unrelated to the reasoning task, inflating input token counts and increasing computational cost. To address these challenges, we propose VisKG, a reinforcement learning (RL) framework in which models learn to translate visual content into question-specific knowledge graph (KG) representations. This process filters out perceptual noise while preserving the entity-relation structure needed for chain-of-thought reasoning, following the principle of minimum sufficient information. To ensure stable RL post-training, VisKG adopts Group reward-Decoupled Normalization Policy Optimization (GDPO). In addition, we strengthen the supervision stage with negative rationale samples, exposing the model to incorrect reasoning paths before RL post-training. Experimental results across science, mathematics, and general visual understanding benchmarks show that VisKG achieves performance comparable to or better than baselines, while requiring fewer tokens than caption-based representations. Moreover, training VisKG with GDPO improves accuracy by 2% over its GRPO-trained counterpart on average. These results suggest that KG representations are a promising approach for supporting multi-step reasoning and open up future work on adaptively selecting the most suitable representation for a given task.

1 Introduction

Complex VQA requires models to integrate visual, linguistic, and background knowledge, but these inputs lack a shared representation. VisKG addresses this with question-specific knowledge graphs and training choices intended to retain relevant information and support reliable reasoning.

1 Introduction

  • VQA becomes more challenging as questions require multi-step inference, relational reasoning, or world knowledge that must be integrated with visual and linguistic information.The example described requires combining the question, the spatial configuration of angles, and mathematical background knowledge.Source: §1 · p. 1
  • Visual content, linguistic input, and background knowledge lack a shared representation, making it difficult for VLMs to align them into coherent reasoning.Source: §1 · p. 1
  • Captions lack an explicit mechanism for selecting question-relevant entities and relations, whereas KGs can align them with linguistic and background knowledge while pruning irrelevant nodes.Source: §1 · p. 1
  • Captions can include irrelevant details, while GRPO can be unstable under heterogeneous advantages and positive-only SFT omits common reasoning errors.The paper frames these as limitations of representation, optimization, and supervision, respectively.Source: §1 · pp. 1–2
  • VisKG combines question-specific knowledge graphs, GDPO-based RL training, and negative-rationale supervision to address representation, optimization, and supervision challenges.The framework trains models to generate KG representations of visual content before reasoning, with positive and negative rationales included in SFT.Source: §1 · p. 2, Figure 1 · p. 2

2 Related Work

Prior work explores language-aligned visual representations, RL post-training for VLMs, and SFT rationales. VisKG situates its approach as question-specific KG representation, GDPO optimization for VQA, and SFT that includes negative rationales.

Efficient Visual Representations for VQA

  • Task-agnostic visual summaries can expose models to redundant details, while VisKG uses KGs to capture minimal, question-relevant reasoning signals.Prior approaches include image or dense captions, region-level features, object lists, and token-compression or visual-summarization techniques.Source: §2 · p. 2

RL Post-Training in VLMs

  • GDPO is described as a more robust alternative to GRPO, but prior work had explored it only in single-modality reasoning, not VQA.The cited RL approaches include explicit image interpretation and separate visual-perception and language-reasoning stages.Source: §2 · p. 2

SFT Supervision

  • Existing SFT supervision focuses on positive rationales, and VisKG adds negative rationales to its SFT dataset before RL post-training.Source: §2 · p. 2

3 Methodology

VisKG trains VLMs to derive question-specific KGs that retain answer-relevant entities and relations while discarding details that do not contribute to answering the question.

3.1 VisKG Training Framework

  • VisKG uses a standard image-and-question rollout and a KG-conditioned rollout, combining answer and format rewards with KG minimality and sufficiency rewards to update the VLM.The KG-based rollout generates a reasoning trace and answer from the KG and question.Source: Figure 2 · p. 3, §3.1 · pp. 3–4
  • The KG should preserve question-relevant entities and relations while minimizing unnecessary content, with a length penalty introduced after the model learns to construct valid KGs.The penalty uses serialized KG token length normalized against mean KG length in the SFT dataset.Source: §3 · p. 3, §3.1 · pp. 3–4

3.2 Advantage Computation in GDPO

  • GDPO normalizes each reward stream using its own statistics before combining streams and rescaling at batch level, avoiding scale dominance and cross-component cancellation.A curriculum weight gradually increases the KG-based reward's influence, reaching full strength with the length-penalty warmup.Source: §3.2 · p. 4

3.3 SFT Preprocessing

  • SFT teaches VisKG's structured output format and strengthens its reasoning prior with both positive rationales and negative rationales explaining incorrect paths.The SFT stage precedes RL post-training.Source: §3.3 · p. 4

3.4 Theoretical Interpretation

  • The information-bottleneck interpretation treats answer information retained in the KG as sufficiency and question-irrelevant image information discarded as minimality.The framework approximates sufficiency with the KG-based rollout's sufficiency reward and minimality with a length penalty.Source: §3.4 · p. 4, Figure 3 · p. 5

4 Experiments

VisKG is evaluated on general visual understanding and mathematical reasoning, against baselines with different visual-grounding strategies. Results show competitive benchmark performance, benefits from negative-rationale SFT and GDPO, and a shorter KG representation than a short-caption alternative.

4.1 Implementation Details

  • VisKG is initialized with SFT on PN-Rationales, then trained for one epoch with GDPO on Vision-SR1-47K using Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct.PN-Rationales is derived from a randomly selected 9K subset of Vision-SR1-47K and includes positive and negative rationales across general visual reasoning, mathematics, and scientific knowledge.Source: §4.1 · p. 5

4.2 Experimental Setup

  • Evaluation covers MMMU-Pro and the MME cognition subset for general visual understanding, and MathVerse for diagram-centric mathematical reasoning, with accuracy reported except for MME cognition score.MME cognition score sums ACC and ACC+, where ACC+ requires both image-paired questions to be answered correctly.Source: §4.2 · p. 5, Table 2 · p. 6

How does the KG representation perform on VQA tasks compared to baselines?

  • VisKG outperforms Vision-R1, Perception-R1, and Visionary-R1 on MMMU-Pro, trails Vision-SR1 by a small margin, and remains competitive with Vision-R1 and Vision-SR1 on MathVerse.The authors exclude MME from this baseline comparison because leading leaderboard entries are closed-source with undisclosed architectures and training data.Source: How does the KG representation perform… · p. 5, Table 1 · p. 5

What is the impact of SFT training on negative rationales on reasoning robustness?

  • Across settings, negative-rationale SFT improves reasoning robustness over positive-only supervision, though the effect weakens in larger, more reasoning-capable models.The ordering SFT(b) > SFT(p) > SFT(n) holds consistently for Qwen2.5-VL-7B-Instruct but has specified exceptions for Qwen3-VL-8B-Instruct on MMMU-Pro and MathVerse.Source: Table 1 · p. 5, What is the impact of SFT training on… · p. 6, Table 2 · p. 6

To what extent does the IB-inspired reward formulation in GDPO improve reward stability during training relative to GRPO?

  • GDPO improves accuracy by 2 percentage points on average over GRPO and produces more stable reward trajectories in most reported settings.GRPO rewards deteriorate sharply toward the end of training in most settings, with a few configurations showing partial recovery.Source: To what extent does the IB-inspired… · p. 6, Figure 4 · p. 7

How does substituting KG representation with short image description in the SFT stage affect downstream task performance and input token efficiency?

  • Replacing the KG with a short image caption preserves comparable accuracy, while the KG is 16% shorter on average in generated output length.The comparison keeps the rest of the VisKG framework unchanged during both SFT and RL.Source: How does substituting KG… · pp. 6–7, Table 3 · p. 6

5 Conclusion and Future Work

VisKG uses question-specific knowledge graphs (KGs) for visual question answering, with negative-rationale supervision and GDPO for RL post-training. It performs comparably to or better than baselines with fewer tokens than caption representations, while its benefits are limited for questions that do not center on entity relations.

5 Conclusion and Future Work

  • VisKG trains VLMs to generate question-specific KGs that filter irrelevant content and retain the entity-relation structure needed for reasoning.The method also uses positive and negative rationales in SFT and GDPO for RL post-training; GDPO yields a 2% average accuracy gain over GRPO.Source: §5 · p. 7
  • Across science, mathematics, and general visual understanding benchmarks, VisKG performs comparably to or better than baselines with fewer tokens than caption-based representations.The conclusion presents KG representations as an efficient intermediate representation for VQA.Source: §5 · p. 7
  • VisKG offers limited benefit on VQA instances that do not follow an entity-relation pattern, and answer quality still depends substantially on the backbone VLM’s reasoning.The same KG can yield very different performance across VLMs; future work proposes coupling KGs with reasoning and adaptively selecting KG, caption, or hybrid representations.Source: §5 · p. 7

6 Appendix

The appendix details VisKG’s GDPO optimization, PN-Rationales supervision, training settings, and evaluation benchmarks and metrics.

6.1 GDPO Optimization

  • GDPO normalizes standard-rollout and KG-based reward streams independently, combines their advantages with a curriculum weight, then normalizes the resulting advantage batch-wise.The curriculum gradually increases the KG-based reward’s influence; the clipped objective also uses a KL penalty to keep the updated policy close to a fixed reference policy.Source: §6.1 · p. 8

6.2 PN-Rationales Dataset

  • PN-Rationales prompts generate image descriptions, question-specific KGs, and positive and negative chain-of-thought rationales in consistent formats.The appendix points to Table 4 for dataset statistics but the supplied passage does not give its values.Source: §6.2 · p. 8, Figure 5 · p. 9, Figure 6 · p. 10

6.3 Training Hyperparameters

  • RL training uses LoRA on SFT-initialized backbones, four rollouts per question, a 20-question batch, and one dataset epoch.The setting uses a learning rate of 5 × 10^-7, gradient-norm clipping at 1.0, and a frozen SFT reference policy; one setting takes approximately four days on a single NVIDIA H100.Source: §6.3 · p. 9

6.4 Benchmarks and Metrics Details

  • Evaluation covers general visual understanding with MMMU-Pro and MME cognition, and multimodal mathematical reasoning with MathVerse.MMMU-Pro includes a vision-only setting; MME cognition uses ACC and ACC+, while accuracy is reported for MMMU-Pro and MathVerse.Source: §6.4 · pp. 9–10

7 Ablation Experiments

The experiments compare rollout settings, training stability, inference cost, and KG question-specificity. They report stronger performance in several standard-rollout settings, similar cognition scores for caption and KG representations, and lower KG overlap alongside higher cognition scores after RL training.

7.1 Standard Rollout Experimental Results

  • Standard rollout consistently outperforms KG-based rollout, and VisKG beats baselines in several settings, especially SFT(b) + GDPO with Qwen3-VL-8B-Instruct.Tables 5 and 7 cover MMMU-Pro, MathVerse, and MME cognition performance under different training settings.Source: §7.1 · p. 11, Table 5 · p. 12, Table 7 · p. 13

7.3 Training reward curves of Qwen3-VL-8B-Instruct

  • For Qwen3-VL-8B-Instruct, GRPO and GDPO show comparably stable RL training, while GDPO is reported as more stable than GRPO for Qwen2.5-VL-7B-Instruct.The authors suggest the larger model’s parameter scale may mitigate GRPO instability; GDPO’s stability is reported across both model scales.Source: §7.3 · p. 11, Figure 9 · p. 15

7.4 Cost Experiment

  • Caption and KG representations achieve similar MME cognition scores, while average captions are approximately 20% longer in tokens than average KGs.Table 6 compares inference cost and cognition scores under KG-based rollout.Source: §7.4 · pp. 11–12, Table 6 · p. 12

7.5 Analysis of KG Question-Specificity

  • Across both backbones, RL-trained KGs have lower cross-question triple overlap and higher MME cognition scores than SFT-only KGs.The authors interpret this pattern as RL moving generation away from redundant, question-agnostic content toward question-specific KGs.Source: §7.5 · p. 12, Table 8 · p. 16

7.2 VisKG Inference Examples

  • Figure 8 presents qualitative VisKG inference examples comparing KG and caption intermediate representations and their resulting answers.Each example includes the input image and question alongside the generated representation and answer.Source: §7.2 · p. 11, Figure 8 · p. 14
Loading 2609.35942v2…