PaperlayerView exact revision on arXiv

Source-linked AI summary

Asking like Socrates: Socrates helps VLMs understand remote sensing images

Run Shao, Ziyu Li, Zhaoyang Zhang, Linrui Xu, Xinran He, Hongyuan Yuan, Bolei He, Yongxing Dai, Yiming Yan, Yijun Chen, Wang Guo, Haifeng Li

arXiv:2511.22396v2cs.CVcs.AI

TL;DR

Remote-sensing models can rely on a single coarse image pass, producing pseudo-reasoning rather than reasoning grounded in visual evidence. The paper introduces RS-EoT, instilled through SocraticAgent and progressive grounding-plus-VQA reinforcement learning, and reports state-of-the-art results across multiple remote-sensing VQA and grounding benchmarks.

  • Problem

    Remote-sensing models often use a single coarse perception pass, which can leave visual evidence incomplete and produce pseudo-reasoning based on linguistic consistency.

    Source: §1 · p. 2, §2.1 · p. 3

  • Method

    RS-EoT trains iterative evidence seeking through SocraticAgent-generated reasoning traces and progressive reinforcement learning on grounding followed by remote-sensing VQA.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    RS-EoT achieves state-of-the-art performance on multiple remote-sensing VQA and grounding benchmarks, and analyses show iterative reasoning and evidence-seeking cycles.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The final model internalizes an iterative pattern of linguistic deduction and targeted visual evidence seeking, which the authors present as supporting trustworthy geospatial AI.

    Source: §5 · p. 9

Abstract

from arXiv · show

Recent multimodal reasoning models, inspired by DeepSeek-R1, have significantly advanced vision-language systems. However, in remote sensing (RS) tasks, we observe widespread pseudo reasoning: models narrate the process of reasoning rather than genuinely reason toward the correct answer based on visual evidence. We attribute this to the Glance Effect, where a single, coarse perception of large-scale RS imagery results in incomplete understanding and reasoning based on linguistic self-consistency instead of visual evidence. To address this, we propose RS-EoT (Remote Sensing Evidence-of-Thought), a language-driven, iterative visual evidence-seeking paradigm. To instill this paradigm, we propose SocraticAgent, a self-play multi-agent system that synthesizes reasoning traces via alternating cycles of reasoning and visual inspection. To enhance and generalize these patterns, we propose a two-stage progressive RL strategy: first, RL on fine-grained Grounding tasks to enhance RS-EoT capabilities, followed by RL on RS VQA to generalize to broader understanding scenarios. Experiments show RS-EoT achieves state-of-the-art performance on multiple RS VQA and grounding benchmarks. Analyses reveal clear iterative cycles of reasoning and evidence seeking, confirming RS-EoT mitigates the Glance Effect and enables genuine evidence-grounded reasoning. Our code, data, and models are available at https://geox-lab.github.io/Asking_like_Socrates

1. Introduction

The introduction frames remote-sensing reasoning as an iterative search for visual evidence, rather than a single-pass interpretation. It presents SocraticAgent and progressive grounding-plus-VQA reinforcement learning as ways to instill this pattern.

1. Introduction

  • A single coarse perception pass can miss sparse, subtle clues in large-scale imagery, leading models to produce linguistically consistent pseudo-reasoning instead of evidence-based reasoning.The paper attributes this pattern to the Glance Effect and notes that performance can fail to improve or even degrade versus the base model.Source: §1 · p. 2
  • The proposed RS-EoT paradigm uses an evolving reasoning–perception loop to guide repeated image inspection and progressively refine perceptual grounding.This addresses the stated need for iterative rather than static evidence acquisition.Source: §1 · p. 2
  • SocraticAgent uses a Reasoner and a Perceiver in multi-turn self-play to create the RS-EoT-4K traces used for supervised fine-tuning cold-start.The Reasoner poses perceptual questions as needed, and the Perceiver answers them with concise visual observations.Source: §1 · p. 2
  • Progressive reinforcement learning first uses fine-grained grounding to strengthen evidence seeking, then general remote-sensing VQA to broaden it; reconstructed multiple-choice data and a tailored reward support stable VQA training.The VQA reconstruction addresses the susceptibility of simple datasets, such as yes/no questions, to reward hacking.Source: §1 · p. 2
  • RS-EoT achieves state-of-the-art performance on multiple remote-sensing VQA and grounding benchmarks, with analyses showing iterative evidence-seeking cycles that mitigate the Glance Effect.The authors describe clear cycles of reasoning and evidence seeking.Source: §1 · p. 2

2. Related Work

The paper situates RS-EoT against limitations of single-pass remote-sensing vision-language reasoning, then describes iterative evidence seeking and progressive SFT/RL training. Its main comparison reports RS-EoT-7B as best across remote-sensing VQA and grounding tasks.

2.1. Vision-language Model for Remote Sensing; 2.3. Multimodal reasoning model

  • Single-pass global perception can miss regional cues in wide, scale-varying remote-sensing images, while multimodal reasoning models inherit assumptions that fail on tasks requiring iterative, region-specific inspection.The paper positions RS-EoT as a dynamic loop of linguistic deduction and targeted visual inspection.Source: §2.1 · p. 3, §2.3 · p. 3

2.2. Large Language Reasoning Model

  • The paper builds on the SFT–RL reasoning paradigm: SFT cold-starts reasoning patterns, and RL refines and generalizes them.It also situates this approach among language reasoning methods including Chain-of-Thought and GRPO.Source: §2.2 · p. 3

3. Method; 3.1.1. RS-EoT

  • RS-EoT makes language the controller of hypotheses and evidence-seeking steps, while localized visual evidence is repeatedly sought, verified, and integrated.The method cold-starts this pattern with SFT and then uses progressive RL to enhance and generalize it.Source: §3.1.1 · p. 3, §3 · p. 3

3.1.2. Asking like Socrates

  • SocraticAgent uses a text-only Reasoner and an image-aware Perceiver in incremental question-and-inspection cycles to synthesize RS-EoT-4K traces for SFT.The dataset includes RGB, infrared, and SAR imagery.Source: §3.1.2 · pp. 3–4, §3.1.3 · p. 5

3.1.3. Self-play Prompting Mechanism

  • Self-play prompts characterize each agent’s collaborator as weak, encouraging detailed question decomposition and concise, relevant visual answers.The authors report that this produces progressively structured multi-turn traces.Source: §3.1.3 · pp. 4–5

3.2.1. Stage#1: Fine-grained Grounding RL

  • The first RL stage trains on fine-grained grounding tasks to strengthen evidence seeking, using target-box IoU as a reward alongside a strict format reward.The model outputs object locations as [x1, y1, x2, y2].Source: §3.2.1 · p. 5, §3.2 · p. 5

3.2.2. Stage#2: General RS VQA RL

  • For the second RL stage, simple RS VQA data is reconstructed as multiple-choice questions over associated QA pairs, with incorrect pairs created by answer inversion.An equal-weight, symmetric option-level reward gives positive credit for correctly selecting true pairs or rejecting false ones, and zero for the opposite choices.Source: §3.2.2 · p. 5
  • RS-EoT-7B achieves the best results across the reported remote-sensing VQA and fine-grained grounding tasks.The comparison includes multimodal reasoning models based on Qwen2.5-VL-7B and the remote-sensing models Geo-R1 and VHM-RL.Source: Table 1 · p. 6

4. Experiments

Experiments compare RS-EoT-7B with reasoning baselines and examine its training stages and evidence-seeking behavior. The reported results span VQA, grounding, qualitative cases, attention dynamics, and VQA-RL stability.

4. Experiments

  • The evaluation compares models on VQA and grounding, assesses SocraticAgent against distillation, and analyzes cases, attention, reward stability, and training stages.Implementation uses a six-round SocraticAgent dialogue limit and two progressive RL stages.Source: §4 · p. 6

4.1. Comparison with Prior Reasoning Models

  • RS-EoT-7B consistently leads all three VQA metrics and surpasses every baseline by a large margin on IoU@50, IoU@70, and mIoU.The comparison covers remote sensing VQA and fine-grained grounding benchmarks; the VQA metrics are Avg@5, Conv@5, and Pass@5.Source: §4.1 · p. 7

4.2. Comparison with Frontier Model Distillation

  • SocraticAgent-generated traces consistently outperform directly distilled traces when models are fine-tuned and evaluated on FIT-RSFG-VQA.The comparison tests traces from SocraticAgent against direct distillation from Qwen3-VL-plus, Doubao-seed-1-6-vision, and GLM-4.5V.Source: Table 3 · p. 6, §4.2 · p. 7

4.3. Case Study

  • In the VQA case, RS-EoT-7B searches for an unoccupied gate after establishing airport context, while the baseline misses contradictory evidence.For grounding, it scans globally, identifies a candidate water-body region, and re-examines the image to rule out other water bodies before producing coordinates.Source: §4.3 · pp. 7–8, Figure 3 · p. 7

4.4. Attention Dynamics During EoT Reasoning

  • Across eight sampled cases, image-token attention peaks during evidence seeking and falls during language-based reasoning, forming a periodic cycle.The visualization plots image-token attention proportion against token index during decoding.Source: §4.4 · p. 8, Figure 4 · p. 8

4.5. VQA-RL Training Stability Analysis

  • VQA-RL reward rises from approximately 0.62 to around 0.84, with a smooth, consistent trajectory attributed to the reconstructed data and graded reward.The paper reports that the strategy mitigates reward hacking and supplies a stable learning signal without training collapse on simple RS VQA datasets.Source: §4.5 · p. 8, Figure 5 · p. 8

4.6. Stage-wise Training Analysis

  • SFT improves VQA accuracy but reduces grounding; RL-IoU restores and enhances grounding, then RL-VQA further improves VQA while maintaining grounding.The authors attribute localization gains to explicit spatial feedback and describe the stages as complementary.Source: §4.6 · p. 8, Table 4 · p. 8

5. Conclusion

The conclusion presents RS-EoT as an iterative evidence-seeking paradigm for remote sensing understanding. It reports stronger baseline comparisons and analyses supporting an alternating reasoning and visual inspection pattern.

5. Conclusion

  • RS-EoT structures remote sensing reasoning as alternating linguistic deduction and targeted visual evidence seeking.The authors instill the paradigm through SFT followed by progressive Grounding+VQA RL.Source: §5 · p. 9
  • The approach surpasses SOTA baselines in reasoning robustness and grounding accuracy.Source: §5 · p. 9
  • Analyses indicate that the model internalizes the iterative pattern, supporting structured reasoning for trustworthy geospatial AI.Source: §5 · p. 9

6. System Prompts Details

The appendix describes SocraticAgent's three roles and points to their system prompts. The Reasoner asks incremental visual questions, the Perceiver supplies observations, and the Verifier checks answers against ground truth.

6. System Prompts Details

  • The Reasoner is text-only and decomposes queries into atomic visual questions, iteratively gathering evidence before deciding.It follows a Plan–Integrate–Decide paradigm.Source: §6 · p. 12
  • The Perceiver supplies accurate, descriptive visual observations without high-level logical reasoning.Self-play prompting frames it as collaborating with a weak-reasoning partner.Source: §6 · p. 12
  • The Verifier compares the Reasoner's final answer with ground truth to filter incorrect or hallucinated traces.The respective prompts are presented in Figures 6, 7, and 8.Source: §6 · p. 12

7. SFT Training Settings

The SFT settings specify the base model, dataset, optimizer, and training schedule. The authors omit reasoning-specific system instructions and hard-code a start-of-thought token in the response template.

7. SFT Training Settings

  • SFT fine-tunes Qwen2.5-VL-7B-Instruct on RS-EoT-4K for five epochs at 3×10−5 with AdamW and a cosine schedule.The global batch size is 64 and maximum sequence length is 4096 tokens; training took approximately 40 minutes on four A100 GPUs.Source: §7 · p. 12
  • Explicit reasoning triggers caused dependency on those prompts, so reasoning-specific instructions were excluded from SFT and RL prompts.Source: §7 · p. 12
  • The chat template hard-codes <think> at the start of the assistant response to initiate reasoning without relying on specific prompts.Source: §7 · p. 12

8. RL Training Settings

The RL setup combines grounding and multiple-choice VQA training, with task-specific rewards that balance answer accuracy and output format.

8. RL Training Settings

  • The training configuration uses KL regularization with β = 1.0 × 10^-2 and generates four rollout samples per input.The response limit is 4096 tokens.Source: §8 · p. 12
  • Both RL stages use a global batch size of 512 and train for 240 steps; RL-VQA takes about 2.3 days and RL-Grounding about 2.5 days.Experiments run on one node with 8 A100 GPUs.Source: §8 · p. 12

9.1. Grounding Reward

  • Grounding reward combines IoU accuracy with a lightweight format term for a matched reasoning block and a valid bounding-box output.The box is parsed after the closing </think> tag; the format indicators check the reasoning block and four-number box list.Source: §9.1 · pp. 12–13
  • Grounding accuracy uses the IoU between the predicted and ground-truth boxes, with predictions parsed from the text after the reasoning block.If no valid four-number box is found, the prediction is treated as invalid.Source: §9.1 · p. 13

9.2. Multiple-Choice VQA Reward

  • For multiple-choice VQA, the model predicts a subset of option letters, extracted from the last line of the response tail containing independent letter tokens.A letter outside the allowed option set receives zero accuracy reward.Source: §9.2 · p. 14
  • The VQA accuracy reward scores options symmetrically, giving positive credit for correct selections and correct rejections while penalizing either type of error.The final VQA reward combines accuracy and format; the format term checks for a complete, matched reasoning block.Source: §9.2 · p. 14

10. RL Training Dynamics Curves

The figure tracks optimization statistics across the RL-Grounding and RL-VQA stages.

10. RL Training Dynamics Curves

  • Figure 9 plots six optimization statistics for both RL stages, showing raw measurements alongside Gaussian-smoothed trends over training steps.The metrics are mean advantage, actor gradient norm, entropy loss, KL loss, policy gradient loss, and average reward.Source: §10 · p. 14

11. Difference Between Multiple-Choice VQA and Standard VQA

The ablation compares multiple-choice VQA reconstruction with standard free-form supervision. The reconstructed formulation performs better across the reported RSVQA metrics and produces smoother RL reward improvement.

11. Difference Between Multiple-Choice VQA and Standard VQA

  • The ablation applies reinforcement learning to the original dataset and model settings, isolating the effect of multiple-choice reformulation against conventional VQA supervision.Source: §11 · p. 14
  • Multiple-choice VQA reconstruction achieves consistently superior RSVQA performance across all metrics compared with standard free-form supervision.The reported comparison is an ablation of VQA data reconstruction strategies.Source: §11 · p. 14, Table 5 · p. 15
  • Standard free-form VQA training shows slow reward improvement and frequent large oscillations, whereas multiple-choice reconstruction yields smoother, steadily increasing rewards.The figure compares RL training dynamics under the two formulations.Source: Figure 10 · p. 15, §11 · p. 15

12. Case Study

The case studies cover remote-sensing general VQA and fine-grained grounding, illustrating RS-EoT reasoning and localization across diverse queries.

12. Case Study

Loading 2511.22396v2…