PaperlayerView exact revision on arXiv

Source-linked AI summary

RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow

Liang Yao, Fan Liu, Hongbo Lu, Chuanyi Zhang, Rui Min, Shengxiang Xu, Shimin Di, Pai Peng

arXiv:2507.19280v3cs.CV

TL;DR

Complex geospatial queries call for interpreting user intent and spatial relationships, while existing approaches are constrained by supervised fine-tuning and task-specific outputs. RemoteReasoner combines reinforcement learning with a unified multi-granularity inference workflow, and reports state-of-the-art performance across tasks, including a 3.67% pixel-level accuracy improvement over SegEarth-R1.

  • Problem

    Existing remote-sensing approaches rely on supervised fine-tuning and task-specific decoders, while complex queries require interpreting nuanced intent and implicit spatial relationships.

    Source: §1 · pp. 1–2

  • Method

    RemoteReasoner trains an MLLM with reinforcement learning and uses a unified task-transformation pipeline for pixel-, region-, and object-level reasoning.

    Source: §1 · p. 2

  • Results

    RemoteReasoner reports state-of-the-art multi-granularity performance, including a 3.67% accuracy improvement over SegEarth-R1 on pixel-level geospatial reasoning tasks.

    Source: §1 · p. 2, §5 · p. 7

  • Takeaways & Limitations

    The workflow provides a flexible foundation for downstream geospatial intelligence requiring nuanced intent interpretation and multi-format outputs.

    Source: §5 · p. 7

Abstract

from arXiv · show

Remote sensing imagery presents vast, inherently unstructured spatial data, necessitating sophisticated reasoning to interpret complex user intents and contextual relationships beyond simple recognition tasks. In this paper, we aim to construct an Earth observation workflow to handle complex queries by reasoning about spatial context and user intent. As a reasoning workflow, it should autonomously explore and construct its own inference paths, rather than being confined to predefined ground-truth sequences. Ideally, its architecture ought to be unified yet generalized, possessing capabilities to perform diverse reasoning tasks through one model without requiring additional fine-tuning. Existing remote sensing approaches rely on supervised fine-tuning paradigms and task-specific heads, limiting both autonomous reasoning and unified generalization. To this end, we propose RemoteReasoner, a unified workflow for geospatial reasoning. The design of RemoteReasoner integrates a multi-modal large language model (MLLM) for interpreting user instructions and localizing targets, together with task transformation strategies that enable multi-granularity tasks, including object-, region-, and pixel-level. In contrast to existing methods, our framework is trained with reinforcement learning (RL) to endow the MLLM sufficient reasoning autonomy. At the inference stage, our transformation strategies enable diverse task output formats without requiring task-specific decoders or further fine-tuning. Experiments demonstrated that RemoteReasoner achieves state-of-the-art (SOTA) performance across multi-granularity reasoning tasks. Furthermore, it retains the MLLM's inherent generalization capability, demonstrating robust performance on unseen tasks and out-of-distribution categories.

1 Introduction

RemoteReasoner combines reinforcement learning with a unified inference workflow to address complex geospatial queries and support pixel-, region-, and object-level reasoning. The paper reports state-of-the-art multi-granularity performance and generalization to unseen tasks.

1 Introduction

  • Complex geospatial queries require interpreting nuanced user intent and implicit spatial relationships beyond simple object detection or classification.The paper gives identifying flood-risk areas near critical infrastructure after heavy rain as an example.Source: §1 · p. 1
  • Prior reasoning approaches rely on supervised fine-tuning, while SegEarth-R1 supports segmentation alone, limiting autonomy and task breadth.The cited discussion also links SFT-based approaches to out-of-distribution degradation and catastrophic forgetting.Source: §1 · pp. 1–2
  • The proposed workflow uses reinforcement learning for self-directed reasoning and decision-level task transformation instead of redundant task decoders and training.GRPO training balances output formats, localization accuracy, and quantity while aiming to preserve MLLM generalization.Source: §1 · p. 2
  • A single MLLM inference followed by a unified transformation pipeline supports pixel-level, region-level, and object-level geospatial reasoning.The paper identifies region reasoning and contour reasoning as novel tasks built on the pixel-level task.Source: §1 · p. 2
  • RemoteReasoner reports state-of-the-art multi-granularity performance, including a 3.67% accuracy improvement over SegEarth-R1 in pixel-level reasoning and superior results on unseen referring segmentation.Holistic evaluations also report generalization and efficiency through a single forward pass.Source: §1 · p. 2
  • The paper introduces precisely annotated datasets for region reasoning and contour reasoning to support research on the new tasks.Its contribution summary also reports efficiency across multi-granularity tasks through a single forward pass.Source: §1 · p. 2

2 Related Work

Prior reasoning models and remote-sensing multimodal systems provide useful capabilities but remain limited in task breadth or robust reasoning. The section frames these gaps alongside RemoteReasoner’s GRPO-based reasoning and multi-granularity inference workflow.

2.1 Reasoning in Large Language Models

  • Reasoning models extend instruction-following, while remote-sensing systems such as SegEarth-R1 remain tied to narrow task types, fixed granularity, and supervised fine-tuning.The related-work discussion describes general LLM reasoning as largely limited to natural-language and vision tasks.Source: §2.1 · p. 2

2.2 Remote Sensing Multi-modal Models

  • Remote-sensing multimodal models offer diverse spatial capabilities but lack robust reasoning for ambiguous or compositional prompts; Figure 2 outlines GRPO and multi-granularity inference.The figure caption describes GRPO as exploring self-thinking, followed by an inference workflow for multi-granularity tasks.Source: §2.2 · p. 2, Figure 2 · p. 3

3 RemoteReasoner

RemoteReasoner builds a multi-stage workflow for multi-task geospatial reasoning. It combines a reasoning model with transformations that produce pixel masks and target contours from bounding-box predictions.

3.1 Data

  • EarthReason's pixel-only dataset is extended with generated detection and contour annotations, while segmentation and contour labels are reserved for downstream evaluation.It contains 2,371 training, 1,135 validation, and 1,928 testing samples.Source: §3.1 · p. 3

3.2 Reasoning Model

  • The workflow uses Qwen2.5-VL-7B-instruct as its reasoning model and fine-tunes it with reinforcement learning to address remote-sensing domain discrepancy and geospatial reasoning needs.Source: §3.2 · p. 3

3.3 Training

  • GRPO training uses a composite reward for localization, object count, and output format, combining IoU-based box matching, an exponential count penalty, and format compliance.The format reward checks whether outputs follow the specified format, returning 1 or 0.Source: §3.3 · pp. 3–4

3.4 Inference Workflow

  • Bounding-box predictions and center points prompt SAM to produce target masks, from which morphological boundary detection derives target contours.The workflow processes an image and query to produce a reasoning trace and target box before applying these transformations.Source: §3.4 · p. 4

4.1 Dataset and Metrics

Evaluation uses EarthReason labels for pixel-level tasks and converts the original labels into region- and object-level formats; additional test sets and metrics cover grounding, referring segmentation, and contour reasoning.

  • EarthReason validation and test sets supply the evaluation data; pixel-level tasks use the original annotations, while region- and object-level tasks use M2B and M2C formats derived from the original labels.Source: §4.1 · p. 4
  • Visual Grounding is evaluated on RSVG, Referring Expression Segmentation on RRSISD, and unseen-category recognition on 100 images spanning 10 categories from RemoteSAM-270K.The RSVG and RRSISD training sets are excluded from model training.Source: §4.1 · p. 4
  • Region- and pixel-level evaluation uses gIoU, cIoU, and Acc@0.5; contour reasoning uses F1@1, F1@3, ASD, and HD.F1@1 and F1@3 use 1-pixel and 3-pixel dilated kernels, respectively.Source: §4.1 · p. 4

4.2 Implementations Details

The reasoning model is fine-tuned with GRPO and LoRA on Qwen2.5-VL-7B-Instruct using a specified multi-GPU training setup.

  • GRPO fine-tunes Qwen2.5-VL-7B-Instruct with LoRA (rank 8, alpha 16) for 24 epochs on 8 NVIDIA L20 GPUs, using AdamW at a 1e-6 learning rate, bf16, and DeepSpeed Zero3.The global batch size is 512, achieved with per-device batch size 8 and gradient accumulation steps 8.Source: §4.2 · p. 5

4.3 Main Results

RemoteReasoner outperforms reported baselines across pixel-, region-, and contour-level reasoning, including comparisons against methods trained with pixel annotations and reasoning trajectories.

Geospatial Pixel Reasoning

  • 3.67% improvement over SegEarth-R1 on pixel-level geospatial reasoning accompanies a 10.41% higher cIoU than LISA on the validation set.RemoteReasoner surpasses SFT-based methods despite not directly using pixel-level labels during training.Source: Geospatial Pixel Reasoning · p. 5

Geospatial Region Reasoning

  • At least 54.47% improvement in Acc@0.5 over GeoChat and over 22.29% over Qwen-2.5VL-7B and DeepSeek-VL2-tiny is reported for region-level reasoning.The paper attributes the comparisons to GeoChat's weak reasoning on ambiguous queries and the general-purpose MLLMs' limited remote-sensing domain knowledge.Source: Geospatial Region Reasoning · p. 5

Geospatial Contour Reasoning

  • At least 0.34 higher on both F1 scores and significantly lower distance errors than EPOC are reported for contour reasoning.The paper attributes EPOC's gap to extraneous contours and difficulty with indistinct target boundaries.Source: Geospatial Contour Reasoning · p. 5

4.4 Further Analysis

Further analyses assess unseen-task transfer, out-of-distribution recognition, retained MLLM capabilities, and qualitative performance; they also report data-scaling and inference-cost observations.

Can RemoteReasoner Do Other Tasks?

  • Under identical zero-shot settings, RemoteReasoner gains 4.12% on Visual Grounding and 19.32% on Referring Expression Segmentation, reaching 80% of supervised remote-sensing methods' results.The comparison uses methods trained on task-specific data for the supervised results.Source: Can RemoteReasoner Do Other Tasks? · p. 6
  • Multi-turn dialogue evaluation tests retained MLLM capabilities by asking image-content questions after image captioning.The results are described as demonstrating strong generalization capabilities with RL.Source: Can RemoteReasoner Do Other Tasks? · p. 6

Can RemoteReasoner Recognize Other Category?

  • RemoteReasoner shows robust recognition across 10 categories absent from EarthReason, but remains insufficient at detecting small objects such as cars.The evaluation manually selects 100 images from RemoteSAM-270K; EarthReason contains only 28 categories.Source: Can RemoteReasoner Recognize Other… · p. 6

Qualitative Analysis

  • Qualitative examples show the workflow locating and producing masks and contours for a background-blended baseball field and a blurred medical helipad.The figure caption characterizes the examples as autonomous target-category identification and task execution across three granularities.Source: Qualitative Analysis · pp. 6–7, Figure 5 · p. 7
  • Performance increases as data scales exponentially, while Mask2Contour has significantly faster inference speed in the reported cost comparison.Source: Figure 6 · p. 7

4.5 Ablation Studies

The ablations compare pixel-level output formats, training-data scale, and contour-extraction speed. Box outputs followed by SAM and RL improve pixel-level accuracy over textual masks, while Mask2Contour is faster than EPOC.

Different Pixel-level Output Formats

  • 60.25 to 69.13 accuracy with RL for box outputs followed by SAM, compared with 22.01 for textual mask representations.Both SFT and RL textual-mask strategies show significant performance degradation relative to visual-based approaches; these findings motivate Mask2Contour.Source: Different Pixel-level Output Formats · p. 7
  • The full 100% training-data subset achieves the best performance among the tested 20%, 50%, and 100% subsets.Performance improves as data scale increases; the authors report that the full dataset contains no redundancy.Source: Different Pixel-level Output Formats · p. 7

Inference Cost Ablation

  • 0.09s per image for Mask2Contour versus 0.43s per image for EPOC, which has 84.6MB of parameters.Mask2Contour uses morphological operations without additional parameters.Source: Inference Cost Ablation · p. 7

5 Conclusion

RemoteReasoner uses RL and a unified inference pipeline to support autonomous geospatial reasoning across pixel-, region-, and object-level tasks. It achieves reported accuracy gains and generalizes across diverse queries.

5 Conclusion

  • RemoteReasoner uses RL and a unified inference pipeline to transform one MLLM output into pixel-, region-, and object-level tasks without redundant per-task decoders.The conclusion describes the approach as overcoming limitations of SFT-based methods while preserving MLLM generalization.Source: §5 · p. 7
  • RemoteReasoner achieves > 15% and 30% accuracy gains in region- and object-level reasoning, respectively.The conclusion also reports superior generalization across diverse, complex queries.Source: §5 · p. 7
  • The authors present RemoteReasoner as a flexible foundation for downstream geospatial intelligence requiring nuanced intent interpretation and multi-format outputs.The conclusion reports generalization across diverse, complex queries.Source: §5 · p. 7
Loading 2507.19280v3…