PaperlayerView exact revision on arXiv

Source-linked AI summary

RingMoClaw: An Experience-Inspired Multi-Agent Framework for Self-Evolving Research in Remote Sensing

Kaiyue Kang, Qixuan He, Peijin Wang, Yingchao Feng, Chao Ren, Kangxin Wang, Wenhui Diao, Yixiao Wang, Liangjin Zhao, Kaiwen Wei, Nayu Liu, Xian Sun

arXiv:2609.00814v1cs.CV

TL;DR

Remote-sensing model improvement still depends heavily on manual trial and error, while existing agents focus mainly on task execution and workflow orchestration. RingMoClaw addresses this gap with a self-evolving multi-agent loop that combines research execution, independent Critic review, and dual-stream experience accumulation. Across four downstream tasks, it reports consistent gains, including a 1.84% mAP_50 improvement for object detection, while reducing required evolution steps by over 40%.

  • Problem

    Existing remote-sensing agents mainly support task execution and workflow orchestration, leaving autonomous research iteration for continuous performance optimization limited.

  • Method

    RingMoClaw combines a research branch, heterogeneous quality-control Critic, and dual-stream dynamic experience bus integrating external knowledge with internal experimental experience.

  • Results

    Across object detection, scene classification, semantic segmentation, and change detection, RingMoClaw achieves consistent gains, including 69.9% mIoU on iSAID and 90.54% F1 on LEVIR-CD.

  • Takeaways & Limitations

    The results support a feasible route from task execution toward continuous research-driven model evolution in remote sensing.

Abstract

from arXiv · show

Remote sensing visual models have continuously advanced various interpretation tasks. However, the research process behind model improvement still heavily relies on manual expertise, requiring extensive trial-and-error iterations in model design, data processing, and performance diagnosis. Existing agent-based approaches mainly focus on task execution and workflow orchestration, while lacking the capability of autonomous research iteration for continuous performance optimization. To address this issue, we propose RingMoClaw, an experience-inspired self-evolving multi-agent framework for remote sensing visual interpretation. RingMoClaw integrates a research branch, a quality-control branch, and a dual-stream dynamic experience bus to establish a closed-loop optimization process covering strategy generation, experiment execution, independent review, and experience accumulation. The heterogeneous Critic mechanism provides stage-wise diagnosis and feedback, while the dual-stream experience bus incorporates external knowledge and internal experimental experience to guide strategy evolution and eliminate ineffective searches. Extensive experiments on four remote sensing downstream tasks, including object detection, scene classification, semantic segmentation, and change detection, demonstrate the effectiveness and generalization of RingMoClaw. Compared with the corresponding baseline models, RingMoClaw improves performance by 1.84\% mAP$_{50}$ on object detection and achieves consistent gains across the other three tasks, while reducing the required evolution steps by over 40\% compared with existing research automation frameworks. These results suggest that RingMoClaw offers a feasible route from task execution toward continuous research driven model evolution in remote sensing.

I. INTRODUCTION

Remote-sensing agents have advanced task execution and workflow orchestration, but remain limited in continuous research iteration, independent quality control, and experience-guided strategy evolution. RingMoClaw addresses these gaps with a unified self-evolving loop for sustained model optimization.

  • Research Gap: Existing remote-sensing agents mainly automate task solving, workflow orchestration, and capability extension rather than continuous performance improvement.This limitation is especially relevant to remote-sensing vision tasks involving long-tailed categories, small-object misses, and domain-specific constraints.
  • Research Gap: A closed loop connecting hypothesis generation, experiment execution, performance analysis, and strategy updates is largely missing.Most existing systems are designed for one-pass task completion, making sustained model optimization difficult.
  • Research Gap: Independent and structured quality control is limited, with insufficient diagnosis and quantitative feedback for plan design, data processing, and experimental bottlenecks.The lack of targeted feedback makes correction during iteration difficult.
  • Research Gap: External knowledge and internal experimental experience are not jointly integrated into a unified optimization process for actionable strategy updates.Existing systems explore retrieval and memory, but experience accumulation and knowledge transformation remain difficult.
  • Proposed Framework: RingMoClaw combines a research branch, quality-control branch, and dual-stream dynamic experience bus into a unified loop for sustained remote-sensing model optimization.The framework supports plan generation, data processing, experiment execution, independent diagnosis, external knowledge injection, and internal experience accumulation.

B. Tool-Augmented Geospatial Agents and Workflow Orchestration

Geospatial agent research has progressed from tool calling toward multi-step workflows and adaptive interaction, but sustained self-optimization in complex remote-sensing settings remains underexplored. RingMoClaw organizes research execution, independent review, and experience-driven search refinement into one architecture.

  • Workflow Orchestration: Recent geospatial systems frame Earth-observation analysis as multi-step workflows involving data discovery, tool selection, reasoning, execution, and verification.Domain-specific agents also use interactive environments and specialized-agent orchestration for complex remote-sensing pipelines.
  • Continual Improvement: General-purpose agent frameworks explore continual improvement, long-horizon evaluation, adaptive workflow optimization, and multi-agent coordination.These approaches mainly target software, web, or other non-remote-sensing environments.
  • Open Challenge: Existing Earth-observation agent studies typically operate offline in batch-processing modes and lack continuous self-optimization or online adaptation.Integrating multimodal perception, reliable tool orchestration, and continual performance self-evolution remains an open challenge.
  • RingMoClaw: RingMoClaw couples a self-evolving research pipeline, Critic mechanism, and dual-stream dynamic experience bus.The architecture maps external retrieval-augmented knowledge and internal experimental feedback onto a structured search space.
  • Research Pipeline: The research pipeline uses Main, Data, and Vision Agents to decompose objectives into staged evolution trajectories and instantiate solutions from an evolving knowledge base.It extends optimization across data processing, training strategies, model selection, model fusion, and module modification.
  • Closed-Loop Control: The Critic independently reviews plan, data, and result stages, while the experience bus prunes unpromising paths and refines optimization directions.Fixed, version-controlled prompts and schema validation support reproducible agent interactions.

B. Critic Mechanism: Heterogeneous Review and Targeted Feedback

RingMoClaw uses a structurally separate Critic to review intermediate research outputs rather than only final results. Its multidimensional scoring and rule-based decisions convert diagnosis into targeted revision, rejection, escalation, and reusable experience.

  • Targeted Feedback: The object-detection trajectory retains improvements, rejects harmful directions, records failures, and accumulates validated knowledge for later evolution.The procedure includes baseline diagnosis, data-level optimization, few-shot directions, resampling, and module integration.
  • Heterogeneous Review: The heterogeneous Critic uses a distinct model and functional separation to reduce homogeneous bias from same-model self-evaluation.The execution chain uses GLM-5 Turbo, whereas the Critic uses Minimax 2.7.
  • Staged Review: The Critic intervenes at plan, data, and result nodes, providing independent diagnosis, quantitative evaluation, and targeted feedback throughout the execution chain.This staged review is designed to identify anomalies and accuracy bottlenecks that standard automated systems may miss.
  • Structured Diagnosis: The Critic score combines task completion, bottleneck impact, and historical experience relevance into a comprehensive evaluation.Task completion measures progress toward the stage objective, while the other dimensions characterize bottleneck severity and experience relevance.
  • Structured Diagnosis: Stage-dependent weights emphasize bottleneck severity during plan review and task completion during result review.The fixed weight tuples are (0.30, 0.50, 0.20) for plan review, (0.35, 0.40, 0.25) for data review, and (0.50, 0.30, 0.20) for result review.
  • Targeted Feedback: Explicit rules escalate outputs with missing critical evidence, accept outputs meeting the score and bottleneck thresholds, revise intermediate cases, and reject others.Acceptance requires Score_t ≥0.75 and B_t ≤0.30; revision applies when Score_t ≥0.45 but the acceptance condition is unmet.

3) Feedback Driven Decision Update and Global Experience Accumulation:

RingMoClaw routes Critic diagnoses through the Main Agent for decision updates and standardizes each iteration into experience memory. Its dual-stream bus combines external knowledge with internal experience to refine the structured search space and guide future strategies.

  • Feedback-Driven Decision Update: The Main Agent receives structured Critic diagnoses and scores, then performs global coordination and decision making rather than directly commanding subordinate agents.The diagnosis represents the current system state and reviewed output.
  • Global Experience Accumulation: Every iteration is standardized and written into the global experience pool, preserving plans, experimental configurations, scores, and diagnostic conclusions.
  • Global Experience Accumulation: Critic feedback simultaneously optimizes the current task and accumulates reusable global experience, improving later baselines, initial plans, and convergence speed as tasks accumulate.
  • Dual-Stream Experience Bus: The dual-stream experience bus integrates literature and code knowledge with task and cross-task memory in global and local experience pools.
  • Structured Search Space: The structured search space spans architecture, training, data processing, and system integration, with 23 predefined operators linked to bottleneck tags and constrained parameters.
  • External Knowledge Flow: Retrieval-augmented extraction maps textual strategies and reusable code operators into executable instructions, allowing external methods to extend the search space on demand.

2) Internal Experimental Flow:

RingMoClaw evaluates candidate strategies through short verification runs, Critic review, and memory-based filtering before full training. The experiments cover four remote sensing tasks with task-specific datasets, metrics, and settings.

  • Agile Verification: Each candidate undergoes a constrained five-minute training run before promotion decisions.The verification protocol is evaluated alongside the framework’s broader experiments.
  • Agile Verification: The verification score combines task-specific performance gain with a training-stability term, using the final 50 losses’ coefficient of variation as a lightweight stability surrogate.
  • Candidate Selection: A candidate reaches full training only after positive gain, failure-free verification, fscore(m) > 0.55, result-stage Critic approval, and negative-memory screening.
  • Candidate Selection: The evolution stops after 20 rounds or after two consecutive full-experiment rounds without improving the best-so-far result.
  • Experimental Settings: Experiments evaluate oriented object detection, scene classification, semantic segmentation, and change detection using RingMo as the primary backbone.
  • Evaluation Protocols: The study reports mAP50 for object detection, Top-1 accuracy for scene classification, mIoU for semantic segmentation, and changed-class F1 and IoU for change detection.

B. Overall Performance and Evolution Process

Across object detection and scene classification, RingMoClaw progressively improves performance through staged optimization and complementary model fusion. The evolution process reduces weak-category errors and shows only mild late-stage overfitting in the final detector.

  • Object Detection: 47.30% to 49.14% mAP50: staged object-detection evolution improves FAIR1M performance.Data augmentation and training adjustments provide moderate gains; few-shot training does not improve performance, while fusion reaches 49.11% and combined module innovation reaches 49.14%.
  • Object Detection: Missed detections and false positives decrease across evolution stages, particularly for weak categories on FAIR1M.The same trajectory shows consistent gains on several weak categories as the optimization path responds to intermediate feedback.
  • Object Detection: The final detector’s validation loss reaches 3.2132 at epoch 14 before rising slightly to 3.3053 at epoch 20.Training loss decreases throughout 20 epochs, indicating only a mild late-stage overfitting tendency.
  • Scene Classification: 94.97% accuracy: Soft Multi-Model Fusion outperforms the NWPU-RESISC45 baseline by 1.79 percentage points.Individual candidate classifiers do not consistently surpass the baseline, but their complementary category strengths support decision-level fusion.
  • Semantic Segmentation: 69.9% mIoU: RingMoClaw improves iSAID segmentation from 67.2% through backbone and scale-aware inference changes.The final upscale-only test-time augmentation configuration achieves the best overall result.

3) Semantic Segmentation:

RingMoClaw combines staged strategy exploration with architecture-level module innovation, producing gains across remote sensing tasks and a closed-loop comparison against research-automation baselines. The jointly retained MSFR-CPR configuration improves detection while adding limited computational complexity.

  • Change Detection: 90.54% F1 and 89.97% IoU: RingMoClaw improves LEVIR-CD validation performance from 89.53% F1 and 87.10% IoU.Temporal-Swap Augmentation, Bidirectional ChangeStar, and EvoChangeStar contribute successive changes along the evolution trajectory.
  • Change Detection: The final EvoChangeStar reaches 90.54% F1 and 89.97% IoU after bidirectional temporal modeling and additional change-interaction exploration.Bidirectional ChangeStar reaches 90.18% F1 and 88.89% IoU, while Deep Change Interaction alone reaches 89.65% F1 and 88.57% IoU.
  • Research Automation: RingMoClaw uses experience retrieval, heterogeneous Critic review, agile verification, and memory updates to form a closed-loop evolution process.The comparison retains AutoResearchClaw through experiment execution and excludes its later paper-generation and publishing steps.
  • Evaluation Protocol: Performance gain is defined as ∆P = Pfinal − Pbaseline, while ER denotes completed evolution steps for each task.Higher gains and fewer evolution rounds indicate better research-automation effectiveness and efficiency.
  • Module Innovation: 49.14/37.10 mAP50/mAP50:95: the jointly retained MSFR-CPR configuration adds 0.730 M parameters and 0.876 G FLOPs.The modules are generated from bottlenecks identified in previous evolution rounds and instantiated on Oriented R-CNN.

C. Practical Cost and Search Efficiency

RingMoClaw’s Critic and dual-stream experience mechanisms improve search efficiency by reducing ineffective exploration and accelerating convergence. Ablations show that removing either experience stream or the Critic requires more rounds and lowers detection performance.

  • Search Efficiency: RingMoClaw reduces unnecessary experimental trials and accelerates evolution despite additional planning, Critic diagnosis, and experience summarization.The framework may consider multiple candidate proposals per round, unlike AutoResearch’s one-candidate-per-round process.
  • Critic Mechanism: 7 rounds and 49.14% mAP: the full framework outperforms the Critic-removed variant, which needs 11 rounds and reaches 47.43%.The ablation attributes the difference to structured diagnosis and reduced ineffective exploration.
  • Evaluation Metrics: “Rounds to Conv.” counts full evolution rounds until accuracy stops clearly improving, while “Failed” counts iterations without improvement over the previous round.“mAP” reports detection accuracy at convergence.
  • Dual-Stream Experience Bus: 7 rounds, 2 failed iterations, and 49.14% mAP: the full dual-stream framework converges fastest and achieves the highest ablation performance.Removing the external stream requires 9 rounds and reaches 48.44% mAP; removing the internal stream requires 11 rounds and reaches 47.62% mAP; removing both requires 14 rounds and reaches 47.53% mAP.
  • Dual-Stream Experience Bus: The internal and external streams surface complementary directions: class-aware sampling and focal loss address imbalance, while BFPA comes from external literature.Each technique improves a different subset of weak categories rather than producing redundant effects.

3) Validation of the Five-Minute Verification Protocol:

The five-minute verification protocol is evaluated as a low-cost screening stage for prioritizing candidate strategies before full training. It preserves performance rankings reasonably well and reliably identifies clearly detrimental directions.

  • Ten object-detection strategies are evaluated under five-minute verification and subsequently trained using the full budget, including candidates that the protocol would reject.
  • Spearman’s rank correlation measures agreement between five-minute and full-training rankings, while precision and recall assess detrimental-candidate screening.For each candidate, di denotes the rank difference between the two evaluation settings.
  • ρ = 0.64 with p-value 0.03 for mAP50:95 gains, indicating positive rank consistency between five-minute and full-training results.The protocol also achieves ρ = 0.59 for mAP50.
  • Precision = 1.00 because all five candidates identified as detrimental during five-minute verification were confirmed detrimental after full training.
  • The protocol is used to prioritize promising directions and screen unfavorable candidates rather than precisely predict final performance values.This allows computational resources to focus on candidates selected for full-scale training.

APPENDIX A IMPLEMENTATION DETAILS OF MODULE INNOVATION

The module-innovation appendix describes two architectural modules generated during self-evolution and instantiated on Oriented R-CNN. MSFR refines multi-scale features, while CPR refines foreground classification using prototype similarities.

  • Multi-Scale Feature Refinement: MSFR replaces Oriented R-CNN’s standard FPN neck while preserving output pyramid levels, feature dimensions, and downstream interfaces.
  • Multi-Scale Feature Refinement: MSFR aligns FPN features to a reference level, applies convolution and channel-spatial attention, then redistributes the refined feature across pyramid levels.Adaptive max pooling and nearest-neighbor interpolation perform downsampling and upsampling, respectively.
  • Multi-Scale Feature Refinement: γ_l is a learnable residual gate initialized to zero in the MSFR redistribution process.
  • Category Prototype Refinement: CPR is inserted into Oriented R-CNN’s rotated RoI classification branch to refine foreground-category logits using similarities to category prototypes.It leaves background classification and bounding-box regression branches unchanged.
  • Category Prototype Refinement: CPR maps the shared 1024-dimensional RoI feature into a normalized 128-dimensional embedding and compares it with normalized category prototypes.

APPENDIX B STRUCTURED SEARCH-SPACE INSTANTIATION

The structured search space represents optimization choices as bottleneck-linked operators with constrained parameters. Historical experience and failed configurations then guide candidate selection and pruning.

  • The FAIR1M oriented-detection base search space contains 23 operators spanning architecture, training, data-processing, and system-integration dimensions.The dimensions contain 5, 8, 6, and 4 operators, respectively.
  • Each evolution round matches operators to the current bottleneck and permits revisions only to parameters declared by the corresponding operator.Externally retrieved methods may enter after filtering and are converted into the same structured representation.
  • Historical-task similarity combines task, bottleneck, dataset, model, and metric factors to retrieve relevant experience.Dataset similarity uses class counts, sample counts, and class-imbalance ratio when available.
  • Sim ≥0.75 marks historical entries as transferable experience, 0.60 ≤Sim < 0.75 as references, and Sim < 0.60 as ignored.At most five highest-scoring entries are supplied to the decision process.
  • A candidate is pruned when its operator-level similarity to a recorded failed configuration reaches 0.80.Similarity is zero across different search-space dimensions or operators; otherwise it compares numerical or categorical parameter agreement.

APPENDIX D RETRIEVAL SOURCES AND FILTERING RULES

The retrieval pipeline limits external search, filters duplicate and unsuitable results, and ranks retained papers and repositories before passing candidates to later decision stages.

  • At most three queries per evolution round are generated from the task description, current bottleneck, and search-space dimension.Queries retrieve related papers and open-source repositories.
  • Retrieval uses arXiv and supplementary Semantic Scholar searches for literature, plus GitHub repository search for implementations.Sources are handled independently so one unavailable or rate-limited service does not interrupt the others.
  • At most 20 papers and 20 repositories are retained before screening, with duplicates removed using normalized titles or repository names.
  • Items are scored once for relevance, implementation feasibility, and novelty, with relevance below 0.70 causing removal.The three scores are normalized to [0, 1].
  • The five highest-ranked surviving items become candidate directions, with relevance receiving the largest ranking weight.The Main Agent can select only from these retained retrieval results.

APPENDIX E AGENT PROMPTS FOR RINGMOCLAW

The appendix specifies fixed, versioned prompts and role boundaries for RingMoClaw’s agents, including planning, data assessment, implementation, and independent critique. It also defines critic scoring, stage-specific weights, decision rules, and structured outputs.

  • Prompt and role design: Fixed, versioned system prompts define each agent’s role boundary, available information, prohibited actions, and structured output schema.The reported experiments use these prompts to constrain agent behavior.
  • Prompt and role design: The Main Agent selects exactly one candidate using retrieved experience but does not train models, modify data, write code, review results, or query external resources.The Experience Bus supplies the candidate pool and relevant positive and negative experience.
  • Prompt and role design: The Data Agent evaluates training readiness from program-computed statistics and determines required preprocessing, sampling, or augmentation actions.Inputs include class counts, imbalance, empty images, malformed annotations, and unmatched class-name tokens, treated as ground truth.
  • Prompt and role design: The Vision Agent makes a selected candidate runnable through configuration overrides, newly written modules, or an explicit infeasibility declaration.It must not reference nonexistent configuration paths, and infeasibility is preferable to inventing a path.
  • Structured outputs: Agent outputs are serialized as JSON, including critic reviews, data assessments, and implementation feasibility, with the Data Agent returning structured readiness fields.The specified schemas include scores, diagnosis, decisions, memory writes, preprocessing plans, flags, and runnable implementation outputs.
  • Critic mechanism: The critic accepts only when final_score >= 0.75 and Bt <= 0.30, revises when final_score >= 0.45, and otherwise rejects if evidence is sufficient.Missing critical evidence or insufficient confidence triggers escalation instead of false precision.
Loading 2609.00814v1…