PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding

Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, Dacheng Tao

arXiv:2610.29552v1cs.CV

TL;DR

Fine-grained multimodal tasks require both understanding a designated region and localizing it from language, while existing methods commonly train these directions separately. Generalized CycleGRPO couples them through reconstruction across spatial, temporal, and spatio-temporal regions and introduces DOVE to test occurrence-distinctive video captions. It improves matched base policies across all three settings, including SAMTok’s GroundingSuite result.

  • Problem

    Fine-grained region understanding and localization are commonly trained with separate objectives, and existing video evaluations do not test whether captions distinguish a target occurrence from similar events.

    Source: §1 · pp. 1–2

  • Method

    Generalized CycleGRPO uses reciprocal description and localization rollouts in a shared region-text cycle, with reconstruction overlap as supervision and no reference captions or external judge during reinforcement learning.

    Source: §1 · p. 3, §3.1 · p. 5

  • Results

    57.5% to 68.2% overall gIoU on GroundingSuite for SAMTok, with improvements over matched base policies across spatial, temporal, and spatio-temporal settings.

    Source: §1 · p. 3, §7 · p. 17

  • Takeaways & Limitations

    The results support region-text cycle reconstruction as a practical, verifiable training signal across the three evaluated region spaces.

    Source: §7 · p. 17

    The cycle assumes the initial policy can perform both mappings reasonably well; unreliable localization can give captioning a noisy learning signal, and nested rollouts cost more than one-directional GRPO.

    Source: §6 · p. 17

Abstract

from arXiv · show

Fine-grained multimodal intelligence requires models to both understand a designated region and localize it from language. Existing methods learn these directions with separate objectives and task-specific annotations. We present Generalized CycleGRPO, the first unified reinforcement learning framework coupling region understanding and localization across spatial, temporal, and spatio-temporal domains. Given a region, a multimodal large language model generates candidate descriptions and then localizes each description to reconstruct the input region. Reconstruction overlap provides a verifiable reward for captioning, while individual localization rewards improve the inverse mapping. A common region-space formulation extends this cycle from image masks and boxes to video intervals and timestamped spatial evidence, without reference captions or an external language-model judge during reinforcement learning. In videos, the cycle encourages descriptions that distinguish a target event from similar recurrences. We introduce DOVE (Distinctive Occurrence-level Video-caption Evaluation), a human-verified contrastive benchmark for this temporal distinctiveness. CycleGRPO improves SAMTok's overall gIoU from 57.5% to 68.2% on GroundingSuite and average gIoU from 71.2% to 78.7% on GRES. For temporal tasks, it achieves 51.9% tIoU on Charades and 33.3% Temporal Evidence Coverage on DOVE, compared with 45.6% and 14.8% for the Qwen3-VL initialization. These results establish region-text cycle consistency as a general self-evaluation principle for multimodal understanding and localization. Models, code, and DOVE are available at https://devinxzhang.github.io/Generalized-CycleGRPO-Page/.

1 Introduction

Generalized CycleGRPO couples region understanding and localization through reconstruction across spatial, temporal, and spatio-temporal regions. The introduction motivates this cycle with the separation of existing objectives and reports gains across all three settings alongside DOVE, a benchmark for occurrence-distinctive video captions.

1 Introduction

  • Existing methods commonly optimize region understanding and localization separately, although fine-grained descriptions must distinguish a target from its confounders.A fluent caption need not uniquely identify its source region, and localization provides no direct signal to improve captions.Source: §1 · pp. 1–2
  • Generalized CycleGRPO uses one policy to describe a region and localize the description back, with reconstruction overlap rewarding both mappings across three region spaces.The framework covers spatial masks and boxes, temporal intervals, and spatio-temporal trajectories, using region annotations without reference captions or an external judge during reinforcement learning.Source: §1 · pp. 2–3
  • In videos, recurring events make generic captions ambiguous; the cycle favors occurrence-specific cues and, for spatio-temporal regions, spatial evidence aligned to timestamps.The method evaluates spatial overlap only at aligned timestamps, so a similar object at the wrong time or the right time without relevant spatial evidence is insufficient.Source: §1 · p. 2
  • DOVE is a human-verified contrastive benchmark that tests whether captions identify a target occurrence and exclude similar events in the same video.Its Temporal Evidence Coverage summarizes order, state, and outcome evidence.Source: §1 · pp. 2–3
  • 57.5% to 68.2% overall gIoU on GroundingSuite, with improvements over matched base policies across all three region spaces.SAMTok’s average gIoU on GRES rises from 71.2% to 78.7%.Source: §1 · p. 3, Figure 2 · p. 3
  • The paper validates the framework across three region spaces with single-task and reward-side ablations, backbone comparisons, and analyses spanning more than ten benchmarks.The analyses include temporal distinctiveness and failure modes.Source: §1 · p. 3

2 Related Work

Related work treats fine-grained understanding and localization as inverse tasks, but generally trains them with different objectives or paired annotations. Generalized CycleGRPO instead couples both directions through region-text reconstruction and evaluates temporal distinctiveness with DOVE.

Image Region Understanding and Grounding

  • Image-region architectures can consume and emit masks, but descriptions and grounded masks are still optimized with different objectives or annotation sources.The paper extends its earlier spatial region→text→region cycle to temporal and spatio-temporal regions.Source: §2 · pp. 3–4

Temporal and Spatio-Temporal Understanding and Grounding

  • Video grounding and captioning commonly use paired timestamp-caption supervision or language-to-region objectives, whereas this framework also learns region-to-language mapping through reconstruction.Repeated events make temporal distinctiveness important because generic descriptions may match multiple intervals.Source: §2 · p. 4

Verifiable Reinforcement Learning and Cycle Consistency

  • Prior multimodal reinforcement learning uses geometric rewards, separate answerers, or other video feedback, but not same-policy language-to-region reconstruction to train region-to-language mapping.CycleGRPO uses reconstruction overlap for understanding and rewards localization rollouts against the known region without reference captions or an external judge during reinforcement learning.Source: §2 · p. 4

Benchmarking Fine-Grained Region--Text Correspondence

  • Existing video benchmarks assess grounding or reference-caption similarity, but do not test whether a caption is faithful to its interval and discriminative against recurrences in the same video.DOVE uses human-verified repeated-occurrence groups and private Order, State, and Outcome probes, assessed by a fixed text-only judge.Source: §2 · p. 4

3 Generalized CycleGRPO

Generalized CycleGRPO uses a shared region–text–region cycle to train region understanding and localization across spatial, temporal, and spatio-temporal tasks. Candidate descriptions are rewarded by how well localization reconstructs their source region, while localization rollouts are also trained against the target.

3 Generalized CycleGRPO

  • The same region–text–region cycle covers masks, boxes, temporal intervals, and spatio-temporal tubes through region-specific serialization and consistency functions.The learning principle is separated from the representation of a region.Source: §3 · p. 4, §3.4 · p. 7

3.1 Region--Text Duality over Measurable Spaces

  • A region space supplies a serialization for model inputs and outputs and a bounded consistency score measuring whether a predicted region recovers the target.The score can be geometric overlap or, for compact region tokens, a training-time token-consistency surrogate.Source: §3.1 · p. 5

3.2 Nested Region--Text--Region Rollouts

  • For each target, the model samples candidate descriptions, then uses each description to condition multiple localization rollouts that reconstruct the region.Averaging inner rollouts estimates how reliably a description identifies its source region.Source: §3.2 · p. 6

3.3 Cycle Rewards and Joint GRPO Optimization

  • Group-relative optimization compares descriptions by cycle closure and localization samples by accuracy under a fixed parent description.Localization rollouts are optimized against the known region, and their aggregate accuracy supplies a reference-free reward for region understanding.Source: §3.3 · p. 7

3.4 Instantiations across Region Spaces

  • Spatial tasks use masks or boxes, temporal tasks use intervals and temporal IoU, and spatio-temporal tasks use timestamped boxes with matched joint-support overlap.The spatio-temporal instantiation requires structured regions but no paired textual descriptions during reinforcement learning.Source: §3.4 · pp. 7–8

4 DOVE

DOVE evaluates whether a caption identifies a designated occurrence among visually similar recurrences, using human-verified probes and a contrastive setup. Its primary comparison shows higher temporal evidence coverage for CycleGRPO than the shared Qwen3-VL initialization.

4.1 Task Definition

  • DOVE tests whether a caption preserves temporal evidence that identifies one designated occurrence among visually similar events, beyond reference-caption similarity.Each group contains a shared query and distinct same-video occurrences, with the remaining intervals serving as sibling distractors.Source: §4.1 · p. 8

4.2 Benchmark Construction

  • The benchmark draws groups from three video sources, uses Gemini proposals followed by iterative group-level human review, and verifies multi-frame Order, State, and Outcome probes.Reviewers check occurrence validity, distinctness, fact assignments, negatives, and probe answers against targets and sibling distractors.Source: §4.2 · p. 9
  • The final benchmark contains 180 distinct-video groups and 530 target intervals, with 1,223 verified probes covering 529 intervals across Order, State, and Outcome.The remaining interval has no retained probe and is excluded from reported metrics.Source: §4.2 · pp. 9–10

4.3 Evaluation Protocol

  • Evaluation captions are scored with frozen private probes and a text-only judge; TEC macro-averages Order, State, and Outcome coverage.The judge sees captions and probes but not the video; all methods share the prompt, probes, judge, and decoding protocol.Source: §4.3 · p. 10, Table 5 · p. 11
  • Against the shared Qwen3-VL initialization, CycleGRPO raises TEC by 18.5 percentage points (95% CI: 15.8–21.3), Temp-Cov by 18.7 points (15.9–21.5), and Clip-Complete by 9.1 points (6.4–11.9).Intervals are paired 95% percentile bootstrap intervals from resampling whole source-video/query groups across 20,000 replicates.Source: §4.3 · p. 10

5 Experiments

Experiments evaluate region–text–region learning across spatial, temporal, and spatio-temporal tasks, with matched benchmarks and reward-side comparisons. The results show gains in localization and region-conditioned understanding, while temporal outcomes depend partly on initialization.

5.1 Main Results

  • DLC-Bench captioning reaches 67.8% average accuracy versus 61.9% for SAMTok, alongside gains in regional recognition and interleaved generation and localization.GAR-Bench-VQA overall accuracy is 64.9% versus 64.2% for SAMTok, while continuous-box results improve across the tested model families and scales.Source: §5.1 · pp. 11–12
  • 68.2% overall gIoU on GroundingSuite and 78.7% average gIoU on GRES, up from 57.5% and 71.2% for SAMTok.GRES gains hold across validation, Test A, and Test B; no-target accuracy rises from 58.7% to 92.5%.Source: §5.1 · p. 12
  • With Qwen3-VL-4B-Instruct initialization, Charades tIoU rises from 45.6% to 51.9%; with the OMTG initialization, tIoU improves in 11 of 12 reported entries.The Qwen-initialized policy improves 10 of 12 entries, with its largest gains on Charades; OMTG-initialized tIoU rises from 50.5% to 51.8% on Charades, 46.8% to 49.0% on ActivityNet, and 62.2% to 64.5% on QVHighlights.Source: §5.1 · pp. 12–13
  • On V-STaR, mLGM rises from 47.2% to 49.8%; TVGBench mIoU rises from 21.1% to 28.5%, while reported video-understanding scores also improve.VideoMME Overall increases from 61.0% to 62.3%, WorldSense Overall from 39.3% to 40.2%, and VideoMMMU Overall from 58.0% to 59.7%; VideoMMMU Perception declines from 76.0% to 75.7%.Source: Table 7 · p. 13, §5.1 · p. 13, Table 8 · p. 14
  • On DOVE, TEC rises from 14.8% to 33.3% over Qwen3-VL-4B-Instruct, with improved temporal coverage and clip completeness.CycleGRPO also exceeds Open-o3-Video-4B (RL) at 13.6% TEC and GPT-5.6-sol at 17.3% TEC; N/M falls from 75.1% to 50.7%.Source: §5.1 · p. 13

5.3 Visual Analysis

  • Visual examples show descriptions retaining target-specific details across masks, intervals, and trajectories, including interval-specific evidence that distinguishes recurring events.The figure marks targets in blue, errors or omissions in red, and target-specific evidence in green.Source: Figure 5 · p. 14, §5.3 · p. 16

5.2 Ablation Study and Analysis

  • The full cycle gives the strongest joint temporal results: 51.9% Charades tIoU and 33.3% DOVE TEC, outperforming caption-only and localization-only variants.In the controlled Open-o3-Video comparison, the full cycle also leads on the reported aggregates across V-STaR, VideoMME, WorldSense, VideoMMMU, and TVGBench.Source: Table 9 · p. 15, §5.2 · pp. 15–16
  • A controlled Qwen2.5-VL-7B comparison also improves V-STaR mAM/mLGM from 33.4/45.7% to 34.4/47.5%; temporal gains nevertheless vary with initialization and target domain.The authors note that cycle consistency does not substitute for broad coverage of target video domains.Source: §5.2 · p. 16

6 Limitations and Future Work

The method uses region reconstruction as a proxy for caption quality, but that proxy has limits and the closed loop depends on adequate initial mappings. The authors identify computational cost and DOVE’s current evaluation scope as additional boundaries.

Limitations

  • Reconstructability does not guarantee captions are complete, fluent, or free of unsupported details, and validity rules cannot enumerate every implicit shortcut.The rules discourage obvious leakage of coordinates or timestamps.Source: §6 · pp. 16–17
  • The closed loop assumes the initial policy can perform both mappings reasonably well; unreliable localization gives captioning a noisy learning signal.The authors connect this dependence to variation in gains across initializations and target domains.Source: §6 · p. 17

Limitations and Future Work

  • Nested N × K rollouts cost more than one-directional GRPO, while DOVE covers visually distinguishable events in supplied intervals, not audio-dependent evidence or event-boundary discovery.Future work proposes semantic-coverage, factuality, and uncertainty signals, plus rollout-efficiency methods and broader DOVE settings.Source: §6 · p. 17

7 Conclusion

Generalized CycleGRPO extends cycle learning from image regions to video intervals and spatio-temporal trajectories. The paper presents reconstruction as a practical, verifiable signal for jointly learning visual description and localization.

7 Conclusion

  • A single policy learns captioning and localization through reciprocal rollouts across masks, intervals, and spatio-temporal trajectories, using reconstruction quality without reference captions or a separate critic.The conclusion reports improvement over matched base policies across all three settings and presents DOVE for testing discrimination among repeated same-query occurrences.Source: §7 · p. 17

Appendix Overview

  • The appendices document training configurations, additional spatial and temporal examples, continuous boxes, a larger backbone, failure cases, and DOVE statistics.They provide supplementary implementation details, controlled analyses, and benchmark statistics.Source: Appendix Overview · p. 24

A Detailed Experimental Settings

The appendix details training configurations and temporal-data preparation, including offline screening of OMTG intervals and reward components for mixed-task training.

A Detailed Experimental Settings

  • 18,742 examples were assigned to RL after source videos were randomly split between bidirectional co-SFT and RL for the Qwen3-VL initialization.Source: A Detailed Experimental Settings · p. 24
  • 18,735 of 36,892 candidate OMTG intervals were retained after offline VLM-assisted screening for event completeness and visual distinctiveness.Screening judgments and generated descriptions were used only for data quality control, not as training targets or RL rewards.Source: A Detailed Experimental Settings · p. 24
  • The appendix supplies configuration tables for spatial, temporal, and mixed-task settings; the spatial configuration carries over unchanged from the ECCV version.The training code is publicly available.Source: Table 13 · p. 24, A Detailed Experimental Settings · p. 24, Table 14 · p. 25
  • Mixed-task Open-o3-Video training combines correctness rewards for multiple-choice video QA, reference-based ROUGE rewards for free-form answers, and format and repetition checks.Temporal reward shaping supplements overlap feedback with interval-center, boundary, and duration agreement.Source: A Detailed Experimental Settings · p. 25

B Additional Spatial Analyses

Additional analyses show spatial gains across region captioning and localization examples, a larger-backbone comparison, closed-loop rollouts, and temporal boundary comparisons.

B Additional Spatial Analyses

  • CycleGRPO better captures target texture, shape, and color, localizes disjoint or occluded GRES targets, covers part and amorphous regions, and tightens continuous boxes in the examples.The comparisons include SAMTok and ground truth for region captioning and mask localization, and Qwen3-VL and ground truth for continuous boxes.Source: B Additional Spatial Analyses · p. 26, Figure 6 · p. 26, Figure 7 · p. 26
  • 68.1% DLC-Bench average and 64.5% GroundingSuite average gIoU were achieved with SAMTok initialized from Qwen3-VL-8B, up from 63.7% and 63.1%, respectively.The reported gains persist with the larger spatial backbone.Source: Table 16 · p. 28, B Additional Spatial Analyses · p. 28
  • More precise descriptions generally produce higher reconstruction IoU across the four-description, six-rollout mushroom example.The rollout illustrates reconstruction accuracy as a learning signal for description quality; hallucinated or insufficiently discriminative details yield low-overlap localization rollouts.Source: B Additional Spatial Analyses · p. 28, Figure 10 · p. 28

C Temporal Grounding Visualization

  • On TimeLens examples spanning brief gestures to longer activities, CycleGRPO aligns predicted temporal boundaries more closely with references, including short events embedded in long videos.The base model often misses targets or predicts broad intervals, while OMTG can retain irrelevant context or truncate events.Source: C Temporal Grounding Visualization · p. 29, Figure 11 · p. 29

D Additional V-STaR Qualitative Analysis

V-STaR examples illustrate closer temporal–spatial predictions alongside remaining errors in retrieval, localization, semantic robustness, and long-video event detection.

D Additional V-STaR Qualitative Analysis

  • CycleGRPO narrows predictions toward reference events in several V-STaR examples, including an 8–10 s prediction for a reference from 8–10.9 s.Other panels show a more complete event-end recovery, but one prediction remains late and overextended.Source: D Additional V-STaR Qualitative Analysis · p. 30, Figure 12 · p. 30
  • Failures include wrong-time retrieval despite correct semantics, over-broad timing with imprecise localization, semantic hallucination for an unusual interaction, and a missed late event.The examples include a cup placed at 0–10 s rather than 37.1–44.1 s and a bear missed near the end of a long video.Source: Figure 13 · p. 30, D Additional V-STaR Qualitative Analysis · p. 31

E DOVE Statistics

DOVE statistics characterize short target intervals, repeated same-query occurrences, diverse query domains, and multi-axis temporal evidence.

E DOVE Statistics

  • The median source video lasts 100.9 seconds and the median target interval 6.8 seconds; 180 targets (34.0%) last under five seconds.Figure 14A plots source-video and target-interval durations on a logarithmic time axis.Source: E DOVE Statistics · p. 31, Figure 14 · p. 31
  • 48 groups (26.7%) contain four or five same-query occurrences, while 195 intervals (36.8%) have probes covering all three evidence types: Order, State, and Outcome.Across 1,223 probes, covered intervals have a median of two probes.Source: E DOVE Statistics · p. 31, Figure 14 · p. 31
  • DOVE queries span domains including food, speech, crafts, transportation, sports, personal care, digital interfaces, medical scenes, and animals.The plotted categories come from a reproducible keyword taxonomy rather than additional human labels.Source: E DOVE Statistics · p. 31, Figure 14 · p. 31

Related papers

Loading 2610.29552v1…