PaperlayerView exact revision on arXiv

Source-linked AI summary

Momentum-Coupled Rubric Adaptation for Detailed Image Captioning

Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu

arXiv:2609.36893v1cs.CL

TL;DR

Detailed image captioning needs reliable feedback on factual accuracy and visual coverage, while separate or staggered rubric roles can become inconsistent with an evolving caption Policy. MoCo Rubric trains one model for captioning, rubric construction, and judging, then uses online rubrics and EMA transfer while updating only the Policy. Across five benchmarks, it reports a 72.83% average pairwise win rate, the best blind-ranking mean rank, and the highest average caption-based question-answering score.

  • Problem

    Detailed captioning needs feedback on factual accuracy and visual coverage, but separate role models and fixed or staggered judging can leave rubric interpretation and evaluation out of step with evolving Policy candidates.

    Source: §1 · pp. 1–2

  • Method

    MoCo Rubric uses role-conditioned shared multi-role SFT, then constructs online rubrics from current Policy candidates and transfers Policy updates by EMA to a shared Generator–Judge model while optimizing only the Policy.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    72.83% average pairwise win rate across five benchmarks; MoCo Rubric also achieves the best mean rank in blind ranking and the highest average caption-based question-answering score.

    Source: §1 · p. 3, §4.2 · p. 8

  • Takeaways & Limitations

    The results support coordinated rubric adaptation as an approach to reinforcement learning for detailed image captioning, with stronger overall performance across complementary evaluations.

    Source: §5 · p. 10

    The tracking bound assumes bounded Policy updates, while the smoothing analysis additionally assumes locally constant drift and independent perturbations across synchronization intervals.

    Source: §3.4 · p. 6

Abstract

from arXiv · show

Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy, information coverage, and clarity. Compared with conventional methods that rely mainly on high-quality supervision or holistic rewards, rubric-based reinforcement learning decomposes these requirements into explicit criteria and provides targeted, structured feedback. However, existing methods often use separate models for caption generation, rubric construction, and judging, which may lead to inconsistent interpretations across roles. Some dynamic rubric methods alternate updates between the caption policy and rubric generator while keeping the judge fixed, but staged optimization may still leave rubric construction and judging out of step with policy optimization. We propose MoCo Rubric, a two-stage framework that coordinates these roles. First, role-conditioned, shared-parameter multi-task supervised fine-tuning equips a single vision--language model to serve as the Caption Policy, Rubric Generator, and Rubric Judge. Then, the Generator constructs rubrics online from captions sampled by the current Policy, reference captions, and image evidence. The Judge provides rubric-based rewards, and only the Policy receives GRPO updates. As Policy updates change the candidates being evaluated, we use an exponential moving average of the Policy parameters to update one momentum model shared by the Generator and Judge. This gradual transfer lets both rubric roles track Policy updates without separate RL optimization while smoothing parameter changes that could disrupt their rubric capabilities under direct synchronization. Across five captioning benchmarks, MoCo Rubric achieves an average pairwise win rate of 72.83\%, the best mean rank in blind ranking, and the highest average score in caption-based question answering.

1 INTRODUCTION

Detailed captioning needs reliable feedback on factual accuracy and visual coverage because one image admits many valid descriptions and references are limited. MoCo Rubric aligns caption generation, rubric construction, and judging through shared role-conditioned training and momentum transfer, achieving strong results across five benchmarks.

Motivation

  • Separate role backbones can interpret criteria inconsistently, while alternating Policy–Generator updates with a fixed Judge may leave rubric construction and judging out of step.The paper illustrates a Generator accepting a broad “grid of cars” criterion while the Judge misses a candidate’s incorrect column count.Source: §1 · pp. 1–2

Framework

  • Shared role-conditioned multi-task SFT equips one vision–language model to generate captions, construct rubrics, and judge them from a common learned foundation.The resulting checkpoint initializes both the online Caption Policy and the momentum model shared by the Generator and Judge.Source: §1 · p. 2

Policy RL with Momentum Transfer

  • Candidate-conditioned rubric construction and joint EMA transfer to the Generator and Judge adapt feedback to current Policy outputs without separate RL optimization for either rubric role.The Judge and Generator share a lagged parameter version during candidate-group scoring; only the Policy receives GRPO updates.Source: §1 · pp. 2–3

Main results

  • 72.83% average pairwise win rate across five detailed image captioning benchmarks, with a 2.90-percentage-point lead over RubiCap.MoCo Rubric also achieves the best mean rank in blind ranking and the highest average score in caption-based question answering.Source: §1 · p. 3

Evaluation

  • The framework’s ablations support complementary effects from Shared Multi-Role SFT, candidate-conditioned rubrics, and joint momentum transfer.The paper also analyzes how the momentum coefficient and synchronization interval affect parameter tracking and update smoothness.Source: §1 · p. 3

2 RELATED WORK

Prior captioning research has expanded from reference-based supervision and holistic rewards to explicit rubric criteria and broader evaluation tools. MoCo Rubric focuses on adapting structured rewards and their generating and judging parameters as the caption Policy changes.

Image Captioning and Evaluation

  • Captioning methods use richer supervision and rewards for detailed descriptions, while evaluation ranges from reference agreement to semantic, fine-grained, and preference-based assessment.The paper positions its focus as adapting structured rewards during training rather than external caption evaluation.Source: §2 · p. 3

Rubric-Based Reinforcement Learning

  • Rubric-based RL decomposes open-ended requirements into explicit criteria, with construction approaches using references, response contrasts, examples, recursive refinement, or online evidence.Online approaches can condition evaluation criteria on current responses, external evidence, or previously collected rubrics.Source: §2 · pp. 3–4
  • Existing methods train rubric roles through alternating or joint objectives, whereas MoCo Rubric starts from shared multi-role initialization and transfers Policy updates to both rubric roles through EMA.This removes separate RL updates and associated supervision construction for the rubric roles.Source: §2 · p. 4

3 METHODOLOGY

MoCo Rubric uses one role-conditioned model foundation, then adapts online rubrics and Policy parameters as caption candidates evolve. Its EMA scheme coordinates the shared Generator and Judge while the Policy alone receives RL updates.

3.1 FRAMEWORK OVERVIEW

  • The Policy generates captions for inference using only the image and instruction, while training uses image-specific rubric feedback because references may not cover every relevant detail.The framework has two stages: shared multi-role SFT followed by Policy RL with Momentum Transfer.Source: §3.1 · pp. 4–5

3.2 SHARED MULTI-ROLE SFT

  • Role-conditioned, sample-aligned SFT trains one vision–language model on caption, rubric, and judgment records linked by image and caption instances.The resulting checkpoint initializes both Policy parameters θ and the shared Generator–Judge momentum parameters ϕ.Source: §3.2 · p. 5

3.3 RUBRIC-GUIDED POLICY OPTIMIZATION WITH MOMENTUM ADAPTATION

  • For each update, the Generator builds a rubric from references, image evidence, and a random subset of four sampled Policy captions; that rubric is applied to all 16 candidates.Rubric items specify a criterion, its satisfaction test, and a positive weight; the selected subset exposes candidate differences while references provide a quality anchor.Source: §3.3 · p. 5
  • The Judge makes criterion-level decisions from each caption and the image-grounded rubric, and normalized weighted rewards supply group-relative advantages for the Policy’s GRPO update.The Judge does not directly access the image, and its rationales do not contribute to the reward; the Generator and Judge receive no RL gradients.Source: §3.3 · p. 5
  • EMA transfers Policy changes to the shared rubric model at synchronization boundaries, retaining prior parameters while incorporating updated Policy parameters.Both rubric roles use the same parameter version for each candidate group, and the momentum coefficient and synchronization interval jointly control tracking and smoothing.Source: §3.3 · p. 6

3.4 TRACKING–STABILITY ANALYSIS

  • With shared initialization and Policy updates bounded by δ, the Policy–momentum parameter gap is bounded by mHδ/(1 − m).This bound depends on the stated bounded-update assumption.Source: §3.4 · p. 6
  • Under the local stochastic assumptions, EMA reduces update covariance by (1 − m)/(1 + m) versus periodic full copying while preserving long-run mean drift.Larger m smooths rubric-role updates but increases tracking lag; the analysis describes parameter tracking and smoothing, not guaranteed improvement in rubric quality.Source: §3.4 · p. 6

4 EXPERIMENTS

MoCo Rubric combines shared-role supervised fine-tuning with Policy-only reinforcement learning and momentum transfer to the rubric roles. Across five benchmarks, it leads in pairwise caption wins, blind ranking, and caption-based question answering; analyses examine component effects and transfer settings.

4.1 EXPERIMENTAL SETUP

  • Experiments use Qwen3-VL-8B-Instruct, train shared-role SFT and Policy-only GRPO, and evaluate pairwise wins, blind rankings, and caption-based QA.The comparison includes captioning and rubric-based baselines.Source: §4.1 · pp. 6–8

4.2 MAIN RESULTS

  • 72.83% average win rate across five benchmarks, exceeding Qwen3-VL-32B by 12.03 percentage points despite MoCo Rubric's 8B backbone.The comparison is against Qwen3-VL-8B across the five captioning benchmarks.Source: Table 1 · p. 7, §4.2 · p. 8
  • 67.17 average score is highest across five caption-based QA benchmarks; MoCo ranks first on CaptionQA, TextVQA, and DocVQA, and second on BLINK and ChartQA.Captions are the sole visual input to a fixed text-only model.Source: Table 2 · p. 7, §4.2 · p. 8

4.3 ANALYSIS

  • Online rubrics improve average win rate by 0.85 pp (M3→M4); EMA raises shared-initialization results from 66.61% to 72.14%, but lowers separate-initialization results from 69.22% to 63.43%.The ablations indicate complementary contributions from candidate-adapted rubrics and shared initialization with momentum transfer.Source: Table 3 · p. 8, §4.3 · p. 9

4.2 MAIN RESULTS

  • 2.450 is the best mean rank, with the highest first-place share at 35.9%; MoCo leads in accuracy and completeness, with clarity comparable to Qwen3-VL-32B.It also improves on RubiCap's mean rank (3.055) and hallucination penalty (1.859 versus 1.614).Source: Figure 3 · p. 8, §4.2 · p. 8

4.3 ANALYSIS

  • 72.14% joint EMA exceeds Generator-only EMA (69.07%) and full synchronization (68.73%); Judge-only EMA yields 65.84%.The reported comparison suggests Judge updates help when coordinated with Generator updates through momentum transfer.Source: Table 4 · p. 8, §4.3 · p. 9
  • At step 600, Policy–momentum distance is 0.084 for m = 0.99 and 0.605 for m = 0.999, versus 0.867 for the frozen reference on the m = 0.99 Policy trajectory.Full synchronization is the zero-distance boundary reference.Source: Figure 4 · p. 9, §4.3 · p. 9
  • 72.14% at (m, H) = (0.99, 1) is close to 71.79% at (0.99, 5); longer intervals reduce win rates at m = 0.99, while H is non-monotonic at other coefficients.The analysis motivates selecting momentum coefficient and synchronization interval jointly.Source: Table 5 · p. 9, §4.3 · p. 9, Figure 4 · p. 9

5 CONCLUSION

MoCo Rubric coordinates caption generation, rubric construction, and judging to address cross-role mismatch in detailed image captioning. Experiments show stronger overall performance across complementary evaluations, with gains in caption quality and downstream utility.

5 CONCLUSION

  • MoCo Rubric coordinates caption generation, rubric construction, and judging to address cross-role mismatch in detailed image captioning.The framework combines shared multi-task SFT with online rubrics and EMA parameter updates as the Policy evolves.Source: §5 · p. 10
  • Experiments show stronger overall performance than existing methods across complementary evaluations, with gains in caption quality and downstream utility.Source: §5 · p. 10
  • The results support coordinated rubric adaptation as an approach to reinforcement learning for detailed image captioning.Source: §5 · p. 10

A IMPLEMENTATION AND EVALUATION DETAILS

The appendix specifies prompts and evaluation protocols, along with implementations of two comparison methods and the caption-based question-answering evaluation. The prompts define role inputs and constrain rubric generation and judging to image-grounded, criterion-based decisions.

A.1.1 CAPTION GENERATION

  • Caption generation prompts the Policy to produce a detailed, flowing plain-text description of the image.Source: §A.1.1 · p. 15

A.1.2 RUBRIC GENERATION

  • The Generator uses the image, two references, and four current-Policy captions to create binary, weighted rubrics grounded in image-verifiable quality differences.The image is authoritative; rubrics prioritize reliable, important distinctions, preserve precise facts when relevant, and avoid uncertain, incidental, overlapping, or easy criteria.Source: §A.1.2 · pp. 15–16

A.1.3 RUBRIC-BASED SCORING

  • The text-only Judge returns a reason and binary decision for each rubric, evaluating only its stated criterion and accepting semantic equivalents.It must not penalize style, fluency, grammar, or length unless a rubric explicitly requires them.Source: §A.1.3 · p. 17

A.1.4 PAIRWISE EVALUATION FOR WIN RATE

  • Pairwise evaluation compares two captions with the image and returns A, B, or Tie, emphasizing precision and salient information rather than style or length.Evaluators are instructed to penalize absent image content and consider detail, assistive description, and whether the caption evokes the depicted image.Source: §A.1.4 · pp. 18–19

A.1.5 BLIND RANKING

  • Blind ranking scores five unnamed captions on accuracy, completeness, clarity, and hallucination penalty, then orders them by total score.The total is (accuracy + completeness + clarity) / 3.0 - hallucination_penalty x 1.5.Source: §A.1.5 · pp. 19–21

A.2.1 RUBICAP

  • RubiCap uses one-time, offline rubrics and a fixed Judge while optimizing the Policy with GRPO from the shared-SFT initialization.Gemini 3.1 Pro generates rubrics, and GPT-5.6-terra judges captions against them; neither rubrics nor Judge are updated during training.Source: §A.2.1 · p. 21

A.2.2 EVOLM

  • The EvoLM adaptation alternates 50 Policy updates with 50 Generator updates while keeping the Judge fixed and training both roles with GRPO.Generator preferences compare current captions with captions from 50 Policy-update steps earlier; its reward weights Judge-score difference at 0.7 and format validity at 0.3, across 600 total updates.Source: §A.2.2 · p. 21

A.3 CAPTION-BASED QUESTION ANSWERING PROTOCOL

  • Caption-based question answering gives a fixed text-only answering model the generated caption and question, using the description as its sole visual information source.The protocol evaluates CaptionQA, BLINK, TextVQA, DocVQA, and ChartQA with the answering model held fixed across captioning methods.Source: §A.3 · pp. 21–22

B.1 EVALUATION WITH GPT-5.6

Under GPT-5.6 Sol, MoCo Rubric ranks first on all five benchmarks and maintains the highest average win rate across both evaluators. The comparison holds captions, image-caption pairs, prompt, baseline, and metric fixed while changing the evaluator.

B.1 EVALUATION WITH GPT-5.6

  • The GPT-5.6 Sol comparison changes the evaluator while keeping generated captions, image-caption pairs, comparison prompt, Qwen3-VL-8B-Instruct baseline, and win-rate metric fixed.Source: §B.1 · p. 22
  • 68.79% average win rate; MoCo Rubric ranks first on all five benchmarks under GPT-5.6 Sol and exceeds RubiCap by 6.63 percentage points on average.It also exceeds EvoLM by 10.31 points and Qwen3-VL-32B by 18.12 points on average.Source: Table 6 · p. 22, §B.1 · p. 22
  • MoCo Rubric has the highest average win rate under both judges: 72.83% with Gemini 3.1 Pro and 68.79% with GPT-5.6 Sol.The passage reports 67 of 31,920 attempted comparisons excluded by the evaluator's content filter.Source: §B.1 · p. 22

B.2 HUMAN ANNOTATION AGREEMENT

The study compares two LLM judges with human preferences on 500 caption pairs, using blinded independent annotations reconciled into final labels. Both judges show high agreement with those labels, supporting the metric as a proxy for human preferences in this setting.

B.2 HUMAN ANNOTATION AGREEMENT

  • Two annotators independently labeled 500 Qwen3-8B versus GPT-5.6 caption pairs, blinded to caption sources and each other's judgments, before reconciling final A/B/tie labels.The set contains 250 PixMo-Cap and 250 DenseFusion examples.Source: §B.2 · p. 22
  • Both LLM judges show high agreement with the final human labels on this comparison set.Source: §B.2 · p. 23, Table 7 · p. 23
  • The reported agreement supports using the LLM-judge metric as a proxy for human caption preferences in this setting.Exact agreement is defined as the share of valid LLM-judge responses matching final labels, excluding invalid responses.Source: §B.2 · pp. 22–23

B.3 TRAINING REWARD DYNAMICS

The training diagnostics compare RubiCap and MoCo Rubric on reward trajectories and the share of rollout groups with no within-group reward variation. Qualitative examples highlight differences in visual details and text rendering across three model comparisons.

B.3 TRAINING REWARD DYNAMICS

  • Figure 5 plots mean training reward and the fraction of rollout groups whose captions all receive the same reward; dark traces are centered nine-point moving averages.The figure marks evaluated step-600 checkpoints; both runs contain 126 logged points over 624 update steps.Source: Figure 5 · p. 23, §B.3 · p. 23
  • Across steps 5–600, zero-standard-deviation groups averaged 3.59% for RubiCap and 0.46% for MoCo Rubric; over steps 301–600, the rates were 4.33% and 0.58%, respectively.These are single-run diagnostics; different rubrics, judges, and SFT checkpoints mean reward magnitudes are not directly comparable measures of caption quality, and the curves do not isolate the cause.Source: §B.3 · p. 23
  • RubiCap’s mean reward rose from 0.652 over steps 1–100 to 0.763 over steps 201–300, then stayed near 0.758 through step 600; MoCo Rubric recorded 0.588 and 0.577 over steps 201–300 and 301–600.Step 600 is the checkpoint used for external evaluation.Source: §B.3 · p. 23

C QUALITATIVE CASE STUDIES

  • Figure 6 selects examples contrasting MoCo Rubric with Qwen3-VL-8B-Instruct in object pose, leaf shape, product layout, text alignment, and label position.Source: Figure 6 · p. 24
  • Figures 7 and 8 illustrate fabricated facial blurring and text-layout errors; Figure 8 also shows border-detail and transcription errors, while Figure 7 highlights typography errors.Source: Figure 7 · p. 25, Figure 8 · p. 26

D PROOFS AND ADDITIONAL MOMENTUM ANALYSIS

The proof proceeds from the momentum update at synchronization boundaries, using recursive bounds and the condition that Policy changes are bounded between synchronizations.

D.1 PROOF OF PROPOSITION 1

  • At synchronization boundary s, the proof starts from the momentum update and substitutes Eq. (5) into d_s.Source: §D.1 · p. 27
  • Taking norms and applying the triangle inequality yields a bound that is recursively expanded.Source: §D.1 · p. 27
  • The bound uses synchronization every H Policy updates, with each update satisfying |θ_t+1 − θ_t| ≤ δ; substitution and a geometric-series evaluation then lead to the momentum-update magnitude result.Source: §D.1 · p. 27

D.2 LOCAL DRIFT–NOISE ANALYSIS

The local drift–noise analysis models Policy displacement between synchronization boundaries and derives stationary tracking behavior for momentum transfer. It identifies a trade-off between parameter-space tracking and stability.

D.2 LOCAL DRIFT–NOISE ANALYSIS

  • The local model assumes approximately constant drift and independent noise across non-overlapping synchronization intervals, with noise covariance HΣ.Source: §D.2 · p. 28
  • The momentum displacement recurses from the prior displacement, the H-step drift, and interval noise; its stationary expected displacement and covariance are given in Eq. (21).The stated limits are E[d_s] → mH/(1−m)v and Cov(d_s) → m^2H/(1−m^2)Σ.Source: §D.2 · p. 28
  • EMA reduces stationary update covariance relative to full copying at the same synchronization interval H.Source: §D.2 · p. 28
  • Increasing momentum coefficient m raises mean tracking lag and tracking variance, while increasing synchronization interval H raises systematic displacement and accumulated noise.These results establish a parameter-space tracking–stability trade-off, not that Policy optimization necessarily improves rubric-generation or judging quality.Source: §D.2 · p. 28

Related papers

Loading 2609.36893v1…