PaperlayerView exact revision on arXiv

Source-linked AI summary

G$^3$-LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA

Jia Song, Wenhow Li, Lichen Bai, Bada Ye, Zeke Xie

arXiv:2609.35189v1cs.CVcs.LG

TL;DR

Heterogeneous reward-weighted video categories may induce incompatible updates, making it uncertain which should share an adapter. G3-LoRA groups categories by residual gradient compatibility, trains group-specific LoRA experts, and consolidates them. On Wan2.1, the grouped adapter’s mean VBench2.0 score exceeds both the base model and joint LoRA.

  • Problem

    Post-training commonly mixes heterogeneous categories into one adapter despite uncertainty about whether their optimization directions are compatible.

    Source: §1 · p. 1

  • Method

    G3-LoRA removes the shared global gradient direction, clusters categories by residual compatibility, trains group-specific LoRA experts, and consolidates them by merging and on-policy distillation.

    Source: §1 · p. 2

  • Results

    Grouped LoRA reaches 48.18 mean score, +1.11 over the base model and +1.19 over joint LoRA.

    Source: §3.3 · p. 7

  • Takeaways & Limitations

    Residual gradient compatibility offers a practical diagnostic for organizing reward-weighted post-training data.

    Source: §1 · p. 2, §6 · p. 10

    The study covers two text-to-video backbones up to 2B parameters, uses VBench2.0-derived categories and coupled vision-language rewards, and reports no human study.

    Source: §5 · p. 10

Abstract

from arXiv · show

Post-training foundation video models on heterogeneous reward-weighted data usually assume that all data categories induce compatible updates. This assumption is fragile when categories correspond to different skills, domains, or evaluation dimensions. We study this problem in text-to-video post-training, where VBench2.0 dimensions define data buckets and an external multimodal reward pipeline assigns sample weights. We propose G$^3$-LoRA (Gradient-Guided Grouped LoRA), a data organization procedure that probes category-level gradients induced by reward-weighted video samples, removes the shared global update direction, clusters categories by residual gradient compatibility, trains group-specific LoRA experts, and consolidates them into one adapter by weight merging followed by on-policy distillation from the experts. We motivate this procedure by viewing reward-weighted flow matching as velocity-field regression: incompatible reward dimensions may prefer different denoising directions in overlapping noisy latent regions, causing shared LoRA training to average capabilities. On Wan2.1-T2V-1.3B-Diffusers, the merged grouped adapter improves the matched VBench2.0 evaluation over the base model, a joint reward-weighted LoRA baseline, and random, semantic, and raw-gradient partitions trained with the same pipeline; an independent evaluator agrees, and on CogVideoX-2B grouping avoids the negative transfer of joint training. The gain is not uniform: merging compresses the largest specialist gains, distillation recovers part of this loss, and camera motion and several local-quality dimensions remain challenging. Together, these results suggest that gradient compatibility can serve as a practical diagnostic for organizing reward-weighted video post-training data.

1 INTRODUCTION

The paper asks whether heterogeneous reward-weighted video categories should share an adapter, rather than assuming their update directions are compatible. It proposes grouping categories by residual gradient compatibility and evaluates that approach on two video backbones.

1 INTRODUCTION

  • A single mixed adapter risks degrading conflict-heavy dimensions, while one adapter per dimension avoids interference but loses data sharing.Source: §1 · p. 2
  • Reward-weighted categories may prefer incompatible velocity directions in overlapping noisy latent regions, so shared MSE training can average capabilities.This can improve average quality while degrading dimensions with greater conflict.Source: §1 · p. 2
  • G3-LoRA removes the shared global update direction, clusters residual category gradients, trains group-specific LoRA experts, then merges and distills them.On-policy distillation uses the merged adapter’s sampling trajectories.Source: §1 · p. 2
  • The study uses 17 non-diversity VBench2.0 dimensions and Qwen3-VL-based QA rewards, while the procedure is presented as applicable to other reward-bearing category partitions.Source: §1 · p. 2
  • The proposed residual compatibility measure reveals conflicts reproducible across probe sets and backbones and predicts held-out transfer.Source: §1 · p. 2
  • Experiments use two video backbones, matched grouping controls, an independent evaluator, and paired bootstrap intervals to examine merging losses and distillation recovery.Source: §1 · p. 2

2 METHOD

The method frames reward-weighted video post-training as category organization: estimate category gradients, remove their shared direction, and group compatible categories before training and consolidating LoRA experts.

2.1 PROBLEM SETTING

  • Each category contains self-generated videos, prompts, and scalar weights from a dimension-specific multimodal QA pipeline; the implementation uses 17 non-diversity VBench2.0 dimensions.Source: §2.1 · p. 2
  • Reward weights multiply the flow-matching loss, changing which samples shape category gradients; joint LoRA instead minimizes the average loss across categories.Source: §2.1 · p. 3
  • Under MSE, reward dimensions favoring different target velocities can yield a conditional-mean direction that softens motion or attribute changes.The paper uses this as a local explanation for improved mean scores alongside degradation on conflict-heavy dimensions.Source: §2.1 · p. 3

2.2 GRADIENT-GUIDED GROUPED LORA

  • For each category, the method estimates a reward-weighted LoRA gradient from a fixed probe subset, fixing timesteps and noise seeds to reduce probe noise.Probing can be repeated at several stages of a reference joint-training trajectory.Source: §2.2 · p. 3
  • It removes each category gradient’s projection onto the global mixed gradient because that shared update can mask category-specific conflicts.Direct subtraction produces the same partition in the reported comparison (adjusted Rand index 1.0).Source: §2.2 · p. 4
  • Residual compatibility is averaged across probe stages and clustered by average-linkage hierarchical clustering into five groups in the experiments.Source: §2.2 · p. 4
  • Each group trains an expert, whose dense weight delta is group-size-weighted and recompressed into one LoRA adapter; merging may cancel or otherwise dilute specialist gains.Source: §2.2 · p. 4
  • On-policy distillation fine-tunes the merged adapter to match the prompt’s group expert on latents from its own sampling trajectories, while anchoring it to the merged adapter.The loss uses neither reward weights nor training videos.Source: §2.2 · p. 4

3 EXPERIMENTS

The experiments test residual-gradient grouping on Wan2.1 and CogVideoX-2B, finding that the grouped adapter improves Wan2.1’s mean score over the base model and joint LoRA, while retaining dimension-specific tradeoffs. On CogVideoX-2B, grouping avoids joint training’s large degradation; expert distillation recovers part of the loss from merging.

3.1 EXPERIMENTAL SETUP

  • The setup uses 17 non-diversity VBench2.0 dimensions, approximately 560 generated videos per dimension, and Qwen3-VL-8B-Instruct reward weights, with matched prompts, seeds, and evaluator settings.The main comparisons include joint LoRA, random, semantic, and raw-gradient partition controls, plus Grouped LoRA and Grouped LoRA + OPD.Source: §3.1 · pp. 4–5

3.2 GRADIENT GROUPING ANALYSIS

  • Residual-gradient grouping yields five groups, with M = 5 raising within-group mean similarity from 0.345 to 0.473 and leaving no negative within-group pairs.Compared with M = 4, it also improves within-minus-between separation from 0.559 to 0.633; Figure 2 visualizes the residual similarities after removing the global mixed-gradient direction.Source: §3.2 · p. 5, Figure 2 · p. 6

3.3 VBENCH2.0 RESULTS

  • The grouped partition outperforms the tested alternatives: random partitions score 46.38 and 45.82, while semantic and raw-gradient partitions score 47.20 and 47.35.The semantic partition puts 13 negatively related pairs together, versus none for residual grouping, indicating the partition matters beyond simply splitting dimensions among experts.Source: §3.3 · pp. 6–7
  • 48.18 mean score: Grouped LoRA gains +1.11 over the base model and +1.19 over joint LoRA on Wan2.1.The gain over the base model is positive in 96.6% of paired bootstrap resamples, but its 95% CI is [−0.08, +2.26].Source: Table 1 · p. 6, §3.3 · p. 7

3.4 SECOND BACKBONE AND INDEPENDENT EVALUATOR

  • On CogVideoX-2B, joint training falls from 39.87 to 27.95, while the grouped adapter reaches 40.99; VideoScore-v1.1 also favors Grouped LoRA over the base model on all five aspects.The CogVideoX margin mainly reflects avoided negative transfer rather than a large gain over its base model; the independent evaluator scores the same Wan2.1 videos.Source: §3.4 · p. 8, Table 2 · p. 8

3.5 DIMENSION-LEVEL DIAGNOSTICS

  • Group-only experts average 48.06 on assigned dimensions versus 48.18 for the merged adapter and 47.07 for the base, but the merged adapter trails experts on all six dimensions where experts beat the base.Distillation raises the mean from 48.18 to 49.91 (+1.73; paired 95% bootstrap CI [+0.42, +3.05]), improving 12 of 17 dimensions over the base; camera motion is unchanged and composition decreases slightly.Source: §3.5 · p. 9

4 RELATED WORK

The paper places G3-LoRA at the intersection of gradient-conflict methods and weight-space merging, using residual category gradients to group experts before consolidation. It organizes reward-weighted video categories rather than proposing a new reward objective.

4 RELATED WORK

  • The paper uses VBench2.0 dimensions as both evaluation metrics and post-training data categories, against a broader multi-axis video-evaluation backdrop.Related work also covers reflective sampling methods that improve generation without retraining.Source: §4 · p. 9
  • Unlike RLHF/DPO-style alignment and diffusion reward-optimization methods, the goal is organizing reward-weighted video data, not introducing a new objective.CRAFT studies composite-reward filtering followed by supervised fine-tuning; this work focuses on which reward-weighted categories should share updates.Source: §4 · p. 9
  • G3-LoRA probes residual category gradients before training specialists, then merges the resulting LoRA experts into one deployable adapter.This connects gradient-conflict and weight-space-merging approaches.Source: §4 · p. 9
  • The consolidation follows on-policy distillation and multi-teacher work, but constructs teachers through residual-gradient grouping rather than single-reward specialization.MaineCoon also consolidates specialized LoRA experts but uses preference optimization and reinforced on-policy distillation for streaming audio-visual generation.Source: §4 · p. 9

5 LIMITATIONS

The study is limited to two open text-to-video backbones of at most 2B parameters and a VBench2.0-derived taxonomy. Evaluator coupling, absent human evaluation, added distillation cost, and limited controls qualify the findings.

5 LIMITATIONS

  • The study covers two open text-to-video backbones of at most 2B parameters and a VBench2.0-derived category taxonomy.The CogVideoX-2B gain over the base model is small; broader tests across model families, reward sources, and evaluation suites remain a next step.Source: §5 · p. 10
  • Qwen3-VL rewards and VBench2.0 judgments are coupled, VideoScore agreement reduces but does not remove that coupling, and no human study was run..Source: §5 · p. 10
  • On Wan2.1 the merged adapter’s 95% interval over the base includes zero, while OPD adds about 205 GPU-hours and was not applied to control partitions or CogVideoX.The compatibility measure is a lightweight practical probe, not a complete account of denoising dynamics.Source: §5 · p. 10

6 CONCLUSION

The paper presents G3-LoRA as a way to organize heterogeneous reward-weighted post-training data by gradient compatibility. Results support the approach with qualifications: grouping improves mean performance and reduces some negative transfer, while distillation recovers part of merging losses.

6 CONCLUSION

  • Grouped LoRA improves mean VBench2.0 score over the base model and joint LoRA, reduces the worst dimension-level drop, and avoids joint training’s negative transfer on CogVideoX-2B.Distillation recovers part of the loss from merging.Source: §6 · p. 10
  • G3-LoRA computes category gradients, removes the shared global direction, clusters residual compatibility, trains group-specific LoRA experts, then consolidates them by merging and on-policy distillation..Source: §6 · p. 10
  • The paper frames reward-weighted video SFT as inducing multiple conditional velocity fields and asks which reward-weighted samples should share update directions.The results position gradient compatibility as a practical organizing signal for reward-weighted video post-training.Source: §6 · p. 10

ETHICS STATEMENT

The method organizes post-training data rather than introducing a new video-generation backbone. Its potential benefits are data efficiency and diagnosability, while improved generation can also improve misleading or harmful media.

ETHICS STATEMENT

  • G3-LoRA is a post-training data-organization procedure, not a new video-generation backbone..Source: ETHICS STATEMENT · p. 10
  • The method may make post-training more data-efficient and diagnosable by exposing interference among training categories..Source: ETHICS STATEMENT · p. 10
  • Improved video generation can also improve misleading or harmful generated media, so releases should respect base-model and benchmark-asset constraints and document sample filtering..Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement describes the paper’s methods and evaluation protocols and identifies planned release artifacts.

AI USE STATEMENT

The appendix documents AI-tool use, evaluation and reward procedures, partition controls, and dimension-level results. The rank-40 grouped adapter is the main grouped comparison.

AI USE STATEMENT

  • The authors report using generative AI for writing and formatting, with its use in data generation, reward construction, distillation, and evaluation described in the paper.Source: AI USE STATEMENT · p. 10

A TRAINING, DISTILLATION, AND EVALUATION PARAMETERS

  • The main LoRA comparisons use 25 prompts per dimension and two videos per prompt under a matched quick VBench2.0 protocol.The protocol holds generation and evaluator settings identical across base, joint, partition-control, grouped, and OPD comparisons.Source: A TRAINING, DISTILLATION, AND… · p. 14

B QWEN-BASED REWARD CONSTRUCTION

  • Qwen3-VL generates dimension-specific yes/no questions and evaluates generated videos; matching expected answers earns sample credit, while samples without valid questions are skipped.The reward audit covers all 17 dimensions; camera motion has the largest low-QA filtering loss, retaining 512 of 560 samples.Source: B QWEN-BASED REWARD CONSTRUCTION · p. 15, Table 7 · p. 15

C PARTITION CONTROLS

  • Random and semantic partitions use the same grouped-LoRA training and merge pipeline to test whether gains depend on residual-gradient grouping rather than smaller adapters alone.The raw-gradient control clusters category-gradient cosines without removing the global direction.Source: C PARTITION CONTROLS · pp. 15–16

D DIMENSION-LEVEL VBENCH2.0 TABLE

  • Rank-8 merging scores 47.41 versus 48.18 for rank-40 on the average score, so Grouped r40 is the main grouped adapter.Rank-8 improves mechanics, material, and thermotics but sharply drops multi-view consistency and human clothes consistency.Source: D DIMENSION-LEVEL VBENCH2.0 TABLE · p. 16, Table 8 · p. 16

E GRADIENT COMPATIBILITY DIAGNOSTICS

The diagnostics examine the stability and transfer relevance of residual-gradient compatibility, then describe how group experts are merged into one LoRA adapter.

E GRADIENT COMPATIBILITY DIAGNOSTICS

F DENSE DELTA-SVD LORA MERGE

  • The five group experts’ dense weight deltas are averaged with group-size weights, then recompressed by truncated SVD into a rank-40 LoRA adapter.For Wan2.1, five rank-8 experts span at most 40 dimensions per layer, so recompression reproduces the merged update up to numerical error.Source: F DENSE DELTA-SVD LORA MERGE · pp. 19–20

G MERGE RETENTION EVALUATION

The retention evaluation compares group experts with the merged adapter on assigned dimensions; the OPD details report how recovery varies across dimensions.

G MERGE RETENTION EVALUATION

  • The rank-40 merge preserves most average group-only performance, but Groups 4 and 5 lose 3.92 and 6.37 points after merging.Groups 1 and 2 score better after merging; multi-view consistency and mechanics show the largest dimension-level compression.Source: G MERGE RETENTION EVALUATION · p. 20, Table 14 · p. 20, Table 15 · p. 20
  • A post hoc choice of the better assigned expert or merged adapter gives a diagnostic mean of 51.07, while Group 5’s small learned delta and 2/17 merge weight are consistent with mechanics dilution.The diagnostics rule out rank truncation and broad cancellation as the main causes of specialist compression.Source: G MERGE RETENTION EVALUATION · p. 20, Table 16 · p. 21
  • Multi-view consistency compression is more consistent with averaging or activation-level interference because its Group 4 norm share is not small.Source: G MERGE RETENTION EVALUATION · p. 21

H ON-POLICY DISTILLATION DETAILS

  • OPD recovers 8.07 of 21.61 lost points on multi-view consistency and 0.71 of 15.14 on mechanics, while composition moves 0.50 farther from its expert.Its paired 95% bootstrap interval for mean change relative to the merged adapter is [+0.42, +3.05].Source: H ON-POLICY DISTILLATION DETAILS · p. 21, Table 17 · p. 21

I COGVIDEOX-2B EXPERIMENT

On CogVideoX-2B, grouped training outperforms joint LoRA, whose best retained checkpoint remains below the base model; the section also reports paired bootstrap and VideoScore-v1.1 evaluations.

I COGVIDEOX-2B EXPERIMENT

  • The CogVideoX-2B experiment uses the same 9,520 videos, prompts, reward weights, and LoRA and optimizer configuration for joint and grouped training.Five disjoint rank-128 experts are merged into a rank-640 adapter; scores are VBench2.0 macro means over 17 dimensions, with multi-view consistency equal to 0 for all methods.Source: I COGVIDEOX-2B EXPERIMENT · p. 21
  • Joint LoRA falls below the base model at 25% exposure, and its best retained checkpoint remains below both the base model and grouped adapter.Table 18 reports Joint LoRA across retained checkpoints.Source: I COGVIDEOX-2B EXPERIMENT · p. 21, Table 18 · p. 22
  • +13.04 points is the grouped–Joint difference on CogVideoX-2B, with a paired prompt-cluster 95% bootstrap interval of [+9.08, +16.92].Source: I COGVIDEOX-2B EXPERIMENT · p. 22
  • Joint LoRA improves human anatomy (48.5 to 98.4), human identity (70.9 to 97.9), and thermotics (46.8 to 56.0), but degrades six other listed dimensions.The deteriorating dimensions are composition, camera motion, mechanics, human interaction, motion order, and motion rationality.Source: I COGVIDEOX-2B EXPERIMENT · p. 22

J INDEPENDENT EVALUATION WITH VIDEOSCORE

  • On matched Wan2.1 evaluation videos, VideoScore-v1.1’s derived mean has a paired 95% CI of [0.140, 0.192] and P(∆> 0) = 1.000.The derived mean averages five aspects; Table 19 reports the evaluation.Source: Table 19 · p. 22
  • For Grouped LoRA versus Base on VBench2.0, the paired bootstrap estimate is +1.11 points, with a 95% interval of [−0.08, +2.26] and P(∆> 0) = 0.966.The protocol resamples prompts as clusters, keeps each prompt’s two videos paired, and uses 10,000 resamples.Source: J INDEPENDENT EVALUATION WITH VIDEOSCORE · p. 22
Loading 2609.35189v1…