Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Models

Motivation

• The premise remains untested because gradient conflict is measured on probe batches while the trade-off is evaluated across independent benchmark families after training.The introduction notes that prior work modified conflict without improving generation CLIPScore (p > 0.5), while no study measured conflict–trade-off correlation across a controlled population of runs.Source: §1 · p. 2

Audit design

• GRIDUMM enables an exact trade-off audit across 63 configurations while reproducing key structural features of unified multimodal model training.The audit combines observational correlations, bootstrap intervals, partial correlations controlling for training progress, reliability ceilings, and a causal intervention that monotonically suppresses conflict while holding data, schedule, and parameters fixed.Source: §1 · p. 2

Audit result

• Directional conflict metrics are weak, fragile predictors of the trade-off, and suppressing conflict does not improve it.The norm ratio is associated with generation failure, while training loss tracks the trade-off strongly and functional interference measures predict it better than conflict metrics, including when generation is mastered.Source: §1 · p. 2

Contribution

• The released audit protocol is intended to help practitioners test whether conflict reduction is a valid intermediate target and identify what to measure instead.The paper releases the GRIDUMM world, protocol, and measurement code.Source: §1 · p. 2

Scope

• The controlled testbed cannot establish that any specific production-scale conflict-metric claim is false; it defines what a validity claim must demonstrate.The authors state that the audit makes such a demonstration routine, while noting that the literature lacks it.Source: §1 · p. 2

Unified architectures

• Unified multimodal models share a trunk, parameter space, and two objectives despite varied tokenization and modeling designs; GRIDUMM abstracts these ingredients with a computable trade-off.Examples include Show-o (Xie et al., 2025a), Emu3 (Wang et al., 2024), Janus and related systems (Wu et al., 2025; Chen et al., 2025; Team, 2025; Zhou et al., 2025).Source: §2 · p. 3

Conflict-motivated UMM design and diagnosis without validation

• Prior UMM designs use gradient conflict as a premise, but the cited work does not validate conflict metrics as diagnostics of the understanding–generation trade-off.Conflict-driven examples include Uni-X (Hao et al., 2026), Symbiotic-MoE (Liu et al., 2026), and Pareto LoRA (Wei et al., 2026); Rao & Rachuri (2026) find no DPO variant improves generation CLIPScore despite near-orthogonal gradients.Source: §2 · p. 3

Multi-task optimization

• UMM optimizer interventions draw on multi-task learning methods that adapt loss weights or reshape gradient geometry, which GRIDUMM uses to vary conflict metrics across methods.The literature includes uncertainty-based weighting (Kendall et al., 2018), gradient surgery (Liu et al., 2021), and Pareto optimization.Source: §2 · p. 3

Modality imbalance

• Joint multimodal training can favor the faster-learning modality, and balancing methods adjust gradients or prototypes; the paper links norm-ratio diagnostics to generation failure, not directional conflict.The passage cites Wang et al. (2020) on greedy dynamics and balancing methods by Peng et al. (2022), Fan et al. (2023), and Wei et al. (2025b).Source: §2 · p. 3

Audits of benchmark validity

• The protocol follows audits of evaluation validity and addresses a stated gap: no analogous audit of training-time conflict metrics in UMMs.Prior audits include rebuilt test sets (Recht et al., 2019; Vasudevan et al., 2022) and controlled metric-learning re-evaluation (Musgrave et al., 2020).Source: §2 · p. 3

3.1 WHY A CONTROLLED TESTBED

• GRIDUMM tests two objectives on one shared decoder-only transformer, using a fixed-seed 5×5 colored-grid world to measure generalization on understanding questions and autoregressive scene generation.Generation is mastered before training ends by configurations that train its gradient at natural scale, while conflict magnitude is concentrated in the learning phase.Source: §3.1 · p. 4

3.2 TRAINING PROTOCOL

• Each run uses separate objective backward passes and captures per-group gradients; conflict statistics are measured every 50 steps on the same fixed probe batch across configurations.Runs use batch size 64 and a 900-step AdamW schedule; generation student forcing ramps to 0.400 after step 200.Source: §3.2 · p. 5

3.3 CONFLICT METRICS

• At each snapshot, the audit measures directional conflict, projected-away gradient energy, and the generation-to-understanding gradient-norm ratio over shared parameter groups.The norm ratio uses generation over understanding; at convergence, generation has the larger token budget but the smaller gradient norm.Source: §3.3 · p. 5

3.4 DOWNSTREAM TRADE-OFF

• The downstream trade-off is summarized by the mean of understanding and generation accuracy, with the two accuracy axes and empirical Pareto frontier also analyzed directly.Understanding is evaluated on 512 held-out questions and generation by object-cell accuracy on 256 held-out descriptions; min–max normalization is avoided because generation saturates.Source: §3.4 · p. 5

3.5 FUNCTIONAL INTERFERENCE MEASURES

• Four functional measures are compared with gradient geometry: probe separability, feature-subspace overlap, effective rank, and self-consistency on questions about model-generated scenes.Self-consistency is the fraction of questions about the model’s own generated scenes that it answers correctly.Source: §3.5 · p. 5

3.6 HYPOTHESES

• The hypotheses test conflict–trade-off correlation, training-loss validity, functional-measure prediction, and whether depth or stage weighting rescues conflict metrics; separate experiments assess causal suppression and replication at a second scale.E1 tests correlations across 63 configurations with bootstrap intervals and partial correlations; E2 tests loss prediction, E4 compares predictors, E3 intervenes on conflict, and E5 tests a second model scale.Source: §3.6 · pp. 5–6

4.1 SETUP

• The audit covered 63 base-scale configurations, 12 intervention configurations, and 18 larger-scale configurations: 93 training runs and 372 measured checkpoints.Source: §4.1 · p. 6

4.2 E1: DOES CONFLICT PREDICT THE TRADE-OFF?

• At 25% of training, no directional conflict metric predicted the final Pareto score at |ρ| ≥0.3 with a confidence interval excluding zero; Figure 2 reports ρ = −0.198 [−0.463, 0.079].Among generation-mastering configurations, directional metrics remained weak and sign-inconsistent (largest |ρ| = 0.23), while the norm ratio was null (ρ = +0.02).Source: §4.2 · pp. 6–7, Figure 2 · p. 7

4.3 E2: IS TRAINING LOSS A VALID SURROGATE?

• Final joint probe loss correlated with Pareto score at ρ = −0.926 across configurations, while the predictive correlation using loss at 25% of training was ρ = −0.424.Among 47 generation-mastering configurations, the loss association was ρ = −0.92; nevertheless, 74 of 1081 loss-ordered pairs reversed.Source: §4.3 · p. 7

4.5 E4: WHAT DOES PREDICT THE TRADE-OFF?

• Effective rank correlated with the Pareto score at ρ = 0.702 (CI 0.603–0.782), versus ρ = −0.187 for the best directional conflict metric.Within generation-mastering configurations, self-consistency reached ρ = 0.854 versus 0.230 for the norm ratio; self-consistency was ρ = −0.117 in the full population.Source: Table 2 · p. 8, §4.5 · p. 8

4.4 E3: INTERVENTION—SUPPRESSING CONFLICT CAUSALLY

• Completely suppressing conflict moved the joint score by -0.002 (understanding -0.004; generation object-cell +0.001), a shift within the un-intervened configuration’s seed-to-seed spread of 0.037.Source: §4.4 · p. 8

4.6 E5: DOES THE CONCLUSION SURVIVE A SCALE CHANGE?

• At the larger scale, directional conflict metrics remained within noise (largest |ρ| = 0.15, CI [−0.33, 0.04]), while the norm-ratio association was ρ = −0.42 (CI [−0.64, −0.20]).Source: §4.6 · p. 8

4.7 RELIABILITY CEILING AND STATISTICAL POWER

• Understanding and generation accuracy had ICCs of 0.982 and 1.000, respectively; the minimum detectable |ρ| was 0.362 at N = 63 and 0.178 at N = 252.After disattenuation, the configuration-level correlation was −0.324, and the audit was powered to detect moderate associations.Source: §4.7 · p. 8

What the audit does and does not show

• Directional conflict metrics weakly predicted the eventual trade-off (largest |ρ| = 0.14, interval crossing zero), and suppressing conflict did not move it.Larger concurrent configuration-level associations were sign-inconsistent, dissolved under the regime cut, and failed reliability-ceiling, progress-control, and per-layer-reweighting checks; production-scale behavior remains open.Source: §5 · p. 9

Why the field adopted an unvalidated proxy

• The field adopted conflict metrics because they are cheap, continuous, and measurable every step; the audit protocol costs a fraction of intervention methods’ compute and returns a go/no-go signal.The results do not make the proxy useless; they indicate that its validity depends on setting, which the settings where it was adopted had not checked.Source: §5 · pp. 8–9

The method population exhibits the pathologies the field fears

• Under an order-of-magnitude task-norm imbalance, MGDA yielded understanding 0.389 and generation 1.000; SOAP yielded object-cell accuracy 0.000 and joint score 0.271 despite healthy-range directional metrics.Method choice changed the two axes without any conflict metric tracking those differences; only norm ratio flagged SOAP's generation failure.Source: §5 · p. 9

Functional measures won—carefully interpreted

• Eff rank (ρ = 0.702) was the best functional predictor; self consistency (ρ = 0.854) stayed strong among generation-mastering configurations, while norm ratio (ρ = −0.651) was null there.The norm ratio functioned as a generation-failure detector; the authors interpret functional measures as leading candidates for capturing interference in representation geometry and cross-task consistency, not as settled answers.Source: §5 · p. 9

Limitations

• The audit used 3.2M–6.7M-parameter synthetic models and seven gradient-combination strategies; at 63 configurations, the minimum detectable correlation was 0.362.Generation was mastered at natural-scale weighting, so trade-off variation mostly lay on understanding; architecture and data-scheduling interventions were not swept. Checkpoint-level analysis with 252 rows tightened detection to 0.178.Source: §5 · p. 9

6 CONCLUSION

• Across 93 training configurations, directional conflict metrics gave weak, statistically fragile signals about the trade-off, and suppressing conflict causally did not improve it.Functional interference measures predicted the trade-off better, while training loss tracked it strongly, locating the failure specifically in gradient-conflict geometry.Source: §6 · p. 10

AI USE STATEMENT

• AI tools assisted with implementation, runs, analysis, and manuscript drafting, but not the core hypotheses or unverified literature selection.The authors state that citations were checked, claims were read against raw measurement tables, and they take responsibility for the final content.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

• The audit is reproducible from a released artifact containing model and world definitions, methods, intervention, seeds, run manifest, and analysis script.The complete compute budget was 18.8 aggregate run-hours on a single Apple M4 Max laptop.Source: REPRODUCIBILITY STATEMENT · p. 11

ETHICS STATEMENT

• The audit uses a fully synthetic testbed with no human subjects, personal data, or new data collection, and its protocol is publicly released for retesting.Claims were restricted to what the controlled setting demonstrates.Source: ETHICS STATEMENT · p. 11

Related work

• The references include unified multimodal models for understanding and generation, alongside work on modality conflict and cross-task consistency.Listed examples include unified autoregressive visual generation and understanding, Uni-X, and benchmarking cross-task consistency in unified multimodal models.Source: B. Efron. Bootstrap methods: Another… · pp. 12, 14
• The bibliography also lists statistical association, rater-reliability, and neural-network similarity or metric-evaluation references.These include Spearman’s work on association, Shrout and Fleiss on intraclass correlations, and surveys of neural-network similarity measures.Source: B. Efron. Bootstrap methods: Another… · pp. 12–13
• Prior work includes methods for resolving or modulating gradient conflict and balancing tasks in multitask and multimodal learning.Examples include conflict-averse gradient descent, prototypical modal rebalance, and on-the-fly gradient modulation.Source: B. Efron. Bootstrap methods: Another… · pp. 12–13
• The bibliography lists Wang et al.’s survey of unified multimodal understanding and generation models and Zhou et al.’s Transfusion model.The passages provide titles and publication details, not findings about these works.Source: B. Efron. Bootstrap methods: Another… · p. 15
• The bibliography also lists Zhuang et al.’s VarGPT work on unified understanding and generation in a visual autoregressive multimodal language model.The passage provides the title and publication details, not findings about the work.Source: B. Efron. Bootstrap methods: Another… · p. 15

Unified architectures, in detail

• Existing unified architectures share a trunk across objectives with different token budgets and output structures, the structural feature isolated by GRIDUMM.Examples include shared-transformer autoregressive and diffusion systems, early-fusion models, and unified vision-language-audio-action architectures.Source: A EXTENDED RELATED WORK · p. 15

Conflict-motivated UMM design, in detail

• Conflict-motivated designs use layer separation, expert disentanglement, gradient shielding, orthogonal projection, Pareto integration, or task-aware routing to address interference.Their reported benefits include training efficiency, benchmark scores, loss sample efficiency, and wall-clock speed; these do not establish that reducing conflict improves the trade-off.Source: A EXTENDED RELATED WORK · pp. 15–16

Diagnosis without validation, in detail

• Prior studies characterize conflict, inconsistency, and forgetting or mitigate their symptoms, but none validates a conflict metric against the trade-off across a controlled population.Rao & Rachuri (2026) report near-orthogonal gradients and 11–14× magnitude imbalance in Janus-Pro preference tuning, but do not measure conflict–trade-off correlation across architectures.Source: A EXTENDED RELATED WORK · p. 16

Multi-task optimization, in detail

• Multi-task optimization spans adaptive loss weighting, gradient rescaling or projection, bargaining solutions, and component alignment; GRIDUMM samples faithful implementations across these families.This population varies methods designed specifically to reshape gradient conflict.Source: A EXTENDED RELATED WORK · p. 16

Modality imbalance, in detail

• Prior modality-imbalance work shows gradient magnitude imbalance can be consequential; GRIDUMM finds its norm-ratio statistic diagnoses generation failure and is null where it would be deployed.Related approaches rebalance gradients during training, at the decision level, or through instantaneous probes; modality-laziness analyses study when a model ignores one modality.Source: A EXTENDED RELATED WORK · p. 16

Audits of benchmark validity, in detail

• Benchmark-validity audits expose proxy failures through controlled re-evaluation; this audit fixes the conflict-metric family, measures trade-off without benchmark idiosyncrasy, and scopes conclusions with reliability and power statements.Precedents include rebuilt ImageNet and CIFAR-10 test sets, controlled metric-learning re-evaluation, and reviews of unvalidated proxies in medical imaging.Source: A EXTENDED RELATED WORK · pp. 16–17

• Table 3 lists training and evaluation hyperparameters; large scale E5 differs from the base scale only in the listed rows.Source: Table 3 · p. 17
• The 108-token vocabulary combines grid symbols, position markers, cell tokens, and text tokens, with scenes describing 6–8 rectangular objects and three understanding-question types.The question types are cell-color lookup, full-scene color count, and color-pair adjacency.Source: B HYPERPARAMETERS AND IMPLEMENTATION… · p. 17
• Fixed probe and evaluation sets are identical across runs, while reliability resamples and student-forcing rollouts use specified seeds for comparable probe statistics.Probe and evaluation seeds are 1234 and 777; reliability resamples use 5101–5105, and probe rollouts use a run-independent seed per snapshot.Source: B HYPERPARAMETERS AND IMPLEMENTATION… · p. 17

Implementation setup

• Each method receives independent understanding and generation gradients for embeddings, six blocks, and the output head; naive joint training sums them (Vandenhende et al., 2021; Senushkin et al., 2023).Method descriptions follow published algorithms and standard reviews.Source: C METHOD IMPLEMENTATIONS AND… · p. 17

Weighting methods

• Token balancing uses a square-root-normalized variant with weights summing to 2 based on answer- and generation-token counts; GradNorm-style weights follow rate-residual gradients and are clamped to [0.1, 10].GradNorm reference norms are frozen after 100 steps.Source: C METHOD IMPLEMENTATIONS AND… · p. 17

Gradient combination methods

• PCGrad projects each task gradient onto the other’s normal plane when their inner product is negative, while CAGrad caps orthogonal components at c/K around the average gradient (Liu et al., 2021).PCGrad uses the original form without magnitude renormalization (Vandenhende et al., 2021; Senushkin et al., 2023); CAGrad uses δ=0.5.Source: C METHOD IMPLEMENTATIONS AND… · p. 17

Documented deviations

• MGDA-UB’s convex combination is rescaled to the average-gradient norm to avoid update-magnitude collapse, but this does not fix its minority-direction pathology when task gradient norms differ (Sener & Koltun, 2018).The two-task convex weights are solved in closed form.Source: C METHOD IMPLEMENTATIONS AND… · pp. 17–18
• The SOAP-style variant uses Adam moments in covariance eigenbases for 2-D matrices and RMS-normalizes updates, omitting momentum-translation scaling to keep learning rates comparable across optimizer families (Vyas et al., 2025).It refreshes bases every 50 steps; 1-D parameters use plain Adam.Source: C METHOD IMPLEMENTATIONS AND… · p. 18

Conflict intervention

• The α-intervention subtracts the generative gradient’s projection onto the normalized understanding gradient before naive combination, monotonically suppressing each group’s cosine conflict; α=0 reproduces matched E1 exactly.Cosine scaling is by (1−α), up to a norm correction.Source: C METHOD IMPLEMENTATIONS AND… · p. 18

D PROTOCOL DETAILS

• Student-forced generation ramps from zero before step 200 to 0.400 by step 500, capped at 16 samples per batch, while targets remain ground-truth colors.Student-forced colors are sampled at temperature 1.0 from the color alphabet at true marker anchors; rendering loss is computed at marker and end-of-stream positions. Probe snapshots use the same mixture and a run-independent generator seed.Source: D PROTOCOL DETAILS · p. 18
• Three unit-checked invariants verify causal-mask equivalence, scene-token round trips, and batched rendering-loss alignment against per-row computation.Cached and full-forward greedy decoding must match exactly in float64; a 500-scene round trip checks grid-value/color conventions, and loss checks account for varying description lengths in padded batches.Source: D PROTOCOL DETAILS · p. 18
• The trade-off score is the mean of raw understanding and generation accuracies, and the final-checkpoint Pareto frontier uses raw accuracy axes without min–max normalization.A configuration is dominated when another matches or exceeds it on both axes and is strictly higher on one; normalization was avoided because a saturating generation axis could amplify resample noise.Source: D PROTOCOL DETAILS · p. 18
• The audit uses Spearman correlations, percentile-bootstrap confidence intervals with B=5000, permutation-free normal-approximation p-values, partial Spearman correlations, and ICC(1,1).Pairs are resampled by configuration and checkpoints by run cluster; partial correlations rank-regress out training step. ICC(1,1) uses five resamples (Shrout & Fleiss, 1979); attenuation analysis uses relu relg, and minimum detectable effects use the Fisher-z two-sample approximation at α=0.05 and power 0.8 (Spearman, 1904; Efron, 1979).Source: D PROTOCOL DETAILS · p. 18

Weighted and per-layer analyses (HA-4)

• The shallow-weighted variant has ρ = −0.29, CI [-0.55, -0.01], below the |ρ| ≥0.3 threshold; blocks 5 and 6 have intervals excluding zero in the concurrent per-layer profile.The per-layer profile does not establish the shallow-and-deep signature reported by Uni-X (Hao et al., 2026), and is an association analysis rather than a prospective layer-wise prediction test.Source: E AUDIT DESIGN AND SUPPLEMENTARY… · p. 19, Figure 5 · p. 21

Training dynamics

• Throughout training, conflict rate stays elevated while the generation-to-understanding norm ratio decays as the generation objective converges.Figure 6 reports the median and interquartile range across E1 configurations.Source: E AUDIT DESIGN AND SUPPLEMENTARY… · p. 19, Figure 6 · p. 21

Dose–response detail

• Across intervention doses, effective conflict follows the dose while raw conflict shows co-adaptation, but the downstream Pareto score stays flat.The conflict manipulation check averages learning-phase snapshots at steps 200–500; the causal readout is the final-checkpoint Pareto score.Source: E AUDIT DESIGN AND SUPPLEMENTARY… · p. 19, Figure 7 · p. 22

Predictor competition and cross-scale replication

• At the larger scale, directional conflict metrics remain within noise (largest |ρ| = 0.15, CI [-0.33, 0.04], against 0.19 at base scale), while the norm-ratio association persists (ρ = −0.42, CI [-0.64, -0.20]).Figure 8 visualizes checkpoint-level predictor competition; Figure 9 shows the per-metric cross-scale correlations.Source: E AUDIT DESIGN AND SUPPLEMENTARY… · p. 19, Figure 8 · p. 22, Figure 9 · p. 23

Pareto plane

• The final-checkpoint empirical Pareto frontier contains 3 configurations (pcgrad (2), cagrad (1)); method choice moves configurations on both axes without directional conflict metrics tracking those differences.Figure 10 plots the understanding/generation plane across E1 configurations.Source: E AUDIT DESIGN AND SUPPLEMENTARY… · p. 19, Figure 10 · p. 23