Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Models

• Connector-based models avoid joint optimization of understanding and generation, but fine-tuning a large diffusion generator such as Wan2.1-14B [44] remains prohibitively expensive.This overhead makes it challenging to achieve strong semantic alignment and high visual fidelity together.Source: §1 · pp. 1–2
• VR-Bench evaluates inferred-intent alignment with generated video across eight dimensions, including physical-world reasoning, commonsense reasoning, and embodied interactions.Source: §1 · p. 3
• During training, Lumos-Nexus aligns only a lightweight diffusion generator with the understanding block; at inference, UPFB progressively hands generation to a pretrained high-capacity generator.The shared homogeneous latent space enables coarse-to-fine refinement, with the lightweight generator encoding semantic cues as structural priors and the large generator contributing high-fidelity synthesis.Source: §1 · pp. 2–3
• Lumos-Nexus significantly improves visual realism and temporal coherence on VBench while maintaining strong reasoning-guided generation on VR-Bench.The reported VR-Bench evaluation concerns alignment between inferred intent and generated video content.Source: §1 · p. 3

Video Generation Models

• Diffusion transformers with spatiotemporal attention improve motion consistency and high-resolution scalability, while autoregressive models enable explicit temporal modeling but can incur latency and accumulated errors [56, 16, 21, 27, 42, 61, 18, 25, 8, 47, 50, 37, 2, 45, 28, 59, 30, 20, 22, 3, 63, 44, 41, 62, 34, 55].Earlier diffusion methods extended 2D U-Nets to video, limiting temporal expressiveness.Source: §2 · p. 3

Video Unified Models

• Video unified models support semantic grounding and logical reasoning in synthesis, with joint-attention designs offering scalability but higher training cost and connector-based designs offering efficiency but difficulty scaling to large generators [54] [29] [48] [5] [39].This work focuses on the connector-based paradigm under practical computational constraints.Source: §2 · p. 3

3.1 Preliminary

• In connector-based unified models, the understanding block’s representation is transformed by a connector into generator-compatible conditioning, and the generator is fine-tuned to use it.For video, temporal sequences increase input length and computational complexity, making training more costly than in image unified models, especially with large diffusion generators.Source: §3.1 · p. 4

3.2 Overview

• Lumos-Nexus trains only a small generator alongside the understanding block, then uses shared-latent-space UPFB at inference to combine its semantic guidance with a large pretrained generator’s visual detail.This avoids the large-scale fine-tuning cost of integrating the high-capacity generator into unified-model training while aiming to retain reasoning ability and improve visual fidelity.Source: §3.2 · p. 4

3.3 Unified Progressive Frequency Bridging

• UPFB progressively shifts from the small generator’s semantic and global-layout control to the large generator’s late-stage detail refinement using temporal weighting and time-varying frequency decomposition.The bandwidth decays over time: large early values suppress high-frequency noise, while smaller later values restore fine structure.Source: §3.3 · p. 5
• UPFB asymmetrically fuses frequency components, suppresses noisy high-frequency contributions from the small generator with γhf ∈[0.5, 0.8], and uses RMS alignment and re-balancing to stabilize fused velocities.The large generator supplies high-frequency textures while the small generator maintains semantic coherence; RMS normalization mitigates magnitude mismatch and helps prevent over-exposure and unstable activations.Source: §3.3 · pp. 5–6

3.4 VR-Bench: Benchmark for Reasoning-Driven Video Generation

• VR-Bench fills the reasoning-evaluation gap with an eight-dimensional suite for assessing whether video generation translates inferred intent into coherent, semantically aligned content.The benchmark targets reasoning capabilities overlooked by benchmarks focused on visual fidelity and temporal coherence.Source: §3.4 · p. 6
• VR-Bench assesses motion and physical reasoning through Dynamic Reference Frame, Energy Transfer Visualization, Material Memory Consistency, and Preventive Causal Reasoning.These dimensions cover motion and spatial relations across viewpoints, momentum and energy, material deformation and recovery, and anticipatory causal relations.Source: §3.4 · pp. 6–7
• Its other dimensions are Conceptual Action Reasoning, Cultural Commonsense Reasoning, Biological Behavior Reasoning, and Concurrent Action Coordination.They assess abstract action sequences, social and cultural context, biomechanical and ecological plausibility, and simultaneous multi-action dynamics.Source: §3.4 · p. 7

4.1 Experimental Settings

• Lumos-Nexus uses Omni-Video [39] with Wan2.1-T2V-1.3B [44] as its small generator and Wan2.1-T2V-14B as its large generator in the same homogeneous latent space.The baseline Omni-Video generator is already fine-tuned within the unified framework.Source: §4.1 · p. 7
• T2V evaluation combines VBench [17] for perceptual quality with VR-Bench for reasoning alignment across 216 cases and eight dimensions.VR-Bench covers high-level physical-world reasoning, high-level commonsense reasoning, and embodied physical reasoning; each dimension is scored from 0 to 1, and Qwen3-VL-30B-A3B-Instruct [57] is used for all evaluations.Source: §4.1 · p. 7
• UPFB uses γw = 0.3, σmin = 0.35, σmax = 0.70, and γhf = 0.7; inference uses 50 sampling steps and CFG scale 5.Generation is at 480p, with 81-frame training clips corresponding to five seconds at 16 FPS; CFG scale 5 is used for both T2I and T2V.Source: §4.1 · p. 7

5.1 Quantitative T2V Results

• 84.12 overall, Lumos-Nexus achieves the highest VBench-T2V score, surpassing conventional video generation and recent video unified models.It combines Omni-Video’s semantic understanding with Wan2.1-14B’s high-fidelity detail synthesis, alleviating the short-board effect of limited generator capacity.Source: §5.1 · p. 7, Table 1 · p. 8
• 79.28 overall, Lumos-Nexus scores highest among Wan2.1-level models on VR-Bench, with best results in HL-Comm. (77.57) and Emb.-Phys. (81.54).The results demonstrate reasoning-oriented video generation performance across the benchmark’s reasoning categories.Source: Table 2 · p. 8, §5.1 · p. 8
• 81.90 overall, Lumos-Nexus* outperforms Wan2.2-TI2V-5B (75.35) and Wan2.2-T2V-A14B (80.98) on VR-Bench.Lumos-Nexus* reaches HL-Comm. 79.43 and Emb.-Phys. 83.93; reported scores include DRF 97.92, ETV 70.43, CCR 84.87, and CAC 88.99.Source: Table 2 · p. 8, §5.1 · pp. 8–9
• Lumos-Nexus improves semantic alignment from 79.10 to 80.52 while enhancing expressive power and overall video generation quality.The reported results also indicate that the framework generalizes across video generation architectures.Source: §5.1 · pp. 8–9

5.2 Qualitative T2V Comparison

• Lumos-Nexus produces more realistic, physically consistent results and expresses nuanced semantics more effectively than the compared models.Examples include physical regularities such as ripple propagation and nuanced actions such as musical rhythm and tongue–paw contact; Omni-Video shares its semantic priors but has limited visual fidelity.Source: §5.2 · p. 9, Figure 4 · p. 9

6.1 Impact of varying γw on wt

• γw = 0.3 achieves the best reported results across both benchmarks: VBench total score 84.12 and semantic alignment 80.52, and VR-Bench reasoning performance 79.28.It particularly improves High-Level Commonsense reasoning (77.57) and Embodied Physical Reasoning (81.54); larger γw preserves more coarse layout and camera motion, while smaller values favor high-frequency detail.Source: §6.1 · p. 10

6.2 Effect of σmin and σmax in UPFB

• 84.12 is the highest VBench total score, achieved with (σmin = 0.35, σmax = 0.70); overly narrow or wide bandwidths reduce semantic and perceptual performance.Overly small values restrict the semantic foundation, while excessively large values introduce unstable high-frequency components and weaken coherence.Source: §6.2 · pp. 10–11, Table 4 · p. 11

6.3 Ablation of RMS Alignment and Energy Re-Balancing in UPFB

• With RMS alignment enabled versus removed, VBench total score rises from 84.07 to 84.12, quality score from 84.98 to 85.03, and semantic score from 80.43 to 80.52.RMS alignment prevents magnitude mismatch when integrating the generators’ velocity fields, supporting smoother denoising dynamics and visual coherence.Source: §6.3 · p. 11, Table 5 · p. 11

7 Conclusion

• Lumos-Nexus aligns a lightweight generator during training, then uses UPFB to hand generation to a high-capacity pretrained generator for coherent coarse-to-fine synthesis.The framework is designed to preserve reasoning-driven generative capability while enhancing visual fidelity at low training cost.Source: §7 · p. 11

Appendix of Lumos-Nexus

• The appendix covers the UPFB workflow, VR-Bench descriptions and examples, additional T2I results, ablations, and limitations.Source: Appendix of Lumos-Nexus · p. 16

A Algorithmic Workflow of Lumos-Nexus

• UPFB fuses both generators’ velocity predictions through cosine-scheduled temporal gating and time-varying frequency decomposition, then rebalances the fused signal before flow-matching latent updates.Asymmetric fusion lets the lightweight generator guide early semantic formation while the high-capacity generator progressively adds high-frequency detail.Source: A Algorithmic Workflow of Lumos-Nexus · p. 16

B Detailed Descriptions of VR-Bench

• VR-Bench assesses eight reasoning dimensions across physical, commonsense, and embodied categories using eight-question prompts scored by Qwen3-VL-30B-A3B-Instruct [57].Questions progress from basic perception through relational reasoning to causal or semantic reasoning; yes/no responses are aggregated with layer-wise weighting.Source: B Detailed Descriptions of VR-Bench · pp. 17–18

B.1 High-Level Physical World Reasoning

• The physical-world dimensions assess camera-relative spatial consistency (DRF), energy transfer (ETV), and material deformation and recovery (MMC).Source: §B.1.1 · p. 18, §B.1.2 · p. 18, §B.1.3 · p. 18

B.2 High-Level Commonsense Reasoning

• The commonsense dimensions assess purposeful action sequences (CAR), culturally grounded scenes and behavior (CCR), and preemptive action to avert negative outcomes (PCR).Source: §B.2.1 · p. 18, §B.2.2 · p. 19, §B.2.3 · p. 19

B.3 Embodied Physical Reasoning

• Embodied dimensions assess anatomically constrained organism movement and environmental response (BBR), alongside coordination of simultaneous actions (CAC).Source: §B.3.1 · p. 19, §B.3.2 · p. 19

B.4 Summary

• VR-Bench provides an interpretable framework for diagnosing whether video generators understand depicted worlds through physical, causal, biological, and cultural reasoning.Figures 7 and 8 present quantitative comparisons across all eight dimensions.Source: Figure 7 · p. 20, §B.4 · p. 21, Figure 8 · p. 21

C Quantitative T2I Results

• Against Omni-Video’s 0.75 on GenEval, Lumos-Nexus improves Two Objects, Colors, Position, and Attribute Binding, reaching 0.67 vs. 0.56 on Attribute Binding.These gains indicate stronger compositional understanding and instruction adherence.Source: C Quantitative T2I Results · p. 21, Table 6 · p. 22

D.1 Discussion on Bridging Large and Small Generators in the Video Unified Model

• Direct Add, which averages both generators’ noise predictions at every sampling step, often creates duplicated or structurally inconsistent subjects, including birds with two heads.The examples illustrate failures despite prompts specifying a single bird.Source: §D.1 · p. 21, Figure 9 · p. 22
• Lumos-Nexus uses temporally scheduled, frequency-aware fusion to respect the small and large generators’ respective strengths.The passage also attributes the gains to frequency-domain fusion and velocity decomposition.Source: §D.1 · p. 21
• Compared with uncontrolled blending, Lumos-Nexus produces cleaner structures, maintains single-subject consistency, and improves realism and semantic alignment.The qualitative comparison is shown in Figure 9.Source: §D.1 · p. 21, Figure 9 · p. 22

C Quantitative T2I Results

• Lumos-Nexus scores 0.79 overall on GenEval, the best result among unified models, and surpasses prior approaches across most evaluation dimensions.The comparison is reported in Table 6.Source: C Quantitative T2I Results · p. 21, Table 6 · p. 22

D.1 Discussion on Bridging Large and Small Generators in the Video Unified Model

• Lumos-Nexus exceeds Direct Add by +0.84 on VBench (84.12 vs. 83.28) and +1.95 on VR-Bench (79.28 vs. 77.33).Table 7 compares bridging strategies between the small and large generators.Source: §D.1 · p. 21, Table 7 · p. 23

D.2 VR-Bench Cross-Evaluator Robustness

• Lumos-Nexus remains the top-performing major open-source baseline across GPT-5.2 and Qwen3-VL evaluations, whose overall rankings are largely consistent.The evaluation replaces Qwen3-VL-30B-A3B-Instruct [57] with GPT-5.2 and re-evaluates the main baseline methods from Tab. 2.Source: §D.2 · p. 23

D.3 Discussion of Model Efficiency

• +0.43 VBench (84.12 vs. 83.69) and +1.05 VR-Bench (79.28 vs. 78.23) accompany Lumos-Nexus’s 40.5 vs. 35.1 s/step inference latency against Wan2.1-14B.Its FLOPs are 4.568P vs. 3.462P per step, respectively.Source: §D.3 · p. 23
• Training Lumos-Nexus matches the small-generator cost of 2.25 s/it and 26.3GB, versus 9.72 s/it and 94.7GB for the large generator.It avoids large-model training and optimizes only lightweight bridging modules.Source: §D.3 · p. 23

D.4 Disentangling Model Capacity from UPFB Gains

• Omni-Video* with a 14B generator scores 81.73 vs. Lumos-Nexus’s 84.12 on VBench and 70.63 vs. 79.28 on VR-Bench despite higher training costs.Omni-Video* training rises from 2.25 to 7.21 s/it and from 26.3GB to 72.8GB; its inference cost is 36.1 s/step and 3.513P FLOPs.Source: §D.4 · p. 24
• The authors attribute Omni-Video*’s shortfall to Wan2.1-14B’s inability to natively accept VLM tokens, which makes cross-modal alignment difficult for limited LoRA fine-tuning.They conclude that the gains stem from UPFB, which integrates Wan2.1-14B without additional large-model training.Source: §D.4 · p. 24

D.5 More Discussions of Hyperparameters

• With Wan2.2-T2V-A14B, γw = 0.3 yields the best VR-Bench score of 81.90; smaller or larger values reduce performance, especially 0.4 or 0.5.The authors suggest overly strong transition weighting can disturb generation quality.Source: §D.5 · p. 24
• For (σmin, σmax), the (0.35, 0.70) setting scores 81.90 on VR-Bench, outperforming the smaller (0.05, 0.10) and larger (1.00, 2.00) ranges.The reported trend is consistent with the Wan2.1 setting and supports robustness across Wan backbones.Source: §D.5 · p. 24

D.6 Human Evaluation on VR-Bench

• Lumos-Nexus ranked first in both Qwen3-VL* and human evaluation, scoring 86.19 and 69.33, respectively.The evaluation used 20 randomly sampled VR-Bench videos and 13 annotators; the rankings had Kendall’s τ = 0.73, indicating strong positive agreement.Source: §D.6 · p. 24

D.7 Discussion of the Homogeneous Latent Space Assumption

• Lumos-Nexus relies on compatible generator latent spaces; replacing its large generator with HunyuanVideo produced severely degraded, blurry outputs.The paper describes the assumption as generators sharing a compatible VAE latent space, and notes that Wan2.1-T2V and Wan2.2-T2V-A14B share a VAE.Source: Figure 10 · p. 25, §D.7 · p. 25
• MMD from Wan2.1-T2V-1.3B was 0.523 for Wan2.1-T2V-14B, 0.531 for Wan2.2-T2V-A14B, 2.977 for Wan2.2-5B, and 1.031 for HunyuanVideo.Lower MMD indicates better alignment; Wan2.2-5B and HunyuanVideo do not share Wan2.1-1.3B’s VAE and also showed substantially degraded performance as large generators.Source: Table 12 · p. 25, §D.7 · p. 25
• When generators are heterogeneous, training-stage alignment can adapt the small generator to the large generator’s latent space, more efficiently than retraining the large generator.The paper concludes that latent-space homogeneity is a key factor underlying the framework’s effectiveness.Source: §D.7 · p. 25

D.8 Discussion of Lumos-Nexus’s Generality

• The CogVideoX generality test used CogVideoX-2B and CogVideoX-5B, which meet the homogeneous-latent condition, but bridged them only at inference time.Due to time constraints, the authors did not retrain CogVideoX-2B with an attached VLM under the full Lumos-Nexus training pipeline.Source: §D.8 · p. 25
• CogVideoX* generated valid videos and scored 65.79 on VR-Bench, between CogVideoX-2B and CogVideoX-5B.The authors suggest VLM-aligned CogVideoX-2B training within Lumos-Nexus could further improve CogVideoX*, potentially surpassing CogVideoX-5B.Source: Table 13 · p. 26, §D.8 · p. 26

E Q&A Evaluation Examples from VR-Bench

• For each VR-Bench reasoning dimension, a representative Lumos-Nexus video receives a full eight-question diagnostic assessment, with cases matching the examples in Fig. 7 and Fig. 8.The cases support fine-grained inspection of physical, commonsense, and embodied reasoning.Source: E Q&A Evaluation Examples from VR-Bench · p. 26
• The expanded Q&A results provide evidence of reasoning consistency across all VR-Bench dimensions.Source: E Q&A Evaluation Examples from VR-Bench · p. 26

F Limitaion

• VR-Bench omits broader reasoning dimensions, including long-horizon causal chains, which remain unexplored and are left for future work.The authors also leave deeper exploration within video unified models and broader benchmark coverage for future work.Source: F Limitaion · p. 26
• The gamer’s visible mouth movements, including opening and closing, are judged synchronized with the implied speech.Source: F Limitaion · p. 34
• The simultaneous gaming and headset communication are judged plausible for competitive play, where real-time coordination is essential.The passage describes this scenario as standard in competitive gaming.Source: F Limitaion · p. 34
• The clip’s overall action tempo, including speech and controller inputs, is judged natural and free of temporal conflicts or unnatural pauses.Its weighted average score is 1.00.Source: F Limitaion · p. 34