PaperlayerView exact revision on arXiv

Source-linked AI summary

Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edge

Yitong Li, Jincheng Yu, Junsong Chen, Haopeng Li, Shuchen Xue, Haozhe Liu, Ping Luo, Song Han, Enze Xie

arXiv:2609.35110v1cs.AIcs.CVcs.LG

TL;DR

MiniMax-H3's 33-billion parameters and iterative denoising impose substantial inference costs, while cloud throughput and edge memory create different deployment bottlenecks. Sol-H3 combines cross-resolution generation with RSI-guided implementation optimization. It reports approximate 30× acceleration and 20% lower memory across cloud and edge, including a five-second video generated in 1.434 seconds on 8×GB200.

  • Problem

    MiniMax-H3's 33-billion parameters and iterative denoising impose heavy computation, while cloud throughput and edge memory constrain deployment in different ways.

    Source: §1 · p. 2

  • Method

    Sol-H3 combines a low-resolution-to-high-resolution generation schedule and latent-to-latent handoff with RSI searches over implementation choices under numerical verification.

    Source: §1 · p. 2

  • Results

    Sol-H3 reports approximate 30× acceleration and 20% memory reduction across cloud and edge; on 8×GB200, a five-second video with native audio takes 1.434 seconds.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The report presents human architectural design and RSI-driven kernel optimization as complementary, with the latter suited to bounded execution problems that can be verified mathematically.

    Source: §7 · p. 16

    RSI does not provide the open-ended architectural intuition needed for lossy pipeline components, which remain human-designed in this work.

    Source: §7 · p. 16

Abstract

from arXiv · show

Video diffusion models are rapidly scaling and exhibiting enhanced generation capabilities. Among these recent advancements, MiniMax-H3 stands out as a highly capable, production-level open-source model. However, its 33-billion parameters and multi-step iterative denoising process introduce substantial computational overhead. Consequently, their practical production is hindered by generation latency in the cloud deployment like NVIDIA-GB200, alongside strict memory limits that pose further challenges at the edge device like DGX-Spark. To address these diverse hardware bottlenecks from cloud to edge device, we present a full-stack inference pipeline that integrates efficient algorithmic design with optimized operator implementations. Algorithmically, we introduce a cross-resolution two-stage generation scheduler that exploits the step-wise nature of diffusion: early low-resolution steps rapidly establish the global layout, while later high-resolution steps focus refinements of local and perceptual details. These stages are connected by a learned latent-to-latent mapping module, completely eliminating the computationally expensive VAE decode-reencode cycle for resolution transferring cross different resolutions. For operator implementation, we deploy a Recursive Self-Improvement (RSI) loop that searches kernel fusions and memory layouts, evaluating latency together with numerical agreement. Together, these optimizations deliver up to 30x end-to-end speedup and 20% lower memory: a 5-second 1344x768 video with audio is generated 3.5x faster than real time on an 8xGB200 node, and in under a minute fully memory-resident on a single DGX Spark.

1. Introduction

MiniMax-H3's 33-billion parameters and iterative denoising create heavy inference costs, with cloud throughput and edge memory posing distinct deployment bottlenecks. Sol-H3 combines cross-resolution generation with RSI-based system optimization, reporting acceleration and memory reductions across cloud and edge deployments.

Problem

  • MiniMax-H3's 33-billion parameters and iterative denoising burden inference, while cloud throughput and generation cost and edge memory constrain deployment differently.Exceeding edge HBM capacity forces CPU offloading and sharply increases inference latency.Source: §1 · p. 2

Approach

  • The pipeline assigns early low-resolution steps to global structure and later high-resolution steps to detail, connecting stages with latent-to-latent translation instead of pixel decoding and re-encoding.The RSI loop searches kernel fusions, memory layouts, and communication boundaries while checking numerical agreement.Source: §1 · p. 2

Results

  • Approximately 30× acceleration and 20% lower memory are reported across cloud and edge environments; on 8×GB200, a five-second video with native audio takes 1.434 seconds.That 8×GB200 generation is approximately 3.5× faster than real-time playback.Source: §1 · p. 2

Contributions

  • Sol-H3 combines algorithmic and system-level optimization, using a cross-resolution schedule alongside RSI-guided implementation search.The stated contribution includes checks distinguishing bit-exact layout transformations from arithmetic fusions and approximate execution modes.Source: §1 · p. 2

2. Related Work

The report situates Sol-H3 among video diffusion models, acceleration methods, and agentic kernel-optimization systems. It presents the low-resolution-draft/high-resolution-refinement schedule as prior art, while locating its contribution in serving and execution-stack realization.

2.1. Video Diffusion Models

  • The report targets MiniMax-H3 with LTX-2.5 refinement, and notes that its latent handoff combines spatial upsampling with learned translation between H3 and LTX VAE representations.MiniMax-H3's joint audio-video generation also makes preservation of audio fidelity a serving concern.Source: §2.1 · p. 3

2.2. Inference Acceleration for Video Diffusion

  • Low-resolution drafting followed by high-resolution refinement is prior art [27] [29]; the report locates its contribution in serving realization and the end-to-end execution stack.It distinguishes resolution-trajectory changes from caching methods that reduce computation within a fixed trajectory.Source: §2.1 · p. 3, §2.2 · p. 3

2.3. Agentic and Self-Improving Optimization Systems

  • The report extends the Sol video inference engine [34] with a fail-closed RSI constraint: candidates optimize realizations of a fixed generation contract rather than changing it.It contrasts verifiable execution optimization with open-ended architectural design, which it says still requires human engineering.Source: §2.3 · p. 4

3. Full-Stack Optimization for Cloud Deployments

Sol-H3 restructures MiniMax-H3 inference for cloud deployment using a cross-resolution pipeline and RSI-guided system optimizations. The section describes the stage handoff, communication and compute techniques, sparse attention, fusion, and decoder optimizations; its stated 8×GB200 result is a five-second video generated in 1.434 seconds.

3. Full-Stack Optimization for Cloud Deployments

  • A five-second video is generated in 1.434 seconds on 8×GB200, approximately 3.5× faster than real-time playback.The report evaluates applicable configurations separately in Section 6.Source: §3 · p. 4

3.1. Cross-Resolution Generation

  • The cloud pipeline restructures full-resolution denoising into asymmetric stages, with early steps establishing layout and motion and later steps synthesizing detail.Stage 1 uses four 672 × 384 updates with MiniMax-H3; Stage 2 uses three joint audio-video updates on a 1344 × 768 latent canvas with LTX-2.5.Source: §3.1 · p. 4, Figure 2 · p. 4
  • The learned translator replaces an H3 decode–LTX re-encode round trip; spatial upscaling is separate, and the handoff maps the upscaled representation into LTX latent coordinates.Because the VAEs have different channel counts and spatial compression, matching resolution alone does not make their latents interchangeable.Source: Figure 2 · p. 4, §3.1 · p. 5

3.2. Communication Optimization

  • Stride-aware kernels combine QKV exchanges and write directly into exchange buffers, addressing layout-copy overhead in multi-GPU attention communication.A representative profile records 94 ms per denoising step for layout transformations and collectives; the naive PyTorch implementation achieved 0.953× reference throughput.Source: §3.2 · p. 6

3.3. Compute Quantization

  • MXFP8 GEMMs serve attention and FFN projections in validated SM100-family transformer blocks 2–46, while boundary blocks and several other modules retain BF16 computation.The MXFP8 path is approximate, and fused producers write E4M3 activations and swizzled scales directly.Source: §3.3 · p. 6

3.4. Sparse Attention

  • Sol-Attn selects KV blocks using an online threshold and progressively increases sparsity across the three refinement steps, while retaining dense prefix attention and the first two transformer layers.The step schedule sets τ to 1.0, 1.25, and 1.5; prefix KV blocks remain always-attended sinks for target-video queries.Source: §3.4 · pp. 6–7

3.5. Kernel Fusion and Precision Preservation

  • The RSI loop searches fused operator groups, but arithmetic fusion can differ from eager execution through intermediate-precision and floating-point rounding changes.For LoRA, the consumer-fusion path retains separate branches and BF16 rounding points; the cited diagnostic found 86.1–94.4% of nonzero updates rounded back in six inspected projections.Source: §3.5 · p. 7

3.6. VAE Decoding

  • Eight-rank H3 decoding distributes independent tiles and batches them locally: compiling the batched decoder reaches 0.3593 s and 15.6 GB peak memory, versus a 1.1597 s sharded baseline and 18.8 GB.Global batching uses 200 execution slots instead of 224 per-clip slots; reported output deviation and PSNR characterize that test, not a general error bound.Source: §3.6 · p. 8

4. Full-Stack Optimization for Edge Deployments

The edge deployment targets full working-set residency in HBM, because host-memory offloading can make communication dominate request latency. Sol-H3 combines memory, caching, and quantization optimizations; its optimized DGX Spark configuration fits within the shared memory pool.

4. Full-Stack Optimization for Edge Deployments

  • Keeping the complete working set resident matters because otherwise weights must be offloaded to host memory and streamed back, with communication cost dominating request latency.The working set includes the text encoder, drafting model, refiner, and VAE.Source: §4 · p. 8
  • The optimized DGX Spark configuration uses 116.9 GiB of its 119.68 GiB unified memory pool, leaving a 2.8 GiB safety margin; a naive two-stage deployment requires 142.4 GiB.The naive configuration exceeds capacity by 22.7 GiB.Source: §4 · p. 8

4.1. AdaLN precompute

  • AdaLN precomputation caches modulation tables by block and step, replacing repeated timestep-conditioned projection work with a precomputed table.The passage reports about 24 GB of device-memory savings and about 26 GB fewer HBM reads per step.Source: §4.1 · pp. 8–9

4.2. Prompt caching

  • A canonical generic refinement prompt is processed offline and cached as compact video and audio context tensors, so the Stage-2 text encoder and connector need not be loaded online.This frees 16.2 GiB for other modules; the rationale given is that Stage-1 latents already carry sample-specific information for local refinement.Source: §4.2 · p. 9

4.3. Weight quantization

  • The DGX Spark configuration uses an FP8 H3 draft DiT and an NVFP4 AWQ Qwen prompt encoder, with reported resident footprints of 23.4 GiB and 14.6 GiB, respectively.The Stage-2 INT8 Gemma encoder and connector are used offline for cache construction and are not resident during request processing.Source: §4 · p. 8, §4.3 · p. 9

5. Caching Reference Tokens for Reference to Video Generation

In reference-to-video generation, repeated reference-token computation can dominate latency as conditioning materials grow, even though clean reference representations vary only weakly across denoising steps. The section introduces reference KV caching to skip those later repetitions, while noting that caching is an approximation with minor qualitative deviation.

Reference computation redundancy

  • Clean reference latents vary only weakly with timestep embeddings, making repeated reference-token recomputation across denoising steps a potential source of computational redundancy.Unlike target video tokens, reference latents are clean and are not iteratively denoised.Source: §5 · p. 9

Generation with heavy conditioning

  • With extensive reference inputs, reference tokens can exceed generation tokens and their redundant recomputation can dominate inference cost.A request may supply up to 9 reference images or 3 reference videos.Source: §5 · p. 9

Reference KV caching mechanism

  • A full first DiT step caches reference K,V; later steps reuse them unchanged and skip reference recomputation while generated tokens attend to the cache.The first step remains a full computation, establishing the conditioning signal exactly while bypassing only later repetitions.Source: §5 · pp. 9–10

Approximation and qualitative comparison

  • Caching is approximate rather than bitwise-identical because timestep embeddings cause slight intermediate-representation drift, though qualitative comparisons show minor deviation.The passage describes it as practical for reference-to-video generation, especially with heavy reference material.Source: §5 · p. 10

Reference KV caching mechanism

  • After the first denoising step, image and video reference query rows are skipped; text and generation queries attend to cached reference K,V and to each other.The figure contrasts this with full self-attention over all tokens in native MiniMax-H3 inference.Source: Figure 4 · p. 10

6. Experiments

Across the evaluated workloads, Sol-H3 accelerates video generation on GB200 GPUs, DGX Spark, and RTX 5090, while experiments also examine reference caching, adapter selection, latent translation, and LoRA fusion. The results report speed, quality, and numerical-agreement measurements alongside their evaluation scope.

6.1. Cross-Resolution Two-Stage Acceleration

  • For a 5 s 1344×768 workload, latency is 6.852 s on one GB200, 56.17 s on DGX Spark, and 39.80 s on RTX 5090, with respective baseline speedups of 22.2×, 31×, and 26.3×.On Spark, LTX refinement accounts for 44.0% of latency; on RTX 5090, Qwen prompt encoding is the largest block at 34.1%, largely because it is loaded and released per request under CPU offload.Source: §6.1 · p. 10

6.2. Sol-H3 Latency on GB200 GPUs

  • 12.73×–16.99× speedup over Base H3 across evaluated workloads; on eight GB200 GPUs, five-second videos take 1.434 s, about 3.5× faster than real time.The measurements cover 5-, 10-, and 15-second outputs; Table 1 reports Sol-H3 latency against MiniMax-H3 baselines on GB200 GPUs.Source: Table 1 · p. 12, §6.2 · p. 12

6.3. Reference Conditioning and KV Caching

  • Reference KV caching yields 1.14×–1.92× speedups, while reference tokens increase DiT time by up to 3.94×; cached outputs retain scene structure, identity, and motion with minor perceptual differences.The reported 1.71×, 2.83×, and 3.94× DiT-time increases correspond to image, video, and video-plus-image references, respectively.Source: §6.3 · p. 12, Table 2 · p. 12

6.4. Comparison of Few-Step LoRA Adapters

  • Adapter comparisons favor different few-step LoRAs by task and evaluation count: Larry’s Turbo and LightX2V for 4-NFE T2VA, and Alibaba PAI PDD Acc8 and VDN-H3 for 8-NFE T2VA.For Ref2VA, the preferred choices are Alibaba PAI’s 4-NFE PDD recipe and LightX2V at 4 NFE, and HyperFlow v1.0 at 8 NFE; comparisons combine human assessment on a subset with visual-only VLM judgments across 3,000 pairs.Source: §6.4 · p. 13

6.5. Cross-VAE Latent Translation

  • The final latent-translation adapter reaches 30.618 dB PSNR and 0.8962 SSIM on 256 held-out clips, while reducing median conversion latency 202.0× in a separate profile.The quality metrics compare decoded predictions with LTX teacher latents before Stage-2 denoising; the conversion profile measures the handoff module, not an end-to-end speedup or a quality-equivalent VAE replacement.Source: §6.5 · p. 14, Table 3 · p. 14, Table 4 · p. 14

6.6. Precision-Preserving LoRA Fusion

  • Consumer fusion matches native branches byte for byte on the tested inputs and settings, reduces median pipeline time by 0.694 s (2.64%), and remains 0.804 s (3.24%) slower than merged weights.Merged weights produce different numerical results; the reported agreement is scoped to the tested inputs and software environment, not an entirely unaccelerated model.Source: §6.6 · pp. 14–15, Table 5 · p. 15, Figure 8 · p. 15

7. Discussion and Conclusion

Sol-H3 combines a human-designed inference framework with RSI-driven kernel optimization for MiniMax-H3. The discussion assigns RSI bounded, verifiable engineering tasks while reserving open-ended, lossy architectural choices for human design.

7. Discussion and Conclusion

  • Sol-H3 combines a cross-resolution two-stage generation pipeline with a kernel-level optimization stack discovered via RSI.The report presents this as a full-stack inference pipeline for the MiniMax-H3 video-audio generator.Source: §7 · p. 16
  • Human design drives the macro-level inference framework, while the RSI loop systematically executes micro-level implementation optimizations.The report describes this as a complementary relationship between architectural insight and automated optimization.Source: §7 · p. 16
  • RSI handles execution bottlenecks with vast optimization spaces when outcomes can be verified, including communication, fusion, sparse-attention, and LoRA implementations.The listed tasks include reducing redundant HBM traffic and preserving numerical fidelity or generation quality.Source: §7 · p. 16
  • Open-ended, lossy architectural choices remain human-designed, including the two-stage framework and a latent adapter that trades strict scene-identity invariance for efficient stage bridging.The report says RSI cannot supply the paradigm-shifting intuition required for these components.Source: §7 · p. 16
  • Sol-H3’s human-designed architecture and RSI-driven kernel optimization together support faster-than-real-time high-resolution video synthesis.The report says this synergy lowers the barrier to deploying massive video generation models in production.Source: §7 · p. 16

A. Latent Adapter Training and Evaluation

The latent adapter is trained and evaluated on paired H3 and LTX latents, with fixed VAEs and a held-out reconstruction protocol. A separate conversion benchmark compares it with two VAE round-trip implementations.

A. Latent Adapter Training and Evaluation

  • 80,000 source videos are resized identically and encoded with frozen H3 and LTX-2.5 Conv VAEs to form paired training data.For 192-frame clips at 1344 × 768, the passage gives H3 and LTX tensor shapes and describes fixed alignment, a fixed skip projection, and an end-to-end-trained residual network.Source: A. Latent Adapter Training and… · p. 17
  • The final decoder-aware optimization uses 65,536 cached native-resolution video pairs across two phases totaling 7,500 updates.Only the adapter residual path is updated; the affine skip and both VAEs remain fixed, and the reported checkpoint uses trained weights without EMA averaging.Source: A. Latent Adapter Training and… · p. 17
  • Held-out reconstruction evaluation uses 256-video development and test cohorts, but does not assess prompt adherence, audio quality, or subsequent LTX refinement.The cohorts exclude identifiers used in earlier evaluation cohorts, and checkpoint selection uses development data.Source: A. Latent Adapter Training and… · p. 17
  • The conversion benchmark compares full VAE round trips, a Tiny AutoEncoder round trip, and an earlier adapter checkpoint under different warmup and measurement counts.Peak allocation excludes the co-resident generative pipeline and is not an estimate of total Spark memory usage.Source: A. Latent Adapter Training and… · p. 17

B. LoRA Precision Diagnosis and Fusion

The precision diagnosis separates weight-rounding error from GEMM precision, while the implementation documents supported fusion configurations and validates native-versus-fused agreement on audited runs.

B. LoRA Precision Diagnosis and Fusion

  • 35.26% relative RMSE from BF16 merged weights falls to 6.24% with FP32 merged weights, while FP32 separate branches give 6.08% against the same baseline.The measurements are video-prediction discrepancies for one input at one denoising evaluation, not perceptual-quality scores.Source: B. LoRA Precision Diagnosis and Fusion · p. 17
  • The merge diagnostic shows that FP32 GEMMs do not recover updates already lost when weights are stored in BF16.The passage also reports that FP32 separate branches do not exactly reproduce the original mixed-precision computation.Source: B. LoRA Precision Diagnosis and Fusion · p. 17
  • Consumer fusion supports specified PEFT LoRA and FastVideo hybrid formats, but requires inference mode, one scale-1 adapter, ordinary BF16 linear layers, identity dropout, and no projection hooks.Unsupported installation configurations are rejected before consumer patches are installed; unsupported runtime states fall back to native PEFT.Source: B. LoRA Precision Diagnosis and Fusion · p. 18
  • Separate and fused runtime modes require compute-quant=none, and consumer fusion is explicitly selected rather than implicit in every pipeline measurement.The reported byte agreement remains scoped to tested inputs and recorded GPU and software configurations.Source: B. LoRA Precision Diagnosis and Fusion · p. 18
  • In an audited eight-evaluation rerun, native and fused arms matched in every recorded prediction and in the complete decoded video and audio.The comparison used four GB200 GPUs, 362 frames, and tensor and frame-hash checks; these audited runs are separate from warm latency measurements.Source: B. LoRA Precision Diagnosis and Fusion · p. 18

Related papers

Loading 2609.35110v1…