Carnator: Fast Text-to-Video Generation with Generation-Native Compatibility-Guided Cross-Request Reuse

1 INTRODUCTION

• Iterative denoising over long spatiotemporal token sequences makes video generation costly, while single-request acceleration is bounded by within-trajectory redundancy.Continuous serving workloads with shared objects, scenes, and structural patterns offer potential cross-request redundancy.Source: §1 · p. 1
• Semantic similarity alone cannot reliably establish reuse validity, and incompatible reuse can preserve historical scene content that conflicts with the target request.Reuse scope is also unresolved in text-to-video because reusable computation may span latent states and attention connectivity.Source: §1 · p. 2
• Carnator constructs an Early Signature from internal diffusion states and applies risk-aware compatibility decisions to select reusable historical computation beyond coarse semantic similarity.A lightweight early probe produces the signature.Source: §1 · p. 2
• The same generation-native evidence localizes target-specific computation and guides joint reuse of historical latent trajectories and sparse attention connectivity.This preserves necessary target updates while reducing redundant token updates and attention interactions.Source: §1 · p. 3
• Across three backbones, Carnator achieves up to 1.72× end-to-end acceleration while maintaining competitive generation quality and reducing unsafe reuse.The experiments evaluate reuse validity and inference acceleration on VidProM.Source: §1 · p. 3

Efficient diffusion inference

• Most efficient diffusion inference methods optimize a single generation process rather than reusing computation from completed requests.These include step reduction, quantization, token selection, feature caching, sparse attention, and adaptive computation.Source: §2 · p. 3

Intra-request acceleration

• Intra-request methods reuse or selectively refresh computation within one denoising trajectory, whereas Carnator tests whether computation from another trajectory remains compatible with the target.Prior methods include feature caching, token-wise refresh, sparse attention, and attention-based semantic-region recovery.Source: §2 · p. 3

Inter-request acceleration

• Inter-request diffusion methods reuse historical computation across requests, but Carnator applies generation-native evidence to validity filtering and joint latent and sparse-connectivity reuse in text-to-video.Prior approaches include intermediate-noise reuse for text-to-image, several text-to-video reuse strategies, and sparse-attention transfer in image-to-video.Source: §2 · p. 3

3.1 OVERVIEW AND PROBLEM SETUP

• Text-level recall and a lightweight target probe produce an Early Signature that filters historical caches before reuse, with dense generation as the fallback when none is compatible.The same generation-native evidence also guides reuse scope after a cache is accepted.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 REUSE VALIDITY WITH GENERATION-NATIVE EVIDENCE

• The object-centric Early Signature summarizes spatiotemporal support and appearance descriptors extracted from internal cross- and self-attention states.Figure 3 describes early denoising representations as informative signals for compatibility.Source: §3.2 · p. 4, Figure 3 · p. 5
• Compatibility compares context appearance and, for substituted entities, object size, shape, and appearance discrepancies against dimension-specific thresholds.The discrepancy vector covers the applicable dimensions for each candidate pair.Source: §3.2 · p. 5
• Thresholds are chosen to maximize reuse coverage subject to an unsafe-reuse risk constraint, then certified on a disjoint set using an upper confidence bound.The thresholds are accepted when the bound is at most δ.Source: §3.2 · p. 6

3.3 REUSE SCOPE WITH JOINT LATENT-ATTENTION REUSE

• Evidence-derived regions guide latent reuse: evidence tokens receive target updates, a transition band blends states, and tokens outside the active region inherit the historical trajectory.Active tokens use contextual keys and values from the larger context region in selected middle layers.Source: §3.3 · pp. 6–7
• Carnator recomputes target Q, K, and V while reusing historical sparse connectivity, restricting middle-layer attention to active queries and valid contextual tokens.Other layers reuse the historical sparse block structure without that additional region constraint.Source: Figure 4 · p. 6, §3.3 · p. 7

4.1 EXPERIMENTAL SETUP

• The workload uses 1,000 VidProM historical requests and a disjoint 300-request test stream across three backbones, with common model-specific configurations and cache settings for comparisons.All online costs are included in cache-hit end-to-end latency, and cache misses fall back to dense generation.Source: §4.1 · p. 7

4.2 MAIN RESULTS

• Region-guided latent reuse and historical sparse-connectivity reuse reduce redundant token updates and query–key interactions, increasing computation reused from each compatible request.The results identify reuse validity and reuse scope as complementary factors in converting cross-request redundancy into end-to-end savings.Source: §4.2 · p. 8
• Carnator scores 0.6382 on VBench-Q, comparable to dense inference and evaluated baselines, and 0.2913 on CLIP-T.Source: §4.2 · p. 8

4.3 ABLATION STUDIES

• Risk-aware thresholding accepts 50 of 70 certified request-cache pairs with no unsafe cases, and its one-sided Clopper–Pearson upper bound of 4.50 percent meets the 5 percent risk constraint.Source: §4.3 · p. 8

4.2 MAIN RESULTS

• 72.67% cache hit rate still yields 1.36× end-to-end speedup on Wan2.2-TI2V-5B, above the strongest baseline’s 1.24× despite its 80.67% hit rate.Among cache-hit requests, Carnator has the lowest end-to-end and DiT latencies on all three backbones; on Wan2.2-TI2V-5B these are 187.30 s and 159.65 s.Source: §4.2 · p. 8, Table 1 · p. 9

4.3 ABLATION STUDIES

• Generation-level filtering lowers cache hit rate from 77.33% to 72.67%, raises CLIP-T from 0.2725 to 0.2913, and maintains VBench-Q at 0.6382.The passage reports essentially unchanged end-to-end latency from the added filtering.Source: §4.3 · p. 8, Table 2 · p. 9
• Joint latent and attention reuse achieves 155.85 s latency and 1.71× end-to-end speedup, outperforming either reuse mechanism alone.Latent reuse reaches 206.06 s and attention reuse 160.91 s; CLIP-T and VBench-Q remain within a narrow range across variants.Source: Table 3 · p. 9, §4.3 · p. 9

• Carnator builds an Early Signature from early diffusion states for risk-aware cache selection, then uses localized evidence to guide joint latent-trajectory and sparse-connectivity reuse.Source: §5 · p. 9
• Experiments show reduced video-diffusion inference latency with competitive generation quality across diverse requests, while cross-request redundancy complements optimization within individual denoising trajectories.Source: §5 · p. 9

AI USE STATEMENT

• Generative AI assisted with language editing, manuscript organization, and refinement of figures, captions, and terminology; the authors reviewed and verified all AI-assisted content.Source: AI USE STATEMENT · p. 10

A.1 EARLY-STATE GENERATION-NATIVE COMPATIBILITY EVIDENCE

• Similar request pairs have consistently smaller source–target latent discrepancies than incompatible pairs, supporting early diffusion states as evidence for reuse-validity assessment.Source: §A.1 · p. 12, Figure 5 · p. 13

A.2 CROSS-REQUEST ATTENTION SIMILARITY

• Despite object substitution, semantically related requests retain highly similar blockwise attention mass distributions and sparse connectivity patterns, motivating Historical Sparse-Connectivity Reuse.Source: Figure 6 · p. 13, §A.2 · p. 13

B ADDITIONAL QUALITATIVE RESULTS

• Qualitative examples show Carnator preserving target-specific content and overall visual quality while reusing computation from compatible historical requests.Source: B ADDITIONAL QUALITATIVE RESULTS · p. 13, Figure 7 · p. 14

C.1 EVIDENCE REGION CONSTRUCTION

• After four denoising steps, noun-conditioned cross-attention responses seed high- and low-response visual tokens that self-attention propagates into a noun–background contrast score.Fixed fusion coefficients and hysteresis thresholds produce the Evidence Region, using the same configuration throughout evaluation for each model.Source: §C.1 · p. 14
• For each noun, signatures store the Evidence Region, normalized support size, and appearance; one-noun substitutions are compared using area, shape, and CIEDE2000 appearance distances.Appearance is extracted by decoding the latent once, projecting the region onto the final frame, and averaging its CIELAB pixels.Source: §C.1 · p. 14

C.2 EARLY PROBE EVIDENCE REGION

• The Evidence Region concentrates on the target object within the first few denoising steps and then becomes spatially stable, supporting a lightweight early probe.Source: §C.2 · p. 14, Figure 8 · p. 15

C.3 REUSE-SCOPE IMPLEMENTATION DETAILS

• Carnator stores post-scheduler source latents by timestep and reuses them at corresponding target steps, while recomputing target prompt conditioning and guidance.Active and Context Regions are formed by dilating the Evidence Region by 2 and then 3 tokens per latent frame; the intervening band supports gradual latent fusion.Source: §C.3 · p. 15
• Historical self-attention connectivity is reused, but target Q, K, and V are recomputed and cached connectivity is restricted to active queries and valid contextual tokens.Connectivity masks are stored per denoising step, guidance branch, and Transformer layer.Source: §C.3 · p. 15

C.4 OVERALL INFERENCE PROCEDURE

• Carnator probes the target, checks recalled caches in order, and accepts the first compatible one; if none qualifies, it performs dense generation.An accepted cache then guides region construction, target latent updates and fusion, and reuse of historical sparse connectivity.Source: §C.4 · pp. 15–16

D CROSS-ARCHITECTURE PORTABILITY OF CHAI

• CHAI preserves generation quality on OpenSora 1.2, but its direct Wan2.2 adaptation exhibits noticeable visual degradation.The comparison suggests Cache Attention requires architecture-specific adaptation when transferred to models other than OpenSora 1.2.Source: D CROSS-ARCHITECTURE PORTABILITY OF CHAI · p. 16, Figure 9 · p. 16

E LAYER-WINDOW ABLATION

• Beyond 20 layers, generation quality begins to degrade as wider region-update windows provide only modest additional speedup.All configurations use the same accepted request–cache pair, and remaining layers perform historical sparse-connectivity reuse; latency includes the dense probe and speedup is relative to dense inference.Source: E LAYER-WINDOW ABLATION · pp. 16–17, Table 4 · p. 17

F RISK-AWARE COMPATIBILITY DECISION

• 50 of 70 request–cache pairs were accepted with no unsafe cases, yielding 71.43% reuse coverage and a 4.50% one-sided upper bound under the 5% target.The certification uses a one-sided 90% confidence level; Table 5 reports threshold-selection and certification results.Source: F RISK-AWARE COMPATIBILITY DECISION · p. 17, Table 5 · p. 17

G.1 END-TO-END LATENCY

• E2E latency measures wall-clock time from request start until the decoded video is ready, including the denoising loop and VAE decoding.It also includes retrieval, encoding, latent construction, any dense probe prefix, and unhoisted preprocessing.Source: §G.1 · p. 17

G.2 DIT LATENCY

• DiT latency sums CUDA-synchronized transformer forward-pass time across executed steps, including any dense probe prefix but excluding scheduler updates and decoding.All guidance branches are counted, and the metric includes every forward the request actually ran.Source: §G.2 · p. 18