Source-linked AI summary
RedKnot-MLA: Multi-Head Offline-Online Reuse for DeepSeek-V4 Long-Context Serving
Yang Liu, Zhaokai Luo, Huayi Jin, Ruozhou He, Chenchen Hong, Mingxiao Ma, Biao Zhang, Zhiyong Wang, Boyu Wang, Guanjie Chen, Yifei Liu, Tao Xie, Junhao Hu
TL;DR
DeepSeek-V4’s packed MLA lacks the physical per-head cache boundary assumed by conventional head-wise reuse. RedKnot-MLA preserves that packed representation while combining canonical offline Local-head artifacts with online RoPE relocation, Global-head and protected-row recomputation, and shared projection; its archived results show substantial analytic savings and aggregate quality gains, alongside dataset-level and measurement-boundary caveats.
Problem
MLA exposes many logical query heads through one packed latent KV stream, making conventional physical per-head cache reuse incompatible with its native representation.
Method
RedKnot-MLA builds certified canonical-position Local-head artifacts offline, then performs query-side RoPE relocation, Global-head and protected-row recomputation, and shared projection online.
Results
At 256K, the paired three-dataset study reports aggregate F1 +3.24 points, EM +4.16 points, and 78.7–79.5% analytic major-operator arithmetic saving, while one dataset loses 2.81 F1 points.
Takeaways & Limitations
The system demonstrates packed-latent MLA reuse with aggregate quality preservation and substantial analytic omission, while retaining visible per-dataset variation.
Takeaways & Limitations
The evidence combines different units: curated diagnostics, preliminary trace-unbacked QPS, and analytic compute ledgers rather than one homogeneous scaling experiment.
Abstract
from arXiv · showhide
Multi-head latent attention (MLA) exposes many logical query heads through one packed latent KV stream. This representation is memory efficient, but it removes the physical per-head cache boundary assumed by conventional head-wise reuse. We present our system, a DeepSeek-V4 realization of RedKnot's head-aware reuse principle. Each immutable document is processed offline at canonical position zero; certified Local-head contributions are retained as MLA-Off. At serving time, query-side RoPE relocation restores the document's request position, a small Global-head set and protected Local token rows are recomputed as MLA-Online, and the two paths are merged before a single shared output projection. The packed MLA latent is never split. DeepSeek-V4-Flash uses 37 reusable layers and a 56/8 Local/Global partition, giving a 75.29% analytic logical head-row ceiling; the Pro-0813 profile uses 55 layers and 112/16 heads, giving 78.89%. Frozen Flash operating points show hot-artifact TTFT speedups of 2.02-3.84x. At 256K, the archived three-dataset study reports an aggregate F1 change of +3.24 percentage points, an EM change of +4.16 points, and a 78.7-79.5% analytic major-operator arithmetic saving, while one dataset decreases by 2.81 F1 points. A separate author-reported 256K hot-artifact QPS measurement is approximately 2.0x; because its raw concurrency trace is not included in this bundle, we mark it as preliminary rather than archived evidence. We describe the factorization, position repair, token-row closure, sparse-MoE support, TP8 integration, and the measurement boundaries needed to interpret these results.
1 Introduction
RedKnot-MLA adapts head-aware reuse to DeepSeek-V4’s packed MLA by reusing document-stable Local contributions without splitting the physical latent stream. It combines offline artifacts with online position repair, selective recomputation, and evidence-qualified evaluation.
- Motivation: RedKnot-MLA applies head-granular reuse to MLA without creating per-logical-head physical caches.MLA exposes many logical query heads through one packed latent KV stream, so duplicating state would defeat the native representation.
- Method: Local heads are processed per document at canonical position zero, while query rotation, Global-head evaluation, protected-row repair, and segmented merging run online.A group-aligned sliced projection makes Local and Global contributions additive before the shared second output projection.
- Evaluation boundary: The reported evidence separates archived TTFT and quality measurements, preliminary QPS, and analytic compute accounting rather than treating them as one homogeneous experiment.The resulting values describe an attainable operating envelope with distinct measurement boundaries.
- System scope: The system preserves MLA’s packed latent and supports fused prefetch, validation, merge integration, sparse-MoE assistance, and TP8 execution.The contributions include concrete Flash and Pro profiles and a protocol-qualified evaluation separating measured, preliminary, and analytic quantities.
2 MLA-Off/MLA-Online Design
RedKnot-MLA partitions logical MLA heads into reusable Local and online Global roles while retaining one native packed latent stream. Canonical artifacts are relocated and selectively corrected online, with protected rows and adaptive sparse-MoE routing supporting the reuse path.
- Logical heads and latent: Local and Global are disjoint execution roles over logical heads; the shared packed latent remains native and is paged once.The first output projection is sliced into Local and Global channel ranges before the shared projection.
- Logical heads and latent: Equation 4 combines offline and online projected states before applying the shared output projection.The identity is exact under full-recomputation Local and Global outputs; approximation enters when Local state is restored in a changed context.
- Canonical offline artifacts: Each immutable document is built independently at canonical position zero and published as a certified, generation-atomic artifact.Certificates bind checkpoint, tokens, profile, tensor-parallel geometry, and artifact generation.
- Canonical offline artifacts: Reuse falls back to full recomputation when validation detects missing layers, token or profile mismatch, stale generations, incompatible TP geometry, or rank disagreement.The fallback occurs before any reuse-specific collective is entered.
- Online position restoration and merge: Online relocation rotates queries into each document’s canonical frame, attends to canonical latents, restores phase, and merges segment statistics stably.The runtime avoids rewriting or physically concatenating each latent cache.
- Online position restoration and merge: Protected rows include query, new, dirty, boundary, Indexer-selected, and recent-tail rows, with cached Local values replaced rather than double counted.A fixed controller may expand the active budget to 10%, 20%, or 25% of candidate rows.
- Adaptive sparse MoE as a supporting path: Adaptive sparse-MoE routing supports head reuse by retaining the shared expert, preserving native weights, and selecting between three and six routed experts.Full-recompute fence layers retain native Top-6 routing, and savings across namespaces are not additive.
3 Profiles and Runtime Integration
The profiles preserve MLA’s single packed latent while assigning Local heads to reuse and Global heads to online recomputation. TP8 integration fuses restoration, attention, projection, and row replacement to retain the savings.
- Head-factorized profiles: Flash uses 3F–37R–3F with 56/8 Local/Global heads, while Pro-0813 uses 3F–55R–3F with 112/16 heads.
- Head-factorized profiles: 75.29% for Flash and 78.89% for Pro are structural logical head-row ceilings, not whole-model FLOP or time savings.
- Runtime integration: The TP8 fast path prefetches artifact tiles by layer group, TP rank, and projected output-channel range while device events guard consumption.
- Runtime integration: The consumer fuses RoPE views, Global-head and protected-row attention, sliced projection, Local restoration, and row replacement before one shared Wb projection.
- Evidence boundary: Profile geometry is analytic, whereas the v6 quality bars are paired measurements.
A. Head-factorized profiles
RedKnot-MLA divides logical heads into reusable Local and online Global roles while retaining one physical packed KV stream. Flash and Pro use distinct layer and head profiles, with explicit runtime accounting for several non-interchangeable savings namespaces.
- Head roles: Local and Global are RedKnot-MLA execution roles, not physical MLA KV-head types.
- Flash profile: Flash uses 56 Local and 8 Global heads with one packed KV stream.
- Pro profile: Pro uses 112 Local and 16 Global heads with one packed KV stream, but its profile has no paired long-context claim.
- Measurement namespaces: The runtime distinguishes logical head-row saving, physical token-row saving, analytic arithmetic saving, and measured wall-clock timing.
4 Evaluation
Evaluation separates archived hot-artifact latency and paired quality measurements from modeled projections and preliminary throughput. Flash shows multi-length TTFT speedups and aggregate 256K quality preservation, but dataset variation and evidence boundaries remain material.
- Hot-artifact performance: 2.91×, 3.20×, 2.02×, and 3.84× are archived hot-artifact TTFT speedups at 64K, 128K, 256K, and 440K.
- Hot-artifact performance: The four length points are not a controlled scaling law because their evidence units and fixed overheads differ.
- Analytic boundaries: 75.29% is the length-independent Flash structural ceiling, while 70.20–73.17% is the 256K active-row sensitivity range.
- Throughput evidence: Approximately 2.0× hot-artifact QPS at 256K is preliminary because the bundle lacks its raw concurrency trace and workload details.
- 256K quality: The aggregate changes from 0.5723 to 0.6047 F1 and from 0.4167 to 0.4583 EM, while MultiFieldQA-en decreases by 2.81 F1 points.
- Sparse-MoE support: Across 50 8K examples, sparse-MoE K3 changes aggregate F1 from 0.5905 to 0.5793 with unchanged EM of 0.42; on five 256K prompts, prefill improves 1.074×.
5 Reproducibility and Limitations
The release packages frozen suites, environment and profile metadata, benchmark automation, and record-level provenance. The present claim boundary excludes population-level scaling, reproducible QPS, achieved hardware counters, and paired Pro long-context validation.
- Reproducibility: The public repository packages frozen JSONL suites, profile files, sparse-FFN parameters, the benchmark driver, environment manifest, and a one-command wrapper.
- Reproducibility: The wrapper validates environment and data hashes, enforces a TP8 release lock, preserves failure logs, and restores the prior GPU holder.
- Limitations: The four length points are curated diagnostics, so larger randomized per-dataset samples and AB/BA repetitions are needed for population estimates.
- Limitations: QPS requires fixed-output multi-concurrency sweeps before it can be called a reproducible measurement.
- Limitations: Analytic compute ledgers do not report achieved FLOPs, HBM traffic, collective time, energy, or utilization, and Pro needs paired long-context runs.
6 Conclusion
RedKnot-MLA adapts head-aware reuse to DeepSeek-V4 MLA while preserving the packed latent representation and combining offline and online contributions before one shared projection. Its evidence separates archived TTFT and quality results, analytic compute accounting, and preliminary QPS.
- RedKnot-MLA adapts head-aware reuse to DeepSeek-V4 MLA without fragmenting the packed latent KV representation.
- Local-head artifacts are relocated with query-side RoPE, combined with Global-head and protected-row recomputation, and merged before one shared projection.
- Archived Flash operating points demonstrate multi-fold hot-artifact TTFT reduction, while the 256K paired study reports analytic arithmetic omission with aggregate F1/EM preservation and a per-dataset regression.
- The preliminary approximately 2× QPS result remains qualified because its raw concurrency trace is an explicit release task.
- Figure 5 provides supporting sparse-execution evidence rather than end-to-end RedKnot-MLA performance.