Source-linked AI summary
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA
Mind Lab, :, Vin Bo, Asher Cai, Jingwei Cao, Song Cao, Vic Cao, Amelia Chen, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Kaixuan Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Pyke Han, Nolan Ho, Ori Hong, Hailee Hou, Piers Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Qiuyu Jin, Fancy Kong, Kuss Koo, Jaron Lee, Andrew Lei, Alexy Li, Dawn Li, Lucian Li, Ray Li, Ricardo Li, Smith Li, Theo Li, Allen Lin, Elliot Lin, Fan Lin, Chen Ling, Kairus Liu, Kieran Liu, Logan Liu, Neo Liu, Xiang Liu, Yuxin Lu, Maeve Luo, Pony Ma, Verity Niu, Cole Qiao, Guian Qiu, Vince Qu, Sentry, Niko Song, Vincent Wang, Bo Wu, Rio Yang, Evelyn Ye, Fiona Ye, Ina Ye, Regis Ye, Josh Ying, Atlas Zeng, Danney Zeng, Salmon Zhan, Anya Zhang, Di Zhang, Mia Zhang, Sueky Zhang, Wei Zhao, Ada Zhou, Adrian Zhou, Yuhua Zhou, Juno Zhu, Murphy Zhuang
TL;DR
Continual learning for deployed agents remains limited by changing environments and fixed post-release systems. Macaron-V1 addresses this with recursive model-harness revision and Mixture-of-LoRA specialists, achieving 87.8 on UI4A-Bench while showing mixed results across external benchmarks.
Problem
Deployed agents face changing environments, while current training largely tunes models against snapshots and holds them fixed after release.
Method
Macaron-V1 combines recursive model-harness revision with a frozen base model and per-turn selection of specialist LoRA adapters.
Results
87.8 Final Score on UI4A-Bench supports clear, accurate, interactive UI generation, while performance leads TerminalBench 2.1 and trails the largest listed values on several other suites.
Takeaways & Limitations
Macaron-V1 provides an initial implementation of a growable agent-model decomposition with separately versioned but jointly evaluated system layers.
Takeaways & Limitations
The released snapshot does not demonstrate compounding over many generations, and cross-generation lift is not yet measured.
Abstract
from arXiv · showhide
Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through recursive improvement of versioned model-harness pairs, where experience from one configuration is evaluated under an external contract and used to construct its successor. Collaboration is pursued via the Mixture-of-LoRA (MoL) architecture that freezes a base model, composes specialist LoRA adapters, and selects one LoRA per user turn. The flagship Macaron-V1-Venti combines a 744B GLM-5.2 base with four LoRAs for chat, agent, coding, and GenUI; the Qwen3.6-based Macaron-V1-Tall (50B) uses the same design for local deployment. This report presents Macaron-V1 as a co-designed system spanning architecture, algorithms, and infrastructure. The MoL architecture supports continual learning through extensible LoRA specialists. The algorithm combines Model-Harness Co-design and recursive self-improvement loop, including the UI4A component-native GenUI harness, a stateful action substrate, versioned HCP contract, and the agentic RL framework MindForge. The supporting infrastructure includes the post-training platform MinT, the long-context RL method LongStraw, and stability techniques for sparse MoE and DSA base models. We evaluate Macaron-V1 on Personal Intelligence, GenUI, and general capability benchmarks against frontier baselines. Our results validate the current system, while compounding gains from continual learning and collective intelligence remain open questions.
1 Introduction
Macaron-V1 targets experiential intelligence through two complementary dimensions: recursive adaptation of versioned model–harness pairs and collaborative composition of specialist LoRA adapters. It presents these ideas as a co-designed open-model system spanning architecture, harnesses, infrastructure, and interaction-trajectory benchmarks.
- Motivation: Post-training outcomes depend on the data, tools, users, interactions, and deployment environment surrounding a model.The paper contrasts this environment dependence with centralized post-training over bounded tasks and harnesses.
- Design goals: Macaron-V1 operationalizes experiential intelligence as adaptation and collaboration, combining learning from deployment experience with modular capability composition.Experiential intelligence requires learning from real-environment experience and continuing after deployment.
- Adaptation: Adaptation recursively evaluates experience from one versioned model–harness configuration under an external contract to construct a successor revision.The longer-term objective measures retention, transfer, and cumulative improvement across successive revisions.
- Collaboration: Mixture-of-LoRA keeps a large base model frozen, layers specialist LoRA adapters on top, and selects one specialist per user turn through a shared runtime.This composition supports adding specialists without retraining the base and registering specialists from different teams or users.
- System framing: Macaron-V1 treats adaptation and collaboration as complementary: capabilities are selected and composed modularly while specialists, routing policies, and harnesses improve through experience.Improving these elements across generations while retaining prior capabilities remains a longer-term research direction.
- Contributions: The report contributes Model–Harness Co-design and RSI, the UI4A and REPL-based harnesses, MinT and LongStraw infrastructure, and trajectory-based Personal Intelligence benchmarks.Macaron ChatBench evaluates context-conditioned agent conversation, while Macaron LivingBench evaluates stateful simulated personal-life assistance under evolving conditions.
2 Mixture of LoRA
Mixture of LoRA keeps one frozen base model and composes specialist adapters through an observable per-turn proxy, enabling modular serving and separately versioned specialists. Its own-view continuity and shared-base deployment support multi-specialist service, while reported quality, cache, and comparative-latency claims remain bounded by the evaluated configurations.
- Architecture: MoL freezes one large base and layers specialist LoRA adapters, with the MoL Proxy making adapter selection an observable action on each user turn.This architecture replaces a monolithic model with a serving-layer composition of specialists.
- Quality and KV reuse: 0.650 ± 0.030 was the Venti routed reuse-off Vita score versus 0.636 direct and 0.632 ± 0.019 routed reuse-on, showing no detected quality loss at this scale.The Tall scores were 0.410 ± 0.030 direct, 0.398 ± 0.035 routed KV-off, and 0.386 ± 0.054 routed KV-on.
- Serving and continuity: The Proxy reconstructs each specialist’s own-view from the append-only timeline, retaining its full trace while summarizing other specialists’ turns into 192-token messages.This construction supports deterministic specialist re-entry and cross-adapter continuity without feeding the summary scaffold into reusable prefixes.
- Deployment boundaries: The validated deployment envelope requires contiguous-prefix reuse, while specialist switches invalidate adapter-specific KV state and arbitrary non-contiguous GPU KV splicing is not implemented.The default deployment uses the emergent path with unmodified engines; route-decode remains opt-in and experimental.
- Deployment efficiency: MoL stores about 774.8B logical parameters versus 2.976T for four replicated merged bases, making it 26.0% of that layout and reducing stored parameter values by 74.0%.The Venti configuration uses one nominal 744B base plus about 30.8B adapter values.
- Extensibility and limitations: A shared frozen base and portable adapter registry allow separately versioned specialists and potential cross-owner composition, but the release evaluates only four shipped specialists and does not test personalization quality or privacy.The base, specialists, and harness can move on independent release cadences; intended cross-owner and personalized-adapter uses remain unevaluated.
3 Algorithm: Model–Harness Co-design and Recursive Self-Improvement
Macaron-V1 treats the harness as a first-class training target and co-designs it with the model through executable substrates, UI4A, versioned HCP artifacts, and recursive self-improvement. Adaptive configuration search reaches every task without weight updates, while parameter updates are reserved for transferring selected behavior into adapters.
- Harness as a training target: The harness defines the agent’s action interface and runtime contract, so train–serve divergence is treated as a source-level bug rather than a modeling problem.Changing only the substrate can alter success rate and token cost, while expressive substrates keep intermediate computation resident instead of returning it through the model as text.
- UI4A: UI4A lets agents write ordinary frontend code with imports, components, state, functions, and runtime-enforced Actions, with an optional specialist adapter for rendering and component binding.When adapters are unavailable, the harness runs against the base model at higher token cost and latency.
- Stateful action substrate: The agent harness uses a stateful Python REPL, keeping intermediate values resident across tool calls and supporting validated helper reuse through save_tool and promote_tool.A helper becomes callable by later queries only after passing a private validation run against a held-out reference.
- Harness Context Protocol: The versioned TOML Harness Context Protocol reconstructs runtime configuration from portable, auditable artifacts, while separating independently paced release clocks for the base, specialists, and harness.HCP carries no gradients; models can rewrite the harness configuration it describes, and harness changes can be evaluated and released without a weight update.
- Recursive self-improvement: 122/122 tasks achieve cumulative unique coverage after 69 chronological jobs and 450 task attempts, without a weight update.Single portfolio configurations pass 4/122 (3.3%) and 11/122 (9.0%), whereas the final 21 jobs reach 81.2% and close the remaining 62 tasks with 3 errors; the subsequent adapter update is not executed or tested for generalization.
4 Infrastructure
Macaron-V1’s infrastructure manages versioned LoRA state, enables response-only long-context updates, and applies model-path-specific controls for sparse-base consistency. These mechanisms provide lifecycle and execution support, while the reported receipts and experiments have explicit evidence boundaries.
- MinT: MinT separates immutable adapter revisions from mutable policy records, preserving model-state lineage across training, rollout, evaluation, serving, resumption, and audit.Reconstructability still requires joining the adapter revision and compatible base with HCP and MindForge artifacts.
- MinT: 18.3× and 2.85×: adapter-only handoff reduces measured training–serving handoff time versus merged checkpoints for Qwen3-4B and Qwen3-30B probes.The probes used rank-32 and rank-16 adapters, respectively, with the base already resident in the sampler.
- MinT: 106 entries: MinT built a packed Qwen3-30B rank-1 adapter catalog with zero build errors and audited 256 entries across all 100 storage shards.Serving selects bounded working sets while actors maintain smaller CPU caches and GPU-active windows.
- LongStraw: LongStraw bounds autograd graph lifetime by replaying responses serially after prompt-state capture, making live graph memory depend on the longest response rather than all group members.The method retains architecture-specific state and does not eliminate prompt-dependent forward computation or memory.
- LongStraw: LongStraw execution receipts completed fixed-hardware response-only checks at exactly 2,097,152 and 4,456,448 positions, but were not Macaron-V1 reruns or quality evaluations.The evidence does not establish cross-system throughput, learning curves, or training of Macaron-V1-Venti or Macaron-V1-Tall at these context lengths.
- Sparse-base controls: R3, DSA alignment, and residual filtering are alternative model-path controls for route provenance, implementation mismatch, and probability outliers, respectively.The report provides no controlled Macaron-V1-Venti ablation attributing benchmark gains to these controls.
5 Benchmarks
Macaron-V1 evaluates Personal Intelligence, GenUI, and broader capabilities using interaction-centered benchmarks designed for context-dependent judgment and dynamic stateful behavior. The suite combines reproducible within-benchmark comparisons with co-designed evaluation logic, while UI4A-Bench tests clear, accurate, usable, interactive interfaces in a shared runtime.
- Benchmark scope: The benchmark suite covers Macaron ChatBench, Macaron LivingBench, UI4A-Bench, and broader transfer evaluations, each serving a distinct evidentiary role.The internal suites contain 46 ChatBench cases, 40 LivingBench scenarios, and 161 UI4A-Bench cases.
- UI4A-Bench: UI4A-Bench evaluates whether models generate clear, accurate, usable, interactive cards under a fixed UI4A runtime, case set, viewport, judge, and scoring policy.Unlike evaluations centered on agent action or reference-design similarity, it treats the interface as a structured artifact users can read, correct, and continue using.
- Personal Intelligence: Personal Intelligence evaluation targets interaction trajectories because personalized quality depends on user identity, scenario, relationship stage, changing state, and limited user patience.These conditions make context-free correctness and static or single-turn evaluation difficult to define.
- Macaron ChatBench: Macaron ChatBench makes conversational-quality judgment reproducible by combining fixed axioms with persona- and scenario-conditioned criteria across seven conversation scenarios.Its score remains an LLM-mediated judgment rather than an objective correctness measure.
- Macaron LivingBench: Macaron LivingBench simulates dynamically unfolding multi-turn behavior under realistic noise, enabling observation and scoring of state maintenance and replanning.Its simulator adapts trajectories to tested models, so different models need not receive identical event sequences.
6 Results
Macaron-V1-Venti shows strong results on Personal Intelligence, coding and terminal, and GenUI benchmarks, while agent performance varies by benchmark. Macaron-V1-Tall exceeds its Qwen3.6 base across all seven reported rows, but the evidence does not establish a single cross-benchmark ranking or causal attribution for the gains.
- Personal Intelligence: Personal Intelligence scores are 58.3 on Macaron ChatBench and 64.0 on Macaron LivingBench, exceeding GPT-5.5 by 2.8 points and Opus 4.8 by 0.2 points, respectively.Both comparisons use the same cases, judge stack, sampling policy, and aggregation rule, although ChatBench uses a private GLM-5.2 judge.
- Agent: Agent results vary: Macaron-V1-Venti scores 60.0 on VitaBench, 46.0 on VitaBench2, and 69.3 on τ 3-Bench, with different protocols limiting direct comparison across rows.On VitaBench, Qwen 3.7 Max scores 61.2; on τ 3-Bench, Macaron-V1-Venti scores 69.3 versus 69.1 under pass@1.
- Coding and terminal: Macaron-V1-Venti records the largest reported TerminalBench 2.1 score at 87.6, while its DeepSWE score is 58.4, below GPT-5.5’s 70.0 and above Opus 4.8’s 58.0.On SWE-Verified and SWE Atlas QnA, Opus 4.8 has the largest reported values: 88.6 and 57.3, respectively.
- Generative UI: Under the common UI4A-Bench protocol, Macaron-V1-Venti achieves an 87.8 Final Score and leads baselines by 12.0 points in Constraint Adherence, 6.6 in Visual Quality, and 1.6 in Interaction.The layer-level comparisons are 94.2 versus Opus 4.8’s 82.2, 90.0 versus GPT-5.5’s 83.4, and 95.0 versus Opus 4.8’s 93.4.
- Macaron-V1-Tall at deployable scale: Macaron-V1-Tall has a larger point estimate than its Qwen3.6 35B-A3B base on all seven reported rows, with gains ranging from 1.3 points on Macaron LivingBench to 25.4 on UI4A-Bench.This comparison is not a parameter-matched ablation because the systems differ in parameter footprint, adapters, routing, and potentially harness behavior.
- Limitations: The results do not define one cross-benchmark model rank, and component-level causal attribution for gains from specialization, routing, the harness, or cross-generation compounding remains unresolved.The reported Personal Intelligence and UI4A results are targeted point estimates without interval or judge-sensitivity analysis.
7 Discussion and Conclusion
Macaron-V1 presents continual learning as a jointly evaluated loop of model, harness, environment, and infrastructure, with MoL enabling specialist composition on a shared base. The release demonstrates the system design and execution checks, but longitudinal improvement, collective intelligence, broader evaluation, and safety remain open.
- System framing: Macaron-V1’s architecture combines a frozen shared base, portable LoRA specialists, a first-class harness, and recursive self-improvement over versioned configurations.MindForge evaluates trajectories under the same harness that ships in production, supported by MinT and LongStraw.
- Limitations: The released checkpoint is one snapshot: cross-generation improvement has not been measured, so compounding RSI effects remain unverified.The snapshot cannot distinguish compounding RSI from a single round of self-generated-data training.
- Limitations: MoL provides an interoperability path for specialists trained by different teams or personalized for users, but this release tests only four shipped adapters.Current tests cover routing, per-specialist conversation views, and adapter registration without establishing robust broad-workload switching or population-level collective intelligence.
- Limitations: Evaluation remains limited by narrow scenarios, focused internal suites, imported leaderboard values, and incomplete documentation of data governance, reproducibility, and safety scope.The report interprets current results as systems characterization rather than evidence for safety-critical use.
- Roadmap: The roadmap prioritizes additional specialists, third-party and personalized adapters, more UI4A renderers, and tighter consent-based integration of real interactions into the RSI loop.Planned specialists include domain-specific research and non-English long-form; Flutter and native mobile are the next UI4A targets.
A Author List
The section presents the paper’s author list, with names arranged alphabetically.
- The authors’ names are listed alphabetically.
- The list includes Vin Bo, Asher Cai, Jingwei Cao, Song Cao, Vic Cao, Amelia Chen, and other contributors.
B Evaluation Protocols and Training Details
The appendix specifies evaluation protocols for twelve benchmark rows in Table 8, using within-row comparisons on a 0–100 scale rather than cross-metric averages. Unstarred models share benchmark-specific evaluation settings, while starred values provide contextual references without uncertainty estimates.
- Evaluation Protocols: The appendix defines protocols for twelve benchmark rows in Table 8, with scores reported on a 0–100 scale.Comparisons are made within benchmark rows rather than averaged across different metrics.
- Evaluation Protocols: Within each row, unstarred models use the same task set, harness or simulator, judge stack, sampling policy, and aggregation rule.These shared settings are benchmark-specific.
- Evaluation Protocols: Starred values come from public leaderboards or model reports as contextual references, and the table reports point estimates without confidence intervals or hypothesis tests.These values are not described as results from the common unstarred evaluation protocol.
B.1 Personal Intelligence
Macaron-V1 evaluates Personal Intelligence with ChatBench and LivingBench, two internally targeted benchmarks built around product behaviors and failure distributions. Their scores support within-benchmark comparison but characterize a targeted distribution rather than a general-purpose public leaderboard.
- ChatBench: ChatBench evaluates 46 de-identified real multi-turn cases, split evenly between interaction quality and task understanding/completion, using a common privately deployed GLM-5.2 judge.Each model–case pair is sampled three times and averaged, with case-specific 1–5 criteria derived from six axioms.
- LivingBench: LivingBench covers 40 everyday scenarios across Chinese and English, with up to 10 turns, six sandbox roles, and 37 semantic tools spanning 14 domains.Write operations include per-tool permission flags such as auto-execute versus requested approval.
- Internal-benchmark scope: ChatBench and LivingBench target product behaviors and failure distributions from the RSI process, enabling direct within-benchmark comparison through common case sets and evaluation stacks.The benchmarks characterize a targeted Personal Intelligence distribution rather than a general-purpose public leaderboard.
B.2 A Full LivingBench Trace
The LivingBench trace documents a 10-turn Macaron-V1-Venti case involving elderly-fall triage and care coordination. It achieves full need coverage while recording a Process score of 0.593 through replanning, deadline management, delegation, transport recovery, and social leverage.
- Case outcome: Need 1.000 and Process 0.593 are reported for the 10-turn elderly-fall triage and care-coordination case.The case concerns a 74-year-old fall victim with possible hip injury, competing work obligations, transport constraints, and refusal to seek care.
- Adaptive replanning: The agent identifies a language mismatch in its initial hospital recommendation and replans to a female Sinhala-speaking home visit followed by possible X-rays.The revised plan accounts for the patient’s preferences and proposes a 750 m trip to Asiri Diagnostic Centre if needed.
- Refusal and safety: The trace closes the checklist by using a neighbor as social leverage to overcome the mother’s refusal while addressing safety and the user’s guilt.The agent advises against pulling the patient by the arms and recommends a stool-based, sideways transfer method.
- Competing obligations: The agent converts the user’s work conflict into three explicit options, including delegation, remote attendance, or leaving the patient with Renuka.It later drafts a delegation message to Priyantha when the user chooses that path.
- Execution recovery: When ride-hailing fails, the agent recovers transport by directing the user to call Chaminda and sequences the remaining actions through hospital arrival.The six-step sequence includes contacting Priyantha, calling Chaminda, enlisting Sunethra, gathering documents and medications, safely helping Amma into the car, and routing to hospital.
B.3 Agent
The Agent section evaluates models across VitaBench, VitaBench2, τ 3-Bench, PinchBench, and ClawGym using specified judges, simulators, APIs, and reporting protocols. Results are designed for within-set comparability, while some leaderboard values are imported or use different aggregation settings.
- VitaBench: VitaBench reports macro-average task success across Delivery, In-Store, OTA, and Cross under a reproduced GLM-5.1 judge-and-user protocol.Every model in Table 8 was rerun under the same protocol, making rows directly comparable within the evaluation set; scores characterize the reproduced GLM-5.1 variant.
- VitaBench2: VitaBench2 evaluates Chinese personalization under Rewrite memory and reports Avg@1 across user sequences.Subtask rewards are averaged within each sequence and then across users; official leaderboard Avg@4 values are a different reporting setting.
- τ 3-Bench: τ 3-Bench uses GPT-5.2 with reasoning_effort=low as the user simulator and reports pass@1 under a single-trial task-success protocol.All unstarred models share this setup, while the starred Gemini 3.1 value is imported from Mind Lab (2026c).
- PinchBench: PinchBench uses Claude Haiku 4.5 as judge, Perplexity as the search API, and reports the best observed score.Unstarred models use this setup, while starred baseline values are imported from pinchbench.com for public leaderboard context.
- ClawGym: ClawGym evaluates every model with GPT-5.4 as judge under the same evaluation setting and reports pass@1.The shared setting applies across all evaluated models.
B.4 Coding and Terminal
The coding and terminal evaluations use Claude Code-based harnesses, with benchmark-specific environments, retry or sampling policies, timeouts, and pass metrics. Public leaderboard baselines are marked separately from scores produced under the stated evaluation configurations.
- SWE-Verified: SWE-Verified uses the Claude Code harness; evaluation-error cases are retried up to three times, while valid attempts are not retried for score selection.Evaluation errors occur at a rate of approximately 0.8%, and the successful evaluated attempt is reported.
- TerminalBench 2.1: TerminalBench 2.1 runs through Harbor and the Claude Code Agent Harness in sandboxes with a four-hour timeout, reporting pass@1.Starred baselines come from the public TerminalBench leaderboard.
- DeepSWE: DeepSWE uses Claude Code, samples up to three attempts, and reports the best attempt.Starred baselines are imported from the public DeepSWE leaderboard.
- SWE Atlas QnA: SWE Atlas QnA uses Claude Code with a Claude Opus 4.8 judge and reports pass@3.Unstarred scores use the common harness and judge configuration, while starred baselines come from the Scale leaderboard.
B.5 Generative UI
UI4A-Bench evaluates code-native interactive UI generation from natural language across diverse experience domains without schema guidance. Its rubric reports five Layer Scores using case-relevant checks from a registry of more than 200 criteria.
- UI4A-Bench: UI4A-Bench contains 161 cases spanning eight experience domains.
- UI4A-Bench: The benchmark asks models to generate code-native interactive UIs from natural language without schema guidance.
- UI4A-Bench: Its rubric registry has more than 200 general and domain-specific checks, applying only case-relevant checks.
- UI4A-Bench: Five Layer Scores measure engineering viability, task quality, visual quality, interaction, and constraint adherence.
B.6 Training Hyperparameters
The four public specialists retain a shared adapter architecture and post-training configuration, with release metadata documented for Venti and Tall. Their common recipe uses AdamW, a 5 × 10−6 learning rate, and a linear-warmup cosine schedule, with specialist-specific batch-size and epoch changes.
- Shared release configuration: The public releases retain the adapter architecture and post-training configuration across four specialists, with Table 11 documenting shared metadata for Venti and Tall.Stored-value counts sum tensor shapes in public adapter_model.safetensors headers and represent retained release tensors, not active parameters or device-memory use.
- Adapter target modules: Venti targets down_proj, gate_proj, kv_a_proj_with_mqa, kv_b_proj, o_- proj, q_a_proj, q_b_proj, and up_proj, while Tall adds Qwen3.6-specific projection modules and excludes gate, lm_head, and shared_expert_gate.Neither release saves embedding or language-model-head modules through modules_to_save.
- Common training recipe: All four specialists use AdamW with learning rate 5 × 10−6.The recipe is stated relative to the L2 (Coding) configuration and is shared across the specialists.
- Common training recipe: The common schedule uses batch size 4, four epochs, and linear-warmup cosine learning-rate decay with warmup ratio 0.1.These values define the baseline L2 (Coding) configuration.
- Specialist-specific changes: L0 (Chat) matches L2, L1 (Agent) uses a single epoch, and L3 (GenUI) uses batch size 2 with a single epoch.L1 and L3 otherwise match the L2 configuration.
C MoL Deployment Details
The deployment appendix reports scoped validation observations across H20 and B300 systems, vLLM and SGLang backends, and varied parallelism layouts. It frames the measurements around weight residency, capacity, latency scaling, and correctness while excluding unmatched quantitative comparisons.
- Scope: Measurements cover H20 and B300 systems, vLLM and SGLang backends, and multiple parallelism layouts at clearly specified operating points.The appendix retains measurements supporting weight residency, capacity, latency scaling, or correctness.
- Interpretation limits: Cross-engine snapshots with different request loads, LoRA residency, or speculative-decoding settings are not treated as quantitative comparisons.No row should be interpreted as a paired latency or throughput comparison against separately deployed merged specialists.
- Capacity: The H20 observations characterize feasible long-context workloads through engine-reported peak KV usage rather than throughput comparisons.Table 12 is explicitly framed as capacity observations for stated workloads.
- Latency scaling: B300 TP8 scaling measurements hold the workload and engine configuration constant while varying DCP degree and concurrency, reporting p50 TTFT and TPOT.The stated workload uses 1K input and 256 generated tokens; TTFT is in seconds and TPOT in milliseconds.
- Speculative decoding: Validated B300 TP8 EAGLE-enabled DCP8 operating points use fixed 8K-input, 1K-output, page-size-64 stress settings and report TPOT statistics plus aggregate output throughput.EAGLE uses five speculative steps, top-k 1, and six draft tokens; TPOT is reported as mean and p99.
- Correctness and capacity: Additional validation establishes CP LayerSplit effects, EAGLE-related logical KV capacity changes, and deployment boundaries from correctness and capacity constraints.These observations cover cold 900K-context TTFT, B300 TP8 FP8 capacity, and scoped constraints on the validated MoL configuration.