PaperlayerView exact revision on arXiv

Source-linked AI summary

SemBridge: Compiling Consumer Observations into Cross-Stack Communication Plans

Genlang Chen, Junyi Zhu, Yuanshan Lin

arXiv:2609.08231v1cs.DC

TL;DR

Distributed-tensor layouts and collective optimization do not specify what a remote consumer must observe across disjoint vendor communicator domains. SemBridge compiles graph and runtime facts into checked, observation-aware communication plans. Source projection sharply reduces result traffic and improves throughput in the reported workloads.

  • Problem

    Distributed-tensor and collective abstractions do not state what a remote consumer must observe at boundaries between disjoint native communicator domains.

    Source: §1 · p. 1

  • Method

    SemBridge compiles graph and runtime facts into a typed contract for consumer-visible results and delivery obligations, then lowers and checks backend-neutral communication plans.

    Source: §1 · pp. 1–2

  • Results

    Source projection reduces result traffic by more than 99.97% and improves throughput throughout the measured MiniMax load range.

    Source: §1 · p. 2, §9 · p. 11

  • Takeaways & Limitations

    Consumer observation provides a semantic layer between placement and collective execution for cross-stack communication.

    Source: §1 · p. 1, §9 · p. 11

    When required graph or runtime facts are unavailable, the frontend falls back to the layout-materializing plan, so log-probability requests retain complete-logit redistribution.

    Source: §3.4 · p. 4

Abstract

from arXiv · show

Distributed-tensor systems specify where values reside, while collective systems optimize how requested operations execute. At a boundary between vendor runtimes that cannot share a native communicator, neither abstraction states what a remote consumer must observe. SemBridge fills this gap by compiling graph and runtime facts into a typed contract for the consumer-visible result and its delivery obligations. The contract captures provenance, substitutability, completion, authority, demand, and native-domain locality. A deterministic lowerer constructs backend-neutral communication plans, and a symbolic checker validates each plan before execution across CUDA/NCCL and CANN/HCCL. An independent layout-only planner handles all 72 structural transitions but establishes only 54 complete obligations; a byte-only minimizer proposes 40 semantically invalid candidates, all rejected by SemBridge. On nine real edges, SemBridge produces distinct observation-aware plans that reduce startups on all nine and payload bytes on the three result edges. A live CUDA/CANN run derives and executes full-logit reconstruction, source projection, and owner-token delivery from log-probability, token-only, and owner-scoped requests. On a measured two-host 1-GbE capacity-spillover deployment, source projection cuts result traffic by more than 99.97% and increases throughput by 8.92-80.20% across Dense, MoE, and MiniMax workloads. All 18 MiniMax restart pairs at concurrency 1, 8, and 16 favor source projection. A Qwen3-14B MLP slice additionally verifies bitwise activation-shard delivery and HCCL completion of row-parallel partials. These results establish consumer observation as a semantic layer between placement and collective execution.

1 Introduction

At mixed-vendor boundaries, placement and collective abstractions do not specify what remote consumers must observe. SemBridge compiles graph and runtime facts into checked communication plans, enabling distinct legal plans for different consumer requests.

1 Introduction

  • SemBridge adds a typed consumer-observation contract and symbolic checker between fixed placement and physical collective execution.The contract separates the requested result from delivery obligations such as provenance, completion, and authority.Source: §1 · pp. 1–2
  • Token-only requests permit source-side projection, while log-probability requests retain complete-logit transfer and reconstruction.Full-logit transfer, source projection, and owner-token delivery can preserve the same requested token ID with different payloads.Source: §1 · p. 1, Figure 1 · p. 2
  • Across 72 layout transitions, layout-only planning establishes 54 complete obligations; SemBridge reduces bytes in 30 and startups in 35 of those cases.It also reduces bytes and startups in 10 and 13 of the remaining 18 authority cases, respectively; on nine real edges it reduces startups on all nine and bytes on the three result edges.Source: §1 · p. 2
  • Source-side projection reduces result traffic by 99.979–99.996% and improves throughput by 8.92% on Dense TP4 and 26.39–62.87% on MoE TP2.MiniMax throughput improves by 18.64–80.20% across concurrency 1, 8, and 16, with all 18 restart pairs positive.Source: §1 · p. 2

2 Cross-Domain Communication Semantics

Cross-domain communication is expressed as a backend-neutral graph between endpoints in distinct native collective domains. SemBridge combines placement with consumer demand and result obligations to determine which values and copies are required.

2.1 Execution Model

  • Native collective domains group endpoints that can participate directly in one vendor runtime; cross-domain operations connect different domains in a backend-neutral graph.SemBridge sits between placement and physical collective execution and identifies a logical graph that preserves each consumer’s required observation.Source: §2.1 · p. 3

2.2 Placement and Consumer Observation

  • Layouts specify shard mappings and partial reductions, but consumer demand, result surface, substitutability, completion, and decision ownership require additional application-boundary facts.The same fixed endpoint layout can require shard-preserving transfer, representative delivery, reduction, owner-sourced decisions, or token projection.Source: §2.2 · p. 3

2.3 Consumer Surfaces and Delivery Obligations

  • Consumer observation factors into a result surface and delivery obligations for provenance, substitution, completion, or authority.Complete-value, log-probability, and stochastic-sampling requests retain complete logits because the current contract does not carry RNG state.Source: §2.3 · p. 3

2.4 Scope of the Current Backend

  • The current CUDA/CANN backend exercises replicated forward values, complete-logit reconstruction, token projection, owner-sourced tokens, exact shards, and partial completion.The Qwen3-14B MLP path uses the production transport for lane-preserving shards and HCCL completion of row-parallel partials.Source: §2.4 · pp. 3–4

3 Consumer-Observable Contracts

SemBridge represents each communication boundary with a contract describing demanded consumers, native domains, value lineage, completion, authority, and the requested result. A symbolic checker tests whether lowered plans preserve the required consumer observations.

3.1 Boundary Contract

  • The boundary contract records producer and demanded-consumer sets, native domains, value kind, substitutable classes, reduction state, owner, and requested result surface.Graph and runtime APIs supply these facts; a separate execution tag identifies the request instance to which a delivered frame belongs.Source: §3.1 · p. 4

3.2 Consumer Observation

  • A consumer observation comprises surface, value, completeness, provenance, and authority, and a legal plan must match the registered observation at demanded consumers.Token-only executions are compared after the registered projection, while authority requires decisions to originate at the runtime-selected owner.Source: §3.2 · p. 4

3.3 Legality Invariants

  • The checker tracks availability, surface, completeness, and provenance through operations, accepting only plans that satisfy coverage, authority, completion, and native-domain constraints.It rejects unavailable sources, missing consumers, unfinished reductions, invalid provenance, non-owner decisions, and cross-domain operations marked as local.Source: §3.3 · p. 4

3.4 Semantic Inputs

  • Optimization-authorizing facts come from graph structure and runtime APIs; when a required fact is unavailable, the frontend emits the layout-materializing plan.In particular, log-probability requests retain complete-logit redistribution, and runtime byte-equality checks validate executions without authorizing lowering.Source: §3.4 · p. 4

3.5 Contract Construction and Static Validation

  • Contract construction combines endpoint placement, demand, result surface, owner selection, shape, and dtype, then validates mappings, domains, reduction lineage, and projection preconditions.Registered reductions are completed before exposing a full value; static validation also checks endpoint membership and observation compatibility.Source: §3.5 · pp. 4–5

3.6 Soundness Argument

  • Under valid graph/runtime facts, correct registered projections and native collectives, and reliable tagged delivery, every accepted plan preserves demanded consumers’ conservative observations.The proof sketch establishes this by induction over operations that preserve or discharge value, provenance, completion, authority, and coverage requirements.Source: §3.6 · p. 5

4 Checked Semantic Lowering

SemBridge lowers contracts through ordered semantic passes, checks the resulting backend-neutral plan, and accepts it for runtime realization only if legal. Its evaluation compares against conservative, layout-only, and semantic-oracle planning approaches.

4.1 Lowering and Checking Procedure

  • The lowerer orders completion, result-surface transformation, source selection, and cross-domain delivery, then checks coverage, completion, provenance, authority, and locality.Token-only edges may project at the producer; complete-value and log-probability requests retain layout redistribution, while replica, shard, partial, and owner cases use their respective delivery rules.Source: §4.1 · p. 5

4.2 Reference, Layout-Only Baseline, and Semantic Oracle

  • The Layout-Only baseline uses layouts and endpoint meshes without demand, observation, authority, lifetime, checker, or cost semantics; the Semantic Oracle shares SemBridge’s semantic IR and checker.A separate byte-only minimizer reduces remote transfer without consumer semantics and supplies legality counterexamples.Source: §4.2 · pp. 5–6

4.3 Bounded Validation Oracle

  • Within a bounded family of small topologies, the production lowerer is Pareto efficient and lexicographically optimal for its measured plan footprint.The oracle’s search is constrained to at most four producers, four demanded consumers, and three consumer domains, with restricted delivery operations and no overlap schedule.Source: §4.3 · p. 6

4.4 Complexity and Operation Coalescing

  • Lowering groups demanded consumers by domain and substitutable producer class; its checker tracks endpoint state in O(|P| + |C| + m) space for m emitted operations.Payload coalescing affects estimated startup cost but does not remove constituent-edge legality checks.Source: §4.4 · p. 6

4.5 Relation to Collective Synthesis

  • Collective synthesis optimizes execution after SemBridge selects the logical requirement; SemBridge first determines whether and what must cross the boundary.A backend can optimize the representative transfer and domain-local broadcast operations emitted by SemBridge.Source: §4.5 · p. 6

5 Cross-Stack Realization

The prototype connects independent CUDA/NCCL and CANN/HCCL serving engines through checked logical plans, while retaining each vendor runtime for local collectives. Its adapter realizes observation-specific result delivery and coordinates cross-host request retirement.

5.1 Runtime Architecture

  • The prototype connects independent CUDA/NCCL and CANN/HCCL engines, retaining native runtimes for local tensor-parallel collectives and mapping accepted plans to runtime switches.The adapter controls forward aggregation, result aggregation, and node-local result fanout; current constraints couple hidden and residual aggregation and require one authoritative decision-return edge.Source: §5.1 · p. 6

5.2 Data, Result, and Control Paths

  • Forward plans use mapped shard delivery or representative transfer with native fanout; result plans select full-logit reconstruction, source projection, or owner-sourced delivery.Log-probability requests retain full-logit redistribution; deterministic token-only requests project to int32 IDs before transfer, while unique authority permits one owner value per remote domain and local broadcast.Source: §5.2 · pp. 6–7

5.4 Ordering and Failure Boundary

  • Cross-host retirement waits for every local rank to drain its current collective slot before host acknowledgements advance the shared epoch.Frames carry request, token, microbatch, and state-epoch fields; transport closure and peer timeouts surface as service-layer execution errors.Source: §5.2 · p. 7, §5.4 · p. 7

5.3 Plan Realization

  • The checked plan generates runtime settings without configuration identifiers, composing shard, representative, owner-token, and result-surface choices.The compared replica campaigns emit forward/result frame pairs of (2, 4), (1, 4), and (1, 2) per logical step; Equation 6's end-to-end effect depends on workload communication-to-computation ratio.Source: §5.3 · p. 7

5.5 Backend Scope and Portability

  • The logical IR remains backend-neutral, while the exercised backend pair uses point-to-point transport, native fanout and reduction, projection, and completion reporting.The Qwen3 MLP slice uses lane-preserving frames for exact shards and HCCL AllReduce for partial completion; socket framing and buffer layout are not logical-plan requirements.Source: §5.5 · p. 7

6 Evaluation

The evaluation tests whether layout alone establishes consumer obligations, whether checked plans preserve them on CUDA/CANN engines, and whether graph differences affect inference. Source projection and representative delivery improve results in the measured capacity-spillover setting, with effects varying by workload and load.

6 Evaluation

  • The evaluation follows an evidence chain from layout-only observation coverage, through checked CUDA/CANN execution, to end-to-end inference effects.Performance estimates use uninstrumented runs; correctness probes and instrumented runs are excluded.Source: §6 · p. 7, §6.1 · p. 8

6.1 Methodology

  • In the two-host 1-GbE CUDA/CANN deployment, six fresh-process pairs per comparison use fixed workload protocols and compare matched configurations.The setup includes A100 and Ascend 910B4-1 hosts; MiniMax load sweeps hold prompt and output length fixed while varying concurrency across 1, 8, and 16.Source: §6.1 · pp. 7–8

6.2 Compiler Comparison with Layout-Only

  • Layout-Only handles all 72 transitions but establishes 54 complete obligations; SemBridge lowers bytes in 30 and startups in 35 of those, and rejects all 40 invalid byte-only candidates.Across nine real edges, SemBridge selects different plan shapes, reduces startups on all nine, and reduces bytes on the three result edges.Source: §6.2 · pp. 8–9, Table 2 · p. 9

6.3 Projection Placement Changes End-to-End Execution

  • Source projection removes 130 result AllGathers on Dense and 332 on MoE, improving p95 TPOT by 10.46% and 41.95%, respectively.The same token-only surface uses identical forward traffic and result-frame counts; source projection applies argmax before transfer rather than reconstructing logits at the consumer.Source: §6.3 · p. 9, Figure 3 · p. 9
  • All 18 pair effects were positive; p95 total latency fell by 15.71%, 44.54%, and 43.29% at concurrency 1, 8, and 16, respectively; throughput rose 76.63% at concurrency 16.Source: §6.3 · p. 10

6.4 Exact Shards and Partial Completion on Real Backends

  • The Qwen3-14B MLP slice verifies bitwise activation-shard delivery and identical completed outputs after HCCL AllReduce of distinct row-parallel partials.Relative L2 error against the full FP32 downprojection reference is 0.207–0.244%.Source: §6.4 · p. 10, Table 3 · p. 10

6.5 Legal Plans Have Workload-Dependent Effects

  • Representative broadcast raises throughput by 38.57% on Dense TP4 long output, 13.68% on a 16-request MiniMax burst, and 19.17% on long-output MoE at concurrency 8.It has no replica advantage at TP1 and reduces host traffic by 50.6–52.5% at TP2 and 75.2–76.2% at TP4.Source: §6.5 · p. 10, Figure 4 · p. 10

7 Related Work

Related systems address placement, serving, or collective execution; SemBridge focuses on the consumer-visible requirements at mixed-vendor communication boundaries.

7.1 Placement and Distributed Tensors

  • Placement frameworks express sharding, while verification systems address whole-graph equivalence; SemBridge checks consumer observations at mixed-vendor boundaries.Distributed-tensor systems represent shard, replicate, and partial states, and cross-mesh resharding optimizes layout transfers.Source: §7.1 · pp. 10–11

7.2 Heterogeneous LLM Serving

  • Heterogeneous serving systems optimize placement or routing, while SemBridge derives boundary values and copies required by a fixed placement.Unlike selected homogeneous sampler paths, SemBridge derives when token handoff is eligible and retains complete logits for log-probability requests.Source: §7.2 · p. 11

7.3 Collective Synthesis and Mixed-Vendor Communication

  • Collective systems synthesize, tune, or realize communication programs, whereas SemBridge selects the consumer-observable requirement.The checked logical graph is then lowered to communication.Source: §7.3 · p. 11

8 Discussion

In the measured capacity-spillover deployment, projection sharply reduces result traffic while improving throughput and lowering p95 total latency. SemBridge separates selecting a legal consumer-observation plan from choosing among legal plans for the fabric.

8.1 Compiler Boundary

  • Full-logit transfer and source-side projection have identical forward traffic and result-frame counts, so their end-to-end difference comes from the compiler decision.The contract makes token-only projection, complete-logit retention, representative delivery, and source ownership explicit before transport.Source: §8.1 · p. 11

8.2 Performance and Fabric-Aware Selection

  • 99.9788–99.9959% lower result traffic, 8.92–80.20% higher throughput, and 8.12–44.54% lower p95 total latency were measured across evaluated model and load points.Every MiniMax restart pair favored source projection at concurrency 1, 8, and 16; representative delivery also raised throughput by 13.68–38.57% on communication-sensitive workloads.Source: §8.2 · p. 11
  • A fabric-aware selector chooses among checker-accepted plans when transfer savings exceed projection and local-collective cost.The model accounts for cross-domain bytes and frames, effective payload rate, startup cost, and non-overlapped local work; restart pairs measure net outcome while component timers identify transfer and local work.Source: §8.2 · p. 11

9 Conclusion

SemBridge makes consumer observations explicit at cross-stack boundaries, compiling and checking plans for execution. Across three model families, token projection reduces result traffic by more than 99.97% and improves throughput throughout the measured MiniMax load range.

9 Conclusion

  • Across three model families, token projection reduces result traffic by more than 99.97% and improves throughput throughout the measured MiniMax load range.The compiler/runtime path also derives and executes full-logit reconstruction, owner-token delivery, representative delivery, exact-shard transfer, and partial completion.Source: §9 · p. 11
Loading 2609.08231v1…