PaperlayerView exact revision on arXiv

Source-linked AI summary

Beacon: LLM Multi-Agent Driven Hardware Design Space Exploration for Heterogeneous Multi-Chiplet Deep Learning Accelerators

Boyu Li, Zongwei Zhu, Qianyue Cao, Xi Li, Xuehai Zhou

arXiv:2608.30932v1cs.AR

TL;DR

Simulator evaluations for heterogeneous multi-chiplet HW-DSE are costly, while conventional search relies mainly on final metrics and predefined states rather than detailed diagnostic reports. Beacon uses report-driven hierarchical LLM agents to guide hardware search and reduces the composite latency-energy-monetary-cost objective by 25.1% to 93.5% versus random search, Bayesian optimization, and reinforcement learning under the same limited evaluation budget.

  • Problem

    Heterogeneous multi-chiplet HW-DSE has costly simulator evaluations, while conventional search relies on final metrics and predefined states rather than explicitly modeling diagnostic information in evaluator reports.

    Source: §I · p. 1

  • Method

    Beacon uses hierarchical LLM agents to localize bottlenecks, diagnose root causes, and generate hardware modifications from reports, with an Analysis Toolbox and RAG Memory supporting closed-loop search.

    Source: §I · p. 2, §VI · p. 13

  • Results

    25.1% to 93.5%: Beacon reduces the composite latency-energy-monetary-cost objective versus random search, Bayesian optimization, and reinforcement learning under the same limited evaluation budget.

    Source: §I · p. 2

  • Takeaways & Limitations

    Beacon's best designs trade off chiplet count, dataflow, bandwidth, and execution parameters according to workload behavior and cost feedback rather than scaling all resources linearly.

    Source: C. Result Analysis · p. 10

Abstract

from arXiv · show

Heterogeneous multi-chiplet accelerators allow chiplets to be configured independently to better match different operator characteristics and improve inference efficiency. However, heterogeneity makes simulator evaluation expensive, limiting the number of iterations affordable for hardware design space exploration (HW-DSE). Mainstream data-driven methods rely mainly on final metrics and a few predefined states, and require many search iterations to implicitly learn the relationships between input parameters and optimization objectives, making them less effective in this setting. In practice, evaluators also generate detailed reports on execution timelines, resource utilization, memory accesses, and communication behavior. Large language models (LLMs) can combine domain knowledge with these reports to explicitly identify bottleneck locations, degradation causes, and parameter adjustment directions, thereby improving each design decision under limited iteration budgets. Based on this observation, we propose Beacon, a report-driven LLM multi-agent framework for heterogeneous multi-chiplet HW-DSE. Beacon employs hierarchical agents for bottleneck localization, root-cause diagnosis, and hardware candidate generation, together with an Analysis Toolbox and RAG memory for closed-loop search. Under the same limited iteration budget, Beacon reduces the composite latency-energy-monetary-cost objective by 25.1\%--93.5\% compared with random search, Bayesian optimization, and reinforcement learning.

I. INTRODUCTION

Beacon uses evaluator reports and hierarchical LLM agents to guide heterogeneous multi-chiplet hardware design exploration under limited evaluation budgets. The introduction motivates this approach by the high cost of simulator evaluation and the diagnostic information omitted by conventional data-driven methods.

I. INTRODUCTION

  • Simulator evaluation dominates each HW-DSE iteration and grows linearly with heterogeneous chiplet count, limiting a run to tens of iterations.Evaluation includes workload mapping search, chiplet-level simulation, and inter-chiplet communication modeling; even on a high-performance multicore server, a complete evaluation typically takes tens of minutes.Source: §I · p. 1
  • Conventional data-driven methods rely mainly on final metrics and predefined states, requiring repeated trials to learn parameter–objective relationships and often thousands of iterations to converge.This makes them unsuitable for heterogeneous multi-chiplet exploration with a limited iteration budget.Source: §I · p. 1
  • LLMs can use fine-grained evaluator reports and hardware knowledge to locate bottlenecks, diagnose degradation causes and parameter interactions, and suggest targeted adjustments.Reports include execution timelines, per-layer latency and energy, resource utilization, mapping behavior, DRAM accesses, and inter-chiplet communication.Source: §I · pp. 1–2
  • In a 324-point example, the LLM reached the optimum in five evaluations, while Bayesian optimization required 86.The LLM identified QKV generation as the initial bottleneck, selected WS, 16K MACs, and an 8 MB buffer for Chiplet A, then increased Chiplet B compute after the bottleneck shifted.Source: Figure 1 · p. 2, §I · p. 2
  • Beacon structures decisions as model-level bottleneck discovery, layer-level root-cause diagnosis, and architecture-level candidate generation, supported by ReAct-style interaction, an Analysis Toolbox, and RAG memory.Its report-driven LLM-agent paradigm targets architecture-level exploration of heterogeneous multi-chiplet accelerators under limited evaluation budgets.Source: §I · p. 2
  • Beacon reduces the composite latency-energy-monetary-cost objective by 25.1%–93.5% versus random search, Bayesian optimization, and reinforcement learning under the same limited evaluation budget.The evaluation covers large-model inference workloads.Source: §I · p. 2

II. RELATED WORK … A. Heterogeneous Multi-Chiplet Accelerator

Prior accelerator-search and chip-design agents provide useful foundations, but leave a gap in report-driven, architecture-level combinatorial optimization for heterogeneous multi-chiplet HW-DSE. The paper situates this gap against a configurable chiplet abstraction in which chiplets can be tailored to different layer workloads.

II. RELATED WORK — A. Hardware DSE for Accelerators

  • Accelerator HW-DSE spans offline surrogate-model methods and online search methods, which train predictors from samples or evaluate candidates iteratively.Examples include Mind Mappings, GANDSE, VAESA, and AIRCHITECT for surrogate approaches, and DiGamma, ConfuciuX, HASCO, and Spotlight for online search.Source: A. Hardware DSE for Accelerators · pp. 2–3
  • Large-scale data-sampling dependence makes these general accelerator-search methods difficult to apply directly when heterogeneous multi-chiplet evaluations are expensive.Source: A. Hardware DSE for Accelerators · p. 3

II. RELATED WORK — B. DSE for Multi-Chiplet Accelerators

  • Prior multi-chiplet accelerator studies use manual configuration, multi-objective genetic search, grid search, or Bayesian optimization over predefined hardware spaces.Examples are Simba, MOHaM, Gemini, and Compass, respectively.Source: B. DSE for Multi-Chiplet Accelerators · p. 3
  • These multi-chiplet studies support evaluation and mapping, but predefined candidates or relatively simple search methods limit hardware-space exploration.Source: B. DSE for Multi-Chiplet Accelerators · p. 3

II. RELATED WORK — C. Agents for Chip Design

  • LLM agents have been applied to chip design through EDA automation, hierarchical chiplet design, analog layout, circuit retrieval, and circuit analysis.The cited systems are ChatEDA, MAHL, LayoutCopilot, CROP, and Atelier, respectively.Source: C. Agents for Chip Design · p. 3
  • This work targets architecture-level combinatorial hardware-parameter optimization with hierarchical LLM collaboration over unstructured data, unlike EDA-flow or circuit-level optimization.Existing LLM-agent work typically generates scripts, code, layout operations, or circuit candidates and relies on compilation, simulation, rule-checking, or human feedback.Source: C. Agents for Chip Design · p. 3

III. PRELIMINARIES — A. Heterogeneous Multi-Chiplet Accelerator

  • The accelerator abstraction derives from common features in prior chiplet DSE studies and represents different designs by changing configuration parameters.Multiple compute chiplets connect through a network on package; boundary chiplets access external DRAM through IO dies or memory interfaces, and the abstraction supports homogeneous or heterogeneous configurations.Source: A. Heterogeneous Multi-Chiplet… · p. 3
  • Each compute chiplet has accelerator-like buffers, PE array, NoC, control, and postprocessing units; layer workloads are tiled by dataflow and can use chiplet-specific compute or memory configurations.Different dataflows and configurations let chiplets better match layers with different compute intensity and architectural preferences.Source: A. Heterogeneous Multi-Chiplet… · p. 3

B. Hardware Design Space

Beacon searches architecture-level designs for heterogeneous multi-chiplet accelerators by varying chiplet count, each chiplet’s configuration, and shared system parameters. The search proceeds from chiplet count and array shape to per-chiplet choices and system-level settings, covering homogeneous through multidimensionally heterogeneous designs.

B. Hardware Design Space

  • The design space varies chiplet count, each chiplet’s configuration, and system parameters shared across the accelerator.The search selects chiplet count and derives the corresponding two-dimensional array shape before configuring chiplets and system parameters.Source: B. Hardware Design Space · pp. 3–4
  • Each chiplet independently selects compute capacity, on-chip buffer capacity, and dataflow architecture.Compute capacity sets compute-unit count and PE or MAC organization; buffer capacity sets on-chip memory resources; dataflow determines data reuse and execution pattern.Source: B. Hardware Design Space · p. 4
  • Shared system parameters include hardware resources such as NoP and DRAM bandwidth, plus mapping choices such as tensor parallelism and micro-batch size.Mapping parameters are included because they closely interact with hardware configuration and substantially affect execution efficiency.Source: B. Hardware Design Space · p. 4
  • The search sequence can represent configurations ranging from homogeneous designs to multidimensionally heterogeneous multi-chiplet accelerators.It selects chiplet count and derives array shape, independently configures each chiplet, then chooses system-level hardware and mapping parameters.Source: B. Hardware Design Space · p. 4

IV. METHODOLOGY · A. Overview

Beacon specializes agentic search for HW-DSE as a closed loop of decision making, memory, and action/observation. Its framework combines hierarchical bottleneck analysis with evaluation feedback and cross-iteration experience reuse.

IV. METHODOLOGY

  • Beacon organizes HW-DSE as a closed loop in which decisions use evaluation feedback, memory reuses trial-and-error experience, and actions turn decisions into evaluable configurations.The evaluator environment returns feedback to guide subsequent search decisions.Source: §IV · p. 4
  • Beacon decomposes decisions into where to optimize, why it is bottlenecked, and how to modify hardware, making decisions traceable to evidence while reducing context pressure and premature changes.The hierarchy follows hardware architects’ progression from end-to-end metrics and timelines to fine-grained compute, memory, and communication behavior, then parameter updates.Source: §IV · p. 4

A. Overview

  • The framework takes a model workload, hardware search space, and search configuration, and outputs an optimized configuration plus a trace of each search iteration.Inputs include workload structure and inference scenario, legal architecture parameters, initial configuration, objective, evaluation budget, and output settings; the trace records metrics, bottleneck states, cases, proposals, tool calls, and modifications.Source: A. Overview · p. 4
  • The Agentic Search Engine uses Model-Level, Layer-Level, and Solution Agents for global bottleneck discovery, layer-level diagnosis, and hardware candidate generation, respectively.These agents form one of the framework’s four main components.Source: A. Overview · p. 4
  • Within each online iteration, the Evaluator Adapter produces metrics and detail reports, and agents use the Analysis Toolbox to progress from global discovery to layer-level diagnosis and candidate generation.The proposed solution is materialized as a legal hardware candidate and evaluated in the next iteration.Source: A. Overview · p. 5
  • Across iterations, the Layer-Level Agent’s BSD queries RAG Memory for historical cases, and evaluated transitions are written back to accumulate successful and unsuccessful experience.Retrieved cases are provided to the Solution Agent for hardware candidate generation.Source: A. Overview · p. 5

B. Agentic Search Engine · 1) Prompt and ReAct Design: · 2) Model-Level Agent:

Beacon gives each agent a role-specific prompt and uses a shared ReAct workflow to gather evidence before producing structured output. Its Model-Level Agent selects layers for deeper diagnosis by combining global execution views, without proposing hardware changes.

1) Prompt and ReAct Design:

  • Each agent uses a role-specific prompt and output schema within a shared two-stage workflow: ReAct evidence collection followed by structured synthesis.During evidence collection, the agent alternates between LLM decisions and tool calls; synthesis produces output from the collected observations.Source: 1) Prompt and ReAct Design · p. 5
  • The system prompt defines the agent’s role and evidence sources, specifies a shared ReAct protocol, and serializes Analysis Toolbox tools with their descriptions and argument schemas.The protocol requires strict JSON at each step and a choice between one tool call and termination; placing tool descriptions in the system prompt reduces sensitivity to later context compression.Source: 1) Prompt and ReAct Design · p. 5
  • Each ReAct step requests one tool observation or ends tool use; once evidence is sufficient or the step budget is reached, the system requests and validates a final agent-specific JSON response.Report parsing, hardware normalization, and state storage are handled by deterministic components, while the LLM selects and interprets evidence and generates decisions.Source: 1) Prompt and ReAct Design · pp. 5–6

2) Model-Level Agent:

  • The Model-Level Agent compares current, previous, and best configurations using search-context tools, then selects an analysis base because the latest configuration may be less informative.After a regressive search move, the best configuration can offer a better basis for the next decision.Source: 2) Model-Level Agent · p. 6
  • From the selected base, the Model-Level Agent uses global analysis and layer-ranking tools to examine latency, energy, monetary cost, objective changes, dependencies, utilization, group contribution, and layer ranks.This stage identifies layers or operator regions for further diagnosis; it does not propose hardware changes.Source: 2) Model-Level Agent · p. 6
  • The Model-Level Agent combines global evidence to select candidates for detailed diagnosis, since high latency alone does not establish that a layer is a good search target.A high-latency layer may not lie on the critical path or contribute much to energy or monetary cost.Source: 2) Model-Level Agent · p. 6

3) Layer-Level Agent: · 4) Solution Agent: · C. Analysis Toolbox

The Layer-Level Agent turns detailed reports into a bottleneck state that guides the Solution Agent’s hardware choices. The Analysis Toolbox structures evidence access from search-state selection through diagnosis to legal hardware changes.

3) Layer-Level Agent:

  • The Layer-Level Agent diagnoses candidate layers across compute, memory, and communication, examining array pressure, MAC utilization, compute density, buffer use, DRAM access, and NoP overhead.It receives candidate layers, selected-base detail reports, hardware context, search state, and toolbox interface; it can use hardware tools when placement or heterogeneous layout affects communication.Source: 3) Layer-Level Agent · p. 6
  • The Bottleneck State Description (BSD) compresses detailed reports into a stable diagnostic state and serves as the RAG Memory retrieval key for bottleneck similarity.Historical cases are retrieved by bottleneck similarity rather than hardware JSON similarity alone.Source: 3) Layer-Level Agent · p. 6

4) Solution Agent:

  • The Solution Agent combines the BSD, retrieved historical cases, current hardware configuration, search state, and hardware analysis tools to produce the next solution.Source: 4) Solution Agent · p. 6
  • The Solution Agent maps diagnosed bottlenecks to candidate parameters, checks similar historical cases, then uses hardware tools to select a legal candidate.Examples include chiplet size for compute or buffer pressure, DRAM bandwidth or micro-batch size for memory bottlenecks, and NoP bandwidth, chiplet layout, or tensor parallelism for communication bottlenecks.Source: 4) Solution Agent · p. 6
  • The final hardware candidate is normalized and validated through hardware tools and the Evaluator Adapter, rather than taken directly from freeform LLM output.Source: 4) Solution Agent · p. 6

C. Analysis Toolbox

  • The Analysis Toolbox structures compact, traceable tools across search-state comparison, global metrics, layer diagnosis, and hardware modification, keeping agents from parsing raw reports.Its design separates global metrics from detailed layer evidence and ties outputs to evaluation files or generated artifacts.Source: C. Analysis Toolbox · p. 6
  • The toolbox’s four groups select the analysis target, provide whole-model evidence, explain layer bottlenecks, and summarize hardware state or edit legal candidates.Together, the evidence chain moves from coarse observations to concrete hardware actions, with each tool call recorded in the agent trace.Source: C. Analysis Toolbox · pp. 6–7

D. RAG Memory · E. Evaluator Adapter

Beacon’s RAG Memory reuses historical search cases to inform Solution Agent candidate generation, while the Evaluator Adapter standardizes evaluator inputs, outputs, and candidate preparation.

D. RAG Memory

  • RAG Memory injects similar historical search cases into the Solution Agent context, which generates candidates using the current BSD and hardware search space.Retrieved cases inform generation; the Solution Agent remains responsible for producing the final hardware candidate.Source: D. RAG Memory · p. 7
  • After each evaluation, RAG Memory stores a key-value case containing the current BSD, before-and-after configurations, proposed action, and measured metric feedback.The key supports similarity retrieval, while the value stores the full search feedback.Source: D. RAG Memory · p. 7
  • Dual bottleneck and hardware keys match cases by root-cause and hardware-context similarity; cosine retrieval returns the top-k cases, including successful and unsuccessful experience.The bottleneck channel captures semantic bottleneck similarity, while the hardware channel limits transfer from cases with substantially different configurations.Source: D. RAG Memory · p. 7

E. Evaluator Adapter

  • The Evaluator Adapter isolates evaluator-specific formats, scripts, and reports behind a unified abstraction, allowing connection of evaluators with compatible workload, candidate, metric, and report interfaces.The Agentic Search Engine and Analysis Toolbox operate on this unified abstraction rather than evaluator-specific organizations.Source: E. Evaluator Adapter · p. 7
  • For each workload and hardware candidate, the adapter returns unified latency, energy, monetary-cost, and objective metrics plus fine-grained reports used by the Analysis Toolbox.Reports preserve execution intervals, compute/NoP/DRAM time, energy and hardware cost breakdowns, and the evaluated hardware snapshot; the toolbox derives timelines and workload and hardware summaries.Source: E. Evaluator Adapter · p. 7
  • Before evaluation, the adapter materializes the Solution Agent’s high-level modification intent into a legal candidate, fills derived fields, and checks for duplicate configurations.Candidate legality and evaluation-file organization are handled deterministically, while the LLM proposes the search direction.Source: E. Evaluator Adapter · p. 7

V. EXPERIMENTAL EVALUATION … 4) Baseline Setup:

The evaluation compares Beacon against search baselines across three model and accelerator scales, using a simulator-based DSE setup and a shared hardware-editing action space. The setup defines a soft compute-budget penalty and specifies workloads, hardware, and baseline policies.

1) DSE Setup:

  • The evaluation covers GPT3-7B, GPT3-13B, and LLaMA3-70B at 64, 512, and 2048 TOPS, respectively, spanning edge-side through high-end cloud inference scales.GPT3 uses the conventional Transformer architecture; LLaMA3 includes pre-layer normalization and grouped-query attention.Source: 1) DSE Setup · p. 8
  • The objective minimizes latency, energy, and monetary cost while applying a symmetric multiplicative soft penalty for compute capability above or below target.The penalty avoids requiring an exact compute-budget match, reducing wasted search steps and allowing design flexibility.Source: 1) DSE Setup · p. 8

2) Evaluation Framework:

  • A modified GEMINI simulator supports LLM inference, while Compass’s genetic algorithm searches mappings with population size 120 and 100 generations.The simulator is a representative multi-chiplet accelerator DSE framework; the genetic algorithm settings match the original setting.Source: 2) Evaluation Framework · p. 8
  • Inference workloads use GovReport sequence lengths and chunked prefill with chunk size 2048 and batch size 128, comprising one prefill and 127 decode requests.These request-organization settings follow the default vLLM setting.Source: 2) Evaluation Framework · p. 8

3) Hardware Parameters:

  • DSE-irrelevant hardware settings fix TSMC 12 nm, an organic substrate, 1 GHz clock, GRS mesh NoP with XY routing, and four DRAM chips.Two DRAM chips are placed on each side of the chiplet array.Source: 3) Hardware Parameters · p. 8

4) Baseline Setup:

  • Random search, BO, and RL serve as baselines because offline methods require substantial training data unsuitable for the evaluation-budget-limited setting.The baselines share an action space comparable to Beacon toolbox editing, with all design points reachable, while retaining their own selection policies.Source: 4) Baseline Setup · pp. 8–9
  • The shared action space edits chiplet count, per-chiplet MAC count, GLB size or dataflow, and system-level NoP, DRAM bandwidth, tensor parallelism, or micro-batch size.Each edit moves to an adjacent legal value, except chiplet dataflow changes, which select a type.Source: 4) Baseline Setup · p. 9
  • BO uses a Gaussian-process surrogate and expected improvement, whereas RL sequentially selects legal edits using structured hardware and performance states.BO combines an RBF kernel with Compass’s topology-aware layout similarity for matching chiplet-array shapes; RL follows Chiplet-Gym and ConfuciuX, with an actor-critic policy-gradient model updated online using a clipped PPO objective.Source: 4) Baseline Setup · p. 9

5) Experimental Environment and Hyperparameters:

Beacon’s experiments use a specified LLM and retrieval setup, and compare methods from the same starting hardware over 100 search rounds under a limited evaluation budget.

Experimental Environment and Hyperparameters

  • Beacon uses DeepSeek-V4-Pro as its LLM backend, selected for its open weights, strong capability, and relatively low API cost.The paper describes these properties as making the model suitable for practical use.Source: 5) Experimental Environment and… · p. 9
  • RAG Memory uses BGE-small-en-v1.5 embeddings and retrieves the top-5 historical cases as a tradeoff between search effectiveness and context length.The retrieval setting was determined through preliminary experiments.Source: 5) Experimental Environment and… · p. 9
  • All methods start from the same hardware and run 100 search rounds on a server with two Hygon 7285 processors and four A100 GPUs.The simulator and baselines mainly use CPUs; GPUs mainly support embedding inference and RL model updating.Source: 5) Experimental Environment and… · p. 9

B. Comparison with Baselines · C. Result Analysis

Beacon outperforms the traditional baselines across three compute scales, while its selected heterogeneous designs balance latency, energy, monetary cost, and workload-specific dataflow needs.

B. Comparison with Baselines

  • The baseline gap is largest at 64TOPS and 512TOPS; at 2048TOPS, Beacon still has the best objective but a smaller relative advantage.At 64TOPS, random search, BO, and RL objectives are 10.51×, 4.79×, and 5.31× Beacon’s; at 512TOPS, they are 6.40×, 4.35×, and 15.34×. At 2048TOPS, more chiplets reduce latency but raise monetary cost, so Beacon balances latency, energy, and MC.Source: B. Comparison with Baselines · p. 9
  • As compute scale grows, dominant overhead shifts from DRAM and I/O toward inter-chiplet communication, chiplets, and packaging.At 64TOPS, DRAM is 75.5% of energy and chiplet/I/O are 42.1%/41.9% of MC; at 512TOPS, DRAM energy is 63.3% and NoP energy 21.3%, while chiplet/package costs are 60.4%/24.9% of MC. At 2048TOPS, DRAM/NoP energy shares are 44.5%/38.4%, package cost is 38.4%, and I/O cost is 5.9%.Source: B. Comparison with Baselines · pp. 9–10
  • Beacon reduces the objective by 79.1%, 77.0%, and 25.1% versus the strongest traditional baseline on the 64TOPS, 512TOPS, and 2048TOPS tasks, respectively.Fig. 5 compares normalized latency, energy, monetary cost (MC), and the composite objective; lower values indicate better designs.Source: B. Comparison with Baselines · p. 9, Figure 5 · p. 10

C. Result Analysis

  • Beacon does not scale resources linearly, instead trading off chiplet count, dataflow, bandwidth, and execution parameters according to workload behavior and cost feedback.Table VI summarizes the best hardware configurations found by Beacon.Source: C. Result Analysis · p. 10, Table VI · p. 10
  • At 64TOPS, Beacon selects two heterogeneous chiplets and micro-batch size 64, keeping MC to $15.55.The design pairs one WS and one OS chiplet; adding chiplets or bandwidth at this small compute target raises MC quickly for limited latency benefit, so it keeps DRAM/NoP bandwidth cost low.Source: C. Result Analysis · p. 10
  • At 512TOPS, Beacon uses eight chiplets—seven WS and one OS—to retain weight reuse for GEMMs while improving attention-related computation.The design does not maximize OS chiplets: too many can reduce dominant-GEMM weight reuse and introduce load imbalance, so heterogeneity follows bottleneck layers and cost feedback.Source: C. Result Analysis · p. 10
  • The 512TOPS schedule pipelines stages across chiplets and micro-batches, enabling weight reuse and assigning the long MB0 MHA AV stage to OS chiplet C2.With tensor-parallel degree 8, QKV Gen, Out Proj, and FFN partition evenly across chiplets; MB0 is chunked prefill and has longer stage fragments than decode micro-batches. Its AV matrix shapes (2048×2048 attention scores and 2048×128 value matrix) suit OS dataflow.Source: Figure 6 · p. 10, C. Result Analysis · pp. 10–11
  • At 2048TOPS, Beacon uses 32 chiplets—25 WS and 7 OS—to reduce latency, while MC reaches $593.69 and makes chiplet and package costs major constraints.The high-throughput design must balance parallelism, communication cost, and dataflow heterogeneity rather than simply increase parallelism.Source: C. Result Analysis · p. 10

D. Design Process Analysis · E. LLM Backend Selection

Beacon’s search trajectories show task-scale-specific hardware adjustments guided by bottleneck diagnosis. On the 512TOPS task, backend choice changes search quality and resource use: Gemini finds the lowest objective, while DeepSeek has the lowest API cost.

D. Design Process Analysis

  • Beacon usually reduces the objective rapidly in the first dozen iterations, then refines cost and heterogeneous layout, while random search lacks direction and BO and RL face diagnosis or learning limits.BO fits trends from scalar objectives but cannot directly distinguish bandwidth cost, dataflow mismatch, or unsuitable micro-batch settings; RL struggles to learn a stable policy within the limited budget.Source: D. Design Process Analysis · p. 11
  • At 64TOPS, Beacon restores two chiplets after a single-chiplet trial worsens latency, then reduces latency and energy by increasing micro batch size and lowering TP from 16 to 8.It subsequently reduces NoP and DRAM bandwidth to lower MC, then reintroduces one OS chiplet for attention without increasing cost.Source: D. Design Process Analysis · p. 11
  • At 512TOPS, Beacon shifts from eight WS chiplets to a heterogeneous layout, ultimately retaining one OS chiplet for attention AV while most chiplets serve QKV and FFN as WS.Increasing micro batch size and reducing tensor parallelism aligns shards with chiplet count; diagnosis identifies the FFN2 critical tail and MHA QKT/AV utilization imbalance. Reducing DRAM and NoP bandwidth trades a small latency increase for significant MC reduction, while too many OS chiplets cause load imbalance.Source: D. Design Process Analysis · p. 11
  • At 2048TOPS, increasing micro batch size from 1 to 16 reduces latency from 7.3×10^7 to 2.2×10^7 and the objective by 66.5% relative to the initial design.Larger batches are not always effective, and attempts to increase parallelism, bandwidth, buffers, or chiplet count do not become new best designs; iteration 56 makes a small improvement by adding local OS dataflow and a larger buffer at a key position.Source: D. Design Process Analysis · p. 11

E. LLM Backend Selection

  • Gemini achieves the lowest objective at 0.90× DeepSeek’s; GPT is 0.03% higher, while Claude’s final objective is 1.28× DeepSeek’s.Gemini selects four chiplets, reducing MC from $48.71 to $30.26 while latency rises from 2.54×10^7 to 3.77×10^7. Claude’s six-WS/two-OS design has lower latency but MC rises to $70.97; GPT and DeepSeek converge to low bandwidth, TP=8, and MB=32.Source: E. LLM Backend Selection · pp. 11–12, Figure 8 · p. 12
  • Gemini has the shortest total API time at 4.09 hours, versus 7.97 hours for Claude, 11.56 for DeepSeek, and 11.45 for GPT; API time does not directly track search quality.The backends have similar per-agent call distributions, with the Model-Level Agent usually accounting for the most calls.Source: E. LLM Backend Selection · p. 12
  • Gemini uses 23.4M total tokens, below DeepSeek’s 35.9M, GPT’s 36.8M, and Claude’s 37.3M; input tokens dominate, and the Solution Agent has higher average input tokens because it reads retrieved RAG cases.DeepSeek tends to produce longer responses, especially during the Solution Agent stage, while Gemini’s outputs are more concise.Source: E. LLM Backend Selection · p. 12
  • DeepSeek has the lowest estimated API cost at $4.94, versus Gemini at $48.98, GPT at $149.03, and Claude at $224.47; it is the cost-efficient choice, while Gemini offers better search quality and shorter wall-clock time.Costs are estimated from measured token usage and providers’ official API prices; both backends are practical under different cost and runtime priorities.Source: E. LLM Backend Selection · p. 12

F. Ablation Study … VI. CONCLUSION

Beacon’s ablations show that hierarchical agents, RAG Memory, and ReAct improve search quality, while overhead analysis quantifies the runtime and API costs. The conclusion presents Beacon as a report-driven framework for making effective hardware decisions under limited evaluation budgets.

F. Ablation Study

  • Ablations use the 512TOPS task to assess hierarchical agents, RAG Memory, and ReAct.The paper identifies Table VII as reporting the ablation results.Source: F. Ablation Study · p. 12

1) Single Agent vs. Multi-Agent:

  • The single-agent variant’s objective is 1.499× Beacon’s, the largest degradation among the ablations, showing the value of hierarchical agents for validating parameter interactions and handling global and layer-level bottlenecks.Its similar hardware choice raises DRAM bandwidth to 32 and reduces one WS chiplet’s GLB capacity to 4096, increasing latency, energy, and MC.Source: 1) Single Agent vs. Multi-Agent · p. 12

2) Effect of RAG Memory:

  • Without RAG Memory, the objective is 1.301× Beacon’s, as higher NoP and related resource costs offset latency and energy gains.The variant uses eight WS chiplets and keeps NoP bandwidth and micro-batch size at 64; MC rises to 71.06.Source: 2) Effect of RAG Memory · p. 12

3) Effect of the ReAct Paradigm:

  • With all tool calls chosen upfront, the best design’s objective is 1.159× Beacon’s, demonstrating that ReAct’s result-driven decisions improve search effectiveness.This variant uses eight chiplets, TP8, and micro-batch size 32, but switches to four WS and four OS chiplets and DRAM bandwidth 32; energy and MC reach 2.72 × 1011 and 55.12.Source: 3) Effect of the ReAct Paradigm · p. 12

G. Overhead Analysis

  • The evaluator accounts for 74.5% of total runtime, while the Solution Agent has the highest total agent overhead and an average call time of 27.7 seconds.The Solution Agent’s calls integrate bottleneck summaries, historical cases, the hardware search space, and candidate actions; 100 RAG Memory writes and retrievals take approximately 16.4 seconds total.Source: Table VIII · p. 13, G. Overhead Analysis · p. 13
  • The 512TOPS run makes 2270 LLM API calls using about 35.9M tokens and has an estimated API cost of about $4.94.Prompt tokens total 33.0M, output tokens 2.9M, and 27.7M prompt-cache-hit tokens correspond to an 84.0% cache hit rate.Source: G. Overhead Analysis · p. 13
  • Beacon trades LLM analysis overhead for higher-quality evaluation decisions, with decision overhead on the same order as simulator evaluation time and better designs found within tens to hundreds of evaluations.Source: G. Overhead Analysis · p. 13

VI. CONCLUSION

  • Beacon converts fine-grained evaluation reports into optimization guidance using hierarchical agents, an analysis toolbox, and RAG Memory to localize bottlenecks, diagnose causes, and propose hardware changes.Experiments on large-model inference workloads reduce the joint latency, energy, and monetary-cost objective versus random search, Bayesian optimization, and reinforcement learning, while discovering heterogeneous dataflow designs.Source: §VI · p. 13

Related papers

Loading 2608.30932v1…