PaperlayerView exact revision on arXiv

Source-linked AI summary

Beacon: LLM Multi-Agent Driven Hardware Design Space Exploration for Heterogeneous Multi-Chiplet Deep Learning Accelerators

Boyu Li, Zongwei Zhu, Qianyue Cao, Xi Li, Xuehai Zhou

arXiv:2608.30932v1cs.AR

TL;DR

Heterogeneous multi-chiplet HW-DSE is constrained by expensive simulator evaluations and data-driven methods that require many iterations to learn from limited feedback. Beacon addresses this gap with report-driven hierarchical LLM agents, an Analysis Toolbox, and RAG memory, reducing the composite objective by 25.1%–93.5% against several search baselines under the same budget.

  • Problem

    Expensive simulator evaluations limit HW-DSE iterations, while conventional methods mainly use final metrics and require many samples to learn parameter–objective relationships.

  • Method

    Beacon uses hierarchical agents for bottleneck localization, root-cause diagnosis, and hardware candidate generation, supported by an Analysis Toolbox and RAG memory.

  • Results

    25.1%–93.5%: Beacon reduces the composite latency-energy-monetary-cost objective versus random search, Bayesian optimization, and reinforcement learning under the same limited evaluation budget.

  • Takeaways & Limitations

    Beacon shows that fine-grained evaluation reports can support targeted heterogeneous dataflow and hardware search for large-model inference workloads.

Abstract

from arXiv · show

Heterogeneous multi-chiplet accelerators allow chiplets to be configured independently to better match different operator characteristics and improve inference efficiency. However, heterogeneity makes simulator evaluation expensive, limiting the number of iterations affordable for hardware design space exploration (HW-DSE). Mainstream data-driven methods rely mainly on final metrics and a few predefined states, and require many search iterations to implicitly learn the relationships between input parameters and optimization objectives, making them less effective in this setting. In practice, evaluators also generate detailed reports on execution timelines, resource utilization, memory accesses, and communication behavior. Large language models (LLMs) can combine domain knowledge with these reports to explicitly identify bottleneck locations, degradation causes, and parameter adjustment directions, thereby improving each design decision under limited iteration budgets. Based on this observation, we propose Beacon, a report-driven LLM multi-agent framework for heterogeneous multi-chiplet HW-DSE. Beacon employs hierarchical agents for bottleneck localization, root-cause diagnosis, and hardware candidate generation, together with an Analysis Toolbox and RAG memory for closed-loop search. Under the same limited iteration budget, Beacon reduces the composite latency-energy-monetary-cost objective by 25.1\%--93.5\% compared with random search, Bayesian optimization, and reinforcement learning.

I. INTRODUCTION

Heterogeneous multi-chiplet accelerators improve workload matching but make simulator evaluation expensive, limiting HW-DSE iterations. Beacon uses report-driven LLM agents to diagnose bottlenecks and guide targeted hardware changes, achieving lower composite cost under the same budget.

  • Motivation: Heterogeneous multi-chiplet designs improve adaptation to inference stages and operator characteristics, but their HW-DSE evaluations are expensive.Simulator runtime is dominated by workload mapping, chiplet-level simulation, and inter-chiplet communication modeling.
  • Motivation: Traditional data-driven methods often need thousands of iterations because they treat evaluations mainly as configuration, state, and final-metric samples.They must gradually learn parameter–objective relationships from subsequent samples rather than directly using diagnostic report information.
  • Motivation: LLMs combine evaluator reports with hardware knowledge to identify bottlenecks, diagnose degradation causes, and suggest parameter-adjustment directions.This can distinguish individual-parameter effects from parameter interactions and track bottleneck shifts after hardware modifications.
  • Beacon: Beacon organizes HW-DSE into bottleneck localization, root-cause diagnosis, and architecture modification using hierarchical agents, an Analysis Toolbox, and RAG memory.The framework transforms fine-grained evaluation evidence into targeted and traceable search actions.
  • Results: 25.1%–93.5%: Beacon reduces the composite latency-energy-monetary-cost objective versus random search, Bayesian optimization, and reinforcement learning under the same limited budget.The evaluation uses large-model inference workloads.

A. Hardware DSE for Accelerators

Prior accelerator DSE studies rely on sampled surrogates, predefined candidates, or relatively simple search methods, which are difficult to apply when evaluations are expensive. The paper positions Beacon as architecture-level combinatorial optimization for heterogeneous multi-chiplet accelerators.

  • Prior HW-DSE: Offline surrogate methods train predictors from collected design samples, while online methods evaluate candidates and update search policies iteratively.Examples include differentiable surrogates, generative models, variational autoencoders, MLPs, contrastive learning, reinforcement learning, and Bayesian optimization.
  • Prior HW-DSE: Large-scale data sampling makes existing DSE methods difficult to apply directly to heterogeneous multi-chiplet designs with expensive evaluations.This limitation motivates methods that can make more effective decisions with fewer evaluations.
  • Multi-chiplet DSE: Multi-chiplet accelerator studies support workload mapping and evaluation but commonly use manual configuration, genetic algorithms, grid search, or predefined candidates.These approaches provide foundations for multi-chiplet design exploration.
  • LLM agents: Existing LLM chip-design studies primarily automate EDA flows or circuit and layout operations rather than architecture-level hardware-parameter optimization.Beacon instead targets hierarchical agent collaboration for combinatorial HW-DSE using unstructured evaluation information.
  • Accelerator abstraction: The accelerator abstraction connects compute chiplets through a network on package and supports heterogeneous dataflows or micro-architectures.Each chiplet includes a global buffer, processing-element array, network on chip, control unit, and postprocessing unit.

B. Hardware Design Space

Beacon’s hardware design space combines chiplet count, independently chosen chiplet configurations, and shared system-level hardware and mapping parameters. The search proceeds hierarchically from chiplet count to per-chiplet and system-wide choices.

  • Design variables: The design variables include chiplet count, each chiplet’s configuration, and system parameters shared across the accelerator.This space can represent homogeneous through multidimensionally heterogeneous multi-chiplet designs.
  • Chiplet-count selection: The search first selects a legal chiplet count and derives the corresponding two-dimensional array shape.This establishes the hardware array before configuring individual chiplets.
  • Per-chiplet configuration: Each chiplet independently selects legal compute capacity, on-chip buffer capacity, and dataflow architecture.Compute capacity determines compute-unit and PE/MAC organization; buffer capacity determines on-chip memory resources; dataflow determines reuse and execution pattern.
  • System parameters: System-level parameters include globally shared hardware resources and mapping choices such as NoP bandwidth, DRAM bandwidth, tensor parallelism, and micro-batch size.These mapping parameters are included because they interact closely with hardware configuration and affect execution efficiency.

IV. METHODOLOGY

Beacon frames HW-DSE as a closed loop of decision making, memory, and action/observation, specialized for heterogeneous multi-chiplet accelerators. Its framework combines workload and search-space inputs with hierarchical agents, analysis tools, and evaluator feedback to produce hardware candidates within a budget.

  • Beacon organizes HW-DSE as a closed loop combining decision making, memory, and action/observation.
  • The framework accepts model workload, hardware search space, and search configuration as its inputs.
  • The Agentic Search Engine uses Model-Level, Layer-Level, and Solution Agents for bottleneck discovery, diagnosis, and candidate generation.
  • The Analysis Toolbox exposes search-context, global-analysis, layer-analysis, and hardware-analysis tools for structured evidence access.
  • Beacon evaluates the current hardware, guides hierarchical analysis toward a new legal candidate, and carries search experience across iterations.

B. Agentic Search Engine

Beacon’s agents use a constrained ReAct interaction to collect structured evidence before synthesizing role-specific decisions. Tool use, JSON schemas, validation, and deterministic report handling constrain the LLM’s interface with downstream search components.

  • Prompt and ReAct Design: Each agent invocation alternates between ReAct-style evidence collection and final synthesis into a structured output.
  • Prompt and ReAct Design: The system prompt combines an agent-specific role, a shared ReAct protocol, and serialized Analysis Toolbox interfaces.
  • Prompt and ReAct Design: At each ReAct step, the agent requests one tool observation or terminates tool use before receiving a schema-constrained final-output prompt.
  • Prompt and ReAct Design: Report parsing, hardware normalization, and state storage remain deterministic, while the LLM selects evidence, interprets it, and generates decisions.

2) Model-Level Agent:

Beacon decomposes diagnosis into global bottleneck selection, layer-level root-cause analysis, and hardware candidate generation. The Bottleneck State Description connects diagnostic evidence to retrieval and constrained solution construction.

  • Model-Level Agent: The Model-Level Agent compares search states and selects candidate bottleneck layers from whole-model evidence.
  • Model-Level Agent: Global analysis combines latency, energy, cost, timelines, dependencies, utilization, and layer contributions rather than relying on latency alone.
  • Layer-Level Agent: The Layer-Level Agent diagnoses candidate layers across compute, memory, and communication dimensions.
  • Layer-Level Agent: The resulting Bottleneck State Description compresses detail reports into a stable diagnostic state and retrieval key for historical cases.
  • Solution Agent: The Solution Agent maps diagnosed bottlenecks to hardware parameters, consults prior cases, and produces a candidate that is normalized and validated before evaluation.
  • Analysis Toolbox: The Analysis Toolbox separates search-state, global, layer, and hardware responsibilities to provide compact structured evidence throughout the search loop.

D. RAG Memory

RAG Memory reuses evaluated search experience by retrieving cases using both bottleneck similarity and hardware context. The Evaluator Adapter standardizes evaluator interaction and enforces legal candidate materialization before simulation.

  • RAG Memory: RAG Memory stores historical cases containing BSDs, hardware states, proposed actions, and measured feedback for later retrieval.
  • RAG Memory: Retrieval uses a joint representation with bottleneck and hardware channels, matching root causes while reducing transfer from dissimilar configurations.
  • RAG Memory: Cosine similarity returns the top-k historical cases to the Solution Agent, including both successful and unsuccessful search experience.
  • Evaluator Adapter: The Evaluator Adapter converts workload and hardware candidates into standardized metrics and fine-grained reports across evaluator implementations.
  • Evaluator Adapter: Before evaluation, the adapter materializes high-level solution intents into legal candidates, fills derived fields, and checks duplicates.

V. EXPERIMENTAL EVALUATION

The evaluation covers three model and hardware compute scales using a modified GEMINI simulator and a composite latency-energy-cost objective with a soft compute-capability penalty.

  • DSE Setup: Experiments match 64, 512, and 2048 TOPS accelerators to GPT3-7B, GPT3-13B, and LLaMA3-70B, respectively.The workloads include conventional Transformer and newer LLaMA3 features such as pre-layer normalization and grouped-query attention.
  • DSE Setup: The objective combines latency, energy, monetary cost, and a symmetric multiplicative penalty for over- or under-provisioned compute capability.The soft penalty replaces a hard exact-compute constraint and is intended to avoid wasted search steps.
  • Evaluation Framework: The simulator is a modified GEMINI framework extended for LLM inference, with mapping solutions searched using Compass’s genetic-algorithm settings.The mapping search uses a population size of 120 and 100 generations.
  • Workload Setup: The workload uses GovReport sequence-length distributions with chunked prefill, a chunk size of 2048, and a batch size of 128.Each batch contains one chunked prefill request and 127 decode requests.
  • Hardware Setup: DSE-irrelevant hardware settings include TSMC 12 nm, a 1 GHz clock, GRS mesh NoP routing, and four DRAM chips.Two DRAM chips are placed on each side of the chiplet array.

4) Baseline Setup:

The baseline study compares random search, Bayesian optimization, and reinforcement learning within a unified legal action space and a fixed 100-round budget, alongside Beacon’s selected LLM backend and memory configuration.

  • Baseline Setup: Random search, BO, and RL serve as baselines under a unified action space that makes all design points reachable.Each baseline uses its own policy to choose actions at each step.
  • Action Space: The action space modifies chiplet count, chiplet-level MACs, GLB size, dataflow, NoP, DRAM bandwidth, tensor parallelism, and micro-batch size.Each variable changes to an adjacent legal value.
  • Baseline Implementations: Random search selects one action uniformly from the available action space at each step.
  • Baseline Implementations: BO uses a Gaussian-process surrogate with expected improvement and topology-aware layout similarity for matching chiplet-array shapes.
  • Baseline Implementations: RL encodes hardware resources and performance feedback into a structured state and selects one legal modification per iteration.Its policy follows a sequential decision process based on the previously evaluated configuration.
  • Experimental Budget: All methods start from the same hardware and perform 100 search rounds, while Beacon uses DeepSeek-V4-Pro and retrieves five historical cases through RAG Memory.The backend is selected for capability and relatively low API cost; the retrieval count balances effectiveness and context length.

B. Comparison with Baselines

Beacon outperforms random search, BO, and RL across three compute scales, while its selected heterogeneous designs and resource tradeoffs vary substantially with workload scale.

  • Overall Comparison: 79.1%, 77.0%, and 25.1%: Beacon reduces the composite objective versus the strongest traditional baseline at 64, 512, and 2048 TOPS, respectively.The comparison uses normalized latency, energy, monetary cost, and composite objective values.
  • Overall Comparison: At 64 TOPS, random search, BO, and RL reach objectives 10.51×, 4.79×, and 5.31× Beacon’s objective; at 512 TOPS, they reach 6.40×, 4.35×, and 15.34×.
  • Component Analysis: As compute scale grows, dominant overhead shifts from DRAM and I/O toward inter-chiplet communication, chiplets, and packaging.DRAM energy falls from 75.5% at 64 TOPS to 44.5% at 2048 TOPS, while package cost reaches 38.4% at 2048 TOPS.
  • Design Analysis: Beacon’s best configurations trade off chiplet count, dataflow, bandwidth, and execution parameters rather than scaling every resource linearly.
  • Design Analysis: At 64 TOPS, Beacon selects two heterogeneous chiplets and achieves an MC of $15.55, avoiding costly extra chiplets or bandwidth with limited latency benefit.
  • Design Analysis: At 512 TOPS, Beacon uses seven WS and one OS chiplet, preserving weight reuse for QKV and FFN while assigning OS resources to attention-related computation.The design uses eight chiplets and tensor parallelism to partition stages across chiplets.
  • Design Analysis: At 2048 TOPS, Beacon uses 32 chiplets with 25 WS and 7 OS chiplets, accepting $593.69 MC as chiplet and package costs become major constraints.
  • Execution Analysis: The 512TOPS timeline shows pipeline-like micro-batch execution that enables resident-weight reuse and reduces repeated weight-loading overhead.The sole OS chiplet handles the long MHA AV computation in the first micro-batch.

D. Design Process Analysis

Beacon’s search process rapidly improves designs through diagnosis-driven adjustments, while backend choice changes search quality, runtime, token usage, and API cost.

  • Search Curves: Beacon usually reduces the objective rapidly within the first dozen iterations before refining cost and heterogeneous-layout choices.Random search lacks direction, BO cannot directly diagnose regression causes, and RL struggles to learn a stable policy within the budget.
  • 64TOPS Process: At 64 TOPS, Beacon restores two chiplets after testing one, then increases micro-batch size and reduces tensor parallelism from 16 to 8 to address granularity mismatch.Later it reduces NoP and DRAM bandwidth for monetary cost, then reintroduces one OS chiplet for attention behavior.
  • 512TOPS Process: At 512 TOPS, hierarchical diagnosis changes the layout to eight WS chiplets, aligns tensor-parallel shards with chiplet count, and assigns an OS chiplet to attention AV.
  • 2048TOPS Process: At 2048 TOPS, increasing micro-batch size from 1 to 16 reduces latency from 7.3×107 to 2.2×107 and lowers the objective by 66.5% relative to the initial design.Later diagnosis adds local OS dataflow and a larger buffer for attention energy and a memory-bound FFN2 tail.
  • LLM Backend Analysis: On 512 TOPS, Gemini reaches 0.90× DeepSeek’s objective and 4.09 hours of API time, while Claude reaches 1.28× DeepSeek’s objective and 7.97 hours.GPT is 0.03% higher than DeepSeek’s objective, and DeepSeek and GPT take 11.56 and 11.45 hours, respectively.
  • LLM Backend Analysis: DeepSeek costs $4.94 versus $48.98 for Gemini, $149.03 for GPT, and $224.47 for Claude, making backend selection a cost-quality-runtime tradeoff.Gemini uses 23.4M total tokens, while input tokens dominate usage and the Solution Agent has the highest average input-token count.

F. Ablation Study

The ablation study shows that Beacon’s hierarchical multi-agent design contributes most to search quality, while RAG Memory and ReAct also improve the objective. The 512-TOPS run further quantifies Beacon’s runtime and API-cost overhead.

  • Ablation Results: The single-agent variant reaches 1.499× Beacon’s objective, producing the largest degradation among the evaluated ablations.It struggles to validate interactions among parameter changes while handling global bottlenecks, layer behavior, and hardware decisions.
  • Ablation Results: Removing RAG Memory increases the objective to 1.301× Beacon’s value.Latency and energy improve to 2.38 × 10^7 and 2.50 × 10^11, respectively, but monetary cost rises to 71.06.
  • Ablation Results: The non-ReAct variant achieves 1.159× Beacon’s objective by selecting all tool calls from the initial context.Its altered design increases energy and monetary cost relative to Beacon.
  • Overhead Analysis: The evaluator accounts for 74.5% of total runtime in the 512-TOPS Beacon experiment.The Solution Agent has the highest average call time at 27.7 seconds, despite not making the most calls.
  • Overhead Analysis: The 512-TOPS run makes 2270 LLM API calls, uses about 35.9M tokens, and costs approximately $4.94.Prompt-cache hits total 27.7M tokens, corresponding to an 84.0% cache hit rate.
  • Overall Assessment: Beacon trades LLM analysis overhead for higher-quality evaluation decisions within tens to hundreds of design evaluations.The decision overhead remains on the same order as simulator evaluation time.
Loading 2608.30932v1…