PaperlayerView exact revision on arXiv

Source-linked AI summary

Divide and conquer: Scalable performance and energy in MCM GPUs

Mario Ibáñez Bolado, Borja Pérez Pavón, Jose Luis Bosque Orero, Julio Ramón Beivide

arXiv:2610.03061v1cs.AR

TL;DR

MCM GPU disaggregation's performance and energy trade-offs remain underexplored, limiting understanding of scaling across compute, chiplet count, and network choices. The paper evaluates scale-up, scale-out, and hybrid designs and their network demands. A 16-chiplet Torus with 256 SMs delivers 2.40× the performance of a state-of-the-art MCM architecture with the same compute capability while reducing energy consumption by 4.45×.

  • Problem

    How compute, chiplet count, and interconnect choices affect MCM GPU performance scalability and energy consumption remains insufficiently understood.

    Source: §I · p. 1

  • Method

    The paper evaluates scale-up, scale-out, and hybrid MCM GPU configurations and analyzes how network scalability affects performance, energy, and efficiency.

    Source: §I · p. 2, §VIII · p. 11

  • Results

    A 16-chiplet Torus with 256 SMs delivers 2.40× the performance of a state-of-the-art MCM architecture with the same compute capability while reducing energy consumption by 4.45×.

    Source: §VIII · p. 11

  • Takeaways & Limitations

    Disaggregation and interconnect scalability are first-order considerations in MCM GPU design, and compute, memory, and interconnect resources must be co-designed and scaled jointly.

    Source: §VIII · p. 11

    The benefits of disaggregation depend on a network capable of sustaining the resulting communication demands.

    Source: §VIII · p. 11

Abstract

from arXiv · show

Multi-chip-module (MCM) GPUs offer a promising path to scale compute capability beyond monolithic designs by integrating multiple chiplets on a common package. However, the impact of disaggregation on performance scalability and energy consumption remains underexplored. The design space grows rapidly across dimensions such as SMs per chiplet, chiplet count, and interconnection network. The inter-chiplet network is particularly critical, as it determines whether additional compute resources translate into performance gains. This limited understanding leaves industry and research without clear guidance on the performance and energy trade-offs of MCM GPU scaling. In this work, we investigate whether distributing compute and memory capability across multiple chiplets offers a more scalable alternative to concentrating resources. We quantify their effects on performance, energy, and efficiency and examine how inter-chiplet topology influences scalability at different system sizes. Our results demonstrate that a 16 chiplet Torus configuration with 256 SMs delivers a remarkable $2.40\times$ performance improvement over a state-of-the-art MCM architecture with the same compute capability, while simultaneously reducing energy consumption by $4.45\times$. These substantial gains provide evidence that disaggregation is a first-order architectural factor and will be critical to unlocking the performance and energy-efficiency potential of next-generation GPUs.

I. INTRODUCTION

As monolithic GPUs approach physical and economic limits, MCM designs distribute compute and memory resources across chiplets but introduce performance and energy trade-offs. This work evaluates scaling strategies and reports substantial gains for a highly disaggregated design when communication facilities are adequate.

I. INTRODUCTION

  • MCM GPUs distribute compute, memory, and communication resources across chiplets to address the physical and economic limits of monolithic designs.The stated pressures include slowing transistor scaling and the area wall, which makes larger monolithic dies increasingly difficult and costly.Source: §I · p. 1
  • Evaluating MCM scalability requires measuring execution time and energy as the system grows, rather than judging designs by aggregate compute capability alone.The paper identifies performance and energy-efficiency trade-offs across scaling strategies as a central design challenge.Source: §I · p. 1
  • A 16-chiplet Torus with 256 SMs improves performance by up to 2.40× and reduces energy consumption by 4.45×, assuming adequate communication facilities.The paper also reports that highly disaggregated designs can match or outperform more compact configurations with 2× or 4× as many compute resources.Source: §I · p. 2
  • The paper analyzes scale-up, scale-out, and hybrid configurations, evaluates their performance and energy-efficiency implications, and develops the open-source MANGO simulator for MCM GPUs.MANGO is intended to enable evaluation of novel MCM designs and optimizations.Source: §I · p. 2

II. BACKGROUND AND MOTIVATION

MCM GPUs can ease monolithic-die constraints, but distributing compute and memory creates communication and scaling challenges. The section contrasts those benefits and challenges with scale-up and scale-out results that show limited gains or degradation under the evaluated configurations.

A. Multi-Chip-Module GPUs

  • MCM GPUs partition a GPU across chiplets in one package, easing monolithic scalability, manufacturing, and cost limits and enabling up to 2× higher compute density than comparable monolithic designs.The passage also notes that smaller dies generally achieve higher manufacturing yield and that chiplet reuse can support different product configurations.Source: A. Multi-Chip-Module GPUs · p. 2
  • Remote memory accesses traverse the inter-chiplet network and introduce NUMA effects, increasing latency and network traffic as compute and memory are distributed.The passage also calls this Non-Uniform Bandwidth Architecture (NUBA) in this context.Source: A. Multi-Chip-Module GPUs · p. 2

B. The Challenge of Chiplet-Based Scalability

  • Prior scalability analyses often use small or fixed-size systems, commonly four chiplets with a ring, and do not independently vary SM count, chiplet count, and topology.This makes it difficult to attribute observed trends to added compute, memory resources, or communication overhead.Source: B. The Challenge of Chiplet-Based… · p. 3
  • With four chiplets, scaling from 128 to 1024 SMs raises performance by only 1.23× on average despite 8× more computational resources; Kmeans reaches approximately 2.05×.The reported average scaling efficiency is approximately 15% relative to linear scaling; Kmeans' gain is attributed to better data-placement alignment and locality.Source: B. The Challenge of Chiplet-Based… · p. 3
  • With 512 SMs fixed, increasing ring-connected chiplets from 4 to 16 or 64 reduces average performance to approximately 0.76× or 0.59×, respectively.The passage attributes the degradation to increased communication distances, inter-chiplet traffic, and ring contention.Source: B. The Challenge of Chiplet-Based… · p. 3
  • Scale-up is limited and application-dependent, while scale-out can incur communication overheads that outweigh disaggregation benefits, so resource allocation and network design must be considered together.The passage frames both strategies as limitations of the typical ring-based architecture.Source: B. The Challenge of Chiplet-Based… · p. 3

III. ARCHITECTURAL SCALING

The section frames MCM GPU scaling as a joint resource and network design problem. Scale-out can reuse a fixed chiplet design, but growing communication distances and contention can constrain resource utilization and performance.

III. ARCHITECTURAL SCALING

  • Open questions include whether added compute yields proportional performance and energy-efficiency gains, whether the network becomes the dominant bottleneck, and how scaling strategies affect energy.The passage identifies actionable design guidance for future MCM GPU systems as a reason to resolve these questions.Source: §III · p. 3

A. Scale-out

  • Reusing a fixed chiplet design supports product modularity, lets its power, cost, and performance be characterized in advance, and can reduce design and verification effort.The same building block can support products with different performance, power, and cost targets.Source: A. Scale-out · p. 3
  • As chiplet count grows, more memory requests traverse the network and communication distance and link contention increase, potentially preventing additional compute resources from being effectively utilized.The inter-chiplet network is identified as scale-out's main limitation.Source: A. Scale-out · p. 3
  • In ring-based scale-out, average communication latency increases by 3.84× with 16 chiplets and 16.22× with 64 chiplets versus four chiplets, while performance degrades despite linear compute growth.The passage attributes this to longer communication distances and growing contention that prevent additional resources from being fully utilized.Source: A. Scale-out · p. 4
  • Workload distribution and data placement can concentrate remote requests on destinations and links, creating hotspots, queuing delays, and network saturation before compute resources are fully used.The passage cites Kmeans as an example where better alignment of data partitions and execution blocks improves locality and resource utilization.Source: A. Scale-out · p. 4

B. Scale-up

Scale-up concentrates additional compute, memory, and supporting resources within a fixed number of chiplets. It can preserve architectural balance, but growing chiplet size brings interconnection, power, complexity, and manufacturing costs.

B. Scale-up

  • Scale-up adds SMs, cache, memory banks, schedulers, and bandwidth within each chiplet while keeping chiplet count constant.It concentrates resources rather than distributing them across more chiplets.Source: B. Scale-up · p. 4
  • Crossbar area and wiring complexity can grow quadratically with port count, driving rapid power growth as chiplets scale up.The ring-based architecture shows a more moderate power increase by distributing communication across simpler interconnects.Source: B. Scale-up · p. 4
  • Larger chiplets occupy more wafer area and have a higher probability of containing at least one manufacturing defect.They may also require more engineering effort and more advanced packaging, power delivery, and cooling.Source: B. Scale-up · p. 4
  • Beyond a certain chiplet size, marginal performance gains may no longer offset rising implementation and economic penalties.These include internal interconnection overhead, power consumption, design complexity, and manufacturing cost.Source: B. Scale-up · p. 4

C. Hybrid scaling

Hybrid scaling combines more capable chiplets with deployment of multiple chiplets, balancing the costs of scale-up and scale-out. Its network effects depend on workload traffic and topology, while effective configurations require systematic evaluation of a complex design space.

C. Hybrid scaling

  • Hybrid scaling balances moderately more capable chiplets with multiple-chiplet deployment to limit both internal crossbar growth and inter-chiplet communication overhead.This balance mitigates scale-up area and energy costs and scale-out bandwidth, latency, and topology requirements.Source: C. Hybrid scaling · p. 4
  • Hybrid design choices require systematic evaluation because scale-up and scale-out interactions can be nonlinear and workload-specific.The design space grows rapidly, and gains in one dimension may introduce bottlenecks or inefficiencies in another.Source: C. Hybrid scaling · p. 4

D. Inter-chiplet Network

  • For most workloads, four-chiplet ring peak bandwidth stays near 2048 GB/s as SM count rises from 128 to 1024; GEMM reaches about 0.75× and 0.86× at 128 and 256 SMs.Kmeans bandwidth falls substantially at 1024 SMs because memory-access and page-placement alignment reduces remote traffic.Source: D. Inter-chiplet Network · p. 5, Figure 4 · p. 5
  • With ring-based scale-out, CUTCP normalized performance falls to approximately 0.45 at 64 chiplets, while all evaluated benchmarks degrade as disaggregation increases.Scale-out also increases communication flows and routing, and can add network hops, latency, and energy; topology and traffic locality therefore matter.Source: D. Inter-chiplet Network · p. 5
  • Hybrid network effects lie between scale-up's traffic concentration and scale-out's communication overhead, but the best design depends on workload mapping and network capacity.Traffic amount and locality also shape the optimal configuration.Source: D. Inter-chiplet Network · p. 5

IV. METHODOLOGY

The evaluation compares MCM GPU configurations across system, energy, and network metrics. It varies architecture and resource distribution under workloads designed for comparable execution and sustained GPU occupancy.

IV. METHODOLOGY

  • The evaluation collects system-, energy-, and network-level metrics across MCM GPU configurations to identify scalability constraints.Configuration parameters vary independently or jointly depending on the experiment.Source: §IV · p. 5, A. Architectural Configurations · p. 5

A. Architectural Configurations

  • Scale-out evaluates 4, 16, 36, and 64 chiplets with 16 SMs each and memory controllers and L2 banks increasing linearly with chiplet count.The 36-chiplet design preserves a regular square layout and edge-symmetric network; these configurations maintain a one-to-one ratio of compute chiplets to memory dies.Source: A. Architectural Configurations · p. 5
  • Scale-up and hybrid studies compare 128, 256, 512, and 1024 SMs partitioned across 4, 16, or 64 chiplets.This permits comparison at equal aggregate compute capability and analysis of both total SM growth and distribution across chiplets.Source: A. Architectural Configurations · p. 5

B. Workloads and metrics

  • Workloads from Rodinia, Parboil, and FNNDSC cover varied compute and communication patterns, with a fixed instruction count across scenarios.Inputs and launch parameters remain unchanged under strong scaling, and datasets achieve average GPU occupancy above 95%; collected metrics include IPC, energy, energy efficiency, and peak bandwidth.Source: B. Workloads and metrics · pp. 5–6, Table II · p. 6

C. Inter-chiplet Network Configurations

The study evaluates Ring, 2D-Torus, Mesh, and Flattened Butterfly networks, whose port requirements, distance, and routing properties shape cost and performance. It models finite buffering, flow control, backpressure, and contention in the inter-chiplet interface.

C. Inter-chiplet Network Configurations

  • The evaluated networks are Ring, 2D-Torus, Mesh, and Flattened Butterfly; topology cost depends on router ports, while throughput under random symmetric traffic depends on total ports and average router distance.Average distance also dictates standard packet latency, while network diameter bounds its maximum value.Source: C. Inter-chiplet Network Configurations · p. 6
  • A bidirectional ring uses two ports per router but its average distance and diameter grow linearly with network size, limiting its size under heavy traffic.A 2D torus uses four ports and reduces diameter and average distance quadratically relative to a ring, but both topologies require deadlock-avoidance mechanisms.Source: C. Inter-chiplet Network Configurations · p. 6
  • A 2D mesh removes torus wraparound links, while a square Flattened Butterfly uses complete row and column graphs and has diameter two.The mesh is deadlock-free under DOR without extra resources; the Flattened Butterfly is also deadlock-free under DOR but requires many links of varying lengths. Fig. 5 marks horizontal links in red and vertical links in yellow.Source: C. Inter-chiplet Network Configurations · p. 6, Figure 5 · p. 7
  • The network model represents each unidirectional channel at 2 GHz with 256-B flits and ideal raw bandwidth of 512 GB/s.It captures finite buffering, credit-based flow control, backpressure, and contention.Source: C. Inter-chiplet Network Configurations · p. 7

V. SIMULATION ENVIRONMENT

The paper introduces MANGO, integrating GPU execution and configurable inter-chiplet network simulation to model MCM GPUs while retaining unmodified CUDA workloads. Its framework also supports evaluating performance and energy in the same configuration.

V. SIMULATION ENVIRONMENT

  • Existing frameworks have constraints for topology exploration or MCM GPU comparison: MGPUSim abstracts router details, gem5 uses a crossbar, and LEGOSim requires explicit message passing.The cited frameworks also differ in execution models, communication granularity, and supported chiplet counts.Source: §V · p. 7
  • MANGO combines GPGPU-Sim’s NVIDIA-like execution model with BookSim’s configurable network models to simulate communication within and across chiplets without workload modifications.GPGPU-Sim supports unmodified CUDA applications; BookSim models routers, routing algorithms, buffers, and topologies.Source: §V · p. 7
  • The integrated framework evaluates GPU and network energy alongside performance, using AccelWattch and BookSim power models.The network model accounts for dynamic, clock, and leakage power in channels, buffers, crossbars, and output circuitry.Source: §V · p. 7

VI. EVALUATION

The scale-out evaluation finds that network topology determines whether adding chiplets sustains performance. Flat-B scales most consistently at high chiplet counts, while workload communication patterns also shape the gains.

A. Scale-out

  • At 36 chiplets, Flat-B delivers approximately 1.32× and 1.77× the performance of Torus and Mesh, respectively, and at high counts reaches tenfold improvement over Ring.At 64 chiplets, Flat-B achieves performance comparable to an equivalent monolithic design; its gains are supported by the strongest GEMM bandwidth scaling.Source: Figure 6 · p. 8, A. Scale-out · p. 8
  • Ring IPC plateaus or decreases as chiplet count grows, eliminating scale-out benefits beyond a small number of chiplets.The evaluation fixes each chiplet at 16 SMs and compares four topologies from 4 to 64 chiplets against monolithic and linear-scaling references.Source: A. Scale-out · p. 8
  • Topology differences grow with system size: Mesh, 2D Torus, and Flat-B perform comparably at 16 chiplets, while Flat-B sustains scaling where other topologies show diminishing returns.The findings identify interconnection-network scalability as the constraint on scale-out beyond the four-chiplet industry standard.Source: A. Scale-out · pp. 8–9
  • BFS gains modest IPC beyond 16 chiplets because irregular memory accesses generate traffic that limits the benefit of added compute and memory resources.The passage concludes that scale-out alone cannot overcome workload-dependent communication bottlenecks.Source: A. Scale-out · p. 8
  • At 64 chiplets, Flat-B delivers twice monolithic performance for FFT, while 2D-Torus and Mesh fall 46% and 64% below monolithic, respectively.For GEMM, Flat-B exceeds the linear reference by 66% at 36 chiplets and 50% at 64; its 64-chiplet peak bandwidth is about 13.5× the four-chiplet Ring baseline.Source: A. Scale-out · p. 8, Figure 7 · p. 9

B. Scale-up and Hybrid Scaling

The scale-up and hybrid-scaling evaluation shows that increasing compute density alone cannot sustain performance as systems grow. Greater disaggregation can outperform larger, less-disaggregated systems when paired with scalable interconnects.

B. Scale-up and Hybrid Scaling

  • At 16 chiplets, Torus and Flat-B match or exceed monolithic performance, while Ring falls below 50% of monolithic at 16 chiplets and below 10% at 64.At 64 chiplets, only Flat-B retains monolithic performance; Flat-4x4 remains scalable at lower router and link cost than 2D-Torus.Source: B. Scale-up and Hybrid Scaling · p. 9
  • In the 64-chiplet configuration, greater SM counts increase network traffic and congestion in Mesh and Torus; Flat-B’s greater bisection bandwidth and lower hop-count let it surpass monolithic performance.In the largest design, Torus reaches only about 50% of monolithic performance.Source: B. Scale-up and Hybrid Scaling · p. 9
  • Adding four memory controllers per chiplet to the 16-chiplet, 1024-SM design improves performance by only 11%, far below the corresponding 64-chiplet design.At 16 chiplets, performance remains largely unchanged as SM count rises, whereas 64-chiplet configurations improve with more SMs.Source: B. Scale-up and Hybrid Scaling · p. 9
  • 2.58×: the 16-chiplet, 256-SM design delivers this performance relative to a four-chiplet MCM architecture with equivalent compute capability.Across non-ring topologies, 64-chiplet configurations with 256 or 512 SMs also outperform the 16-chiplet, 1024-SM configuration by up to 1.39× and 2×, respectively.Source: B. Scale-up and Hybrid Scaling · p. 9, Figure 8 · p. 10
  • Network scalability, rather than compute density, drives MCM GPU performance; the paper concludes that compute, memory, and interconnect must be co-dimensioned.The finding states that additional SMs or memory controllers alone provide limited benefit.Source: B. Scale-up and Hybrid Scaling · p. 10

C. Disaggregation and energy

The energy evaluation finds that topology choice changes the balance between performance and efficiency as chiplet count and SM count grow. The related-work discussion positions resource disaggregation as complementary to prior component-level locality optimizations.

C. Disaggregation and energy

  • At 64 chiplets, Flat-B’s performance gains offset much of its higher network energy, whereas Flat-4×4 lowers network energy relative to Flat-B and performs comparably to 2D-Torus.At high disaggregation, the inter-chiplet network becomes the dominant energy source as each chiplet’s internal crossbar contribution falls.Source: C. Disaggregation and energy · p. 10, Figure 9 · p. 11
  • Torus is the most energy-efficient topology in most evaluated configurations, while Flat-4×4 leads for large SM counts; topology preference depends on whether performance or efficiency is prioritized.Monolithic efficiency values are reported as 4.46× to 246.87× lower than corresponding chiplet-based configurations.Source: C. Disaggregation and energy · p. 10, Figure 10 · p. 11
  • The 16-chiplet Torus is identified as the most balanced design point for next-generation GPUs with 256 SMs, while concentration can further improve efficiency at high disaggregation.The authors frame the balance in terms of cost, performance, and energy.Source: C. Disaggregation and energy · p. 10

VII. RELATED WORK

  • Earlier chiplet-network studies examined interposer topologies and scalability largely for CPUs and general-purpose systems, identifying communication challenges also relevant to MCM GPUs.Examples named include ClusCross, Kite, HexaMesh, and modular deadlock-free routing.Source: §VII · pp. 10–11
  • Prior work targets specific locality mechanisms across MCM GPUs, multi-GPU systems, and CPUs; this work complements those approaches by dividing compute and memory across chiplets.The cited examples include thread/data placement, remote-data caching, and cache or routing reconfiguration.Source: §VII · p. 11

VIII. CONCLUSIONS

The 16-chiplet Torus with 256 SMs is the most balanced evaluated design point, combining higher performance with lower energy than a state-of-the-art MCM architecture with the same compute capability. The findings identify network scalability and joint resource scaling as central considerations for MCM GPU design.

VIII. CONCLUSIONS

  • Disaggregation improves MCM GPU scalability, performance, and energy efficiency, but its benefits depend on the scaling strategy and a network that sustains communication demands.The paper concludes that compute, memory, and interconnect resources must be co-designed and scaled jointly.Source: §VIII · p. 11
  • As chiplet count increases, network scalability becomes the primary determinant of performance, while adding SMs or memory controllers in isolation provides limited benefits under bandwidth and contention constraints.Highly scalable topologies enable disaggregated configurations to surpass equivalent monolithic designs beyond the current four-chiplet industry standard.Source: §VIII · p. 11
  • 2.40× the performance and 4.45× lower energy than a state-of-the-art MCM architecture with the same compute capability are delivered by the 16-chiplet Torus with 256 SMs.The paper describes this as the most balanced evaluated design point.Source: §VIII · p. 11
  • The findings establish disaggregation and interconnect scalability as first-order design considerations and motivate future study of adaptive routing, heterogeneous interconnects, and data-placement algorithms.The proposed future work aims to improve locality and reduce inter-chiplet communication.Source: §VIII · p. 11
Loading 2610.03061v1…