PaperlayerView exact revision on arXiv

Source-linked AI summary

PANEM: A Heuristic Latency Model

Lucas Crowthers, Mahesh Madhav

arXiv:2609.29629v1cs.ARcs.PF

TL;DR

Pre-silicon memory modeling must balance predictive fidelity with the speed and configurability needed for large studies. PANEM converts bandwidth-latency data into a dynamic, contention-aware latency model, and the reported results show that fixed-latency assumptions can distort architectural conclusions and sizing decisions.

  • Problem

    Pre-silicon studies need to represent dynamic memory contention across many workloads and system configurations, while cycle-accurate DRAM models can be costly and fixed-latency assumptions can mislead design decisions.

    Source: §I · p. 1

  • Method

    PANEM converts bandwidth-latency characterization data into a request-bytes/latency lookup and updates latency dynamically from observed system load.

    Source: §III · p. 2, A. Offline Curve Transformation · p. 2

  • Results

    Fixed-latency assumptions mispredicted IPC trends by +6.74% optimistically in the moderately loaded regime and -3.85% pessimistically in the heavily loaded regime, and biased LOB provisioning by roughly 3-10%.

    Source: §VI · p. 10

  • Takeaways & Limitations

    Across the reported workloads, PANEM largely enforced bandwidth limits across read/write mixes and reproduced rising latency at higher demand, unlike fixed-latency models.

    Source: §VI · p. 10

    The averaging window interacts with workload periodicity, so no single window size is equally appropriate to every trace in the study list.

    Source: §V · p. 10

Abstract

from arXiv · show

Accurate pre-silicon memory modeling is essential for achieving meaningful representation of workloads on cloud-class many-core processors. Existing options force a poor tradeoff between fidelity and speed: fixed-latency models are fast but misleading, while cycle-accurate DRAM models are costly and difficult to scale across large study spaces or onto single-core environments. This paper presents PANEM, a lightweight event-driven heuristic model that has guided four generations of commercial server-core development at Ampere Computing. PANEM converts bandwidth-latency characterization data into a dynamic request-bytes/latency response, allowing miss latency to adapt to transient demand, queuing pressure, and read/write mix during simulation. Integrated into a single-core flow with configurable system-loading assumptions, PANEM enables realistic bandwidth constraints and contention-aware latency behavior without sacrificing throughput. Across a broad cloud workload trace suite, PANEM avoids the optimistic and pessimistic biases of fixed-latency baselines, yields more reliable conclusions for prefetching and dynamic throttling studies, and materially improves core-resource sizing decisions. These results show that a calibrated, contention-aware abstraction can deliver practical predictive value for industrial design-space exploration at simulation costs similar to fixed-latency models.

I. INTRODUCTION

Memory latency varies with system-wide contention, making realistic memory behavior essential to predictive pre-silicon simulation. PANEM addresses the speed–fidelity challenge with a configurable model intended for single-core studies of many-core systems.

I. INTRODUCTION

  • Contention makes cache-miss latency dynamic, so pre-silicon simulation must capture shared-memory behavior to predict performance meaningfully.Many-core systems have cores competing for shared caches and memory bandwidth.Source: §I · p. 1
  • Cycle-accurate DRAM models can provide high fidelity, but runtime and configuration costs can make large design-space sweeps impractical.Prior work also identifies CPU-memory interface inaccuracies and missing latency as sources of simulation mismatch,.Source: §I · p. 1
  • Commercial server-core studies span many workloads, loading levels, and memory configurations, while fixed miss latency can mislead decisions about buffer sizing and prefetching.The paper argues that choices made with overly simplistic models can be incorrectly sized for a real shipping product.Source: §I · p. 1
  • PANEM transforms empirical bandwidth-latency curves into a request-bytes/latency lookup to determine latency dynamically from system load.The authors report using the methodology to design four generations of AmpereOne® server CPU cores –.Source: §I · p. 1

II. BACKGROUND

  • Single-core studies in Panthera model core, mesh, LLC, and memory-controller behavior, but the mesh and memory resources can be over-provisioned relative to one core.Using cycle-accurate DRAM simulation also requires meaningful bandwidth demand, through multi-core simulation or traffic replication, adding configuration and runtime overhead.Source: §II · p. 2

III. METHODOLOGY

PANEM uses bandwidth-latency characterization data to create a lookup representation for runtime latency decisions. Converting the data through Little’s Law makes the relation usable beyond memory saturation.

III. METHODOLOGY

  • PANEM is a lightweight, event-driven heuristic model that uses standard bandwidth-latency curves to provide a dynamic latency response based on system load.Its methodology has an offline preparation phase and a runtime operational loop.Source: §III · p. 2

A. Offline Curve Transformation

  • Little’s Law converts bandwidth-latency data into a request-bytes/latency lookup relation that remains well-defined beyond the saturation point.Unlike bandwidth as a function of latency, the converted relation remains usable when bandwidth drops as latency continues to rise.Source: A. Offline Curve Transformation · p. 2
  • The lookup can use multiple curves to represent different read/write ratios, and its defined relation under oversaturation distinguishes it from ZSim’s M/D/1 model, which must clamp utilization below 100%.The clamp avoids the asymptote in the analytical model’s formula.Source: A. Offline Curve Transformation · p. 2

B. Runtime Latency Calculation

PANEM updates latency from recent demand and read/write mix, using precomputed curves to model contention. The described experiments show its response tracking the target behavior more closely than ZSim M/D/1 and DAMOV near peak bandwidth.

B. Runtime Latency Calculation

  • The runtime loop tracks in-flight requests in separate read and write queues and averages outstanding requests and read/write ratio over a configurable sliding window.Writes may be acknowledged immediately to the core but remain tracked internally as system load; the default configuration uses a single tumbling 520 ns window.Source: B. Runtime Latency Calculation · p. 2
  • Periodic updates select a precomputed curve from recent demand and read/write mix, interpolate latency, and assign new requests completion times using that latency.As outstanding requests rise, the next window’s lookup latency rises, modeling contention and queuing effects.Source: B. Runtime Latency Calculation · p. 3

C. Comparison and Design Rationale

  • Unlike Mess’s iterative approximation, PANEM directly selects latency from recently observed workload behavior and is described as more reactive to transient behavior.The authors frame its design choices against analytical queuing models such as ZSim’s M/D/1.Source: C. Comparison and Design Rationale · p. 3

M/D/1 PANEM

  • PANEM converts empirical bandwidth-latency curves into a request-bytes/latency relation that remains defined when demand exceeds peak bandwidth.The examples use Intel Skylake curves reported in, with the converted representation shown alongside the source curves.Source: M/D/1 PANEM · p. 3, Figure 2 · p. 3, Figure 3 · p. 3
  • Curve filtering removes points that fail to increase both outstanding request-bytes and latency, then retains only the first point after peak bandwidth.The full family of 26 curves reported in was converted and ingested, although the figures show only a subset.Source: M/D/1 PANEM · p. 3, Figure 4 · p. 4, Figure 5 · p. 4
  • PANEM extrapolates linearly from the last two points to arbitrarily high request-byte values, presuming latency continues to rise as demand increases.The authors state that this represents performance collapse in real hardware, where bandwidth decreases under excessive congestion.Source: M/D/1 PANEM · p. 4
  • Vendor bandwidth-latency curves can configure PANEM for future DRAM technologies as early characterization data becomes available.The authors describe this as a practical benefit for core and system co-design.Source: M/D/1 PANEM · p. 4
  • PANEM more closely replicates target bandwidth-latency behavior than ZSim M/D/1 and DAMOV, especially near peak bandwidth, while constraining throughput below peak and increasing latency under excessive demand.The comparison used two abstract cores and a rate-limited traffic generator across four read/write mixes; very high demand produced increased variance when the memory configuration saturated.Source: M/D/1 PANEM · p. 4

A. Hardware Correlation Comparison

PANEM correlates more closely with measured many-core performance than fixed-latency configurations, including at individual-benchmark granularity. The paper also illustrates PANEM’s response to bandwidth/latency data captured from Intel Skylake.

A. Hardware Correlation Comparison

  • PANEM achieves better overall correlation than either fixed-latency configuration and maintains closer accuracy and consistency at individual-benchmark granularity.The comparison covers SPEC CPU2017 benchmarks against half- and fully-loaded real-world hardware.Source: A. Hardware Correlation Comparison · p. 4
  • Figures 6–7 show PANEM responding to ingested Intel Skylake bandwidth/latency data, with nominal read:write curves for four workloads highlighted.Figure 6 uses an abstract traffic generator; Figure 7 provides finer detail.Source: Figure 6 · p. 5, Figure 7 · p. 5
  • At full load, idle fixed latency drastically overestimates performance, while high fixed latency balances benchmark-level over- and underestimates.For 523.xalancbmk, PANEM’s improvement under MAV characterization is greater than for fixed configurations; meaningful characterization requires a sufficiently representative system environment.Source: A. Hardware Correlation Comparison · p. 5

B. Bandwidth/Latency Distributions

PANEM constrains bandwidth to configured curve parameters, whereas fixed-latency models produce wider bandwidth distributions and biased performance projections. Prefetching changes both bandwidth use and latency, so its impact differs across the memory-model configurations.

B. Bandwidth/Latency Distributions

  • Idle fixed latency overpredicts performance relative to PANEM moderate loading, while high fixed latency underpredicts it even relative to PANEM heavy loading.The high-latency configuration does so despite its high achieved bandwidth.Source: B. Bandwidth/Latency Distributions · p. 5
  • Fixed-latency models produce much wider bandwidth distributions than PANEM, whose configurations constrain bandwidth to their curve parameters.The PANEM configurations also show distinct distributions near peak bandwidth.Source: Figure 8 · p. 5, B. Bandwidth/Latency Distributions · p. 5, Figure 10 · p. 6

C. L2 Prefetcher Impact

  • The PANEM-based model correlates closer to the many-core SoC under test than the fixed-latency models in Table II.Source: Table II · p. 6
  • Re-enabling L2 prefetchers raises average bandwidth by 22% while increasing latency by 10%.Source: C. L2 Prefetcher Impact · p. 6
  • Fixed-model prefetcher sensitivity is 48% higher than PANEM moderate, while PANEM heavy has the lowest sensitivity.Under PANEM, unhelpful prefetches consume bandwidth as well as cache capacity, unlike under fixed-latency models.Source: C. L2 Prefetcher Impact · p. 6
  • With more aggressive prefetching, fixed latency overstates improvement by a factor of 1.9, while PANEM heavy slows down as bandwidth cost outweighs better coverage.Source: C. L2 Prefetcher Impact · p. 6

D. Voluntary Core Throttling

CBusy throttling reduces average latency with a small reduction in bandwidth utilization, but the estimate is pessimistic because the modeled control loop is slower than in a real SoC. PANEM also captures read/write-dependent bandwidth and latency behavior.

D. Voluntary Core Throttling

  • Without L2 prefetching, the greater bandwidth available in the configuration is not fully utilized; disabling prefetchers also frees bandwidth for demand loads while increasing latency.Source: Figure 11 · p. 7, Figure 12 · p. 7
  • The throttling result is pessimistic because utilization measurement and throttling occur between the core and memory controller, lengthening the control loop versus a real SoC.PANEM is described as suitable for evaluating general efficacy, but insufficient for fine-tuning this behavior.Source: D. Voluntary Core Throttling · pp. 6–7
  • Under heavy PANEM loading, more aggressive L2 prefetching regresses because inaccurate prefetches can harm system performance while cores compete for resources.Source: Figure 13 · p. 7
  • Fixed-latency models cannot reasonably represent throttling’s bandwidth-dependent latency response and are likely to show regression when latency cannot improve.Source: D. Voluntary Core Throttling · p. 7
  • CBusy throttling reduces average latency by 10% in exchange for a 1.2% reduction in average bandwidth utilization.This recovers the latency increase from enabling prefetchers while retaining nearly all of the bandwidth improvement.Source: Figure 14 · p. 7, D. Voluntary Core Throttling · p. 7
  • Across read/write scenarios, PANEM largely reproduces the hardware bandwidth/latency relationship; write-heavy traces trend toward lower bandwidth and higher latency.Exceptionally high latency is limited to write-dominated traces; early write acknowledgment, mesh buffering, and scaled-down PANEM bandwidth lump latency into PANEM write processing time.Source: D. Voluntary Core Throttling · pp. 7–8, Figure 15 · p. 8

E. Core Load Depth Sensitivity

Fixed-latency models show little differentiation as load depth changes, whereas PANEM exposes the impact of bandwidth availability. The models also imply different LOB sizing needs at a 3% overall IPC impact threshold.

E. Core Load Depth Sensitivity

  • High and low fixed-latency models hardly differentiate as load depth falls, while PANEM models expose the impact of bandwidth availability.Source: Figure 16 · p. 8
  • Both fixed-latency configurations suggest greater LOB-size sensitivity than either PANEM configuration.Source: E. Core Load Depth Sensitivity · p. 8
  • At a 3% overall IPC impact threshold, idle fixed latency indicates a 3% larger LOB than PANEM moderate, while high fixed latency calls for nearly 10% larger LOB than PANEM heavy.The comparison is between the named fixed-latency and PANEM configurations.Source: E. Core Load Depth Sensitivity · p. 8

F. Runtime Impact

PANEM adds little runtime impact relative to the assessed fixed-latency models, while modeling more cycles per scheduled CPU second. The section reports runtime and modeled-cycle comparisons averaged over 4,300 study-list data points.

F. Runtime Impact

  • Table III reports PANEM runtime impact versus fixed-latency models in scheduled CPU seconds, averaged over 4,300 study-list data points.Source: Table III · p. 8
  • Table IV reports modeled cycles for PANEM-based runs and model efficiency, averaged over 4,300 study-list data points.Source: Table IV · p. 8
  • PANEM-based runs model 20% more cycles per scheduled CPU second than fixed-latency runs.The prose describes the total mean model runtime impact as minimal; Figure 17 summarizes runtime for the four configurations as box plots.Source: F. Runtime Impact · p. 8, Figure 17 · p. 8

V. LIMITATIONS

PANEM’s behavior depends on idle latency captured in its parameterizing curves and on how request patterns interact with measurement windows. The authors describe occasional curve-bound violations, especially for periodic workloads, and select a default window to limit outliers while retaining workload reactivity.

V. LIMITATIONS

  • If an initial request responds before an adjacent request arrives, both can see idle latency, preventing enough simultaneous outstanding requests to raise modeled latency; adding upstream components can reduce idle latency lumped into the curves and introduce error.In single-core simulations, PANEM also subsumes some mesh latency; the authors say curve adjustment offers minimal benefit given the sharp latency rise near bandwidth capacity.Source: §V · pp. 8–9
  • Traces exceeding peak bandwidth account for 0.32% of traces, with the highest-bandwidth trace averaging 4.8% above theoretical peak; traces with lower-than-expected latency account for 0.57%.Source: §V · p. 9
  • Because latency selection follows request measurement, high latency can apply to lower-bandwidth regions and low latency to adjacent higher-bandwidth regions, yielding high average bandwidth at low average latency.The text points to the highest-bandwidth trace in Figures 18–20 as an example.Source: §V · p. 9, Figure 18 · p. 9, Figure 19 · p. 9
  • Averaging-window size changes the observed request pattern: 4 × 520 ns produces highly correlated adjacent-window bandwidth, while 1 × 210 ns exposes workload detail and limits bandwidth through rapid request-rate reaction rather than sustained latency.For single-core simulations, the wider window homogenizes bandwidth to approximate many cores making similar but orthogonal requests; the smaller window reflects the modeled core’s successive requests.Source: Figure 21 · p. 10, Figure 22 · p. 10, §V · p. 10
  • Changing the window size can stop these traces from exceeding peak bandwidth, but the selected window also determines how much single-core workload behavior appears in successive measurements.Source: §V · p. 10
  • Periodic request intensity can make traces exceed peak bandwidth at any reasonable window size, so no single size suits all traces; the default is the smallest window with no trace-average outlier above 5% of nominal peak bandwidth.The authors say workload-specific window adaptation may improve accuracy, but its opportunity seems limited given the low incidence and deviation of non-compliant traces.Source: §V · pp. 9–10

VI. CONCLUSION

PANEM derives a dynamic request-bytes/latency response from bandwidth/latency characterization data to model memory congestion in single-core simulations. Across the workloads, it largely enforces bandwidth limits and increasing latency under demand, while fixed-latency assumptions mispredict performance and architectural sensitivities.

VI. CONCLUSION

  • Across workloads and read/write mixes, PANEM largely enforces appropriate bandwidth limits and reproduces increasing latency at higher demand, unlike fixed-latency models.Source: §VI · p. 10
  • Fixed-latency assumptions mispredict IPC trends by +6.74% in the moderately loaded regime and -3.85% in the heavily loaded regime, distort prefetch sensitivity, and bias LOB provisioning by roughly 3–10%.The conclusion also reports that PANEM enables evaluation of dynamic throttling and a practical latency-bandwidth tradeoff under load.Source: §VI · p. 10
  • PANEM is weakly susceptible to interactions between averaging-window size and workload periodicity, but offers a pragmatic middle ground between fixed-latency abstractions and cycle-accurate DRAM simulation.The authors frame this as improving predictive confidence in high-throughput studies for core and system co-design in commercial cloud SoCs.Source: §VI · p. 10
Loading 2609.29629v1…