PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving DRAM Performance by Parallelizing Refreshes with Accesses

Kevin K. Chang, Donghyuk Lee, Zeshan Chishti, Alaa R. Alameldeen, Chris Wilkerson, Yoongu Kim, Onur Mutlu

arXiv:1712.07754v1cs.AR

TL;DR

DRAM refresh is necessary to preserve data but blocks memory service, and per-bank refresh only partially alleviates that interference. The paper proposes DARP and SARP to increase refresh-access parallelism, reporting larger benefits at higher DRAM densities.

  • Problem

    DRAM refresh delays memory requests, while per-bank refresh leaves cross-bank and within-bank refresh-access parallelism underused.

  • Method

    DARP schedules per-bank refreshes out of order to idle or write-draining banks, while SARP accesses idle subarrays during refreshes in other subarrays.

  • Results

    DSARP improves average performance by 3.3%/7.2%/15.2% over per-bank refresh across 100 workloads using 8/16/32Gb DRAM, respectively.

  • Takeaways & Limitations

    Refresh-access parallelization through DARP and SARP improves performance and energy efficiency, with benefits increasing as DRAM density increases.

  • Takeaways & Limitations

    The evaluation assumes 32ms retention time and estimates DDR3 tRFCpb values from LPDDR2 ratios because commodity DDR lacks REFpb support.

Abstract

from arXiv · show

Modern DRAM cells are periodically refreshed to prevent data loss due to leakage. Commodity DDR DRAM refreshes cells at the rank level. This degrades performance significantly because it prevents an entire rank from serving memory requests while being refreshed. DRAM designed for mobile platforms, LPDDR DRAM, supports an enhanced mode, called per-bank refresh, that refreshes cells at the bank level. This enables a bank to be accessed while another in the same rank is being refreshed, alleviating part of the negative performance impact of refreshes. However, there are two shortcomings of per-bank refresh. First, the per-bank refresh scheduling scheme does not exploit the full potential of overlapping refreshes with accesses across banks because it restricts the banks to be refreshed in a sequential round-robin order. Second, accesses to a bank that is being refreshed have to wait. To mitigate the negative performance impact of DRAM refresh, we propose two complementary mechanisms, DARP (Dynamic Access Refresh Parallelization) and SARP (Subarray Access Refresh Parallelization). The goal is to address the drawbacks of per-bank refresh by building more efficient techniques to parallelize refreshes and accesses within DRAM. First, instead of issuing per-bank refreshes in a round-robin order, DARP issues per-bank refreshes to idle banks in an out-of-order manner. Furthermore, DARP schedules refreshes during intervals when a batch of writes are draining to DRAM. Second, SARP exploits the existence of mostly-independent subarrays within a bank. With minor modifications to DRAM organization, it allows a bank to serve memory accesses to an idle subarray while another subarray is being refreshed. Extensive evaluations show that our mechanisms improve system performance and energy efficiency compared to state-of-the-art refresh policies and the benefit increases as DRAM density increases.

1. Introduction

DRAM refresh interrupts memory service, and existing per-bank refresh only partially overlaps refreshes with accesses. DARP and SARP address these limitations by scheduling refreshes more flexibly and exploiting subarray-level parallelism.

  • Refresh problem: 8.2% average performance degradation results from contemporary DRAM refresh, whose latency delays memory requests.The problem worsens with increasing DRAM density because more rows must be refreshed within the same refresh interval.
  • Refresh problem: Per-bank refresh improves concurrency across banks but restricts refreshes to a sequential order and blocks accesses to the bank currently being refreshed.These limitations leave opportunities for additional refresh-access overlap unexploited.
  • DARP: DARP schedules refreshes out of order to idle banks or banks draining writes, rather than following strict round-robin order.It monitors bank request-queue occupancy and overlaps refresh latency with batched write draining.
  • SARP: SARP allows a bank to serve requests to idle subarrays while other subarrays in that bank are being refreshed.The mechanism relies on the mostly independent subarrays within a bank and requires modest DRAM modifications.
  • Evaluation: DSARP improves average performance by 3.3%/7.2%/15.2% over per-bank refresh for 8/16/32Gb DRAM across 100 workloads.Maximum improvements are 7.1%/14.5%/27.0%, and gains increase with DRAM density.

2. Background

DRAM is organized into independently accessible ranks, banks, and subarrays, while refresh commands trade refresh granularity against memory availability. All-bank refresh blocks a rank, whereas per-bank refresh permits access to other banks during refresh.

  • DRAM organization: Each rank contains multiple banks that can serve requests independently, providing bank-level parallelism.Banks share an internal bus for reading and writing data.
  • DRAM organization: Each bank contains multiple subarrays, which are two-dimensional arrays of cells organized into rows and columns.Physically, DRAM typically has 32 to 64 subarrays per bank, grouped into eight subarray groups in this paper.
  • All-bank refresh: All-bank refresh operates across every bank in a rank, making the rank unavailable for memory requests during tRFCab.Refreshes are staggered across banks to avoid exceeding power-delivery limits.
  • Per-bank refresh: Per-bank refresh splits an all-bank refresh into eight operations across eight banks and issues commands eight times more frequently.The controller does not select the bank; the internal refresh unit follows sequential round-robin order.
  • Per-bank refresh: Per-bank refresh lets non-refreshing banks serve requests while another bank is refreshed, reducing refresh interference across banks.Refresh latency is shorter than all-bank refresh because operations are scattered into non-overlapping bank-level operations.

3. Motivation

Refresh increasingly harms performance as DRAM density and workload memory intensity rise. Per-bank refresh reduces but does not eliminate this impact because it remains restrictive and blocks accesses to the refreshing bank.

  • Increasing Performance Impact of Refresh: Higher DRAM density increases refresh latency, locking the entire rank and raising memory-access latency.The estimated all-bank refresh latency is 350ns for an 8Gb device and is expected to increase with density.
  • Increasing Performance Impact of Refresh: Refresh-related performance degradation becomes more severe as DRAM chip density or workload memory intensity increases.The evaluation uses 100 randomly mixed workloads grouped by memory intensity on an 8-core system.
  • Increasing Performance Impact of Refresh: Per-bank refresh improves over all-bank refresh by allowing accesses to non-refreshing banks, but its performance loss also worsens with increasing density.With 32Gb DRAM chips, per-bank refresh still causes 16.6% average performance loss versus an ideal no-refresh baseline.
  • Increasing Performance Impact of Refresh: Per-bank refresh is limited by sequential round-robin scheduling and by blocking accesses to the bank currently being refreshed.These restrictions prevent it from fully overlapping refreshes with accesses across banks.

4.1. Overview

DARP and SARP hide refresh latency by parallelizing refreshes with accesses across banks and subarrays. DARP reorders and overlaps per-bank refreshes, while SARP uses idle subarrays during refresh.

  • 4.1. Overview: DARP uses out-of-order per-bank refresh and write-refresh parallelization to overlap refreshes with accesses across banks.It can target idle banks and issue refreshes while writes drain in other banks.
  • 4.1. Overview: SARP allows a bank to serve accesses in idle subarrays while other subarrays are being refreshed.The mechanism relies on subarrays having local sense amplifiers and refreshes remaining contained within a subarray.
  • 4.1. Overview: LPDDR2 has shorter per-bank refresh latency than DDR3 because it refreshes fewer rows and uses a 32ms retention time instead of 64ms.The cited comparison explains the refresh-latency difference between the two DRAM types.

4.2. Dynamic Access Refresh Parallelization

DARP replaces rigid round-robin refresh scheduling with request-aware bank selection and overlaps refreshes with writes. Its scheduler uses refresh credits and queue occupancy while preserving refresh-scheduling flexibility.

  • 4.2.1. Out-of-order Per-bank Refresh: Out-of-order refresh selects an idle bank instead of following strict round-robin order, avoiding banks with pending requests.The policy monitors bank request-queue occupancy and makes refresh decisions dynamically.
  • 4.2.1. Out-of-order Per-bank Refresh: Refresh reordering relies on the JEDEC allowance for up to eight postponed or pulled-in all-bank refresh commands.This flexibility lets each bank tolerate up to eight reordered per-bank refreshes.
  • 4.2.2. Write-refresh Parallelization: During writeback mode, DARP refreshes a bank with the fewest pending demand requests while writes drain in other banks.The scheduler selects a candidate only when its refresh credit remains below the maximum pulled-in threshold.
  • 4.2.2. Write-refresh Parallelization: Write-refresh parallelization avoids delaying reads by postponing or pulling in refreshes without extending the writeback period.Writes are buffered and drained in batches, and they are not latency-critical because they result from dirty-cache-line evictions.
  • Implementation: DARP tracks refresh flexibility with four-bit-per-bank refresh-credit counters and adds queue-monitoring and scheduling logic.The storage overhead is 32 bits per rank.

4.3. Subarray Access Refresh Parallelization

SARP enables a DRAM bank to serve an idle subarray while another subarray is refreshed, addressing refresh-access collisions within banks. It requires targeted DRAM and controller changes while constraining activation rates to limit power consumption.

  • SARP allows memory accesses to an idle subarray while another subarray in the same bank is being refreshed.Refresh uses the refreshing subarray’s sense amplifiers and does not transfer data through the I/O bus, while other subarrays remain available.
  • SARP requires concurrent activation of multiple subarrays while connecting only the accessed subarray’s row buffer to the global bitlines.The design addresses shared global row-decoding and column-routing structures that otherwise prevent independent subarray operation.
  • SARP adds refresh-specific address paths and control logic so refreshes and accesses can simultaneously select their designated rows and subarrays.The additions include decoupled refresh-subarray and local-row counters, row-address and subarray selectors, and refresh-control logic.
  • The memory controller tracks the currently refreshing subarray and the number of subarrays to avoid issuing accesses to the refreshing subarray.The controller uses shadow copies of the refresh counters and stores the subarray count in SPD.
  • SARP increases tFAW and tRRD during refresh operations to constrain ACTIVATE activity and reduce power-delivery stress.Using Micron 8Gb DRAM values, the increases are 2.1x during all-bank refresh and 13.8% during per-bank refresh.
  • 0.71% area overhead is estimated for SARP in a 2Gb DDR3 DRAM chip, while additional-component power overhead is negligible.The estimate uses eight subarrays per bank, eight banks per chip, and a 55nm Rambus DRAM model.

5. Methodology

The evaluation uses cycle-level multi-core simulation and workloads spanning several benchmark suites and memory-intensity levels. It tests 8Gb, 16Gb, and 32Gb DRAM systems under a 32ms retention-time setting and summarizes performance across 100 workloads.

  • The mechanisms are evaluated with an in-house cycle-level x86 multi-core simulator and a cycle-accurate DRAM timing model validated against DRAMSim2.
  • The study evaluates 8Gb, 16Gb, and 32Gb DRAM systems using estimated DDR3 tRFCpb values and a 32ms retention time.The 32ms setting represents a typical server and LPDDR configuration.
  • Workloads use SPEC CPU2006, STREAM, TPC, and a random-access microbenchmark, classified by memory intensity using MPKI.Five workload categories contain 0%, 25%, 50%, 75%, or 100% memory-intensive benchmarks, with 20 randomly mixed workloads per category.
  • Figure 12 summarizes multi-core system performance improvement over REFab across 100 workloads.

6. Evaluation

Across diverse systems and workloads, DARP and SARP improve performance and energy efficiency by parallelizing refreshes with accesses. Their combined benefits increase with DRAM density and approach an ideal no-refresh baseline.

  • Multi-Core Results: SARPpb improves average system performance over REFpb by 3.3%/6.7%/13.7% and over REFab by 7.9%/11.7%/18.6% in 8/16/32Gb DRAMs.It enables accesses to idle subarrays in refreshing banks.
  • Multi-Core Results: DSARP provides additive improvement, reaching 27.0% over REFpb and 36.6% over REFab in 32Gb DRAM.Combining DARP and SARP enables more parallelization of refreshes and memory accesses.
  • Multi-Core Results: REFpb can perform worse than REFab because serialized per-bank refreshes may total about 3.5× tRFCab when multiple banks cannot be used effectively.All-bank refresh partially overlaps refreshes across banks with a single command.
  • All Mechanisms’ Results: DSARP performs within 0.9%, 1.2%, and 3.7% of the ideal no-refresh baseline in 8, 16, and 32Gb DRAMs.Across evaluated mechanisms, DSARP captures most of the benefit of eliminating refreshes.
  • Energy and Multi-Core Effects: DSARP reduces energy per memory access and consistently improves multi-core performance, fairness, and energy without unfairly penalizing specific applications.The energy reduction primarily follows from lower average static energy per access caused by performance improvement.
  • Sensitivity Studies: SARPpb’s benefit increases as tFAW decreases and as the number of subarrays increases, because more parallel accesses become possible and requests are less likely to target refreshing subarrays.These sensitivity results are reported for SARPpb over REFpb.
  • DDR4 Fine Granularity Refresh: 2x and 4x FGR reduce average system performance by 3.9%/4.0%/4.3% and 8.1%/13.7%/15.1% versus REFab in 8/16/32Gb DRAMs.Adaptive refresh performs within 1% of REFab, while DSARP avoids the degradation associated with higher refresh rates.

7. Related Work

Prior work reduces refresh overhead through scheduling, retention awareness, pausing, or parallelism, but differs from DARP and SARP in assumptions, scope, or level of parallelization.

  • Refresh Scheduling: Elastic refresh postpones refreshes based on postponed-count and predicted rank-idle time, but is less effective when rank idle periods are shorter than tRFCab.Its refresh latency cannot then be fully hidden during idle periods.
  • Refresh Scheduling: The authors’ mechanisms apply to single-ranked systems by parallelizing refreshes and accesses at bank and subarray levels, unlike a write-draining policy limited to multi-ranked systems.They can also be combined with that prior write-scheduling technique.
  • Subarray-Level Parallelism: SARP differs from SALP by parallelizing refreshes with accesses across subarrays, rather than pipelining multiple accesses to different subarrays within a bank.Both exploit subarray structure for different purposes.
  • Refresh Pausing: Refresh pausing assumes sequential row refreshes with recovery points, whereas current DRAM designs refresh multiple rows in parallel.This architectural assumption limits direct applicability to contemporary DRAM.
  • eDRAM Concurrent Refresh: The cited eDRAM mechanism parallelizes refreshes across banks, while SARP additionally targets parallelization within a bank and cannot be directly transferred because DRAM and eDRAM architectures differ.The architectures also differ in standardization constraints.
  • Retention-Aware Refresh: Retention-aware refresh techniques require accurately profiled cell retention times, but variable retention time and data-pattern dependence make accurate profiling an unsolved problem.This profiling challenge affects their viability.

8. Conclusion

DARP and SARP reduce DRAM refresh penalties by parallelizing refreshes with accesses at bank and subarray levels. Evaluations show improved performance and energy efficiency, with benefits increasing as DRAM density grows.

  • Conclusion: DARP schedules per-bank refreshes to idle banks out of order and during write draining instead of following strict round-robin scheduling.These policies hide refresh latency through bank and write parallelization.
  • Conclusion: SARP lets a bank serve requests from idle subarrays while other subarrays refresh.Together, DARP and SARP enhance refresh-access parallelization at bank and subarray levels.
  • Conclusion: Extensive evaluations show that DARP and SARP improve performance, outperform state-of-the-art refresh policies, and approach the performance of eliminating all refreshes.The evaluation covers a wide variety of systems and workloads.
  • Conclusion: The benefits of DARP and SARP increase as DRAM density increases.The conclusion identifies this trend for modern and near-future DRAM systems.

Erratum

The erratum corrects DARP’s description of when REFpb commands may be postponed. The corrected rule bounds ref_credit from 0 to 8 and resets it after refresh to preserve DRAM refresh guarantees.

  • The original DARP description incorrectly allowed REFpb postponement when a bank’s ref_credit stayed above -8.The erratum identifies this statement as an error.
  • The corrected postponement range is 0 ≤ ref_credit ≤ 8.A REFpb command must be issued when more than eight REFpb commands have been postponed to comply with the standard.
  • After a bank is refreshed, the memory controller resets its ref_credit value to 0.This reset applies to each REFpb command.
  • Using -8 as the lower bound could let a bank operate without refreshes for more than 9 refresh intervals, violating the JEDEC requirement.The correction limits each bank to skipping at most 8 REFpb commands at any time.
Loading 1712.07754v1…