PaperlayerView exact revision on arXiv

Source-linked AI summary

Basic Performance Measurements of the Intel Optane DC Persistent Memory Module

Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R. Dulloor, Jishen Zhao, Steven Swanson

arXiv:1903.05714v3cs.DCcs.PF

TL;DR

Existing NVDIMM research relied on models because real devices lacked detailed performance evidence. This paper measures Intel Optane DC PMM across memory, file-system, and user-space persistence scenarios, finding that deeper stack integration generally delivers faster real-world applications.

  • Problem

    Prior NVDIMM studies lacked detailed evidence about real-device behavior, performance, and architectural integration.

    Source: §1 · p. 12

  • Method

    The paper measures Optane DC PMM performance across Memory and App Direct modes, file systems, user-space persistence, and real-world applications.

    Source: Background (Section 2) · p. 5, §1 · p. 12, §6.2 · p. 51

  • Results

    Across applications and storage configurations, performance improves as Optane DC becomes more deeply integrated into the storage stack.

    Source: §6.6 · pp. 55–56, §7 · p. 57

  • Takeaways & Limitations

    Optane DC can provide DRAM-like application performance in cached mode and substantially accelerate storage workloads through NVMM-aware software integration.

    Source: §6.6 · p. 56, §7 · p. 57

    User-space persistence is more complex because applications must ensure crash consistency rather than relying on the file system.

    Source: Optane DC as Persistent Memory… · p. 10

Abstract

from arXiv · show

Scalable nonvolatile memory DIMMs will finally be commercially available with the release of the Intel Optane DC Persistent Memory Module (or just "Optane DC PMM"). This new nonvolatile DIMM supports byte-granularity accesses with access times on the order of DRAM, while also providing data storage that survives power outages. This work comprises the first in-depth, scholarly, performance review of Intel's Optane DC PMM, exploring its capabilities as a main memory device, and as persistent, byte-addressable memory exposed to user-space applications. This report details the technologies performance under a number of modes and scenarios, and across a wide variety of macro-scale benchmarks. Optane DC PMMs can be used as large memory devices with a DRAM cache to hide their lower bandwidth and higher latency. When used in this Memory (or cached) mode, Optane DC memory has little impact on applications with small memory footprints. Applications with larger memory footprints may experience some slow-down relative to DRAM, but are now able to keep much more data in memory. When used under a file system, Optane DC PMMs can result in significant performance gains, especially when the file system is optimized to use the load/store interface of the Optane DC PMM and the application uses many small, persistent writes. For instance, using the NOVA-relaxed NVMM file system, we can improve the performance of Kyoto Cabinet by almost 2x. Optane DC PMMs can also enable user-space persistence where the application explicitly controls its writes into persistent Optane DC media. In our experiments, modified applications that used user-space Optane DC persistence generally outperformed their file system counterparts. For instance, the persistent version of RocksDB performed almost 2x faster than the equivalent program utilizing an NVMM-aware file system.

How to Use this Document

This document guides readers to sections aligned with their interests in Optane DC memory, applications, file systems, and persistent-memory libraries, while highlighting key findings and noting limits on available design information.

Reader guidance

  • Read Section 3 for Optane DC memory characteristics, Section 4 for large-memory applications, Section 5 for file systems, and Section 6 for persistent-memory libraries and emulation.Section 6 covers prototype libraries on real Optane DC PMMs and comparisons with prior emulation methods.Source: How to Use this Document · p. 3

Document conventions

  • “Observations” boxes identify key Optane DC PMM facts and findings that are broadly useful or statistically important.These boxes highlight findings intended for a wide group of readers and key device statistics.Source: How to Use this Document · p. 3

Limitations

  • The authors may be unable to answer some data or methodology questions because many Intel design details remain undisclosed.The document welcomes questions but notes that publicly unavailable design information limits the information provided.Source: How to Use this Document · p. 3

Versions

The document progressed from an initial release with embargoed results to a full release, followed by an updated release with clarifications, new bandwidth experiments, and errata.

Version 0.1.0

  • Version 0.1.0 was the initial release, with some results still under embargo.Source: Versions · p. 4

Version 1.0.0

  • Version 1.0.0 was the first full release, with all results released from embargo.Source: Versions · p. 4

Version 1.0.1

  • Version 1.0.1 added minor textual clarifications, new experiments on device bandwidth, and errata, including correction of the Memory Mode data-movement cache-line size from 4 KB to 64 bytes.The correction applies to data movement between Optane DC and DRAM memory.Source: Versions · p. 4

Executive Summary

The report evaluates Intel Optane DC PMM as expanded main memory, persistent storage, and byte-addressable persistent memory, showing both substantial capacity and performance benefits and important latency, bandwidth, and software-integration costs.

Executive Summary

  • The report provides the first detailed performance review of commercially available Optane DC PMMs across volatile-memory extension, conventional storage, and nonvolatile main-memory uses.Its goal is to help architecture and systems researchers develop intuition about the technology’s behavior.Source: Executive Summary · p. 5

Background (Section 2)

  • Optane DC PMMs attach through the processor memory controller using a DDR4-compatible protocol and support Memory mode with a DRAM cache or App Direct mode without one.The tested modules were 256 GB, and a Cascade Lake socket can host six modules totaling 3 TB.Source: Background (Section 2) · p. 5

Basic Optane DC Performance (Section 3)

  • Optane DC has lower bandwidth and higher load latency than DRAM: six modules peak at 39.4 GB/s reads and 13.9 GB/s writes, while efficient accesses begin at 256 B.On one DIMM, random load latency is 305 ns versus 81 ns for DRAM, and random-access bandwidth peaks at 2.8 GB/s for loads and 1.5 GB/s for stores.Source: Basic Optane DC Performance (Section 3) · p. 6, Figure 1 · p. 6, Figure 2 · p. 7

Optane DC as Main Memory (Section 4)

Optane DC as Persistent Storage (Section 5)

  • Optane DC improves basic storage performance over SATA and Optane SSDs, while NOVA outperforms legacy file systems and scales better.Application-level effects vary widely; for MongoDB, legacy Ext4 outperforms DAX, possibly because DAX disables the useful DRAM page cache.Source: Optane DC as Persistent Storage… · p. 8, Figure 5 · p. 9

Optane DC as Persistent Memory (Section 6)

  • User-space persistent memory raises RocksDB performance by 3.5× but Redis 3.2 performance by only 20%, while requiring applications to ensure crash consistency.The performance impact varies widely across applications despite the larger potential gains of direct load/store access.Source: Optane DC as Persistent Memory… · p. 10

Conclusion

  • Optane DC introduces a new processor-hierarchy memory technology whose strengths and weaknesses remain incompletely understood, motivating further systems and application research.The presented measurements demonstrate immediate benefits as well as areas requiring additional work.Source: Conclusion · p. 10

1 Introduction

The report presents early measurements of Intel Optane DC PMM, a commercially emerging byte-addressable nonvolatile memory, to replace assumptions from modeled NVDIMMs with real-system evidence. It examines fundamental performance, system integration, applications, software, and emulation across broad benchmarks and microbenchmarks.

Technology context

  • Optane DC PMM provides byte-granularity nonvolatile storage designed to survive power outages and is presented as an emerging scalable main-memory technology.The paper notes that Optane DC PMM is not technically an NVDIMM because it does not comply with the NVMM-F, -N, or -P JEDEC standards.Source: §1 · p. 12

Motivation and contribution

  • The report measures real Optane DC PMM systems to address performance and integration questions that prior NVDIMM research studied without detailed knowledge of actual hardware behavior.Earlier work relied on custom hardware, simulation, altered DRAM frequency, NUMA effects, or treating DRAM as persistent.Source: §1 · p. 12

Research questions

  • It evaluates Optane DC PMM’s basic performance, persistence instructions, use as DRAM extension and storage, system software, NVMM-specific applications, and comparison with prior emulation methods.These questions span memory technology, persistent-memory instructions, applications, file systems, custom NVMM software, and emulation fidelity.Source: §1 · p. 12

Evaluation scope

  • The study covers a wide range of applications, benchmark suites, and microbenchmarks using more than 330 hours of machine time, with underlying data made freely available.The data is provided in the attached anc/ directory.Source: §1 · p. 12

2 Background and Methodology

This section introduces Optane DC PMM as a dense, persistent, byte-addressable memory tier between DRAM and block storage, then defines its hardware, operating modes, evaluation platform, and benchmark configurations. The methodology evaluates Optane DC in both main-memory and persistent-device roles using controlled memory, emulated-memory, and block-storage comparisons.

2.1 Optane DC Memory

  • Optane DC PMM combines higher density and persistence than DRAM with better performance and byte-addressability than external-interface storage, and debuted in 128 GB, 256 GB, and 512 GB capacities.It creates a new tier between volatile DRAM and block-based storage.Source: §2.1 · p. 13

2.1.1 Intel’s Optane DC PMM

  • The PMM connects to Cascade Lake’s integrated memory controllers through DDR-T, translates CPU cache-line accesses into 256-byte media operations, and can incur write amplification through read-modify-write handling.Internal translation supports wear-leveling and bad-block management, while on-DIMM control coordinates access to Optane media.Source: §2.1.1 · p. 13

2.1.2 Operation Modes

  • Memory mode uses adjacent DDR4 as a direct-mapped write-back cache for volatile Optane memory, whereas App Direct mode exposes Optane directly as persistent storage alongside visible DDR4.The study evaluates one mode at a time and uses a consistent NVDIMM configuration across the Optane memory.Source: §2.1.2 · pp. 13–14

2.2 System Description

  • Experiments use a dual-socket Cascade Lake platform with 384 GB DRAM and 3 TB NVMM, running Fedora 27 and Linux 4.13.0 with controlled CPU, firmware, and power settings.NVMe Optane and SATA NAND SSDs provide block-storage baselines.Source: §2.2 · p. 14, Table 1 · p. 15

2.3 Configurations

  • The evaluation examines six configurations spanning Optane’s roles as main memory and persistent storage, summarized in Table 2.Three configurations vary memory behind the DRAM interface, while three vary the device beneath a file system.Source: §2.3 · p. 15, Table 2 · p. 15

2.3.1 Memory Configurations

  • Memory configurations compare DRAM-only MM-LDRAM, DRAM-cached Optane MM-Optane-Cached, and raw uncached Optane MM-Optane-Uncached as system memory.The uncached setup uses App Direct mode, treats Optane as slower NUMA memory, and binds applications to the Optane-only NUMA node.Source: §2.3.1 · p. 16

2.3.2 Persistence Configurations

  • Persistence configurations expose fsdax devices under /dev/pmem, enabling DAX file systems and user-level load/store libraries while retaining block-based access for non-DAX file systems.PM-Optane uses real Optane, while PM-LDRAM and PM-RDRAM emulate persistent memory with local and remote DRAM.Source: §2.3.2 · p. 16
  • File-system experiments additionally compare Optane DC’s load/store interface with an NVMe Optane SSD and a SATA NAND SSD beneath the file system.These baselines contrast comparable Optane media using block storage and a more established storage technology.Source: §2.3.2 · p. 17

3 Basic Performance

Optane DC exhibits substantially higher latency and lower bandwidth than DRAM, with performance strongly shaped by access pattern, concurrency, access size, and read/write mixtures. Sequential and sufficiently large accesses improve utilization, while persistence-oriented instruction choices can reduce small-access costs.

3.1.1 Read Latency

  • 305 ns random-read latency makes Optane DC about 3× slower than local DRAM, while sequential access improves latency by 2× through request merging.The sequential result indicates on-DIMM prefetching and batching, including merging adjacent requests into a 256-byte access.Source: §3.1.1 · p. 18, Figure 7 · p. 19

3.1.2 Memory Instruction Latency

  • For small accesses, clflushopt and clwb outperform clflush and non-temporal stores, while Optane DC loads are slower but stores resemble PM-LDRAM.Store latency is hidden within the ADR domain, whereas load sequences favor DRAM over Optane DC.Source: §3.1.2 · p. 19, Figure 8 · p. 20

3.2.1 Maximum Bandwidth

  • 39.4 GB/sec maximum read bandwidth and 13.9 GB/sec maximum write bandwidth are achieved with six interleaved Optane DC PMMs.Reads continue scaling with threads, reaching maximum throughput at 17 threads, whereas four threads saturate write bandwidth.Source: §3.2.1 · p. 20, Figure 9 · p. 21

3.2.2 Concurrency and Bandwidth

  • A single Optane DC DIMM reaches 6.6 GB/s reads and 2.3 GB/s writes, with non-monotonic scaling that peaks at low thread counts; interleaving moves store+clwb’s peak to twelve threads.DRAM bandwidth is substantially higher and scales predictably with thread count.Source: §3.2.2 · p. 21, Figure 10 · p. 22

3.2.3 Access Size and Bandwidth

  • 256-byte-or-larger accesses are preferred because random Optane DC bandwidth is poor below 256 B; interleaving improves peak read and write bandwidth by 5.8× and 5.6×.A performance dip at 4 KB is attributed to interleaved-access contention at the integrated memory controller.Source: §3.2.3 · p. 22, Figure 11 · p. 23

3.2.4 Bandwidth under Mixed Workloads

  • Mixed reads and writes hurt Optane DC more than DRAM: PM-LDRAM and PM-RDRAM outperform it by up to 12.0× and 7.5×, respectively.Optane DC performs best for all-read scenarios and substantially better on sequential workloads.Source: §3.2.4 · p. 23, Figure 12 · p. 24, Figure 13 · p. 24

3.2.5 Performance under Load

  • 38.9 GB/sec versus 105.9 GB/sec sequential read bandwidth and 10.3 GB/sec versus 70.4 GB/sec random read bandwidth show Optane DC trails DRAM under load.Sequential write bandwidth also reaches about 11.5 GB/sec for Optane DC versus 52.3 GB/sec for DRAM; latency rises sharply near saturation because of queuing.Source: §3.2.5 · p. 25, Figure 14 · p. 26
  • Sequential Optane DC bandwidth is 4× higher than the corresponding non-sequential performance because merged accesses avoid the write-amplification cost of 256-byte device accesses.The section measures latency, bandwidth, store types, persistence costs, and access-pattern effects using MLC and LATTester under PM-LDRAM, PM-RDRAM, and PM-Optane configurations.Source: §3 · p. 18, §3.2.5 · p. 25

4 Optane DC as Main Memory

Optane DC expands affordable main-memory capacity, but its lower bandwidth and higher latency can reduce performance when workloads exceed cache capacity or saturate memory bandwidth. DRAM caching largely preserves performance for small footprints, while write-heavy and multi-threaded workloads expose Optane DC’s limitations.

4 Optane DC as Main Memory

  • DRAM generally outperforms cached Optane DC, which outperforms uncached Optane DC, although caching keeps small-footprint workloads close to DRAM.Uncached Optane DC is especially disadvantaged when concurrent workloads saturate memory bandwidth.Source: §4.1 · p. 27, §4.2 · p. 32

4.1 SPEC CPU 2006 and 2017

  • SPEC CPU shows that cached Optane DC is almost as fast as DRAM for workloads fitting in DRAM, while uncached Optane DC slows substantially in rate tests.The evaluation covers SPEC CPU 2006 and 2017 speed and rate variants, with 24 copies used for SPECrate.Source: §4.1 · p. 27, Figure 15 · p. 28, Figure 16 · p. 29

4.2 PARSEC

  • PARSEC exposes increasing performance gaps between uncached Optane DC and DRAM as thread counts rise and Optane DC bandwidth saturates, while cached Optane DC remains close to DRAM.Most PARSEC workloads fit in the last-level cache and show no performance difference across memory types.Source: §4.2 · p. 32, Figure 19 · p. 33

4.3 Memcached

  • Memcached experiments show that DRAM caching hides read latency more effectively than write latency, while cache coverage changes when the store reaches 288 GB and no longer fits only in DRAM.Experiments use GET-dominant (10% SET) and SET-dominant (50% SET) workloads with a 32 GB object store; write amplification may increase bytes written to Optane DC.Source: §4.3 · p. 34, Figure 20 · p. 34, Figure 21 · p. 35

4.4 Redis

  • Redis benefits from cached Optane DC when the workload fits in DRAM, but the cache’s benefit for SET requests decreases as workload size increases.The cache still reduces accesses reaching the Optane DC media, while the experiment measures pure SET and GET workloads across growing stores.Source: §4.4 · p. 36, Figure 22 · p. 37

5 Optane DC as Persistent Storage · 5.1 File Operation Latency

The study evaluates Optane DC as persistent storage across seven file-system configurations and memory- or SSD-backed setups, then measures basic file-operation latency. Optane DC favors DAX and purpose-built NVMM designs for some writes, but DRAM page caching can make non-DAX systems faster for reads.

5 Optane DC as Persistent Storage

  • The storage setups use DRAM as system memory and vary memory- or disk-backed storage among PM-LDRAM, PM-RDRAM, PM-Optane, SSD-Optane, and SSD-SATA.The section examines Optane DC beneath file systems using raw operations, synthetic I/O, emulated applications, and application workloads.Source: §5 · p. 38
  • The evaluation compares seven file systems or configurations across three memory-backed and two SSD-backed storage configurations when compatible.The broader study asks whether existing systems exploit Optane DC, whether custom NVMM systems outperform block-based adaptations, and whether DAX improves performance.Source: §5 · p. 38
  • NOVA provides crash consistency with log-structured metadata and copy-on-write, while NOVA-Relaxed preserves metadata consistency but permits in-place file-page writes.Both require Optane DC in DAX mode; NOVA-Relaxed relaxes file-data consistency to improve write performance.Source: §5 · p. 38

5.1 File Operation Latency

  • The basic latency benchmark measures single-threaded create, append, 4K write, 512-byte write, and 4K read system-call sequences, including fsync() for writes.The operations open and close files around the specified data-access sequence.Source: §5.1 · pp. 38–39
  • Ext4-DAX and XFS-DAX improve write latency over non-DAX counterparts on DRAM and Optane DC, while NOVA and NOVA-Relaxed outperform conventional systems for create and append operations.NOVA’s page-level copy-on-write makes 512-byte writes slower, whereas NOVA-Relaxed restores performance with in-place writes.Source: §5.1 · p. 39
  • Optane DC increases operation latency over DRAM by 3.3% for XFS-DAX to 156% for NOVA across memory-type storage configurations.The longer Optane DC latency affects every evaluated file system.Source: §5.1 · p. 39
  • DAX-enabled systems have similar read latencies that worsen from DRAM to Optane DC, whereas non-DAX systems hide Optane DC read latency with the DRAM page cache.Thus, non-DAX systems can outperform DAX systems on Optane DC for reads.Source: §5.1 · p. 39
  • File-operation latency results show PM-Optane has the longest memory-backed latencies, while non-DAX systems have shorter reads than DAX systems through DRAM page caching.Figure 23 also indicates that DRAM emulation overestimates Optane DC performance, especially on write paths.Source: §5.1 · p. 39, Figure 23 · p. 40

5.2 FIO Bandwidth · 5.3 Filebench

FIO shows that DAX-aware file systems, especially NOVA and NOVA-Relaxed, improve bandwidth and scaling on Optane DC memory, while Filebench shows their strongest advantages on write-intensive workloads. Small random writes can substantially expose performance differences between DRAM emulation and real Optane DC memory.

5.2 FIO Bandwidth

  • FIO measures 4 KB read and write bandwidth across sequential and random workloads while varying thread counts from one to sixteen.Each workload uses a 512 MB file per thread, 4 KB operations, and runs for 30 seconds; writes issue fsync() after every 4 KB.Source: §5.2 · p. 41
  • Non-DAX Ext4, Ext4-DJ, and XFS read paths copy data twice because FIO invalidates the page cache before each operation.Data is copied first from storage to the page cache and then from the page cache to a user buffer.Source: §5.2 · p. 41
  • NOVA and NOVA-Relaxed generally outperform other file systems in FIO and scale better, uniquely showing Optane DC memory outperforming SSDs.DAX-enabled file systems also outperform non-DAX systems in memory-backed configurations by bypassing the page cache.Source: Figure 24 · p. 42

5.3 Filebench

  • Filebench evaluates fileserver, varmail, webproxy, and webserver workloads that emulate file servers, mail servers, web proxies, and web servers.The workloads combine operations such as creates, appends, reads, writes, syncs, and deletes.Source: §5.3 · p. 43, Table 3 · p. 43
  • NOVA is 1.43×–3.13× faster than other evaluated file systems on average, while NOVA-Relaxed is marginally faster than NOVA in Filebench.Read-intensive workloads perform similarly, but NOVA and NOVA-Relaxed lead when write traffic increases.Source: §5.3 · p. 43, Figure 25 · p. 44
  • Ext4 performs better on block devices than DRAM in Filebench, apparently because inefficiencies affect its byte-granularity code path.This result is described as an anomaly investigated by the authors.Source: §5.3 · p. 43
  • Small random writes cause drastic differences between DRAM emulation and real Optane DC memory, with PM-Optane affecting NOVA and NOVA-Relaxed most on fileserver.The fileserver workload generates many small random writes, causing write amplification on Optane DC PMMs.Source: §5.3 · p. 43

5.4 RocksDB · 5.5 Redis

RocksDB and Redis workloads show that frequent synchronous persistence can make NOVA-type NVMM file systems outperform alternatives. RocksDB relies on WAL synchronization, while Redis uses an append-only log with an “always” fsync policy to ensure recovery safety.

5.4 RocksDB

  • RocksDB combines a DRAM skip-list memtable with WAL logging and layered SSTables, flushing and compacting data to reduce disk accesses.The memtable provides O(log n) inserts and searches; full memtables become SSTables, which are merged across levels during compaction.Source: §5.4 · p. 45
  • NOVA-type file systems perform best for write-dominant RocksDB because optimized sync operations avoid the flushing costs caused by frequent synchronization.RocksDB’s frequent syncs prevent non-NVMM file systems from batching updates, whereas NOVA maintains performance through an NVM-optimized sync mechanism.Source: Figure 26 · p. 45, §5.4 · p. 46
  • The RocksDB experiment measures SET throughput with 20-byte keys and 100-byte values while synchronizing the database after every operation.The workload uses db bench and emphasizes the performance impact of WAL appends and sync operations on NVMM file systems.Source: §5.4 · p. 46

5.5 Redis

  • NOVA-type file systems also achieve the highest Redis throughput on a write-dominant workload involving frequent synchronization.Redis’s frequent sync operations produce the same advantage for NOVA-type file systems observed in RocksDB.Source: Figure 27 · p. 46, §5.5 · p. 46
  • Redis persists writes through an append-only file that is replayed during recovery, with flush frequency trading performance against consistency.Redis is an in-memory key-value store used for caching and message queues, and supports “always”, “everysec”, and “no” fsync modes.Source: §5.5 · p. 46
  • The Redis experiment uses 190-byte MSET operations that generate 335-byte log records, with fsync after every append to prevent recovery data loss.Server and client processes run on the same machine while communicating via TCP.Source: §5.5 · p. 46
  • XFS unexpectedly outperforms DRAM on block-based devices, likely because inefficiencies in its byte-granularity code path affect several benchmarks.This anomaly appears under the safest Redis persistence configuration using the “always” fsync policy.Source: §5.5 · p. 46

5.6 Kyoto Cabinet · 5.7 MySQL · 5.8 SQLite

Across Kyoto Cabinet, MySQL, and SQLite, database performance on Optane DC varies with persistence mechanisms, buffering, and file-layout optimizations. NOVA-Relaxed leads the Kyoto Cabinet and SQLite write-heavy evaluations, while MySQL throughput remains stable across storage configurations.

5.6 Kyoto Cabinet

  • Kyoto Cabinet memory-maps metadata, updates it with load/store instructions, and uses msync plus write-ahead logging for persistence and SET failure atomicity.Each transaction appends an undo record before in-place modification, then flushes updated data and truncates the WAL at commit.Source: §5.6 · pp. 46–47
  • NOVA-Relaxed achieves the best Kyoto Cabinet HashDB throughput on the write-dominant workload because of its fast synchronization mechanism.The comparison uses one million random SET transactions with 8-byte keys and 1024-byte values, persisting each transaction at completion.Source: §5.6 · pp. 46–47, Figure 28 · p. 47
  • Disabling huge pages fixes a greater-than-90% Kyoto Cabinet performance drop caused by msync on transparent huge pages with DAX file systems.The benchmark results therefore report DAX performance with huge pages disabled while the underlying bug remains under investigation.Source: §5.6 · p. 47

5.7 MySQL

  • MySQL TPC-C throughput remains surprisingly stable across file systems and storage devices.The ten-warehouse, five-minute runs use MySQL’s aggressive buffer pool and checkpointing to avoid frequent persistence and hide access latency.Source: Figure 29 · p. 47, §5.7 · p. 48

5.8 SQLite

  • SQLite uses a single-file B+tree and is evaluated with write-ahead redo logging because measurements identify WAL as its best-performing logging mechanism.Mobibench inserts 100-byte values into one table using a single thread.Source: §5.8 · p. 48
  • NOVA-Relaxed performs best on SQLite’s write-dominant SET workload and significantly outperforms regular NOVA.Its in-place file-update optimization is especially effective because randomly distributed writes modify a single B+tree contained in one file.Source: §5.8 · p. 48, Figure 30 · p. 48

5.9 LMDB · 5.10 MongoDB

LMDB is evaluated with synchronous sequential SET throughput, while MongoDB is evaluated with single-threaded YCSB workloads. The results show performance differences between real and emulated Optane DC, block and DAX file systems, and native versus adapted NVMM file systems.

5.9 LMDB

  • LMDB is a lightweight B-tree database that memory-maps its entire database and uses copy-on-write pages with frequent msync calls for atomicity.Its mapped accesses directly load and store database memory, while copy-on-write supports atomic updates.Source: §5.9 · p. 48
  • LMDB throughput is measured for sequential synchronous SET operations using 16-byte keys and 96-byte values, as shown in Figure 31.The benchmark uses LevelDB’s db bench workload and measures a write-dominant sequential-key workload.Source: §5.9 · p. 48, Figure 31 · p. 49

5.10 MongoDB

  • MongoDB 3.5.13 uses the default WiredTiger engine, which maintains in-memory data, journals updates for immediate persistence, and periodically checkpoints it.MongoDB is an open-source document-oriented NoSQL database with pluggable storage engines, while WiredTiger is described in.Source: §5.10 · p. 49
  • MongoDB is evaluated with YCSB-A and YCSB-B using 100 K entries and 100 K operations, measuring single-threaded throughput with client and server on one socket.YCSB-A is write-dominant with 50% reads and 50% updates, whereas YCSB-B is read-dominant with 95% reads and 5% updates; YCSB is cited as. Figure 32 presents the results.Source: §5.10 · p. 49, Figure 32 · p. 50
  • Real Optane DC generally delivers lower application performance than emulated persistent memory, with the gap widening as the file system becomes faster.The observation attributes this behavior to latency differences measured in previous sections.Source: §5.10 · p. 49
  • Block-oriented file systems are not necessarily slower than DAX file systems in real-world benchmarks, particularly for read-oriented workloads.This result indicates that the DRAM page cache can substantially boost application performance.Source: §5.10 · p. 49
  • Native NVMM file systems generally outperform adapted file systems across the studied applications, especially with frequent sync operations.The observation highlights NOVA and NOVA-Relaxed and efficient sync mechanisms, while noting the trend may differ for other applications or workloads.Source: §5.10 · p. 49

6 Optane DC as Persistent Memory

User-space persistence exposes Optane DC through load/store access and cache-line flushes, avoiding an intervening file system. Across applications and benchmarks, deeper integration and persistent data structures improve performance, while Optane’s latency costs vary by workload and access pattern.

6 Optane DC as Persistent Memory

  • Optane DC persistent memory gives applications direct user-space access through memory mapping, ordinary loads and stores, and cache-line flushes for persistence.The evaluated configurations expose the memory as a pmem device using PM-LDRAM, PM-RDRAM, and PM-Optane.Source: §6 · p. 51

6.1 Redis-PMEM

  • PMDK-based Redis outperforms Redis using NOVA-Relaxed file-backed logging, showing the benefit of user-space persistence that bypasses the file system.The comparison uses 4-byte keys and values, 12 clients, and one million random MSET operations.Source: §6.1 · p. 51, Figure 33 · p. 51

6.2 RocksDB-PMEM

  • 73% higher performance on PM-Optane makes persistent-memtable RocksDB substantially faster than volatile memtables backed by NOVA-Relaxed WAL.The persistent skip-list eliminates the WAL by making the memtable crash-consistent with PMDK.Source: §6.2 · pp. 51–52, Figure 34 · p. 52

6.3 MongoDB-PMEM

  • PMem provides similar MongoDB performance to WiredTiger, while switching among PM-LDRAM, PM-RDRAM, and PM-Optane has little throughput impact.The result covers single-threaded YCSB-A and YCSB-B workloads with MongoDB and YCSB on the same socket.Source: §6.3 · p. 52, Figure 35 · p. 53

6.4 PMemKV

  • 2% to 15% higher latency for sequential reads and 45% to 87% for random reads occur on PM-Optane versus PM-LDRAM in PMemKV.Write operations reach up to 2.05× higher latency, while sequential reads remain comparable to DRAM.Source: §6.4 · p. 53, Figure 36 · p. 54

6.5 WHISPER

  • 24% average execution-time increase on PM-Optane versus PM-LDRAM occurs across large WHISPER workloads, with the largest gaps in persistent data structures.The performance gap is lowest for client-server applications and correlates with each benchmark’s share of persistent-memory accesses.Source: §6.5 · p. 54, Figure 37 · p. 55

6.6 Summary

  • Optane DC performance improves as it becomes more deeply integrated into the storage stack, with application-level software changes yielding significant benefits.The global comparison covers devices, file systems, and user-space persistence, and reports the highest performance for deeper hardware and software integration.Source: §6.6 · pp. 55–56, Figure 38 · p. 55

7 Conclusion

The experiments position Intel Optane DC PMM as a new memory tier between DRAM and storage, with performance properties distinct from currently deployed media. Results show benefits in cached, uncached, file-system, and user-space persistence modes, while motivating further adaptation and study.

Conclusion

  • Optane DC PMM creates a memory tier between DRAM and storage whose performance differs significantly from currently deployed media.The report presents a large sampling of performance experiments on Intel’s Optane DC Persistent Memory Module.Source: §7 · p. 57
  • Cached Optane DC memory often matches DRAM for real-world applications while increasing system memory, and uncached use offers faster access than hard drives or SSDs.With NVMM-aware file systems, Optane DC can drastically accelerate many storage applications; NVMM-aware user-space applications can achieve still greater benefits when software overheads are already low.Source: §7 · p. 57
  • The findings challenge assumptions that Optane DC matches local or remote DRAM and show that important questions about these devices and software designs remain open.Further work is needed to adapt persistent-memory designs to real Optane DC memory and complete understanding of the technology.Source: §7 · p. 57

A Observations

Optane DC PMM performance depends strongly on access pattern, request size, concurrency, and software stack integration. Across memory, file-system, database, and application workloads, its lower raw performance than DRAM can be mitigated by caching, sequential access, and persistent-memory-aware software.

Hardware observations

  • 305 ns random-load latency is about 3× slower than local DRAM, while sequential access improves Optane DC memory latency by 2×.Optane DC PMMs merge adjacent requests into a single 256 byte access.Source: A Observations · p. 60
  • Six interleaved Optane DC PMMs reach 39.4 GB/sec read bandwidth and 13.9 GB/sec write bandwidth, with reads scaling to 17 threads but writes saturating at four.Application-level bandwidth is best utilized with accesses of 256 bytes or larger.Source: A Observations · p. 60
  • Optane DC is more sensitive than DRAM to access patterns and mixed reads/writes, while sequential access provides 4× higher bandwidth through request merging.Merged sequential accesses avoid the write amplification associated with the NVDIMM’s 256 byte access size.Source: A Observations · p. 60

System performance

  • The DRAM cache makes cached-mode workloads fitting in DRAM largely unaffected, whereas uncached Optane DC can saturate under multithreaded applications and impose performance overhead.Small-access persistence operations perform better with clflushopt and clwb than with clflush or non-temporal stores.Source: A Observations · p. 60

File systems

  • Non-DAX file systems can outperform DAX systems because the DRAM page cache hides Optane DC read latency, especially for read-oriented workloads.Real Optane DC exposes small-random-write amplification, and NOVA’s copy-on-write consistency mechanism adds latency overhead.Source: A Observations · p. 60
  • Real Optane DC generally slows applications relative to emulated persistent memory, with the gap increasing for faster file systems and workloads containing frequent small persistent writes.Native NVMM file systems, NOVA and NOVA-Relaxed, generally outperform adapted file systems, particularly with frequent sync operations.Source: A Observations · pp. 60–61

Database workloads

  • Switching among PM-LDRAM, PM-RDRAM, and PM-Optane has little throughput impact for MongoDB YCSB-A and YCSB-B, while PMem storage performs similarly to WiredTiger.Client-server communication and MongoDB query-processing overhead likely dominate these workloads.Source: A Observations · p. 61

Application workloads

  • PM-Optane raises PMemKV latency by 2%–15% for sequential reads, 45%–87% for random reads, and up to 2.05× for writes versus PM-LDRAM, while increasing WHISPER execution time by 24% on average.The PM-Optane versus PM-LDRAM gap is greatest for persistent data structures and lowest for client-server applications.Source: A Observations · p. 61
Loading 1903.05714v3…