PaperlayerView exact revision on arXiv

Source-linked AI summary

FDP: The Data Placement Promise of Modern NVMe SSDs

Sijie Lan, Hui Qi, Xing He, Mahmut Kandemir, Javier González, Vivek Shah

arXiv:2610.02676v2cs.OScs.DB

TL;DR

Conventional block interfaces hide flash-management information that applications could use to guide data placement and reduce device WAF. This paper characterizes enterprise FDP SSDs and evaluates mapping data separation in RocksDB and MySQL onto FDP. It reports lower WAF at high utilization, with application-level performance improvements.

  • Problem

    Conventional block interfaces obscure internal flash management, preventing applications from conveying placement information relevant to reducing device WAF and improving SSD lifetime.

    Source: §1 · p. 1

  • Method

    The paper characterizes enterprise FDP SSD isolation and QoS, then evaluates architectural data placement in RocksDB and MySQL through Linux interfaces.

    Source: §1 · p. 2

  • Results

    RocksDB’s device WAF reaches approximately 1 at 90% utilization, while MySQL’s falls to ~1.2 at 80% utilization; evaluations also report application-level improvements.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    FDP supports data separation in RocksDB and MySQL using Linux interfaces, with reported WAF reductions and throughput and latency improvements across single- and multi-tenant settings.

    Source: §1 · p. 2

    An I/O Passthru implementation was impractical for MySQL within the paper’s scope because InnoDB relies on filesystem semantics for its mutable files.

    Source: §5 · p. 8

Abstract

from arXiv · show

NVMe SSDs are now widely deployed as the storage tier in data centers. As SSDs have evolved over the past decade, the commu- nity has continued to debate the interfaces they expose and how operating systems and storage systems should exploit them. The NVMe Flexible Data Placement (FDP) proposal is the latest point in this design space. FDP introduces an interface based on Reclaim Units that enables explicit data placement to reduce device write amplification without the software engineering costs of sequential- write constraints and host garbage collection. FDP-enabled SSDs are emerging in commercial products and early data center de- ployments. Their compatibility with conventional block I/O allows existing applications to run unchanged, allowing a frictionless adop- tion in industry. This paper presents an experimental evaluation of FDP SSDs to characterize their data placement guarantees over the raw device interface. We then revisit two widely deployed and distinct open- source storage systems, MySQL and RocksDB, and examine whether lifetime-based data separation and distinct write patterns built into their architectures can be mapped onto FDP SSDs without inva- sive changes. Our evaluation shows end-to-end WAF reductions at higher device utilization, along with QoS and throughput improve- ments under synthetic and real-world workloads. These results demonstrate that FDP provides a practical and deployable cross- layer mechanism for data placement with open-source ecosystem support on Linux. They also highlight why FDP SSDs are gaining traction in industry.

1 INTRODUCTION

The paper asks whether FDP can expose architectural data separation in RocksDB and MySQL to reduce device write amplification without invasive changes. It characterizes enterprise FDP SSDs and evaluates these systems using Linux placement interfaces.

1 INTRODUCTION

  • FDP addresses a block-interface gap: conventional interfaces obscure flash management and prevent applications from conveying placement information relevant to device WAF and lifetime.Earlier approaches such as Open Channel SSDs and ZNS offer placement control at the cost of software complexity.Source: §1 · p. 1
  • The paper evaluates whether architectural differences in LSM key-value stores and B-tree relational databases can support FDP placement without invasive changes.Stable Linux I/O Passthru supports raw-device access, while filesystem lifetime hints offer a lower-engineering-cost path for systems using filesystem interfaces.Source: §1 · p. 1
  • RocksDB’s device WAF falls to approximately 1 at 90% device utilization, nearly eliminating compaction-induced device write amplification.The paper maps LSM levels with distinct lifetimes to FDP placement hints through filesystem and I/O Passthru interfaces.Source: §1 · p. 1
  • MySQL’s device write amplification falls to ~1.2 at 80% device utilization after filesystem-level FDP hints separate its components.The paper reports extended SSD lifetime without a specialized DBMS-managed translation layer with exclusive SSD ownership.Source: §1 · p. 2
  • The study characterizes enterprise FDP SSD isolation and QoS, then evaluates data placement in RocksDB and MySQL across single- and multi-tenant settings.It reports improvements in application throughput and latency alongside lower device WAF and extended SSD lifetime.Source: §1 · p. 2

2 BACKGROUND AND MOTIVATION

FDP adds host-selected placement tags while retaining conventional block-I/O compatibility and leaving garbage collection under device control. Linux supports both raw I/O Passthru and filesystem lifetime hints, and the paper studies whether storage-system architecture can guide placement.

2.1 FDP In a Nutshell

  • FDP groups NAND flash blocks into Reclaim Units and lets hosts tag writes with Reclaim Unit Handles, while the SSD retains control of their mapping.Writes without a specified handle use a default handle, and standard NVMe admin commands expose device placement topology.Source: §2.1 · p. 2

2.2 FDP Linux Support

  • Linux offers raw I/O Passthru with per-LBA placement hints and filesystem lifetime hints mapped to Reclaim Unit Handles.Filesystem hints propagate from an inode and therefore place data at file granularity; the raw path bypasses VFS and block layers.Source: §2.2 · pp. 2–3, Figure 1 · p. 3

2.3 The Flexible Data Placement Opportunity

  • The paper evaluates whether architectural data separation in RocksDB and MySQL can be used for FDP placement across two distinct storage-system classes.The opportunity rests on systems having components with different lifetimes and write patterns that conventional storage interfaces do not expose to the device.Source: §2.3 · p. 3

2. B-tree based relational database systems (§ 2.3.2)

The section motivates FDP data placement through contrasting RocksDB and MySQL write patterns and explains why device WAF matters for SSD lifetime, QoS, and capacity costs.

2.3.1 Key-Value Stores using LSM tree-based architectures (RocksDB).

  • RocksDB’s WAL and SST files are written sequentially, but the storage interface mixes their data and LSM levels; lower levels are smaller and churn more rapidly.Measured SST lifetimes were relatively short and uniform in L0–L3, diverged from L4 onward, and indicate a placement boundary at L4 across workloads.Source: §2.3.1 · p. 3

2.3.2 B-Tree based Relational Database Systems (MySQL).

  • The study tests whether separating RocksDB’s WAL and LSM levels through filesystem and I/O Passthru interfaces reduces device WAF at high utilization.MySQL’s existing file categories are assigned distinct lifetime hints through the filesystem interface with 31 lines of code changed.Source: §2.3.1 · p. 3, §2.3.2 · p. 4
  • MySQL InnoDB combines random in-place table updates with sequential redo logging and doublewrite buffering, alongside short-lived temporary files and metadata.User-table and doublewrite-buffer files account for 44.9% and 43.7% of host writes, respectively; metadata and logs account for 7.0% and 4.3%, while temporary files contribute less than 0.1%.Source: §2.3.2 · pp. 3–4

2.3 B-tree based relational database systems (§ 2.3.2)

  • Device WAF affects NAND SSD endurance and QoS; even modest increases can reduce lifetime, while capacity overprovisioning is less attractive amid supply constraints and rising NAND costs.The passage also links lower WAF with sustainability, performance stability, power consumption, and fewer garbage-collection-related latency spikes.Source: §2.3.3 · p. 4

3 THE FDP PROMISE OF THE RAW DEVICE

Raw-device FIO experiments evaluate FDP isolation, WAF, IOPS, and latency across concurrent workloads and placement conditions. Across the tests, RUH isolation generally lowers WAF and interference, with benefits varying by workload mix and access pattern.

3.1 RUH Isolation With Sequential Workloads

  • Under uneven partitioning, Conv WAF rises to ~3 while FDP stays near ~1; FDP cuts average latency 70%, tail latency ~77%, and raises IOPS 3.3×.With even partitioning, both modes have WAF ~1 and similar latency and IOPS.Source: §3.1 · pp. 4–5, Figure 4 · p. 5

3.2 RUH Isolation With Shared Sequential Workloads

  • At 64 workloads, FDP WAF peaks at 1.83 versus Conv’s 4.60, while FDP retains 1.9× IOPS (19.9 K vs. 10.5 K).FDP also lowers average write latency at every sharing ratio, including roughly 47% at the 1:8 ratio.Source: Figure 5 · p. 5, §3.2 · p. 5

3.3 RUH Isolation With Mixed Random and Sequential Workloads

  • At 4S4R, FDP WAF is ~1.59 versus Conv’s ~3.49, a 54% reduction; at 8R, the advantage falls to about 9%.FDP’s benefit diminishes as random writes increase GC pressure and reduce spatial locality.Source: Figure 6 · p. 6, §3.3 · p. 6

3.4 RUH Isolation With Mixed Hot and Cold Data

  • With hot/cold data, FDP WAF stays around 1.0 through 90% prefill and reaches 2.43 at 99% utilization; at high utilization it reduces WAF by up to 44%.Without FDP, WAF reaches 1.81 at 90% and 2.51 at 99%.Source: Figure 7 · p. 6, §3.4 · p. 6

3.5 RUH Isolation Under Skewed Accesses

  • At skew=1.4 with an 80/20 LBA split, FDP WAF is 1.25 versus Conv’s 1.77; FDP’s benefit grows as skew increases.Across both modes, WAF rises as more LBA space goes to the uniform-random workload.Source: Figure 8 · p. 6, §3.5 · pp. 6–7

3.6 FDP Data Placement Guidelines

  • Placement groups should separate streams by invalidation behavior, locality, working-set size, and write rate, merging similar classes rather than assigning every writer its own RUH.The guidelines recommend sharing RUHs among compatible streams and using the minimum number of groups that preserves distinct invalidation behavior.Source: §3.6 · p. 7

4 ROCKSDB DATA PLACEMENT

RocksDB’s WAL and SST levels have distinct lifetimes, so the paper maps them to a small set of FDP placement groups through filesystem hints or a raw-device path. The two paths preserve the same placement policy while differing in how they reach the SSD.

4 ROCKSDB DATA PLACEMENT

  • RocksDB placement uses four RUHs for the WAL, L4, L5 and deeper levels, and L0–L3, with filesystem hints or a raw-character-device path.The raw-device path uses stable io_uring_cmd; the filesystem path uses experimental VFS patches.Source: §4 · p. 7

4.1 Data Placement Using Filesystem Hints

  • Filesystem hints assign the WAL to SHORT, L0–L3 to MEDIUM, L4 to LONG, and L5+ to EXTREME, requiring about 25 lines of RocksDB changes.Flush and compaction output files receive the hint for their destination level.Source: §4.1 · p. 7, Table 2 · p. 7

4.2 Data Placement Using I/O Passthru (Filesystem Bypass) - TorFS

  • TorFS implements the RocksDB FileSystem API and sends writes with RUH placement identifiers through io_uring_cmd, bypassing VFS, filesystem, and block layers.It uses xNVMe and comprises approximately 4,000 lines of C++ code.Source: §4.2 · pp. 7–8, Figure 9 · p. 7

5 MYSQL DATA PLACEMENT

MySQL’s redo log, doublewrite buffer, and buffer-manager writes have distinct patterns that the paper separates using VFS lifetime hints. The integration assigns hints by reuse pattern and data lifetime, but does not implement an I/O Passthru path.

5 MYSQL DATA PLACEMENT

  • InnoDB produces sequential redo-log and doublewrite-buffer writes, alongside random in-place buffer-manager writes to user tablespaces.The doublewrite buffer cycles over a smaller region and is reused more frequently than the redo log.Source: §5 · p. 8
  • MySQL assigns SHORT to doublewrite buffers, MEDIUM to redo logs, and EXTREME to user tablespaces; temporary files receive LONG and metadata files no hint.The relative temperature labels are used to separate data and are not semantically important.Source: §5 · p. 8
  • The paper uses only VFS integration for MySQL because InnoDB relies on filesystem semantics for mutable tablespaces, logs, temporary files, and doublewrite buffers.An I/O Passthru implementation was impractical within the paper’s scope.Source: §5 · p. 8

6 EVALUATION

Across RocksDB and MySQL workloads, FDP lowers write amplification, with benefits generally strongest before near-saturation GC pressure dominates. Placement remains effective across filesystems and multiple tenants, while some outcomes depend on workload and configuration.

6.1.1 YCSB-A (Update-Intensive).

  • In the RocksDB YCSB-A 50/50 workload, FDP-XFS and TorFS reach WAF near 1 while Conv rises to about 1.39; FDP-XFS also lowers p99.99 read and update latency from 44,383 and 45,791 µs to 32,959 and 35,775 µs.The 20/80 workload shows consistent behavior, and TorFS further lowers p99.99 read and update latency by 31% and 22% over Conv.Source: §6.1.1 · p. 9, Figure 10 · p. 9, Figure 11 · p. 9

6.1.2 Modified YCSB-E (Insert-Intensive).

  • At 80% utilization, insert-heavy RocksDB reaches WAF 2.09 under Conv-XFS versus 1.25 under FDP-XFS and 1.20 under TorFS; near 92.6%, all reach about 2.9.Beyond roughly 85% utilization, free-space shortage causes sharp degradation and placement alone cannot contain amplification.Source: §6.1.2 · p. 9, Figure 12 · p. 9

6.1.3 SSD Utilization Stress Test.

  • At 70–80% utilization, FDP-XFS and TorFS reduce RocksDB WAF by 33% and 37% over Conv-XFS (1.91 to 1.28 and 1.20), with the gap narrowing beyond 90%.In the TPC-C runs, FDP also improves average TpmC by 9.7% at 6000 warehouses and 13.7% at 12000 warehouses.Source: §6.1.3 · p. 10, Figure 13 · p. 10, §6.2.2 · p. 11

6.1.4 db_bench Workloads.

  • On RocksDB db_bench fillrandom and overwrite workloads, TorFS sustains WAF near 1.05, while Conv-XFS rises to about 1.5–1.6 under fillrandom; TorFS outperforms FDP-XFS on both.FDP’s WAF benefit is filesystem-neutral in the reported comparison, reducing steady-state WAF by 26.0% on ext4 and 27.3% on XFS.Source: §6.1.4 · p. 10, Figure 14 · p. 10, §6.1.5 · p. 10

6.1.6 Multi-Tenant RocksDB Deployment.

  • Across two and four RocksDB instances, FDP achieves steady-state WAF 1.07 and 1.08 versus Conv’s 1.48 and 1.40, reductions of 28% and 23%.Average read and update latencies remain comparable, and sharing RUHs across containers preserves placement by data lifetime.Source: §6.1.6 · pp. 10–11, Figure 16 · p. 10

6.2.1 File Classification Strategy.

  • Policy 3 lowers MySQL WAF from 1.80 to 1.24 (31.1%) using three RUHs, separating user tables and double-write buffers while leaving other files at the default.Policies 1–3 have the same reported WAF of 1.24; adding RUHs for logs, temporary tables, and metadata shows no observable benefit.Source: §6.2.1 · p. 11, Table 3 · p. 11

6.2.3 Filesystem Sensitivity on MySQL.

  • FDP reduces MySQL WAF by 29.6% on ext4 and 14.9% on XFS, with the larger ext4 advantage emerging mainly at higher utilization.The discard mount option has little effect: final WAF differs by less than 2% between discard settings, and FDP reductions are 29.3% with discard and 29.6% without.Source: §6.2.3 · pp. 11–12, Figure 19 · p. 12, §6.2.4 · p. 12

6.2.5 Multi-Tenant MySQL Deployment.

  • In MySQL multi-tenant runs, FDP lowers WAF by 28.7% with two tenants and 27.9% with four, while host write bandwidth rises by 12.7% and 20.0%.Both configurations have the same aggregate database size and reach about 70% SSD utilization.Source: §6.2.5 · p. 12, Figure 21 · p. 12

7 CONCLUSION

The paper concludes that FDP can reduce write amplification in RocksDB and MySQL through lifetime-aware placement using existing Linux interfaces. Its appendix also relates natural access skew to WAF and contrasts that effect with FDP-enforced separation.

7 CONCLUSION

  • Across synthetic and real-world workloads, FDP delivers near-ideal RocksDB WAF and reduces MySQL WAF by 10–15% at high utilization in single- and multi-tenant settings.The paper maps RocksDB SST and WAL files and MySQL data, logs, and metadata to placement streams using filesystem hints or I/O Passthru.Source: §7 · p. 12
  • FDP’s backward compatibility and non-invasive software changes support practical data separation in widely deployed storage systems.The paper identifies stable Linux I/O Passthru support and emerging filesystem lifetime hints as deployment paths.Source: §7 · p. 12

A.1 Skew vs WAF

  • At skew ≥1.2, WAF remains near 1.0, whereas skew ≤0.6 reaches about 4.2 after 15–20 TiB of user writes; FDP can enforce hot/cold separation when natural locality is insufficient.At skew 0.8 and 1.0, WAF saturates at about 1.6 and 1.2, respectively.Source: §A.1 · p. 14, Figure 22 · p. 14

A.2 Homogeneous vs. Heterogeneous RUH Placement

RUH assignment quality determines how well FDP maintains isolation as concurrency grows. Across the tested device capacities, homogeneous placement reduces WAF relative to a conventional baseline.

A.2 Homogeneous vs. Heterogeneous RUH Placement

  • Across 24–64 jobs, homogeneous U-FDP maintains lower WAF and more stable tail latency than heterogeneous H-FDP and Conv; H-FDP’s advantage over Conv disappears by 48 jobs.At WL64, improper stream colocation largely nullifies FDP’s advantage, while homogeneous placement continues to limit WAF growth and maintain predictable latency.Source: §A.2 · p. 15

A.3 Scalability of FDP Across Device Sizes

  • At peak load, U-FDP reduces WAF by 49% on the 3.84 TB device, 40% on the 1.88 TB device, and 59% on the 7.52 TB device.The larger address space limits interference but does not eliminate the benefit of explicit placement.Source: §A.3 · p. 15
  • For the 3.84 TB device, Conv WAF rises from 2.99 to 3.97 as concurrency increases, while U-FDP mitigates the pressure through explicit placement.On the 1.88 TB device, Conv WAF stays at 2.99–3.00 across concurrency levels, yet U-FDP still reduces WAF by 40%.Source: §A.3 · p. 15

A.4 Multiple Containers with Similar Filesystem Workloads

With mixed-lifetime workloads, FDP reduces WAF across four Linux filesystems and improves tail latency or IOPS in the reported comparisons. The largest reported throughput gain occurs with XFS, whose allocation groups align with the placement model.

A.4 Multiple Containers with Similar Filesystem Workloads

  • Across EXT4, XFS, Btrfs, and F2FS, FDP stabilizes WAF near 1.0 while Conv rises to about 1.8 as the device fills.The experiment uses four concurrent lifetime classes with distinct write hints after SSD preconditioning.Source: §A.4 · p. 15, Table 4 · p. 15, Figure 25 · p. 16
  • With FDP, P99.99 latency falls by 34–51% on EXT4, Btrfs, and F2FS while IOPS remain similar; XFS reaches about 83 K IOPS versus 47 K under Conv.The authors attribute XFS’s larger gains to its allocation-group architecture aligning with FDP placement granularity.Source: §A.4 · pp. 15–16, Figure 26 · p. 16
  • FDP restores efficient placement by data lifetime across the tested filesystems, with gains particularly pronounced when filesystem allocation structures align with stream isolation.The paper identifies filesystem layout compatibility with device-level stream isolation as relevant to the observed results.Source: §A.4 · p. 16

B ROCKSDB EVALUATION (CONTD.)

FDP maintains comparable tail latency under update-heavy and high-utilization workloads, while improving extreme-tail READ and UPDATE latency at high device utilization. In the 20/80 workload, it also reduces WAF without an observable latency penalty.

B.1 Update-Intensive Workloads with Different read-update Ratio

  • For the 20/80 update-heavy workload, FDP reduces WAF from 1.45 to 1.05 without observable latency penalty; all configurations have comparable latency across measured percentiles.The comparison covers READ and UPDATE operations at the 95th, 99th, 99.9th, and 99.99th percentiles.Source: Figure 27 · p. 16, §B.1 · p. 17

B.2 Tail Latency Under the SSD Utilization Stress Test

  • At p99.99, FDP-XFS reduces READ latency 26.9% (50 591 µs) and TorFS 25.9% (51 263 µs), versus Conv-XFS at 69 183 µs.These extreme-tail results are reported for YCSB Workload A at high device utilization.Source: Figure 28 · p. 17, §B.2 · p. 17
  • At p99.99, FDP-XFS reduces UPDATE latency 20.7% (56,223 µs) and TorFS 18.4% (57 887 µs), versus Conv-XFS at 70 911 µs.The passage attributes the extreme-tail improvements to cleaner placement within each RU, reducing the frequency and severity of GC-related I/O stalls.Source: Figure 28 · p. 17, §B.2 · p. 17
  • Under high-utilization YCSB Workload A, all three approaches show broadly comparable READ and UPDATE latency at p95, p99, and p99.9.At p99.99, FDP-XFS and TorFS show lower READ and UPDATE latency than Conv-XFS.Source: Figure 28 · p. 17, §B.2 · p. 17

C TORFS VS UFROP

TorFS is a RocksDB plugin for FDP SSDs that bypasses the kernel file system and uses variable-length segments instead of UFROP’s fixed-size blocks. In db_bench, it improves write-heavy performance and space efficiency relative to UFROP.

Design motivation

  • Direct application access avoids the VFS, file-system, and block-layer path, whose software-stack overhead can bottleneck performance.The plugins issue direct I/O using native device semantics.Source: C TORFS VS UFROP · p. 17

Design comparison

  • UFROP separates RocksDB data into streams and bypasses the kernel file system, but fixed-size blocks can waste space and limit maximum file size; TorFS uses variable-length segments.TorFS’s segments match actual file size, avoiding internal fragmentation without an intrinsic file-size limit.Source: C TORFS VS UFROP · p. 17

Performance

  • TorFS raises sequential-write throughput 47% and update throughput 14% over UFROP; random-write and random-read throughput improve 18% and 25%, respectively.Sequential-read performance is nearly identical, with TorFS improving it by 3%; TorFS batches I/O submissions, unlike UFROP’s individual submissions.Source: Figure 29 · p. 17, C TORFS VS UFROP · p. 17

Space efficiency

  • TorFS uses 214 MB of physical space versus UFROP’s 518 MB, a 2.4× difference, and its space use stabilizes at steady state.UFROP’s space use continues growing as fragmented blocks accumulate; TorFS’s segment-based allocation eliminates internal fragmentation.Source: Figure 29 · p. 17, C TORFS VS UFROP · p. 17
Loading 2610.02676v2…