Source-linked AI summary
PredictIO: Decoupling Workload Prediction from OS-Controlled Storage Prefetching
Jun Lu, Yancan Mao, Zhenya Ma, Sheng Yue, Yongjian Fu, Ju Ren
TL;DR
Offset-based readahead misses workload structure that can predict future accesses, while existing workload-aware approaches often tie prediction and execution to particular applications or systems. PredictIO lets user-space policies express predictions as common page-range requests for bounded, OS-controlled execution. It reduces latency across application launch, SQLite scans, and LLM loading, with benefits also reported across storage devices.
Problem
General-purpose readahead relies on recent file-offset locality, but launch, database scans, and LLM loading expose workload structure that can predict accesses beyond that signal.
Source: §1 · p. 1, §2.1 · p. 3Method
PredictIO delivers kernel-observed accesses and application hints to user-space policies, which submit common page-range requests to a bounded kernel executor.
Source: §1 · p. 2, §3 · p. 4, §3.3 · p. 5Results
PredictIO reduces application cold-start latency by 5.3–26.3%, accelerates fragmented SQLite scans up to 2.99×, and reduces LLM loading latency by 8.8–47.9%.
Source: §1 · p. 2Takeaways & Limitations
Workload semantics can improve prefetching without embedding specialized policies in the kernel or moving page-cache management out of the OS.
Source: §8 · p. 13
Abstract
from arXiv · showhide
Storage stalls remain a significant source of application latency even on fast storage devices. Prefetching can hide part of this latency by fetching data before it is demanded, but general-purpose operating-system mechanisms typically infer future accesses from recent file offsets. This approach is ineffective when accesses are determined by workload structure, such as recurring sparse accesses during application launch, logical traversal in database scans, or model-layout metadata in LLM runtimes. Existing workload-aware prefetchers usually embed prediction logic inside individual applications and couple execution to application-level I/O paths, making policies difficult to reuse across workloads and systems. We present PredictIO, a programmable interface that separates workload prediction from prefetch execution. PredictIO delivers kernel-observed accesses and application hints to user-space policies, which express predictions as uniform page-range requests. A kernel-space executor validates, bounds, and filters these requests against cached and in-flight pages before issuing asynchronous prefetch I/O. We implement PredictIO for application launch, SQLite range scans, and LLM model loading. Across these workloads, PredictIO reduces latency by 5.3-26.3%, achieves up to 2.99x speedup for SQLite scans, and reduces LLM loading latency by 8.8-47.9%. We further show that a common prediction policy can be reused across SQLite and LMDB through lightweight adapters, that an application-integrated RocksDB readahead policy can be migrated while preserving its workload selectivity, and that the execution backend determines how much benefit migration provides for different access patterns and storage devices.
1 Introduction
Offset-based readahead misses workload structure, while workload-aware prefetching often ties prediction and execution to particular applications or systems. PredictIO separates user-space prediction from bounded, OS-controlled execution through a common page-range interface.
1 Introduction
- PredictIO addresses reuse gaps in workload-specific prediction and system-specific execution with an OS abstraction separating prediction from prefetch execution.User-space policies emit target files and bounded page ranges; the OS validates, schedules, and executes predictions.Source: §1 · pp. 1–2
- Three policies use launch access history, SQLite B-tree traversal hints, or LLM tensor-layout metadata while sharing the same page-range requests and bounded kernel backend.The policies vary in their event sources, prediction logic, and degree of application involvement.Source: §1 · p. 2
- PredictIO reduces application cold-start latency by 5.3–26.3%, accelerates fragmented SQLite range scans up to 2.99×, and cuts LLM loading latency by 8.8–47.9%.The reported benefits persist across NVMe SSDs, SATA SSDs, and HDDs.Source: §1 · p. 2
- PredictIO reuses a common prediction policy across SQLite and LMDB and migrates RocksDB readahead while preserving workload selectivity; migration benefits depend on the execution backend.The backend's benefit varies with access patterns and storage devices.Source: §1 · p. 2
2 Background and Motivation
Linux readahead relies on recent file-offset locality, but workload structure can expose useful prediction signals that offsets do not capture. Because those signals differ across workloads, prediction and execution are best treated as separate responsibilities.
2.1 Readahead Sees Locality, Not Structure
- In a controlled benchmark of 4,096 page-sized reads, longer contiguous runs improve readahead coverage, but jumps leave early pages on the demand path; fully sequential access nears complete coverage.Figure 1 contrasts short-run and sequential access; jumping patterns retain high P95 demand-read latency.Source: §2.1 · p. 2, Figure 1 · p. 3
- Launch and database scans can be structured and repeatable yet lack contiguous runs, so file-offset observations hide patterns visible to applications or runtimes.Launch sequences are sparse; B-tree logical neighbors can be physically distant.Source: §2.1 · pp. 2–3
2.2 Why One Predictor Cannot Fit All
- Different workloads require different prediction signals: launch offers recurring sparse page sequences, while SQLite traversal exposes logical relationships between physically distant pages.Recurrence suits history-based prediction; upcoming B-tree children require application knowledge.Source: §2.2 · p. 3, Figure 2 · p. 3
- During LLM loading, major faults remain unchanged while filesystem inputs and voluntary context switches fall by 99.6% and 97.8%, respectively.The passages attribute this pattern to faults no longer waiting for storage because required pages are already cached; tensor-layout metadata provides the advance signal.Source: §2.2 · p. 3
2.3 Separating Prediction from Execution
- User-space policies can use workload structure and evolve independently, while the OS can coordinate speculative reads, suppress cached or in-flight duplicates, and bound work.PredictIO applies this policy–mechanism split so diverse predictors share a common execution substrate.Source: §2.3 · pp. 3–4
3 Design
PredictIO separates workload-specific prediction from shared, OS-controlled prefetch execution: events flow to user-space policies, which return page-range requests to a kernel backend.
3 Design
- The event path routes kernel observations or application hints to user-space policies, while the request path carries their predicted page ranges to a shared backend.This separates workload-specific prediction from common prefetch execution.Source: §3 · p. 4, Figure 3 · p. 5
3.1 Design Goals
- Design goals include heterogeneous signals, user-space policies, page-cache-aware kernel execution, and bounded speculative work.The bounds limit queued requests, request sizes, and in-flight prefetches.Source: §3.1 · p. 4
3.2 Workload Event Sources
- Kernel observations support unmodified applications, while application hints expose workload structure such as upcoming B-tree children or tensor layouts.Both sources deliver workload events to the runtime through adapters.Source: §3.2 · p. 4
3.3 User-Space Prediction Policies
- Policies process typed events and emit a common request format identifying a target file and predicted page offsets or ranges.Policies determine what to prefetch but do not manage page-cache state or issue storage I/O.Source: §3.3 · p. 5
3.4 Kernel-Space Execution Backend
- The kernel backend queues bounded work asynchronously and skips pages already resident or in flight before issuing reads.Queue and in-flight limits bound speculative work; late or omitted predictions fall back to normal demand paging.Source: §3.4 · p. 5
4 Policy Instantiations
PredictIO applies a common event and request interface to three forms of workload predictability: recurring launch accesses, B-tree traversal hints, and LLM tensor-layout metadata.
4.1 History-Based Application Launch
- A per-file first-order Markov model aggregates launch access sequences across multiple runs, then a bounded walk predicts unseen pages without modifying the application.The walk starts from an observed page, suppresses repeats, and submits a sorted sparse page list.Source: §4.1 · pp. 5–6
4.2 B-Tree Traversal Range Scans
- For B-tree scans, a TreeChildEnter hint exposes upcoming child pages; the policy selects a bounded window and suppresses recent duplicate predictions.The policy is database-engine independent when an equivalent traversal hint is available.Source: §4.2 · p. 6
4.3 LLM Tensor Access
- For LLM loading, a lightweight runtime adapter forwards tensor-layout metadata that the policy maps into bounded file-page ranges.The policy uses static layout metadata and requires no model-format or kernel-execution-path changes.Source: §4.3 · pp. 6–7
5 Implementation
The Linux prototype combines a user-space runtime, event-source adapters, and a shared asynchronous kernel backend to execute prefetch requests.
5 Implementation
- The Linux prototype includes 269 lines of eBPF observation code, a 2,314-line kernel module, and a Rust runtime with approximately 6,900 non-comment source lines.The implementation also integrates LMDB and RocksDB alongside SQLite and llama.cpp.Source: §5 · p. 7
5.1 Event Observation and Application Adapters
- eBPF probes report file-access events for unmodified applications, while lightweight adapters forward database traversal and LLM tensor-layout or layer-start hints.Events travel to the runtime through a BPF ring buffer or Unix-domain datagram sockets, respectively.Source: §5.1 · pp. 7–8
5.2 Runtime and Policy Architecture
- The Rust runtime dispatches typed events to matching policies, then normalizes their requests into bounded chunks sent through a shared-memory ring to the kernel.When the ring is full, the backend drops the affected chunk rather than blocking the application.Source: §5.2 · p. 8
5.3 Asynchronous Prefetch Execution
- Kernel workers asynchronously issue reads while skipping cached, in-flight, beyond-end-of-file, and sparse-hole pages.Queue occupancy and in-flight prefetch pages are limited; optional batching coalesces physically consecutive pages.Source: §5.3 · p. 8
6 Evaluation
PredictIO’s evaluation tests control-path overhead, end-to-end performance, policy reuse and migration, and sensitivity to storage devices. It finds benefits across workload families, with results varying by workload and access pattern.
6 Evaluation
- The evaluation uses cold-cache trials and separates synthetic control-path microbenchmarks from end-to-end runs of launch, SQLite, LLM, LMDB, and RocksDB workloads.The testbed is a Linux 6.6.138 system with an NVMe SSD; results generally report medians across three to five trials.Source: §6 · p. 8
6.1 Control-Path Overhead
- eBPF events reach about 5 M events/s and application hints about 300–500 K events/s; runtime dispatch handles 6.3–8.8 M matching events/s.Kernel request processing is tens of ns/page for launch and SQLite, and below 1 ns/page for LLM loading when amortized over its predicted pages.Source: §6.1 · p. 9, Figure 8 · p. 9, Figure 9 · p. 9
6.2 End-to-End Performance
- 5.3–26.3% lower mean launch latency and up to 2.99× SQLite scan speedup show gains across two end-to-end workload families.The launch result is reported across five applications; SQLite’s 2.99× speedup occurs at the smallest YCSB-style scan.Source: §6.2 · p. 10, Figure 10 · p. 10, Figure 11 · p. 10
- 8.8–11.5% lower latency with mmap and 17.7–47.9% lower latency without mmap show LLM loading benefits across the tested models.Without mmap, the reported reductions are 17.7% for Q4 and 46.2–47.9% for the two Q8 models.Source: §6.2 · pp. 10–11, Figure 12 · p. 11
6.3 Policy Reuse and Migration
- The same tree-aware policy reduces latency by 34.3% for SQLite and 32.0% for LMDB, while RocksDB migration benefits vary by workload.RocksDB’s long sequential scan reaches 0.22 relative to Baseline 1.0, while YCSB-E improves only slightly and YCSB-C stays close to Baseline.Source: §6.3 · p. 11, Figure 13 · p. 11, Figure 14 · p. 11
- The migrated RocksDB policy preserves selectivity: it helps long sequential streams but stays close to Baseline for YCSB-C point reads.YCSB-E’s short scans also show little improvement because scans may finish before useful prefetches accumulate.Source: §6.3 · pp. 11–12
6.4 Storage-Device Sensitivity
- Across representative workloads, launch gains remain similar across SSDs, SQLite gains decline on slower devices, and LLM gains are small on HDDs.The evaluation attributes the differences to workload semantics and the device’s ability to overlap speculative and demand I/O.Source: §6.4 · p. 12
7 Related Work
Related work spans OS-level prefetching, semantic-aware approaches, and user-space delegation. These lines of work address prediction, cross-layer interfaces, or policy placement in different settings.
OS-Level Prefetching
- OS-level prefetching work explores prediction from past references, cache management, and adaptations to storage technologies, generally within specific OS or storage settings.The section contrasts prefetching with application-controlled caching, which targets what to keep rather than what to prefetch next.Source: §7 · p. 12
Semantic-Aware Prefetching
- Semantic-aware systems expose future accesses or richer cross-layer context, while other approaches target particular workloads such as LLM loading or mobile launches.Examples include CrossPrefetch, HintStor, PPC, FALCON, Frog, Lynx, and semantic locality.Source: §7 · p. 12
User-Space Delegation in Operating Systems
- User-space delegation supports programmable policies for scheduling, paging, caching, and prefetching; FUSE and SPDK illustrate that moving file-system and storage logic can be practical.The cited systems span policy prototyping and production use.Source: §7 · pp. 12–13
8 Conclusion
PredictIO separates workload-aware prediction from kernel-controlled execution. Its evaluated policies cover application launch, B-tree scans, and LLM loading, with reuse and migration demonstrated across storage systems.
8 Conclusion
- PredictIO separates workload-aware prediction from kernel-controlled execution, using user-space page-range predictions and an asynchronous kernel backend.The framework supports prediction from kernel-observed accesses or application hints, and the backend filters requests before execution.Source: §8 · p. 13