PaperlayerView exact revision on arXiv

Source-linked AI summary

PAGE404: Breaking GPU Isolation Through Hardware-assisted Page Migration

Ravan Nazaraliyev, Yicheng Zhang, Nael Abu-Ghazaleh

arXiv:2610.20962v1cs.CRcs.AR

TL;DR

GH200’s hardware-assisted page migration is designed to improve CPU–GPU memory locality, but its security under multi-tenant sharing has been unexplored. PAGE404 reverse-engineers the access-counter and driver mechanisms, then demonstrates leakage and covert channels across MIG partitions. The attacks achieve high-rate signaling and reveal workload and LLM prompt-length information.

  • Problem

    The security of hardware-assisted page migration under co-tenant sharing is unexplored, although these platforms may be shared by potentially mistrusting users.

    Source: §1 · p. 2

  • Method

    PAGE404 reverse-engineers access-counter hardware and UVM interrupt handling, then builds proof-of-concept side- and covert-channel attacks against their shared resources.

    Source: §1 · p. 2

  • Results

    Eviction signaling reaches 8500 bps at a 1.5% error rate; migration-delay signaling reaches 8100 bps at a 4% error rate, and LLM token-length estimates have a 4.6-token mean absolute error.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Shared access counters and driver handling make both leakage paths exploitable across MIG partitions, challenging MIG’s tenant-isolation guarantees.

    Source: §1 · p. 2

    The modified driver used to reverse-engineer the counter structure is a controlled experimental setup unavailable to a real attacker.

    Source: §4.3.1 · p. 9

Abstract

from arXiv · show

NVIDIA's Grace Hopper (GH200) superchip represents a new generation of heterogeneous computing, tightly integrating a 72-core Arm-based CPU with a Hopper GPU on a single module via a high-bandwidth, cache-coherent interconnect. By eliminating the conventional PCIe bottleneck for CPU-GPU data movement, this design establishes a unified memory domain that scales to the capacities demanded by generative AI and high-performance computing (HPC) workloads. The architecture also introduces hardware-assisted page migration: driven by access counters, the system dynamically promotes frequently accessed pages from host CPU memory to GPU DRAM to improve memory locality. We study the security of this emerging architecture against new side- and covert-channel attacks targeting its page migration support. We analyze counter-driven migration across both the hardware mechanisms and the UVM driver's interrupt-handling pipeline, and identify two side-channel vulnerabilities: one rooted in the shared structure of access counters, and the other in the driver's response to migration interrupts. We further show that both channels are exploitable across Multi-Instance GPU (MIG) partitions, a feature intended to provide strong, hardware-enforced isolation between co-located tenants. Using proof-of-concept attacks, we build covert channels that reach roughly 8,500 bits per second through eviction-based signaling and 8,100 bits per second through migration-delay-based signaling across MIG instances. The interrupt-based channel also fingerprints inference workloads across distinct large language models (LLMs) and CPU SPEC2017 benchmarks, and estimates the prompt length processed during LLM inference with high accuracy. These results expose a security gap in hardware-managed heterogeneous memory systems and call into question the isolation guarantees of current MIG-enabled multi-tenant GPU deployments.

1. Introduction

PAGE404 examines whether GH200’s hardware-assisted page migration creates side- and covert-channel risks for tenants sharing a GPU. It identifies two MIG-crossing leakage paths and demonstrates covert signaling and workload inference attacks.

1. Introduction

  • GH200 combines a Grace CPU and Hopper GPU with cache-coherent NVLink-C2C and uses access counters to promote frequently accessed host pages to GPU memory.The driver migrates the corresponding pages when a configurable counter threshold is reached.Source: §1 · p. 1
  • The authors reverse-engineer access counters and the UVM driver’s interrupt handling to characterize page-migration notifications and identify two previously unexplored vulnerabilities.The paper also demonstrates covert-channel attacks between tenants on separate MIG partitions.Source: §1 · pp. 1–2
  • The access-counter channel exploits a shared cache-like counter structure, while the second channel detects driver activity through its effect on migration response latency.Both target GPU-wide shared resources and are exploitable across MIG partitions.Source: §1 · p. 2
  • Eviction signaling reaches 8500 bps at a 1.5% error rate, while migration-delay signaling reaches 8100 bps at a 4% error rate; the latter estimates LLM token length with a 4.6-token mean absolute error.The eviction-based side channel also distinguishes 10 CUDA workloads with 94% accuracy when they use system-allocated memory; the migration-delay channel distinguishes CUDA and LLM workloads with around or above 90% accuracy.Source: §1 · p. 2

2. Background and Threat Model

The background describes heterogeneous memory-management approaches and GH200’s hardware-supported alternatives to conventional fault-driven transfers. The threat model assumes tenants on distinct MIG slices, with containers sharing the GPU driver.

2.1. Background

  • UVM, HMM, and GH200 address unified addressing and memory-placement needs as CPU and GPU workloads outgrow GPU memory, including LLM KV caches and large graphs.The passage identifies unified addressing, on-demand paging, and transparent oversubscription as desired capabilities.Source: §2.1 · p. 2
  • With PCIe ATS, a device requests host-side address translation on an ATC miss; GH200’s NVLink-C2C adds coherence, while access counters trigger driver-managed migration after a configurable threshold.The default access threshold is 256; unlike demand-fault migration, counter-triggered migration is a placement hint and does not stall the GPU thread.Source: §2.1 · pp. 3–4

2.2. Threat Model

  • The threat model gives each user a separate MIG slice and container, but the containers share nvidia.ko; the attacker needs no superuser privileges.The attacker performs experiments to reverse-engineer page-migration behavior and hardware access counters before attacking.Source: §2.2 · p. 4

3. Demystifying HW-assisted Page Migration

Reverse engineering characterizes GH200 page migration’s access counters and UVM driver pipeline. The findings show counter sharing across MIG instances and timing variability in interrupt handling that can leak information across tenants.

3.1. Page Migration Access Counters

  • A 32-byte cache line explains why 16-byte strides double the accesses needed to trigger migration, while 32-, 64-, and 128-byte strides require 256 accesses.Volatile loads bypass GPU caches when accessing system memory.Source: §3.1 · pp. 4–5, Figure 5 · p. 4
  • At approximately 360–380 distractor pages, migration probability degrades similarly across full-GH200 and separate-MIG configurations, showing the counter pool is shared across MIG boundaries.This means MIG process isolation does not extend to the page-migration hardware counters.Source: §3.1 · p. 5, Figure 6 · p. 5
  • 256 hardware access counters form 8 sets of 32; eviction removes the smallest-valued counter below 7, then falls back to LRU.The counter pool is exhausted at approximately 360–380 distractor pages in the single-process experiment.Source: §3.1 · pp. 5–7, Figure 8 · p. 6, Figure 10 · p. 7
  • Each counter is assigned by physical address to a 2 MB range, while accesses share that counter and migration targets the most recently accessed 64 KB region.Hardware-assisted migration operates at 64 KB granularity; the UVM prefetcher promotes pages within a 2 MB region using a tree-based policy.Source: §3.1 · pp. 4, 5, 7
  • Hardware counters index physical addresses, and their state persists across process teardown but resets when a process initializes an otherwise idle GPU.With a persistent GPU context, two launches accumulated 128 accesses each to trigger migration at the 256-access threshold.Source: §3.1 · p. 7

3.2. Migration: Driver

  • The driver handles migration notifications through sequential interrupt, work-queue, service, copy, and counter-reset stages, with shared fault work and locks adding delays.The bottom half shares a work queue with GPU page-fault handlers, and memory operations can hold locks needed by migration.Source: §3.2 · pp. 7–8, Figure 11 · p. 8
  • Migration delay measures accesses from the threshold-triggering remote access until NVLink transfer starts; idle-background delay is typically 20 to 30 system-memory accesses.The driver’s interrupt handling introduces variable delays that form an information-leakage vector across tenants.Source: §3.2 · p. 8

4. Proof-of-Concept Attacks

This section presents proof-of-concept attacks across separate MIG instances that exploit shared access-counter sets and contention in the shared interrupt-handling path, using eviction- and migration-delay signaling to fingerprint GPU, CPU, and LLM workloads and estimate processed token counts. The attacks also identify victim CUDA kernels with close to 95% success and estimate a victim’s access count within a 5% margin of error.

4.2. Attack Challenges

  • 17,000 samples per second with parallel accesses and 15,000 with non-blocking loads: parallel probing enables finer-grained sampling.To avoid UVM prefetching, the attacker touches pages at a 128 KB stride, leaving sibling 64 KB pages unoccupied; after migration, it advances to a fresh page because migrated pages cannot trigger another migration.Source: §4.2 · pp. 8–9

4.3.1. Access Counter Set-based Covert Channel.

  • 8,500 bps with one counter set and 15,500 bps with two: the eviction-based covert channel’s bandwidth nearly doubles but scales sublinearly.The receiver probes pages to approach the migration threshold while the sender evicts its counters; probing multiple pages increases interrupt-handling and migration contention, raising errors. The implementation uses 512 MB huge pages to construct eviction sets despite opaque physical addresses, a controlled driver-assisted reverse-engineering setup unavailable to real attackers.Source: §4.3.1 · pp. 9–10, Figure 13 · p. 10

4.3.2. Migration Delay Variation.

  • 8,100 bps with a 4% error rate: migration-delay signaling is unstable, succeeding in roughly half of runs.A sender creates software-level contention to delay the receiver’s migration; the contention level is difficult to control across runs.Source: §4.3.2 · pp. 10–11

4.4.2. Migration Delay Leakage: CUDA Applications.

  • Nearly 94% accuracy: migration-delay fingerprints identify which of 25 CUDA applications a victim is running.The spy collected 100 traces per application, with each collection lasting around 8 seconds and gathering 131,000 samples.Source: Table 1 · p. 11, §4.4.2 · p. 11

4.4.3. Migration Delay Leakage: SPEC2017 Benchmarks.

  • Up to 89% accuracy: migration-delay leakage identifies victim CPU workloads, including when the victim is not using the GPU.The best reported techniques were XGBoost, LightGBM, and the soft-voting ensemble; the attack exploits migration delays associated with CPU memory activity.Source: Table 2 · p. 11, §4.4.3 · p. 11, §4.4.1 · p. 11

4.4.4. Large Language Model Architecture Fingerprint.

  • Up to 94% accuracy identifies the victim’s LLM, while cross-provider classification reaches 72% accuracy.LightGBM achieved the reported 94% on 20 models; the cross-provider classifier was trained on CloudLab traces and evaluated on Lambda traces.Source: Table 3 · p. 12, §4.4.4 · p. 12, §4.4.5 · p. 12

4.4.6. Input-Output Prompt Length Estimation.

  • R2 = 0.92 with an MAE of 4.6 tokens: migration-delay traces estimate processed token counts from the tear-down signal.The attack locates the tear-down pattern and maps its onset to token count using a linear estimator; evaluation used a held-out 20% test split.Source: §4.4.6 · p. 13

4.4.7. Counter-Set Leakage: CUDA Kernel Fingerprint-

  • The counter-set fingerprinting experiment probes eight sets and records migration success at approximately 2,000 samples per set per second.It uses Polybench kernels from UVMBench and changes their allocation APIs to malloc so the victim exercises the counters; longer migration waits suppress migration-delay noise.Source: §4.4.7 · p. 13
  • A soft-voting ensemble identifies the victim’s executed CUDA kernel with close to 95% success.The evaluation uses 100K samples per set collected over 50 seconds.Source: Table 4 · p. 13, §4.4.7 · p. 13

4.4.8. Future Finer-Grained Attacks.

  • Shared counters for 64 KB pages within a 2 MB region let an attacker estimate victim access counts within a 5% margin of error.The attacker primes a co-located page’s persistent counter near the migration threshold, then probes in small increments and measures the access delta until migration occurs.Source: §4.4.8 · p. 13

5. Potential Mitigations

The vulnerabilities arise from shared access-counter hardware and shared driver code. The proposed mitigations partition counters across MIG instances and isolate driver performance, though vGPU support and manager-side leakage remain caveats.

5. Potential Mitigations

  • Shared counter hardware and access to driver code are identified as the roots of the vulnerabilities.The shared components enable contention and observable delays.Source: §5 · p. 14
  • Partitioning access counters into groups dedicated to each MIG partition is proposed to mitigate hardware contention.Source: §5 · p. 14
  • Driver performance isolation, potentially by MIG partition, is proposed to mitigate observable delays, but vGPU lacks ARM support and manager-side leakage needs experimental confirmation.Guest drivers are multiplexed by the host vGPU manager, where leakage may shift to observable delays.Source: §5 · p. 14

6. Related Work

Related work covers GPU microarchitectural leakage, cross-domain and interconnect attacks, GPU Rowhammer, and isolation between MIG instances. PAGE404 addresses leakage through hardware-assisted page migration on tightly integrated platforms.

6. Related Work

  • Intra-GPU attacks exploit shared caches,, memory buses, functional units and on-chip networks,, TLB state, and DRAM effects,.Source: §6 · p. 14
  • Cross-domain CPU–GPU attacks target shared PCIe and memory resources,, while multi-GPU attacks exploit interconnect contention,.The cited NVLink approach relies on bandwidth saturation, increasingly difficult on the GH200’s 900 GB/s NVLink-C2C [29].Source: §6 · p. 14
  • GPU Rowhammer work reports model-accuracy degradation, privilege escalation,, cross-component attacks, and exploitable bit flips on ECC-protected GPUs.Source: §6 · p. 14
  • Prior MIG work reports shared last-level TLB, cache-partition, uncore, and page-fault leakage; this work demonstrates a novel leakage form compromising MIG isolation.Source: §6 · p. 14

7. Conclusion

PAGE404 analyzes GH200 page migration and identifies leakage through counters shared across MIG instances and delays in the driver’s interrupt pipeline. Its proof-of-concept attacks achieve covert communication and workload inference, underscoring security concerns for multi-tenant environments.

7. Conclusion

  • 8500 bps at 1.5% error and 15500 bps at roughly 5% error were achieved by the eviction-based covert channel; its side channel distinguished 10 CUDA kernels with 94% accuracy.The kernel result applies when workloads use system-allocated memory.Source: §7 · p. 14
  • The counter structure is shared across MIG instances, while the driver’s interrupt-handling pipeline creates migration-delay leakage.Source: §7 · p. 14
  • The migration-delay covert channel achieved 8100 bps at 4% error, and side-channel attacks distinguished CPU, CUDA, and LLM workloads with around or above 90% accuracy.Source: §7 · p. 14
  • LLM processed-token length was estimated with a mean absolute error of 4.6 tokens on average.Source: §7 · p. 14

Appendix A. Supplementary Side Channel Data

The supplementary materials list feature-vector components, evaluation applications and models, prompt statistics, CPU benchmarks, and Polybench materials.

Appendix A. Supplementary Side Channel Data

Appendix B. Research Ethics Considerations

The authors conducted experiments on a machine they accessed exclusively and disclosed their findings to NVIDIA. The program committee prepared the meta-review as part of the 2027 IEEE S&P review process.

Appendix B. Research Ethics Considerations

  • All experiments were conducted on a machine to which the authors had exclusive access.The authors state this ensured no other users were affected or placed at risk.Source: Appendix B · p. 19
  • The authors disclosed their findings to NVIDIA.Source: Appendix B · p. 19

Appendix D. Meta-Review

  • The program committee prepared the meta-review for the 2027 IEEE Symposium on Security and Privacy.Source: Appendix D · p. 19
  • The meta-review was prepared as part of the review process detailed in the call for papers.Source: Appendix D · p. 19

D.1. Summary

The meta-review describes a newly discovered side channel in GH200's hardware-assisted page migration. It attributes the channel to access-counter hardware shared across MIG boundaries and notes attacks involving covert communication and LLM workload analysis.

D.1. Summary

  • The paper details a newly discovered side channel targeting page migration on NVIDIA Grace Hopper (GH200).Source: §D.1 · p. 19
  • The access-counter hardware structure that tracks accesses to trigger CPU-to-GPU page migration is shared across MIG boundaries.Source: §D.1 · p. 19
  • The authors exploit the shared structure to build a covert channel across MIG boundaries.Source: §D.1 · p. 19
  • The attacks include LLM fingerprinting and measuring LLM input and output lengths.Source: §D.1 · p. 19

D.2. Scientific Contributions

The meta-review identifies an impactful vulnerability and characterizes the paper as a valuable step forward in an established field.

D.2. Scientific Contributions

  • The program committee identifies an impactful vulnerability as a scientific contribution.Source: §D.2 · p. 19
  • The committee also identifies a valuable step forward in an established field.Source: §D.2 · p. 19
  • The listed scientific contributions are an impactful vulnerability and a step forward in an established field.Source: §D.2 · p. 19

D.3. Reasons for Acceptance

Reviewers viewed the work as a comprehensive study of GH200 page migration and its security implications across isolated instances. The meta-review describes it as seemingly the first security analysis of Grace Hopper and MIG isolation.

D.3. Reasons for Acceptance

  • Reviewers described this as seemingly the first security analysis of Nvidia’s Grace Hopper GPU and its MIG isolation primitive.Source: §D.3 · p. 19
  • Reviewers appreciated the comprehensive, in-depth study of hardware-assisted page migration in GH200.Source: §D.3 · p. 19
  • Reviewers valued the exposure of security implications from sharing a microarchitectural structure across isolated instances.Source: §D.3 · p. 19

D.4. Noteworthy Concerns

The authors note that stronger LLM attacks and evidence of generalization across system architectures would strengthen the work.

D.4. Noteworthy Concerns

  • The attacks do not recover LLM inputs or outputs or use recovered information in prompt-injection attacks.Source: §D.4 · p. 19
  • It remains unclear whether the attacks generalize across different system architectures.Source: §D.4 · p. 19

Related papers

Loading 2610.20962v1…