PaperlayerView exact revision on arXiv

Source-linked AI summary

Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving

Rana Abu Bakar

arXiv:2609.06853v1cs.CRcs.AI

TL;DR

The paper asks whether contention-related changes in KV-cache timing reliability reflect an interior overlap threshold or an ambient-versus-loaded regime change. Across seven studies, it measures timing reliability and request-level contention on shared serving systems, finding a vLLM collapse that also reproduces on distributed two-GPU vLLM.

Abstract

from arXiv · show

Shared key--value (KV) cache reuse improves large language model (LLM) serving, but it can also create a timing side channel that reveals whether a prefix is already cached. Previous work shows that such attacks are possible, but their reliability under realistic multi-tenant contention is less understood. We study this problem through seven experiments on live shared LLM-serving systems. On a vLLM server running DeepSeek-R1-Distill-Llama-8B on NVIDIA GB10, mean Cohen's d drops from 0.7789 with no synthetic workers to 0.2109 with two workers (t=8.412), while higher worker counts cause no statistically detectable further loss. A 120-run sparse-overlap experiment places the best breakpoint at the boundary of the measured range (tau=0, 95% CI [0.000,0.113]), supporting an ambient-versus-loaded regime change rather than an internal physical threshold. AUROC falls from 0.650 at ambient to 0.531 near 61% overlap and partially recovers to 0.574 at saturation. Concurrency-depth variance is the strongest measured correlate of effect size (r=-0.416) and hit consistency (r=-0.637). An interleaved control preserves the same non-monotonic ordering. The main collapse is also reproduced on a real two-node, two-GPU tensor-parallel vLLM setup, where mean d falls from 3.418 to 0.511 (p<0.01). Two SGLang pilots are statistically inconclusive. Overall, KV-cache timing reliability depends strongly on the load regime and serving stack, and measurements on quiet systems can overestimate operational attack reliability.

I. INTRODUCTION

KV-cache reuse can make cache state observable through timing, but the reliability of that signal under measured multi-tenant contention remains insufficiently characterized. This work studies load-regime dependence on live serving systems and compares its focus with prior attacks and defenses.

I. INTRODUCTION

  • The largest reliability loss occurs when sustained synthetic contention begins, with no detectable monotonic degradation at higher worker counts.The study characterizes this pattern on a live multi-user vLLM stack.Source: §I · p. 1

A. KV-cache timing leakage

  • Prefix caching skips prefill work for a shared prefix, while queueing, active sequences, batching, and prefill/decode interleaving make the timing signal load dependent.In vLLM, prefix caching operates over cache blocks and prefix context.Source: A. KV-cache timing leakage · p. 2

B. Prior attacks

  • PROMPTPEEK measures prompt-extraction completeness under a concurrent-user sweep, while EarlyBird [2] and InputSnatch [3] note noise but do not measure timing reliability as load changes.PROMPTPEEK reports extraction loss with GPU-memory-pressure-driven KV-cache eviction; the other attacks mitigate or filter temporal noise.Source: §I · p. 1, B. Prior attacks · p. 2

C. Defenses and serving systems

  • SafeKV [4] and KVGov [5] focus on defense and governance, leaving reliability curves under directly measured overlap or concurrency structure uncharacterized.The paper frames its narrower gap around statistical reliability, regime-change testing, and cross-deployment replication.Source: C. Defenses and serving systems · p. 2

III. RESEARCH QUESTIONS

The evaluation asks how sustained co-tenant activity affects cache-timing distinguishability, whether degradation follows load smoothly or by regime change, and how overlap, concurrency, and deployment affect the result. It uses a remote-timing adversary and a live shared vLLM target, with metrics spanning detection, repeated probes, and request-layer contention.

III. RESEARCH QUESTIONS

  • Six research questions test load effects, response shape, workload sensitivity, measured overlap, concurrency depth, and transfer across deployments and serving frameworks.The questions distinguish configured load from measured request overlap and ask whether the main result survives a two-GPU deployment.Source: §III · p. 2

A. Threat model

  • The adversary is a remote tenant who submits requests, measures response times, and repeats controlled before/after probes without privileged server, GPU, driver, or scheduler telemetry.Cache-state discrimination uses controlled probe material, not recovery of real users’ prompts.Source: A. Threat model · p. 2

B. Serving platform

  • The primary target is a live vLLM 0.28.0 server running DeepSeek-R1-Distill-Llama-8B on one NVIDIA GB10, with real production traffic present even in the zero-synthetic-worker condition.Study 7 adds a second GB10 node and two-node tensor-parallel deployments of vLLM and SGLang; probes record paired timings and synthetic request intervals.Source: B. Serving platform · pp. 2–3

C. Metrics

  • The study reports AUROC and threshold-based detection measures, repeated-probe majority success, request overlap, and per-probe concurrency-depth summaries.Overlap is a request-layer temporal measure, not GPU occupancy; depth summaries retain information lost by a binary contention indicator.Source: C. Metrics · p. 3

D. Change-point model

The change-point analysis tests whether timing-effect degradation has an interior overlap breakpoint, while the pilot first establishes whether introducing sustained synthetic work changes the signal. The pilot shows a large effect-size drop but cannot determine the response shape.

D. Change-point model

  • Study 5 fits two unconstrained linear segments, selects the minimum-SSE split with at least four points per segment, and bootstraps breakpoint and slope estimates.The authors do not claim an interior physical threshold unless the breakpoint confidence interval lies inside the observed range.Source: D. Change-point model · p. 3

A. Objective

  • The pilot tests whether a small sustained synthetic workload changes the cache-timing signal, using five independent runs per condition..Source: A. Objective · p. 3

B. Results

  • Mean effect size decreases from 1.126 to 0.213 when four synthetic workers are introduced.Source: B. Results · p. 4

C. Interpretation

  • The pilot suggests concurrent activity can destabilize the timing channel, but cannot distinguish smooth worker-count decline from a loss concentrated at contention onset.Study 2 is motivated by this unresolved response-shape question.Source: C. Interpretation · p. 4

A. Design

Study 2 finds that the largest effect-size change occurs when moving from no workers to a loaded condition; later worker-count increases produce much smaller, statistically indistinguishable changes. The evidence supports an ambient-versus-loaded transition, not progressive linear degradation or a localized physical threshold.

A. Design

  • Study 2 fixes synthetic request size and varies worker count, with each condition evaluated in 12 independent runs.Source: A. Design · p. 4

B. Effect-Size Results

  • The largest change occurs at the first loaded condition; later means decrease numerically, but by much less relative to within-condition variance.Each condition contains 12 independent runs.Source: B. Effect-Size Results · p. 4, A. Design · p. 4

C. Welch Tests

  • The zero-to-two-worker transition is large relative to sampling variance, while the two later transitions are not statistically distinguishable at conventional significance levels.The plotted 2-, 4-, and 8-worker means remain within one another’s confidence intervals; ratios are descriptive, with inference based on run-level comparisons.Source: C. Welch Tests · p. 4, Figure 1 · p. 4, B. Effect-Size Results · p. 4

D. Contention-Onset Regime Change Rather Than Linear Degradation

  • The pattern does not support progressively weaker timing signals as worker count rises; Study 2 cannot locate a physical threshold, and Study 5 supports an ambient-versus-loaded regime change instead.Worker count is a configuration knob, so the result is described as a contention-onset transition rather than an interior physical threshold.Source: D. Contention-Onset Regime Change… · p. 4

E. Why Worker Count Is Not a Physical Load Metric

Study 3 varies worker count and maximum generation length independently, finding low mean effect sizes across workload shapes but markedly greater variability for the single-fast-worker condition. Scheduler alignment or instantaneous concurrency depth is proposed as a possible explanation, not established by this study.

E. Why Worker Count Is Not a Physical Load Metric

  • Worker count is only an indirect proxy for instantaneous GPU work, which also depends on request duration, queueing, continuous batching, and ambient traffic.Study 3 independently varies worker count and maximum generation length across three configurations, with 12 runs each.Source: E. Why Worker Count Is Not a Physical… · pp. 4–5, A. Design · p. 5

A. Design

  • Study 3 changes worker count and maximum generation length independently to compare workload shapes.Source: A. Design · p. 5

B. Common Low-Mean Regime

  • All three workload configurations produce low mean effect sizes, consistent with Study 2’s collapsed regime and weakening the explanation that one worker count or token limit caused the result.Source: B. Common Low-Mean Regime · p. 5, Table V · p. 5

C. The Single-Fast-Worker Anomaly

  • The single-fast-worker condition is nearly an order of magnitude more variable and more often alternates between strong, weak, and reversed signals than a consistently weak signal.The key difference is variance, not mean.Source: C. The Single-Fast-Worker Anomaly · p. 5, Table V · p. 5

D. Mechanistic Hypothesis

  • The study hypothesizes that scheduler alignment or instantaneous concurrency depth drives the variance, as short-output workers may alternate activity near the probe cadence.Two continuously issuing workers could maintain a steadier loaded state, but this explanation is plausible rather than established by Study 3.Source: D. Mechanistic Hypothesis · p. 5

VIII. STUDY 4: DIRECT TEMPORAL-OVERLAP MEASUREMENT

Study 4 measures request-interval overlap directly, finding that sustained synthetic workloads saturate probe overlap. Across runs, the overlap-effect-size association largely separates ambient from loaded conditions, while overlap provides little ranking within the loaded cluster.

A. Motivation

  • Study 4 instruments request intervals to measure their overlap with the probe directly; each condition contains four runs.Source: A. Motivation · p. 5

C. Overlap Saturation

  • Measured overlap is near saturation across loaded conditions, leaving little dynamic range for comparing their overlap fractions.The study instruments request intervals because earlier studies inferred contention from configured worker count and request size.Source: C. Overlap Saturation · p. 5, A. Motivation · p. 5

D. Correlation with Effect Size

  • The observed association is largely the separation between ambient O = 0 and loaded high-overlap runs, with little ability to rank reliability within the loaded cluster.The figure shows the loaded cluster at O ≳0.76, while the 1w_small16 condition spans nearly the full observed effect-size range despite similar overlap to other loaded conditions.Source: D. Correlation with Effect Size · pp. 5–6, Figure 2 · p. 6

E. Why “GPU Occupancy” Would Be Incorrect

The section distinguishes request-layer concurrency measures from GPU occupancy and tests whether overlap metrics explain timing variation. It also describes the measured-overlap experiment used to examine these questions.

E. Why “GPU Occupancy” Would Be Incorrect

  • Request-layer intervals do not expose CUDA kernel residency or SM active cycles, and the metric differs from vLLM KV-cache utilization and running-request count.The authors recommend incorporating available server-side metrics in a follow-on experiment.Source: E. Why “GPU Occupancy” Would Be… · p. 6

F. Failure to Explain the Single-Worker Variance

  • Near-identical overlap fractions can mask sharply different timing behavior: one condition ranges from a large negative to a substantial positive effect, while another remains near the collapsed band.Both conditions spend nearly all probe time overlapped with at least one synthetic request.Source: F. Failure to Explain the… · p. 6

A. Motivation

  • Study 4 left open whether the collapse reflects an interior overlap threshold or an ambient-versus-loaded change, and whether concurrency depth explains more residual variance than overlap.Study 5 tests both questions using measured overlap.Source: A. Motivation · p. 6

B. Design

  • The sparse-overlap design calibrates 22 configurations to eight target bins, then collects 15 runs per bin using measured overlap, for 120 runs total.The ambient bin uses no synthetic generator.Source: B. Design · p. 6, Table VII · p. 7

C. Effect-Size and Attack-Reliability Results

Study 5 finds non-monotonic attack reliability across measured overlap and supports an ambient-versus-loaded regime change rather than an interior threshold. An interleaved control preserves the condition ordering, though temporal drift and fixed condition order remain qualifications.

C. Effect-Size and Attack-Reliability Results

  • AUROC and phit decline from ambient through the 60% bin, then partially recover at 80% and 100%; AUROC stays above chance, bottoming at 0.531.With odd probe budgets, Psucc rises monotonically in every bin; at 60%, it increases from 0.525 at k = 1 to 0.562 at k = 9.Source: C. Effect-Size and Attack-Reliability… · p. 7

D. Concurrency-Depth Analysis

  • Concurrency-depth variance correlates with Cohen’s d at r = −0.416 and is the strongest correlate of Cohen’s d and hit consistency; its overlap-adjusted d residual correlation is r = −0.329.The passages identify this as correlational evidence, not proof that burstiness causes instability.Source: Figure 3 · p. 7, D. Concurrency-Depth Analysis · p. 7

E. Segmented Regression: Threshold or Regime Change?

  • The best two-segment fit reaches R2 = 0.536 with τ = 0 and bootstrap 95% CI [0.000, 0.113], supporting an ambient-versus-loaded split rather than an interior threshold.The segment fits are independent, so the improvement partly reflects a level shift.Source: E. Segmented Regression: Threshold or… · p. 7, Figure 4 · p. 8

F. An Important Environmental Caveat

  • Study 5’s ambient bin has mean d ≈ 0.13–0.50 and AUROC 0.650, below earlier ambient mean d values up to 1.126; uncontrolled production traffic may explain the difference.The sequential 2.5-hour blocks also leave the mid-range dip and high-overlap recovery potentially confounded with time.Source: F. An Important Environmental Caveat · p. 8

A. Motivation

  • Study 6 probes ambient, 40%, and 100% conditions back-to-back in each of 15 rounds to separate within-condition temporal drift from between-condition contention effects.Study 5 had run its eight bins as sequential blocks.Source: A. Motivation · p. 8

B. Results

  • Interleaving preserves the ordering 0% > 100% > 40%; 100% has about half its sequential-block mean, while round trends are descriptive and lack significance tests.The 40% condition is approximately flat across rounds, unlike the negative trends reported for ambient and 100%.Source: B. Results · p. 8

C. Interpretation

The central vLLM collapse also appears on a two-node, two-GPU tensor-parallel deployment, while SGLang pilots leave cross-framework generality unresolved. The authors interpret the evidence as regime-dependent, without claiming a universal breakpoint or contended effect size.

C. Interpretation

  • Interleaving supports treating the dip-then-recovery pattern as more likely real than a sequential-block artifact, but fixed within-round order leaves a possible position effect.The order was 0%–40%–100%.Source: C. Interpretation · p. 8

XI. STUDY 7: CROSS-INSTANCE, CROSS-FRAMEWORK, AND MULTI-GPU REPLICATION

  • On two-node, two-GPU tensor-parallel vLLM, mean d falls from 3.418 to 0.511 under four background workers (p < 0.01).The SGLang pilots are not significant and used MPS while co-resident with production vLLM, so their ambient condition is not comparable.Source: Table XI · p. 9, §XI · p. 9

A. Replication Across Experimental Designs

  • Across seven studies, the work tests contention across worker counts and workload shapes, samples sparse measured overlap, checks ordering under interleaving, and replicates across deployments.The second GB10 node reproduces an ambient signal, and the two-GPU vLLM deployment reproduces the collapse.Source: A. Replication Across Experimental… · p. 9

B. A Two-Regime Reliability Model

  • The experiments support dA > dC, but identify neither a universal contended mean nor a universal breakpoint because both depend on the evaluated model, server, accelerator, and ambient workload.The Study 5 breakpoint confidence interval touches the edge of the measured overlap range.Source: B. A Two-Regime Reliability Model · p. 9

C. Why a Smooth Utilization Model Is Unsupported

Contention does not yield a smooth relationship between measured overlap and timing reliability. Concurrency-depth variability is more strongly associated with reliability than binary overlap, while scheduler-state asymmetry remains a proposed explanation for some pair-level variance.

C. Why a Smooth Utilization Model Is Unsupported

  • Measured overlap remains weakly correlated with effect size (r = −0.106), while most loaded runs have O ≳ 0.76 and little dynamic range.The intermediate region added in Study 5 does not support a simple linear overlap model.Source: C. Why a Smooth Utilization Model Is… · p. 10

A. Prefill Savings Versus Scheduler Variance

  • Prefix reuse primarily changes cache-dependent work, while contention can change scheduler and accelerator effects, inflating latency variability even as cache hits continue.This explains how the timing channel can remain physically present yet become statistically weak under contention.Source: A. Prefill Savings Versus Scheduler… · p. 10

B. Pair-Asymmetry Effects

  • Despite occupying the same low-mean regime, 1w_small16 has sd = 0.7112 versus 0.0727 for 2w_tiny4; scheduler-state asymmetry is only a hypothesis.Pair members can encounter different scheduler states, but the responsible event was not directly identified.Source: B. Pair-Asymmetry Effects · p. 10

C. Concurrency Depth as the Missing Request-Layer Variable

  • Binary overlap maps one or many simultaneous synthetic requests to the same state, discarding concurrency depth and burstiness.The overlap metric integrates only whether concurrency depth is greater than zero.Source: C. Concurrency Depth as the Missing… · p. 10
  • Concurrency-depth variance correlates more strongly with d (r = −0.416) and hit consistency (r = −0.637) than measured overlap (−0.106 and −0.504, respectively).It retains a residual correlation of −0.329 with d after regressing out overlap; these remain correlational results.Source: C. Concurrency Depth as the Missing… · p. 10

D. Why Reliability Partially Recovers at Saturation

Attack reliability falls at intermediate overlap and partially recovers at saturation, while quiet-system measurements can show a cleaner signal than live multi-user service. The results describe load-dependent reliability, not a security boundary, and the reported repeated-probe curve applies to a simple attacker.

D. Why Reliability Partially Recovers at Saturation

  • AUROC falls from 0.650 at ambient to 0.531 near 61% overlap, then rises to 0.574 at saturation; interleaving preserves the effect-size ordering 0% > 100% > 40%.The proposed scheduler-transition explanation is plausible but the measurements cannot distinguish it from host queueing or batch-composition effects.Source: D. Why Reliability Partially Recovers… · p. 10

A. Quiet-System Measurements Can Overstate Attack Reliability

  • Mean d falls from 0.7789 in ambient conditions to 0.2109 with the first sustained synthetic load; quiet-system results need not match live-service reliability.A single attack-success number is incomplete unless the serving-load regime is reported.Source: A. Quiet-System Measurements Can… · p. 11

B. Contention Is Not a Defense

  • Contention is not a security boundary: it is uncontrolled and may disappear during low demand, whereas cache isolation or a sharing policy prevents cross-tenant cache-hit signals.Artificial background work would consume capacity and degrade legitimate users.Source: B. Contention Is Not a Defense · p. 11

C. Probe Budget and Residual Exploitability

  • Near 61% overlap, phit = 0.525 yields Psucc(9) = 0.562, versus ambient phit = 0.695 and Psucc(9) = 0.895; this is a simple sign-only voter, not an exploitability upper bound.The channel remains above chance in every tested bin.Source: C. Probe Budget and Residual… · p. 11

D. Reporting Guidance

  • Report concurrent-request structure, prompt and output lengths, cache state, scheduler configuration, latency scope, probe budget, stack and topology, and whether load is controlled or ambient.Run-level distributions are preferable to means alone.Source: D. Reporting Guidance · p. 11

A. Internal Validity

  • Uncontrolled ambient traffic adds variance, and the interleaved control reduces but does not eliminate sequencing concerns because within-round order was fixed.A short-timescale position effect remains possible; randomizing all eight Study 5 bins within each block is proposed for future replication.Source: A. Internal Validity · p. 11

B. Construct Validity

The measured collapse reproduces on distributed two-GPU vLLM, but the study does not establish a universal threshold, framework generality, or a specific mechanism. Its scope and operational interpretation depend on request-layer measures, end-to-end timing, and the evaluated stacks.

B. Construct Validity

  • Worker count is a configuration knob, overlap is request-layer temporal coverage rather than GPU occupancy, and concurrency depth still does not expose internal scheduler state.Timings are end-to-end non-streaming completion, not direct server-side TTFT.Source: B. Construct Validity · p. 11

C. Statistical Conclusion Validity

  • Studies 1 and 4 have small per-condition samples, and correlations are descriptive rather than causal estimates.Study 5 uses 120 runs and 750 paired trials per overlap bin; failure to detect a difference is not evidence that higher-load distributions are equal.Source: C. Statistical Conclusion Validity · p. 11

D. External Validity

  • Numerical effects remain stack-specific; cross-node vLLM replication reduces the single-instance concern, but SGLang generality remains unresolved.The SGLang pilots are statistically inconclusive, and MPS co-residency prevents an equivalent low-contention baseline.Source: D. External Validity · p. 12

E. Reproducibility and Artifact Scope

  • The artifact records paired timings, run-level measures, intervals, and configuration metadata, with scripts that allow overlap and concurrency-depth summaries to be recomputed.Prefix-cache activity was verified from live metrics counters.Source: E. Reproducibility and Artifact Scope · p. 12

A. What the Data Establish

  • Studies 2–6 support an ambient-versus-loaded regime change, reject an interior overlap threshold, and identify concurrency-depth variance as the strongest request-layer correlate.Study 7 strengthens external validity through significant reproduction on distributed two-GPU vLLM; SGLang remains unresolved.Source: A. What the Data Establish · p. 12

B. What the Data Do Not Establish

  • The data establish neither a universal threshold nor that attacks become impossible, frameworks respond identically, or a specific GPU or scheduler mechanism is responsible.Concurrency-depth variance is correlational, and ambient production work is not instrumented; the SGLang pilots do not support an opposite-effect claim.Source: B. What the Data Do Not Establish · p. 12

C. Operational Interpretation

  • Measured side-channel severity can depend on when and how testing occurs; noisy daytime measurements do not prove cache sharing is safe.The vulnerability comes from cross-principal reuse, while contention changes the attacker’s observation quality.Source: C. Operational Interpretation · p. 12

XVII. CONCLUSION

  • On two-node, two-GPU tensor-parallel vLLM, mean d falls from 3.418 to 0.511 (p < 0.01), while SGLang pilots remain statistically inconclusive.The conclusion is stack-sensitive and does not establish that all serving frameworks respond identically.Source: §XVII · p. 12
Loading 2609.06853v1…