PaperlayerView exact revision on arXiv

Source-linked AI summary

Prioritizing Microservice Requests via Retries

Ofek Lavi, Jhonatan Tavori, Hanoch Levy, Anat Bremler-Barr

arXiv:2610.20567v1cs.NIcs.CRcs.DC

TL;DR

Miscoordination can overload microservices and expose valuable transactions to rejection, including during attacks. DiffTry assigns retry limits by transaction class and the paper develops an analysis for service networks. In a live deployment, measured per-class rejection closely matched the numerical prediction, and DiffTry reduced high-priority rejection under overload.

  • Problem

    Miscoordination can overload downstream services and reject valuable transactions, while attackers may exploit this weakness to cause mass rejections of innocent high-value traffic.

    Source: §I · p. 1

  • Method

    DiffTry embeds priority in retry logic through class-specific retry limits, and the paper extends its analysis from a tandem service model to service networks.

    Source: A. DiffTry: Differential Retries · p. 1, §VIII · p. 9

  • Results

    At the heaviest flood, DiffTry reduced high-priority rejection by about 3.5×, from ≈54% to ≈15%, while low-priority rejection rose to ≈61%; live deployment measurements closely matched the analysis.

    Source: 1) End-to-end rejection rate (Fig. 5(b)) · p. 9, §VIII · p. 9

  • Takeaways & Limitations

    DiffTry can be deployed as a drop-in service-mesh plugin without application-code changes, and its measured per-class rejection matches the numerical prediction.

    Source: §VIII · p. 9

Abstract

from arXiv · show

Cloud services rapidly shift towards independently-managed microservices architectures. This independence may lead to temporary microservices miscoordination, resulting in overloads and rejections of valuable transactions. This weakness may further be exploited by attackers to launch overload attacks inflicting performance and cost damages. Retry operations are a key machinery aiming at overcoming seldom rejections. However, during overload periods they: a) Fail protecting valuable transactions from rejections, and b) Trigger large-scale performance-degrading retry storms. We propose simple modifications in retry operations, overcoming these deficiencies by transforming retries into a rejection prioritization mechanism, DiffTry (Differential Retries). It is based on a simple, lightweight, distributed and stateless design and requires neither overload sensing nor microservices coordination. We develop an analytical model capturing DiffTry's effectiveness for two services in tandem, then generalize it to networks of services. Our theoretical framework lets practitioners quantify, for a given network, the resulting prioritization and rejection rates, and yields formal guarantees that DiffTry substantially reduces the impact of both incidental and adversarial orchestrated overloads. We validate these predictions in simulation and on a live Kubernetes deployment, where DiffTry is a simple drop-in service-mesh plugin requiring no application changes; the measured rejection closely matches the analysis.

I. INTRODUCTION

Miscoordination among independently managed services can overload downstream microservices, rejecting valuable transactions and exposing services to attacks that flood them with low-value traffic. DiffTry uses different retry limits by class to prioritize rejections, and the paper analyzes and evaluates this approach from a single service to network deployment.

I. INTRODUCTION

  • Miscoordination can leave downstream microservices under-capacity, causing transaction losses that are especially costly when high-value requests are rejected.Attackers may exploit this vulnerability by sending spikes of cheap, low-value requests that trigger mass rejection of innocent high-value traffic.Source: §I · p. 1

A. DiffTry: Differential Retries

  • DiffTry assigns class-specific retry limits so high-priority requests can receive more attempts, prioritizing rejection rates without explicit inter-service coordination.The mechanism operates at the retry decision point and does not require overload detection.Source: A. DiffTry: Differential Retries · p. 1
  • Under the illustrated overload, high-priority requests have about 15% rejection and account for about 85% of successful completions when both classes supply 50% of input load.The passage reports about 70% throughput improvement for high-priority transactions.Source: A. DiffTry: Differential Retries · p. 2, Figure 1 · p. 2

B. Main Contributions: DiffTry Analysis and Results

  • The network analysis approximates per-class, per-node and end-to-end rejection rates, with guaranteed convergence and uniqueness under stated conditions, including DAG service graphs.For other cases, the general uniqueness question remains open, although experiments yielded a unique solution.Source: B. Main Contributions: DiffTry… · p. 2
  • Network-wide class-level success rates increase monotonically with retry limits, supporting performance lower bounds.The monotonicity result is stated for network-wide class-level success rates.Source: B. Main Contributions: DiffTry… · p. 2
  • A live Kubernetes deployment validated that measured rejection closely matches the analysis, using a drop-in service-mesh plugin with no application-code changes.The paper also reports that the plugin incurs negligible overhead.Source: B. Main Contributions: DiffTry… · p. 2

II. RELATED WORK

Prior approaches to overload control and retry handling have trade-offs in coordination, rejection handling, or class differentiation. The paper presents DiffTry's retry-budget mechanism and analyzes its behavior with a single-service fluid model under overload.

II. RELATED WORK

  • Centralized overload control requires network-wide information, while local auto-scaling lacks a global view; traditional queue priorities act at service rather than rejection, and retry differentiation remains limited in prior work.The passage also notes that retry storms can degrade performance and create security threats.Source: §II · pp. 2–3

III. DiffTry: MECHANISM OPERATION AND MODEL

  • DiffTry assigns different retry-attempt limits by priority class, shifting overload rejections toward low-priority traffic while suppressing retry amplification without overload sensing or coordination.The mechanism is described as non-intrusive during no-overload periods and designed for local, distributed operation.Source: §III · p. 3

IV. ANALYSIS OF DiffTry IN A SINGLE SERVICE

  • The single-service analysis uses a fluid approximation under overload to capture retry dynamics while abstracting queueing details, with simulation used to validate accuracy.The approximation is presented as independent of the specific queueing discipline.Source: §IV · p. 3, A. DiffTry Dynamics Under Overload… · p. 3

A. DiffTry Dynamics Under Overload (Fluid Model)

  • Under persistent overload, capped retry budgets stabilize retry traffic; the model assumes rejection probability is independent of retry stage, an assumption the authors say is likely when retry delays are long relative to service time.The independence assumption is validated through simulation, according to the passage.Source: A. DiffTry Dynamics Under Overload… · pp. 3–4
  • The overload model yields a unique rejection-rate solution determined by retry budgets, arrival rates, and service capacity.The fluid approximation neglects idle capacity and is described as accurate in the overload regime it targets.Source: A. DiffTry Dynamics Under Overload… · p. 4

B. A More Detailed Model for No Overload

For no overload, the fluid approximation yields zero rejection, while a queueing model can provide a more precise rejection probability as a function of utilization. The paper combines these regimes into a continuous approximation across operating conditions.

B. A More Detailed Model for No Overload

  • For utilization ρ≤1, the fluid approximation gives zero rejection; a system-dependent queueing model can instead supply rejection probability as a function of ρ.The passage gives the M/M/1/m blocking probability as an example of a queueing-specific expression.Source: B. A More Detailed Model for No Overload · p. 4
  • When combining queueing-based behavior for ρ≤1 with the fluid approximation for ρ>1, the paper uses a lower bound based on rejection at ρ=1 if idle probability is unavailable.This construction is intended to maintain a continuous approximation over the operating range.Source: B. A More Detailed Model for No Overload · p. 4
  • The resulting rejection function is continuous and monotone, enabling a unified fixed-point equation.Source: B. A More Detailed Model for No Overload · p. 4

C. Per-Stream Throughput and Rejection Rates

Class-specific retry budgets determine each stream’s total load and rejection probability. A residual-time correction relaxes the independent-failure assumption, and single-service evaluation compares simulation with analytical models.

C. Per-Stream Throughput and Rejection Rates

  • For class p, Λp counts original and retry requests, while its rejection probability is s̄^(kp+1); lower retry limits reduce retry-induced load.The expected successful completion rate is Ep = λp·(1 − s̄^(kp+1)).Source: C. Per-Stream Throughput and Rejection… · p. 4

D. Relaxing the Independence Assumption for Highly Variable Service Times

  • When α=1, the blocking job completes before the next retry and failures are independent; smaller α indicates stronger correlation.The residual-time model conditions on when the blocking job finishes relative to retry attempts.Source: D. Relaxing the Independence… · p. 4

V. EXAMINATION OF DiffTry IN A SINGLE-SERVICE

  • The single-service examination uses discrete-event simulation and numerical M/M/1/m and fluid-model evaluations to assess model accuracy and retry-limit effectiveness.The simulator is validated against exact M/M/1/m evaluation, while the fluid approximation is assessed against that evaluation.Source: §V · pp. 4–5, Figure 2 · p. 5

A. Examination in typical cases: Accuracy and Effectiveness

Typical-case simulations closely match analytical predictions and show retry amplification alongside class-based rejection prioritization. Extreme-case analysis examines approximation accuracy near critical load and under correlated retry failures, while the network analysis uses open-network flow equations.

A. Examination in typical cases: Accuracy and Effectiveness

  • Simulation closely matches the M/M/1/m and fluid-model derivations; larger retry budgets increase retry traffic, while the class with the larger budget has substantially lower rejection probability.The lower-priority class absorbs most losses across traffic mixes.Source: A. Examination in typical cases:… · p. 5

B. Examination of the model in extreme cases

  • For small buffers, the fluid approximation can deviate near ρ=1; the mixed approximation is closer there and remains tractable.The fluid-model error is at most 1/(m + 1), and the paper notes that this is negligible for its target buffer sizes in the tens or hundreds.Source: B. Examination of the model in extreme… · p. 5
  • Under short retry timeouts and high service-time variance, the residual model tracks simulated rejection rates despite correlated failures.The study induces high variance with a two-phase service-time distribution and reports similar results for other tested variances and smaller timeouts.Source: B. Examination of the model in extreme… · p. 5

VI. ANALYSIS OF DiffTry ON A GENERAL NETWORK

  • For general service networks, the analysis extends the standard open-network flow model to DiffTry, whose nonlinear equations make the analysis non-trivial.The standard flow model does not require Poisson external arrivals when the analysis uses only rates.Source: §VI · pp. 5–6, A. The Standard Network Flow Model · p. 6

A. The Standard Network Flow Model

  • The standard model obtains class-specific node arrival rates from external arrivals and routing, with λ = γ(I − F)−1 when the routing matrix satisfies the stated convergence condition.The sufficient condition given is ρ(F) < 1.Source: A. The Standard Network Flow Model · p. 6

B. Generalizing the Standard Model to Accommodate DiffTry

The generalized model separates each service’s input and output rates and incorporates class-specific retry limits. Its resulting nonlinear equation set has 2NP unknowns and yields transaction rates when solved.

B. Generalizing the Standard Model to Accommodate DiffTry

C. Solving for the transaction rates for DAG Networks

In DAG networks, the transaction-rate equations can be solved in one topological pass with a unique solution. For general networks, an iterative method is proposed, and convergence is guaranteed when the standard flow equations are solvable.

C. Solving for the transaction rates for DAG Networks

  • For DAG networks, a single forward pass in topological order computes each node’s input and output rates, and the solution is unique regardless of the order chosen.Predecessor output rates and local parameters determine each node’s rates.Source: C. Solving for the transaction rates… · p. 6

D. Solving for the Transaction Rates in General Networks

  • For networks with cycles, the proposed iteration updates λ and recomputes s̄ at each step because s̄ depends on λ.The procedure solves the nonlinear system through successive iterations.Source: D. Solving for the Transaction Rates… · p. 6
  • When the standard flow equations have a solution, the generalized DiffTry equations also have a solution.The proof establishes convergence through an equivalent iterative procedure.Source: D. Solving for the Transaction Rates… · p. 6

E. Uniqueness of the Solution for cyclic graphs

Uniqueness is non-trivial for cyclic networks because DiffTry’s equations are nonlinear and their effective routing weights depend on the solution. The section gives sufficient conditions for uniqueness and supports broader applicability with numerical validation.

E. Uniqueness of the Solution for cyclic graphs

  • DiffTry’s nonlinear equations couple effective routing weights to the solution, making uniqueness non-trivial for networks with cycles.By contrast, the standard linear traffic equations have a unique solution when ρ(F) < 1.Source: E. Uniqueness of the Solution for… · p. 7
  • Theorem 2 gives sufficient conditions for a unique nonnegative solution, including ∥F∥∞ < 1/(P−1), which becomes ∥F∥∞ < 1 for P = 2.The other stated condition requires P ≤ 2.Source: E. Uniqueness of the Solution for… · p. 7
  • When a boundary case has ρ(F) = 1, subtracting an arbitrarily small ε from a row entry can provide a unique solution without expected practical impact in non-ill-conditioned systems.The text says tiny transaction departures are naturally expected at nodes.Source: E. Uniqueness of the Solution for… · p. 7
  • The proof treats the fixed-point map as a contraction when φ is L-Lipschitz and ∥F∥ < 1/L, yielding uniqueness by the Banach Fixed-Point Theorem.The section bounds φ’s Lipschitz constant separately in underload and overload regions, then extends the bound across their boundary.Source: E. Uniqueness of the Solution for… · p. 7
  • The second sufficient condition uses class-independent routing and P ≤ 2 to obtain a blockwise norm with ∥F∥ < 1 when ρ(F) < 1.The text notes that class-independence lets the routing matrix use identical copies across priority classes.Source: E. Uniqueness of the Solution for… · p. 7

2) Numerical verification:

Numerical tests found no evidence of multiple fixed points across roughly 60 million tested cases. The section also describes computing job-completion probabilities from known paths or recursively through the network.

2) Numerical verification:

  • Across roughly 60 million tested cases on cyclic networks with P < 10, fixed-point iteration never converged to two distinct solutions, providing strong empirical evidence for universal uniqueness.The tests used µBench and DGG graph generators, [36], hundreds of parameter configurations per network, and thousands of initializations.Source: 2) Numerical verification · p. 7

F. Transaction (job) Completion Probability

  • For a known job path, completion probability is the product of its success rates across the traversed hops.For an unknown path, the network completion probability is derived recursively from the probability of leaving at a node and downstream neighbors’ completion probabilities.Source: F. Transaction (job) Completion… · p. 8
  • The vector completion probabilities satisfy C = Ŝ + C F̂, and the section gives the solution when the original network is solvable.Ŝ denotes the vector of fractions of jobs leaving the system at each node.Source: F. Transaction (job) Completion… · p. 8

G. System-Wide Monotonicity of DiffTry

The section establishes that increasing a class’s retry budget preserves its dominance through network iterations and at fixed points. It concludes that prioritizing a class raises its successful transactions and total completion rate relative to no prioritization.

G. System-Wide Monotonicity of DiffTry

  • Increasing class p’s retry budget yields consistent dominance at fixed points, even as load shifts between network stages.The theorem compares systems with ordered retry budgets and appropriately ordered initial loads.Source: G. System-Wide Monotonicity of DiffTry · p. 8
  • The proof preserves p-dominance through the node map, the routed iteration, and its fixed point.Routing forms nonnegative mixtures of accepted-rate vectors, which preserve the ordering.Source: G. System-Wide Monotonicity of DiffTry · p. 8
  • Relative to no prioritization, prioritizing class p raises its per-node successful transactions and its total completion rate Γp, even if multiple DiffTry fixed points exist.The comparison uses the class-equal-budget system as the baseline.Source: G. System-Wide Monotonicity of DiffTry · p. 8

VII. NETWORK-LEVEL EVALUATION AND EXPERIMENTAL DEPLOYMENT IN A LARGE KUBERNETES NETWORK

The evaluation uses DeathStarBench’s hotelReservation application to assess network-wide prioritization and how well the analytical model predicts a live deployment. It combines numerical solutions of the network model with experiments on Kubernetes using a service mesh.

VII. NETWORK-LEVEL EVALUATION AND EXPERIMENTAL DEPLOYMENT IN A LARGE KUBERNETES NETWORK

  • The evaluation uses the hotelReservation application from the DeathStarBench microservice benchmark.The study examines whether DiffTry’s local mechanism composes into network-wide prioritization and how well the analysis predicts a real system.Source: §VII · p. 8
  • The study compares numerical solutions of the network model with experiments on a live Kubernetes deployment using a service mesh.Source: §VII · p. 8
  • The evaluation asks whether DiffTry’s local per-node mechanism produces network-wide prioritization and whether the analysis predicts a real system.Source: §VII · p. 8

A. Experiment and numerical examination Setup

The evaluation tests DiffTry in a DeathStarBench hotelReservation deployment, comparing numerical network-model predictions with a live Kubernetes experiment. It reports end-to-end prioritization under overload, lower retry-related latency and internal work, and a drop-in service-mesh implementation supported by an analytical framework.

A. Experiment and numerical examination Setup

  • The experiment uses two priority classes, fixes γ(1)=50, and sweeps low-priority arrivals γ(2) from 50 to 250, reaching total offered load ρ≈2.1.The comparison is DiffTry k=(5, 1) versus a symmetric no-prioritization baseline k=(5, 5).Source: A. Experiment and numerical… · p. 8

B. Experiment and numerical examination results

  • At ρ≈2.1, high-priority rejection falls from ≈54% to ≈15% (∼3.5×), while low-priority rejection rises to ≈61%.Measured live-deployment rejection tracks the numerical fixed-point prediction with an ≈0.03 difference; below capacity (ρ < 1), both classes see ≈0 rejection.Source: Figure 5 · p. 9, 1) End-to-end rejection rate (Fig. 5(b)) · p. 9
  • ≈270 ms baseline median retry-storm latency falls to ≈20 ms for low-priority and ≈70 ms for high-priority under DiffTry (∼3.6× lower than baseline for high-priority).The low-priority retry budget is capped so the flood fails fast; the high-priority latency reflects the cost of k=5 retries.Source: Figure 5 · p. 9, 2) Latency and resource consumption… · p. 9

3) Conclusions:

  • The live deployment implements DiffTry as a stateless Envoy service-mesh filter that sets each hop’s retry budget by request priority, without application-code changes.The physical experiment uses the application’s native fan-out, while the theoretical model relies on probabilistic routing.Source: A. Experiment and numerical… · p. 9, 3) Conclusions · p. 9

B. Experiment and numerical examination results

  • DiffTry reduces internal calls from ≈6.4 to ≈3.3 per request under overload, below the ≈3.8-call fan-out floor.Per successfully served request, internal calls are ≈13.8 under uniform retries versus ≈7.2 with DiffTry; CPU and memory stay comparable because rate limiters reject before service logic.Source: 2) Latency and resource consumption… · p. 9

VIII. CONCLUSIONS

  • The analytical framework quantifies prioritization and rejection rates, proves convergence, establishes uniqueness conditions including DAG service graphs, and proves success rates increase monotonically with retry limits.The monotonicity results provide performance bounds even when uniqueness is not guaranteed.Source: §VIII · p. 9
Loading 2610.20567v1…