PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy-performance tradeoffs in server farms with batch services and setup times

Thu Le-Anh, Tuan Phung-Duc

arXiv:2610.04881v1cs.PFmath.PR

TL;DR

Turning off idle servers can save power, but restart setup times create a trade-off with system performance. The paper analyzes setup queues with batching and variants, and reports that the base model’s special rate-matrix structure enables lower-complexity exact analysis.

  • Problem

    How to save energy by turning off idle servers while managing the performance cost of their setup times is a central data-center trade-off.

    Source: §1 · p. 1

  • Method

    The paper analyzes a fixed-batch setup queue and delayoff and dynamic-batch variants, using matrix-analytic methods and simulations of alternative setup-time distributions.

    Source: §1 · p. 2, §7 · p. 20

  • Results

    The special structure of the M/M/c/SET–BATCH rate matrices permits an exact recursive algorithm with lower computational complexity for stationary distributions and performance measures.

    Source: §7 · p. 20

  • Takeaways & Limitations

    The models are intended to help choose policies as system parameters change while balancing performance and power consumption.

    Source: §1 · p. 2

    The published E[b] expression contained a denominator typo; recomputed numerical results changed plotted curves negligibly and did not affect the conclusions.

    Source: §5.3 · p. 14

Abstract

from arXiv · show

Data centers consume a large amount of energy, much of which is wasted due to idle servers. Turning off idle servers might be an effective power-saving solution; however, there is a trade-off between energy savings and system performance. Hence, we propose a setup queueing model with a batching policy that allows servers to process a set of jobs simultaneously to minimize power consumption while maintaining acceptable performance. We consider an M/M/c/SET--BATCH queue, a multi-server batch service queue with a fixed batch size and setup times, and some variants, including systems in which idle servers delay before turning off or systems in which the batch size is dynamic. We analyze the steady-state probabilities and system performance of the M/M/c/SET--BATCH system and its variants. Our analysis of the M/M/c/SET--BATCH system with lower computational complexity is made possible by utilizing the special structure of the model. In addition, we use simulations to compare the M/M/c/SET--BATCH model with some other variants with different setup time distributions. The results suggest that the model performs better when the setup time has a larger coefficient of variation. Our results indicate that the batching policy enhances the system performance, especially when we allow servers to be idle before turning them off.

1. Introduction

Turning off idle servers can save power, but restart setup times create a trade-off with system performance. The paper proposes batching setup queues and variants to balance these concerns, with analysis and numerical experiments aimed at evaluating efficiency and choosing policies.

1. Introduction

  • Batching lets a server process multiple jobs together, aiming to reduce how often servers enter setup mode.The motivation notes that setup times can be much longer than service times in data centers.Source: §1 · p. 1
  • Prior work often studies setup times and batch services separately, while this paper combines them in a multi-server queueing model.Related models with batch services and vacations do not include setup times in the same way.Source: §1 · p. 2
  • The paper proposes setup queues with batching and variants to balance system performance and power consumption in data centers.The variants include delayoff and dynamic batch-size policies.Source: §1 · p. 2
  • An exact recursive algorithm computes stationary distributions and performance measures by exploiting the special structure of the model’s rate matrices.The introduction also identifies computational complexity and approximation accuracy for finite buffers as evaluation targets.Source: §1 · p. 2
  • The authors use numerical experiments and simulations to evaluate computational efficiency and choose policies for different system parameters.The introduction also identifies simulations of alternative setup-time distributions as part of the study.Source: §1 · p. 2

2. Related works

The related-work review distinguishes research on batch service, setup queues, delayoff and sleep policies, and deterministic setup times. It positions the paper among these lines of work, where combined setup-and-batching models are comparatively uncommon.

2. Related works

  • Most prior work treats setup times and batch services separately, though some models combine batch service with vacations rather than setup times.The review identifies the different queueing mechanisms as an analytical distinction.Source: §2 · p. 2

2.1. Batch service without setup

  • Batch-service research includes general bulk service and models combining batch service with vacations, typically using a minimum queue threshold.The review traces batch-service studies back to Bailey’s single-server M/G/1 model.Source: §2.1 · pp. 2–3

2.2. M/M/c with setup

  • Multi-server setup-queue studies include immediate shutdown, staggered setup, and s-staggered policies, with analyses using several methods.The review cites exact and approximate analyses of setup queues and extensions involving batch arrivals or abandonment.Source: §2.2 · p. 3, §2.3 · p. 3

2.4. M/M/c/Setup/Delayoff

  • Delayoff policies let idle servers wait before shutdown; prior M/M/c setup-queue studies include approximations and models with non-stationary arrivals or abandonment.One cited study develops an approximation for choosing the delay before shutdown.Source: §2.4 · p. 3

2.5. M/M/c/Setup/Sleep

  • Sleep modes use less power than idle servers but require setup times typically shorter than those for powered-down servers, creating a separate energy trade-off.The review also notes PowerNap experiments reporting a 74% average energy reduction.Source: §2.5 · p. 3

2.6. Deterministic setup times

  • Deterministic setup times have received limited study, and prior simulations found a greater adverse delay effect than with exponential setup times.The review also reports that the negative effect in was greater for deterministic than exponential setup.Source: §2.6 · p. 3

3. Model

The models use Poisson arrivals, exponential batch service, and exponential setup times in their Markovian formulation. The paper then specifies fixed-batch, delayoff, and dynamic-batch variants.

3. Model

  • The base model assumes c servers, Poisson job arrivals at rate λ, exponential batch service with mean 1/µ, and exponential setup times with mean 1/α.The finite-buffer version limits the number of jobs in the system to K, where K ≥ cb.Source: §3 · p. 4, §3.1 · p. 4

3.1. M/M/c/SET–BATCH

  • In M/M/c/SET–BATCH, servers process fixed-size batches, turn off when no batch is waiting, and cancel excess setup servers when fewer batches remain.The model defines i, j, and b as active-server count, jobs in the system, and batch size; servers operate FCFS and can be setup, busy, or off.Source: §3.1 · p. 4

3.2. M/M/c/SET–BATCH/Delayoff

  • In the Delayoff variant, an idle server waits an exponential time with mean 1/β before shutdown, but immediately serves an arriving batch during that interval.If no batch requests service during the delay, the server turns off.Source: §3.2 · p. 4

3.3. M/M/c/SET–BATCH/Dynamic

  • In the Dynamic variant, batch sizes range from b1 to b2: setup begins when at least b1 jobs wait, and service takes up to b2 jobs.After service, a server turns off if fewer than b1 jobs remain; otherwise it serves another batch up to b2.Source: §3.3 · p. 5

4. The quasi-skip-free process (QSF)

This section formulates the batch-service models, including the dynamic variant, as quasi-skip-free processes and uses matrix-analytic methods to derive stationary distributions.

4. The quasi-skip-free process (QSF)

  • All three models are formulated as quasi-skip-free processes to the right, with jumps not crossing two or more levels in that direction.This structure captures batch-service queues in which servers can process multiple jobs simultaneously.Source: §4 · p. 5

4.1. M/M/c/SET–BATCH

  • For the base model, stability requires λ < cbµ; in the finite-buffer case, the authors approximate it by the infinite-buffer system to reduce the equations to solve from K+1 to cb.The finite-buffer formulation requires K+1 vector boundary equations, compared with cb for the infinite-buffer case.Source: §4.1 · pp. 5, 8

4.1.1. Homogeneous part

  • In the homogeneous part, the rate matrix R is upper triangular, with diagonal entries given by unique roots in (0, 1) and off-diagonal entries obtained recursively.The recursion computes the diagonal first, then the upper diagonal parts.Source: §4.1.1 · pp. 8–9

4.1.2. Non-homogeneous part

  • In the non-homogeneous part, upper-trapezoidal rate matrices make the linear equations for the boundary rate matrices efficient to solve.Only the matrices R(i) for i = cb, cb − 1, …, 1 need to be found before the rate matrices become homogeneous.Source: §4.1.2 · p. 9

4.2. M

  • The Delayoff model has stability condition λ < cbµ and uses a standard matrix-geometric method to obtain its stationary distribution.Its state tracks busy or idle servers and the number of jobs in the system.Source: §4 · p. 5, §4.2 · pp. 9–10

4.3. M/M/c/SET–BATCH/Dynamic

  • The Dynamic model is analyzed similarly to the base model, uses standard matrix-geometric methods, and is stable when λ < cb2µ.Its Markov-chain state tracks busy servers and jobs in the queue.Source: §4 · p. 5, §4.3 · p. 11
  • The block matrices Q1, Q−b2, and Q(i) become independent of level i beyond the specified thresholds.Q−b2 and Q(i) are stated to be level-independent for i ≥ b1+(c−1)b2.Source: §4.3 · p. 12
  • The stationary distribution is obtained using the matrix-analytic method, with the rate matrix R defined as the minimal non-negative solution of its matrix equation.The passage states that the stationary distribution is the unique solution of the balance equations.Source: §4.3 · p. 12
  • Once R is determined, the subvectors π0 through πb1+(c−1)b2 are solved using the first b1+(c−1)b2+1 equations and the normalization equation.The rate matrix can be solved using formulas similar to those for the Delayoff model.Source: §4.3 · p. 12

5. Performance measures

The section derives steady-state measures for energy and service performance across fixed-batch, delayoff, and dynamic-batch models.

5.1. M/M/c/SET–BATCH

  • The fixed-batch model derives average busy and setup servers, switches, customers in the system, and response time from its steady-state probabilities.Source: §5.1 · pp. 12–13
  • The fixed-batch model evaluates energy–delay product using mean power based on busy- and setup-server counts, assuming setup servers consume the same power as busy servers.Source: §5.1 · p. 13

5.2. M

  • The delayoff model tracks idle, busy, and setup servers, switches, customer count, and response time; it assumes idle-server power is 60% of busy-server power.Source: §5.2 · p. 13

5.3. M /M/c/SET–BATCH/Dynamic

  • Because batch size varies, the dynamic model derives its average at service-initiation epochs before computing batch and setup costs, power consumption, and ERP.The authors revised the average-batch-size expression; for the parameter settings considered, plotted changes were negligible and model comparisons and conclusions were unchanged.Source: §5.3 · p. 14

6. Numerical examples and simulations

The numerical studies compare computational methods and evaluate how setup-time distributions, batching, and policy variants affect energy and service performance. Their results show computational gains from the proposed analysis and policy trade-offs that depend on setup rate and traffic.

6.1. Computational complexity comparison

  • At c = 200, direct inversion takes about 9 times longer for b = 2 and 16 times longer for b = 3; the finite-buffer approximation is 16,000 times faster with about 0.1% error at K = 1000.The approximation’s error is below 1% when the buffer is not too small, and its efficiency and accuracy improve as server count and batch size increase.Source: §6.1 · p. 15, Table 2 · p. 15, Table 3 · p. 15

6.2.1. E

  • Higher setup-time coefficient of variation is associated with lower mean response time and power consumption; deterministic and Erlang setups perform worse than exponential and especially hyperexponential setups.The comparisons hold the setup-time mean fixed at 10 seconds across the four setup distributions.Source: §6.2.1 · p. 16, Figure 4 · p. 16

6.2.2. E

  • Batching consistently improves mean response time and ERP over no batching, but its power advantage depends on setup rate, server count, and batch size.With exceptionally long setup times, no batching can use less power; increasing batch size can also raise response time and ERP.Source: §6.2.2 · pp. 16–17, Figure 5 · p. 17, Figure 6 · p. 17

6.3. Some variant models

  • For the delayoff policy, the power-minimizing rate depends on setup rate, while β = 0 minimizes mean response time and ERP in heavy traffic.As β increases, response time and ERP increase; the direction of the power change varies with β and setup rate.Source: §6.3 · p. 18, Figure 8 · p. 19

6.3.2. M /M/c/SET–BATCH/Dynamic

  • When setup rate is high, the fixed-batch model outperforms the dynamic-batch model on power consumption, E[W], and ERP.The dynamic policy varies its batch-size range with load; the comparison fixes the upper limit at b2 = 5.Source: §6.3.2 · p. 18, Figure 9 · p. 19

6.3.3. Comparison of all policies

  • With a 10-second mean setup time, Delayoff has the lowest Cost, E[W], and ERP; with a 0.1-second mean setup time, it is generally superior except at extremely low traffic.Under light traffic with short setup times, delaying shutdown can consume more power than immediate shutdown followed by restart.Source: §6.3.3 · pp. 18, 20, Figure 10 · p. 19, Figure 11 · p. 20

7. Conclusions

The paper analyzes a batch-service setup queue and variants, finding that batching improves performance especially under high arrival rates and prolonged setup times. Delayoff generally performs best, while dynamic batch sizes help when setup times are long.

7. Conclusions

  • Batching enhances system performance, especially under high arrival rates and prolonged setup times.The delayoff policy generally outperforms the other policies except with short setup times and extremely low traffic intensity.Source: §7 · p. 20
  • A matrix analytic approach yields an exact recursive algorithm for the stationary distribution and performance measures of the M/M/c/SET–BATCH model, with lower computational complexity from its rate-matrix structure.The other models are analyzed using a matrix geometric method.Source: §7 · p. 20
  • Simulations find better results for variable setup times in the batching models, and dynamic batch sizes improve performance when setup times are long.The simulated setup-time distributions include hyperexponential, Erlang, and deterministic cases.Source: §7 · p. 20
  • Combining the considered policies to enhance system performance is possible, although the paper considers them independently.Source: §7 · p. 20

CRediT authorship contribution statement

The contribution statement assigns writing, visualization, validation, software, methodology, investigation, formal analysis, data curation, and conceptualization to Thu Le-Anh. Tuan Phung-Duc is credited with writing, validation, supervision, resources, project administration, methodology, investigation, funding acquisition, and conceptualization.

CRediT authorship contribution statement

Declaration of competing interest

The authors declare that they have no conflicts of interest.

Declaration of competing interest

Appendix A.1. Proof of Lemma 1

The proof applies Rouché’s theorem to establish a unique root of Eq. (2) in (0,1). It verifies the theorem’s analytic and boundary-magnitude conditions for the chosen functions.

Appendix A.1. Proof of Lemma 1

  • Rouché’s theorem states that if analytic functions f and g satisfy |f(z)| < |g(z)| on a simple closed contour, then f(z) − g(z) and g(z) have the same number of zeros inside.Zeros are counted according to multiplicity.Source: Appendix A.1 · p. 21
  • For the proof, f(z) = λ + iµz^b+1 and g(z) = (λ + iµ + (c − i)α)z are analytic on the unit disk, with |f(z)| < |g(z)| on the unit circle.Source: Appendix A.1 · p. 21
  • The theorem application shows that Eq. (2) has a unique root in (0,1).Source: Appendix A.1 · p. 21

Appendix A.2. Proof of Lemma 2

The proof uses induction to establish that the recursive sequence is strictly increasing and remains below 1.

Appendix A.2. Proof of Lemma 2

  • The proof uses induction to verify both the increasing property and the upper bound.Source: Appendix A.2 · p. 21
  • Induction establishes that the recursive sequence is strictly increasing for all positive integers.The proof applies the induction hypothesis to show each successive term exceeds the preceding one.Source: Appendix A.2 · p. 21
  • Induction also establishes that every term of the sequence is less than 1.The base case uses x1 = p < 1, and the induction step assumes earlier terms are below 1.Source: Appendix A.2 · p. 21
Loading 2610.04881v1…