PaperlayerView exact revision on arXiv

Source-linked AI summary

Job Class Thermal Intent Aware Liquid Cooling Allocation for AI Data Centers

Krishna Chaitanya Sunkara

arXiv:2609.30785v1cs.DCcs.PFcs.SYeess.SY

TL;DR

Reactive cooling can lag rapidly changing workloads, while JCTI uses job-class and dispatch information to stage coolant before heat arrives. Across 120 paired trials, it reduced thermal violations and overshoot versus a straight PI loop, with a pumping increase.

  • Problem

    Temperature-triggered cooling can take 30 to 50 seconds to respond, leaving workload changes ahead of the feedback loop.

    Source: §I · p. 1

  • Method

    JCTI uses scheduler job-class and dispatch information to forecast thermal demand and combine coolant pre-positioning with PI feedback.

    Source: §I · p. 1, §V · p. 3

  • Results

    Thermal violations fell 56.4% and overshoot integral fell 60.2% versus the straight PI baseline across 120 paired trials.

    Source: §VII · pp. 4–5

  • Takeaways & Limitations

    JCTI connects workload orchestration and thermal management by passing job class and dispatch timing directly to cooling control.

    Source: §VIII · p. 5

    The lumped model omits microchannels, die-level heat spreading, and sub-second fluid transients, while job class is only a proxy for power trajectory.

    Source: §VIII · p. 6

Abstract

from arXiv · show

GPU-dense AI data centers need to run on liquid cooling as air simply cannot shed the heat at these power densities. Yet the cooling loops themselves are blind to what workloads are about to run; they crank up flow only after a sensor catches a temperature climb, which can take 30 to 50 seconds. We built Job-Class Thermal Intent (JCTI) to close that window. The scheduler already knows a job is coming and what class it belongs to; JCTI feeds that information straight to the cooling controller so it can stage coolant before the heat shows up. We pulled the thermal signatures for each job class out of MLPerf GPU power traces and tuned arrival patterns against Alibaba cluster data. Over 120 paired Monte Carlo trials the numbers come out to 56.4% fewer thermal violations and 60.2% less cumulative overshoot than a straight PI loop. As AI data centers evolving towards gigawatt grid loads with highly fluctuating power swings, thermally-aware scheduling reduces sudden demand and improves load prediction in grid side. Cooling and scheduling have been running as two separate systems for years despite each one knowing something the other needs, JCTI wires them together.

I. INTRODUCTION

GPU power density and workload variability strain cooling systems, while temperature-triggered control can lag rapid inference ramps. JCTI uses scheduler metadata to stage coolant in advance.

I. INTRODUCTION

  • GPU-dense deployments exceed forced-air capacity, while training, inference, and analytics have distinct power profiles.Training is relatively steady with checkpoint spikes; inference swings with batch size and request surges.Source: §I · p. 1
  • 30 to 50 seconds of sensor, controller, and valve delay can leave reactive cooling behind rapid workload changes.Inference jobs can rise from idle to full power in two or three seconds.Source: §I · p. 1
  • JCTI sends job class and dispatch timing to cooling control, using MLPerf power traces and Alibaba arrival statistics to forecast demand.Its readiness metric captures stabilization time and forward-horizon throttling-violation probability; the approach is evaluated in 120 paired Monte Carlo trials.Source: §I · p. 1

II. THERMAL INTENT ARCHITECTURE AND DERIVATION

JCTI carries job descriptors from orchestration to rack-level coolant allocation through a thermal-intent service and a feedforward-plus-feedback controller. Its class models draw on MLPerf power measurements and Alibaba production traces.

II. THERMAL INTENT ARCHITECTURE AND DERIVATION

  • Job descriptors include class, GPU count and type, expected duration, dispatch notice, and priority for resource contention.These are metadata already available to the scheduler.Source: §II · pp. 1–2
  • The four-layer architecture translates orchestration metadata into rack-specific demand signals, then combines feedforward and PI feedback to actuate coolant flow across racks sharing a CDU.A separate thermal-intent service maintains job-class parameter sets and can update models from live telemetry without changing the orchestrator.Source: Figure 1 · p. 2, §II · p. 2
  • The controller computes per-rack flow targets from forecasts and current temperatures, subject to valve limits, shared-CDU constraints, and a safety layer.Feedforward pre-positioning uses dispatch notice, while temperature feedback corrects the flow; signatures come from MLPerf and Alibaba traces.Source: §II · p. 2
  • Training power ramps to 380–420W over 8 to 12 seconds and stays relatively flat, with checkpoint flushes producing brief spikes.Alibaba training jobs use 4 to 32 GPUs and run from two hours to days; the trajectory is modeled as a stochastic process.Source: §II · p. 2
  • Inference power rises from idle to 220–280W in 1 to 3 seconds and varies with batch size and incoming traffic.The model sets inference τc = 2 s, σc = 0.25, λc = 0.5 per minute, κc = 1.5, and Ppeak = 250 W per GPU; analytics ramps take 4 to 6 seconds.Source: §II · p. 2
  • The class policy weight is higher for inference than training because inference latency is what end users feel, and nearly all GPU electrical power becomes heat.The stated weights are αinfer = 1.5 and αtrain = 1.0, with η ≈1.0.Source: §II · p. 2

III. MULTI-RACK THERMAL PLANT

The multi-rack plant models each liquid-cooled rack as a thermal mass controlled through coolant flow, with shared CDU capacity and hardware limits constraining allocation.

III. MULTI-RACK THERMAL PLANT

  • Each rack’s temperature reflects the combined heat from its active jobs and evolves through heat input, passive loss, and liquid cooling.The rack is modeled as a first-order thermal mass with coolant flow as its controllable input.Source: §III · pp. 2–3
  • The model uses Cr = 12,000 J/K, Rr = 0.08 K/W, γr = 4200 W/(kg/s)/K, and Tin = 27 °C.Cr represents server, GPU, cold-plate, manifold, and loop-water thermal mass; γr converts flow into heat removal.Source: §III · p. 3
  • Six racks sharing one CDU compete for pump capacity, and allocation is bounded by total flow, per-rack valve limits, and flow-rate-of-change limits.The rate bound is intended to avoid water hammer and valve wear.Source: §III · p. 3

IV. WORKLOAD THERMAL READINESS

Workload Thermal Readiness supplements standard cooling targets with stabilization latency and forward-looking violation risk, assessed through Monte Carlo rollouts of queued-job power trajectories.

IV. WORKLOAD THERMAL READINESS

  • Steady-state temperature and PUE do not capture whether a rack near the throttling line can withstand a workload burst.The readiness measures address time to settle after a job lands and the chance of hitting the hard limit over a forward horizon.Source: §IV · p. 3
  • Stabilization latency is the first time the rack remains within the target band for duration w, with stability requiring a bounded temperature-change rate.Tset is the target, ∆T the allowed band, and ϵ caps the rate of change.Source: §IV · p. 3
  • A rack can stabilize just inside the target band yet remain vulnerable to a burst that causes a violation.The forward-looking risk uses Tmax as the hard throttling limit and H as the look-ahead horizon.Source: §IV · p. 3
  • Short Monte Carlo rollouts estimate violation risk by counting simulated queued-job power trajectories that cross Tmax.More samples increase compute cost but tighten the estimate.Source: §IV · p. 3

V. CONTROL METHODOLOGY

The controller combines dispatch-informed feedforward cooling with temperature feedback, addressing the delay that limits a conventional PI loop. It also prioritizes inference jobs when flow is constrained and respects rack and shared-CDU limits.

V. CONTROL METHODOLOGY

  • The baseline PI loop adjusts flow from temperature-tracking error, but 30–50 seconds of sensing and valve latency can exceed a job’s 2-second power ramp.Kp supplies an immediate response, Ki reduces steady-state offset, and m0 maintains minimum coolant flow.Source: §V · p. 3
  • JCTI uses dispatch notice to sum anticipated thermal demand across jobs starting within the notice window and convert it into a feedforward flow target.The policy weight αcj prioritizes inference over training when flow is tight.Source: §V · pp. 3–4
  • JCTI pre-positions coolant from dispatch notice and keeps rack temperatures below the 78 °C throttling threshold, unlike the delayed PI controller.The figure reports frequent PI violations above Tmax = 78 °C under a 40-second actuation delay.Source: Figure 2 · p. 4
  • A smaller feedback correction supplements feedforward, while iterative proportional scaling enforces the shared CDU cap when rack demands exceed it.The controller also respects valve limits and hardware safety constraints.Source: §V · p. 4
  • Grid search selected g = 1.5 × 10−4 kg/(s·W) and kf = 0.018 kg/(s·K) while balancing violations, setpoint regulation, and pump energy.The candidates were evaluated over Monte Carlo workload draws.Source: §V · p. 4

VI. EXPERIMENTAL METHODOLOGY

The evaluation uses paired Monte Carlo simulations of a six-rack plant with stochastic workloads and environmental variation. It compares thermal violations, overshoot, and pumping proxy across matched trials.

VI. EXPERIMENTAL METHODOLOGY

  • Each Monte Carlo trial simulates six racks sharing one CDU for 1,200 seconds at 2-second integration steps.Workloads, power jitter, burst events, and ambient drift vary across the simulations.Source: §VI · p. 4
  • Jobs arrive at 0.08 jobs per second per rack with class probabilities 0.4 training, 0.5 inference, and 0.1 analytics, calibrated to Alibaba traces.GPU allocations and log-normal run durations are drawn from class-specific distributions; dispatch notice is uniformly distributed from 20 to 60 seconds.Source: §VI · p. 4
  • Power trajectories use per-class stochastic models with timestep Gaussian jitter, Poisson burst events, and sinusoidal ambient-temperature drift.Ambient temperature follows a 10-minute cycle with added white noise.Source: §VI · p. 4
  • The six-rack plant uses a 1.8 kg/s CDU capacity, 0.35 kg/s per-rack minimum flow, and a 78 °C throttling limit.The setpoint is 62 °C with a 2 °C acceptable band; baseline latency is 40 seconds.Source: §VI · p. 4
  • Performance is measured by violation fraction, overshoot integral in kelvin-seconds, and a pumping proxy based on cubic flow scaling.Violation fraction is the fraction of time any rack exceeds the hard limit.Source: §VI · p. 4
  • The study runs 120 paired trials with shared workload and disturbance seeds, then tests paired differences using 95% confidence intervals and a two-tailed t-test.The test uses 119 degrees of freedom and rejects at p < 0.01.Source: §VI · p. 4

VII. RESULTS

Across 120 paired trials, JCTI reduced thermal violations and overshoot compared with baseline, while increasing pumping effort. It pre-positioned coolant before jobs started and kept both controllers within the CDU capacity constraint.

VII. RESULTS

  • Pumping proxy increased 25.4%, from PP = 137.5±1.0 for baseline to PP = 172.4±1.3 for JCTI.JCTI directs more coolant to racks that need it rather than distributing flow evenly.Source: §VII · p. 5, Figure 4 · p. 5
  • JCTI pre-positions coolant before job heat arrives, whereas baseline flow spikes reactively; both controllers remain within CDU capacity.The proposed controller allocates coolant 20 to 60 seconds before execution based on dispatch notice.Source: §VII · p. 5, Figure 3 · p. 5
  • Violation fraction fell from V = 0.4469±0.0067 to 0.1947±0.0097 (56.4%), while overshoot integral fell from OI = 113,206±1156 to 45,079±1832 K·s (60.2%).These results compare JCTI with baseline across 120 paired trials.Source: §VII · pp. 4–5, Figure 4 · p. 5
  • In a 600-second trial, baseline racks crossed the throttling threshold 426 times, while JCTI racks crossed it zero times.Other trials had nonzero but much smaller JCTI counts.Source: §VII · p. 5
  • Removing dispatch notice raised violation fraction from 0.195 to 0.255; 20% job-class label swaps raised overshoot integral from 45,079 to 58,142 K·s.In the same trial, JCTI peaked at 73.8 °C versus 159.4 °C for baseline; feedback absorbed moderate classification noise, though regulation loosened.Source: §VII · p. 5

VIII. DISCUSSION

JCTI connects workload orchestration to thermal management by sending job class and dispatch timing to the cooling controller. The discussion highlights a representative reduction in violations and notes modeling, workload-drift, and deployment constraints.

VIII. DISCUSSION

  • JCTI pipes job class and dispatch timing into cooling control, bridging scheduling work that treats cooling as fixed and predictive cooling that treats workloads as opaque.Source: §VIII · p. 5
  • 426 baseline violation events versus 0 with JCTI appear in one representative 600 s trial.The figure counts racks simultaneously exceeding Tmax = 78 °C.Source: Figure 5 · p. 6
  • Workload characteristics can drift as architectures and serving patterns change, so online learning requires careful initialization to avoid bad transients.Source: §VIII · p. 6
  • Shared CDU competition currently backs racks off proportionally, while deployment requires cross-layer coordination; the authors propose shadow-mode validation before actuation.Source: §VIII · p. 6
  • Job-class power proxies may miss variation from model architecture, batch size, optimizer, and data; evaluation also uses Alibaba-calibrated workloads rather than raw production replay.Source: §VIII · p. 6

IX. CONCLUSION

JCTI and Workload Thermal Readiness use workload information to forecast heat and stage coolant before jobs run. Across 120 paired Monte Carlo trials, JCTI reduced thermal violations and overshoot versus a standard PI loop, with increased pumping.

IX. CONCLUSION

  • 56.4% fewer violations, 60.2% less overshoot, and a 25.4% pumping increase versus a standard PI loop were reported across 120 paired Monte Carlo trials.Source: §IX · p. 6
  • The system derives thermal signatures from MLPerf GPU power traces and Alibaba cluster patterns, then uses dispatch notice for feedforward coolant staging with temperature-feedback correction.Source: §IX · p. 6
  • A six-rack plant sharing a CDU with hard flow limits represents multi-rack resource competition; ablations identify dispatch notice as the key enabler and find moderate job-class misclassification tolerable.Source: §IX · p. 6
  • Future work includes production-trace validation, hardware testing, multi-zone coupling, online parameter learning, and joint thermal-aware scheduling.Source: §IX · p. 6
Loading 2609.30785v1…