Source-linked AI summary
HELIOS: Guardrailed LLM-Driven Evolution of Autonomous Resource Orchestration Policies for Multi-Cloud Distributed Systems
Guanyu Ding, Ying Wang
TL;DR
Multi-cloud orchestration must balance heterogeneous prices, interruption hazards, egress, outages, and SLO tiers, while per-decision LLM inference is too costly and slow. HELIOS evolves compact policies offline in a trace-driven simulator and deploys the champion behind guardrails; it reduces penalized cost across held-out traces while preserving premium-tier safety under stress.
Problem
Multi-cloud service orchestration spans provider prices, spot hazards, egress costs, outages, and SLO tiers that single-cloud autoscalers do not jointly reason about, while per-decision LLM inference is impractical.
Method
HELIOS uses an LLM offline to evolve small Python policies over a fixed feature interface against measured simulator fitness, then deploys the champion inside mechanically enforced guardrails.
Results
HELIOS attains the lowest penalized cost on all three held-out suites, including −45% versus SingleCloud-BFD and −9.6% versus calibrated Greedy on PlanetLab.
Takeaways & Limitations
The results support placing the LLM off the critical path and using guardrails to make generated orchestration policies auditable and bounded during online control.
Takeaways & Limitations
Results are trace-driven simulations, and replication with other models plus continual re-evolution under rolling prices remains future work with stability untested.
Abstract
from arXiv · showhide
Operating latency-sensitive services across multiple public clouds creates an optimization surface no single provider autoscaler can see: on-demand vCPU prices differ by provider, spot discounts and interruption risks vary by provider and instance type, egress fees penalize state movement, and provider-level failures can take down single-cloud deployments. Large language models (LLMs) are appealing orchestrators because they can synthesize non-trivial decision logic from a natural-language environment description. However, placing an LLM on the critical path of every scheduling decision is impractical: for a 200-service fleet, per-decision inference would cost more than the cloud capacity it manages and add multi-second latency to a millisecond-scale control loop. HELIOS moves the LLM off the critical path. The LLM evolves executable orchestration policies, small Python programs over a fixed feature interface, against a trace-driven multi-cloud simulator. Only the champion program runs in production, wrapped in guardrails that enforce capacity feasibility, SLO-class placement rules, and churn budgets regardless of evolved code's proposal. On real workload traces (PlanetLab, Bitbrains, Azure) combined with 2026 price and interruption-frequency data from AWS, Azure, and GCP, the evolved policy reduces penalized operating cost by 45% versus a single-cloud best-fit baseline and by 9-19% versus calibrated multi-cloud heuristics. It matches an oracle-informed MILP planner's premium-SLO performance at 40% lower cost and outperforms a DQN meta-controller trained with 2.6x more environment interactions. Guardrails are essential: without them, the same policy family degrades to 97-98% premium-tier downtime under a 10x spot-hazard stress test, versus 0.17% when guarded. We release the simulator, policies, LLM prompt/response archive, and per-generation fitness logs for full reproducibility.
I. INTRODUCTION
HELIOS addresses multi-cloud orchestration by using an LLM to evolve compact policies offline rather than making latency- and cost-prohibitive per-decision calls. The champion policy is evaluated across heterogeneous cloud prices, hazards, SLO tiers, and portability mechanisms, then deployed with guardrails.
- Motivation: HELIOS targets continuous multi-cloud service control, where provider prices, spot hazards, egress costs, and outages create a policy problem beyond single-cloud autoscalers.The paper frames the open question as who writes and can trust decision logic that exploits this changing heterogeneity.
- Approach: HELIOS instead evolves Python orchestration programs offline against a trace-driven simulator, then deploys only the champion inside a guardrail layer.The LLM receives a fixed policy interface, real catalog information, and later measured fitness tables to generate mutations, crossovers, and redesigns.
- Contributions: The system contributes a fixed-interface evolutionary synthesis loop, five mechanically enforced safety invariants, and an archive of prompts, programs, and fitness logs.The claimed invariants include feasible packing, premium isolation from preemptible capacity, churn limits, residency and cooldown, and projection of infeasible proposals.
- Positioning: HELIOS extends prior cloud schedulers, sky-computing systems, spot optimizers, learning schedulers, and LLM discovery frameworks to guarded continuous multi-cloud control.Its distinction is combining a domain-specific policy interface and guardrail contract with evaluation of live-system robustness.
III. PROBLEM FORMULATION
The paper formulates SLO-tiered service placement as penalized online optimization over heterogeneous, failure-prone cloud capacity. It then quantifies why LLM inference must be amortized into an offline policy whose marginal control cost is microseconds.
- Catalog: The 24-instance catalog spans AWS, Azure, and GCP with on-demand prices, spot prices where available, capacities, and per-step interruption hazards.AWS hazards use published interruption buckets, while Azure and GCP use assumed mean lifetimes that are swept by up to 10×.
- Objective: A service-step is violated when the service is down or its host’s actual demand exceeds capacity in either resource dimension.Down states include pending, booting, migrating, and restarting.
- Objective: The penalized objective combines operating cost with standard- and premium-tier violation penalties, using βstd = $0.5 and βprem = $5.The 10× penalty asymmetry is used for evolution fitness and headline comparisons, while cost and tier-specific violation rates are also reported separately.
- LLM bottleneck: Naive per-service LLM scheduling would process 57,600 daily decisions for N=200, costing approximately $500–1,000 per day against a $240–520 capacity bill.A single seconds-scale LLM tail would also dominate the measured 1.9 ms complete per-step decision cost and extend interruption-recovery downtime.
- LLM bottleneck: HELIOS amortizes the LLM budget into five offline synthesis rounds that produce an approximately 40-line auditable program with microsecond marginal decision cost.This architecture preserves online control speed while retaining LLM-generated policy logic.
V. HELIOS DESIGN
HELIOS exposes a small Python policy interface and evolves candidate programs using measured simulator fitness. The deployed champion is separated from raw cloud state by a feature compiler and from execution risk by the architecture’s guardrail layer.
- A. Policy interface and context compiler: The policy interface contains four pure functions for global knobs, placement priority, candidate scoring, and migration urgency.The context compiler supplies per-service, per-candidate, and fleet-level feature dictionaries.
- A. Policy interface and context compiler: The context compiler converts raw cloud state into fixed features including demand, tier, residency, provider, price, hazard, capacity, boot, and egress information.Fleet features also include aggregate demand, time of day, pending count, provider price minima, and availability signals.
- B. LLM-driven evolution: The LLM begins from interface, catalog, and guardrail facts, then receives prior fitness tables and metric breakdowns to generate candidate programs with stated rationales.Generation 0 is prior-driven, while later generations use measured archive feedback rather than invented metric values.
- B. LLM-driven evolution: Figure 1 places the LLM in the offline evolution loop and deploys only the champion program within guardrails after context compilation.The artifact run used an in-session Claude Sonnet-class policy-writing LLM with exact inputs and responses archived.
- B. LLM-driven evolution: Each generation evaluates three or four candidate programs on 2-day N=200 training traces, ranks them by mean J, and stops after two generations without improvement.The executed algorithm maintains the archive by adding candidates and logging all metrics.
C. Guardrail layer and safety contract
The guardrail layer intercepts every policy proposal and mechanically enforces feasibility, tier isolation, bounded churn, residency, and projection rules. This converts unsafe generated behavior into bounded, measurable outcomes while the champion applies price- and hazard-aware placement logic.
- Guardrail enforcement: Every policy proposal passes through guardrails that enforce reservation-feasible packing, premium-tier isolation, migration budgets, residency and cooldown, and projection of invalid actions.The contract is designed to hold independently of the evolved program.
- Safety contract: Reservation checks account for max demand and reservation requirements while treating in-flight services as occupying target reservations; invalid or failing proposals are dropped or projected.Exception handling can replace a policy failure with an empty proposal set so mechanism execution continues.
- Guardrail enforcement: Premium services are never offered spot candidates, and migrations are capped at ⌈0.05N⌉ per step with six-step residency and cooldown requirements.The executor sorts migrations by urgency and truncates them at the budget.
- Safety contract: The safety contract turns catastrophic policy bugs into finite fitness penalties and bounds behavior under distribution shift.This empirical role enables safe search over generated programs rather than merely documenting formal invariants.
- Champion policy: The champion places premium services first toward the cheapest on-demand pool and prices standard capacity using amortized interruption cost, with larger-instance tie-breaking reducing exposure.The evolution log reports interruption events falling from 18 to 8 per training episode, while migration rules respond to changing hazard and price conditions.
- Evaluation substrate: The simulator executes 5-minute trace-driven episodes with interruption hazards, downtime, retirement, and catalog billing, using real PlanetLab, Bitbrains, Azure, and provider-price data.The implementation also includes plug-in policies, guardrails, and metrics in the simulator core.
- Evaluation substrate: The MILP baseline uses reservation-oracle information and re-plans when price or availability changes, providing an oracle-informed planning comparison.Its executor follows plans with a dominant affinity bonus.
VII. EXPERIMENTAL SETUP
HELIOS is evaluated through trace-driven multi-cloud experiments using held-out workloads, explicit baselines, and metrics that combine operating cost with SLO violations. Evolution discovers structural policy changes, rejects harmful variants, and remains robust to a simulator accounting correction.
- Trace suites: HELIOS trains on two PlanetLab days and evaluates on held-out PlanetLab, Azure, and Bitbrains workloads with unseen seeds and N=200 services.Bitbrains adds a real-memory dimension absent from the training distribution.
- Baselines and metrics: The evaluation compares HELIOS with single-cloud, autoscaling, calibrated multi-cloud, hand-written, MILP, and DQN baselines using cost, SLO violations, migrations, interruptions, and J.J is reconstructed from per-tier violation rates with below-1% error versus exact per-seed premium counts.
- Evolution dynamics: 20.9% training-fitness improvement over the hand-written seed plateaus after generation 1, while later parameter refinements regress and evolution stops under a two-generation patience rule.Figure 2 tracks all candidates and the best-so-far trajectory on the two-day training trace.
- Evolution dynamics: Structural changes drive the gains: premium-first tier segregation improves placement, while bigbox spawning reduces interruptions from 18 to 8 by shrinking exposed instance count.Seven later parameter-level refinements all regressed, and two bolder proposals were decision-identical to the champion on training.
- Evolution dynamics: Negative candidate families are filtered by measured fitness: utilization draining caused extreme violations and migrations, while removing the premium min-price anchor added $221 in on-demand spend.Guardrails kept failed episodes finite, and the archived analysis extracted a rule that tiebreak coefficients must remain below the relevant pool’s price spread.
- Evolution dynamics: An arithmetic error that over-penalized hazards by about 20× was diagnosed and corrected, and the corrected accounting left every generation ranking and the champion trajectory unchanged.The corrected simulator logs are released with the artifact.
B. Main comparison
Across held-out trace suites, HELIOS achieves the lowest penalized cost while trading raw cost, SLO performance, and robustness differently from competing baselines. Its gains generalize across workloads and depend on both multi-cloud choices and guardrails under hazard stress.
- Main comparison: −45% penalized cost versus SingleCloud-BFD and −9.6%, −9.6%, and −5.7% versus Greedy on PlanetLab, Azure, and Bitbrains, respectively.Table II and Figure 3 report the lowest penalized cost for HELIOS on all three held-out suites.
- Main comparison: HELIOS matches MILP-Plan’s best premium SLO on PlanetLab and Bitbrains while costing 40–67% less on CPU-only suites and 124% less on Bitbrains.MILP-Plan’s premium violation rates are 0.133% and 0.116% on those suites, but its planning is brittle under disturbance.
- Generalization: The advantage transfers from two PlanetLab training days to Azure and memory-dimensioned Bitbrains because tier segregation, exposure minimization, and amortized hazard pricing reflect catalog structure.These genes are presented as properties of the catalog rather than the training trace.
- Ablations: Removing spot capacity raises cost 140%, while restricting the fleet to AWS, GCP, or Azure raises cost 111%, 168%, or 5%, respectively.The Azure-only variant benefits from this price snapshot but lacks provider-outage fallback; migration matters mainly under disturbances.
- The price and value of guardrails: Under 10× hazard stress, unguarded policies produce 97–98% premium downtime, whereas guarded HELIOS holds premium violations to 0.17–0.18% at a graceful 100% cost increase.At calibrated hazards, removing guardrails superficially cuts the bill 70% without statistically changing violations.
E. Sensitivity
Sensitivity analyses show that guardrails preserve SLO behavior under spot-hazard stress, while adaptive risk pricing and exposure-minimizing packing improve robustness as conditions and fleet size change.
- Parameter sensitivity: By 4× hazard, Greedy cost exceeds $1,445 while HELIOS costs $691 because HELIOS adapts its amortized interruption-risk constant.Greedy abandons spot near 2× hazard, whereas HELIOS continues pricing cheap spot capacity according to interruption downtime.
- Parameter sensitivity: At N=400, HELIOS is 9% cheaper in raw dollars than Greedy, while at N=100 Greedy’s simpler policy is better.The crossover reflects the benefit of exposure-minimizing packing at larger fleet sizes.
- Robustness scenarios: During a mid-episode Azure zone outage, HELIOS rebuilds the displaced Azure fleet on GCP and AWS within a few control periods.The case study evaluates the provider-failure fallback that Azure-only operation lacks.
- Deployment overhead: The deployed artifact costs approximately 10 µs per service per step and scales linearly on one CPU core.This overhead is reported as six orders of magnitude below per-decision LLM inference and fits inside 5-minute or 5-second control periods.
- Scope and limitations: Sensitivity conclusions remain bounded by trace-driven simulation, a single 2026-07-03 price snapshot, synthesized memory demand, and untested continual re-evolution.The evaluation also excludes network latency, data gravity, GPU capacity, and intra-provider multi-region placement.
X. CONCLUSION
HELIOS places LLMs off the critical path by evolving small, auditable policies offline and deploying only a guarded champion. The released artifacts support reproducibility and expose both successful policies and measured failures.
- HELIOS evolves small policy programs offline, selects them by measured fitness, and deploys the champion inside guardrails whose safety contract holds regardless of generated code.
- The complete artifact includes the simulator, evolved policies, prompt/response archive, experiment logs, and table-generation scripts for reproducing every reported number.
APPENDIX A GENERATION-0 SYNTHESIS PROMPT (ABRIDGED)
The generation-0 synthesis prompt specifies the fitness definition, policy interface, environment facts, guardrail contract, and a requirement for diverse design hypotheses. Later generations add measured historical fitness and failure information.
- The fitness definition and training configuration are part of the archived synthesis prompt.
- The prompt supplies the policy API, feature fields, provider price and hazard facts, egress rates, downtime constants, and guardrail contract G1–G5.
- The deliverable specification requests three diverse policy programs that prioritize different design hypotheses rather than parameter variants.
- From generation 1 onward, the prompt includes measured fitness tables, cost decompositions, violation rates, and migration/interruption counts for prior candidates.
APPENDIX B CHAMPION POLICY (COMPLETE)
The champion policy combines priority ordering, market-aware scoring, and migration-related penalties, with lineage and measured provenance documenting how its design choices evolved.
- The evolved lineage progresses from gen0_d_segregator through gen1_d_bigbox to gen2_a_amortized_bigbox.
- Measured provenance links premium-first ordering, cheapest-on-demand anchoring, amortized interruption pricing, bigbox spawning, and fill-existing or friction terms to observed fitness or interruption changes.
- The policy exposes a headroom value of 0.08 and defines a migration-urgency function alongside its scoring and priority functions.
- The policy prioritizes premium services using 10.0 × premium plus CPU demand, placing premium status ahead of ordinary resource demand.
- Its score penalizes vCPU price, deviation from minimum on-demand price, boot time, and egress while rewarding available CPU and adding market-dependent adjustments.
APPENDIX C PER-GENERATION FITNESS LOG
Table VII contains all 19 evolved candidates and reports both training fitness and a corrected-accounting rerun, with both logs included in the artifact.
- Table VII covers all 19 evolved candidates and reports training fitness alongside a corrected-accounting rerun.Both logs are included in the artifact.