PaperlayerView exact revision on arXiv

Source-linked AI summary

Cluster Workload Allocation: Semantic Soft Affinity Using Natural Language Processing

Leszek Sliwko, Jolanta Mizeria-Pietraszko

arXiv:2601.09282v2cs.AIcs.DCcs.LGcs.SE

TL;DR

Cluster scheduling often requires rigid configuration that separates human intent from machine-level directives. This paper introduces an LLM-based scheduler extender that translates natural-language allocation hints into soft-affinity scoring logic. The prototype demonstrates high parsing accuracy and simplified intent expression, while synchronous LLM processing remains a production-readiness limitation.

  • Problem

    Existing schedulers require rigid, machine-readable rules and complex YAML, creating a semantic gap between human preferences and low-level configuration.

  • Method

    The paper uses an LLM integrated with a Kubernetes scheduler extender to translate natural-language allocation hints into weighted soft-affinity scheduling logic.

  • Results

    The prototype achieved superior or equivalent placement across six scenarios and required only a single allocation-hint annotation for behaviors needing complex Kubernetes configurations.

  • Takeaways & Limitations

    The results support semantic soft affinity as a viable proof-of-concept for simplifying workload orchestration.

  • Takeaways & Limitations

    Synchronous LLM calls in the /prioritize path introduce unacceptable latency for real-world use, requiring asynchronous architecture for deployment.

Abstract

from arXiv · show

Cluster workload allocation often requires complex configurations, creating a usability gap. This paper introduces a semantic, intent-driven scheduling paradigm for cluster systems using Natural Language Processing. The system employs a Large Language Model (LLM) integrated via a Kubernetes scheduler extender to interpret natural language allocation hint annotations for soft affinity preferences. A prototype featuring a cluster state cache and an intent analyzer (using AWS Bedrock) was developed. Empirical evaluation demonstrated high LLM parsing accuracy (>95% Subset Accuracy on an evaluation ground-truth dataset) for top-tier models like Amazon Nova Pro/Premier and Mistral Pixtral Large, significantly outperforming a baseline engine. Scheduling quality tests across six scenarios showed the prototype achieved superior or equivalent placement compared to standard Kubernetes configurations, particularly excelling in complex and quantitative scenarios and handling conflicting soft preferences. The results validate using LLMs for accessible scheduling but highlight limitations like synchronous LLM latency, suggesting asynchronous processing for production readiness. This work confirms the viability of semantic soft affinity for simplifying workload orchestration and presents a proof-of-concept design.

I. INTRODUCTION

Cluster schedulers traditionally require rigid, machine-readable rules, creating a semantic gap between human workload preferences and low-level configuration. This paper proposes an LLM-based semantic scheduling layer that translates natural-language intent into Kubernetes soft-affinity logic.

  • Traditional schedulers require operators to express scheduling goals in rigid, machine-readable syntax.
  • Complex YAML configuration makes workload placement time-consuming, error-prone, and dependent on deep system expertise.
  • The proposed scheduler uses an LLM to translate natural-language preferences into structured directives for human-aligned placement decisions.
  • The design shifts Kubernetes scheduling from syntax-bound configuration toward semantic, intent-based expressions that reduce cognitive overhead.
  • The prototype combines live cluster-state awareness with dynamic intent logic for nuanced soft-affinity decisions such as colocation and zone spreading.

B. MULTI-OBJECTIVE SCHEDULING

Cluster scheduling is a multi-objective problem in which preferences such as spreading and colocation can conflict. The paper frames allocation hints as natural-language inputs that an LLM translates into quantitative weights for weighted-sum scheduling.

  • Scheduling must trade off conflicting objectives, including fault-tolerance-oriented spreading and latency-oriented colocation.
  • Scalarization converts multi-objective scheduling into a single objective, commonly through a weighted-sum method.
  • The allocation hint captures unstructured user preferences and translates them into quantitative weights for the scheduler’s multi-objective cost function.
  • Intent-based systems aim to let operators specify desired outcomes rather than low-level procedures, reducing configuration burden and cognitive overhead.
  • Natural-language intent input is proposed as a way to avoid the specialized syntax and steep learning curve associated with traditional intent-based systems.

III. PROTOTYPE AND TESTBED DESIGN

The prototype uses a reproducible nine-node Minikube testbed with synthetic multi-level topology, a cluster-state cache, and a Kubernetes scheduler extender. These components support controlled evaluation of topology-aware and resource-aware placement.

  • The testbed is a self-contained Minikube environment with a custom scheduler extender that converts natural-language intents into node-scoring decisions.
  • The testbed emulates a hierarchical data-center topology to evaluate proximity, resource-affinity, and workload-spreading policies.
  • The architecture includes a multi-node cluster, real-time cluster-state cache, stateful score extender, and AWS Bedrock-backed intent analyzer.
  • The Kubernetes 1.31.4 cluster contains nine nodes: one control-plane node and eight worker nodes.
  • The cluster-state cache avoids repeated API-server queries by maintaining an in-memory representation of nodes and pods.
  • Cached metadata enables O(1) access for scoring, compared with O(N) complexity and network I/O for full API queries.

C. SCORE EXTENDER SERVICE

The score extender receives allocation hints, parses them into structured intents, and scores candidate nodes using resource, topology, affinity, and spreading logic. Its intent analyzer uses prompted LLM semantic parsing to produce machine-readable scheduling data.

  • The /prioritize endpoint parses allocation hints, assigns intent weights, and computes final scores for each candidate node.
  • Resource preferences compare requested GPU or memory values with corresponding node metadata from the cluster-state cache.
  • Topology, dynamic-affinity, and spreading intents use node labels and effective pod-state information to guide placement.
  • Semantic parsing translates unstructured natural-language hints into JSON objects containing scheduling intents and metadata.
  • The LLM returns confidence scores and strength multipliers alongside detected intents to represent perceived preference importance.
  • The analyzer prompt matches user phrases to predefined intents and requires strict output formatting for direct scheduler parsing.

IV. SCORING MODEL

The scoring model evaluates each candidate node by summing weighted utilities for the unique scheduling intents extracted by the Intent Analyzer. Utility combines equalized base weighting, analyzer confidence, linguistic strength, and intent-specific node evaluation.

  • Each candidate node receives a raw score formed by summing the weighted utility of every identified scheduling intent.
  • The utility function μ(n, i) applies an intent-specific evaluation to node n.
  • Each intent’s contribution is scaled by analyzer confidence, linguistic strength, and its domain-specific evaluation function.
  • β = 100/|ℐ| dynamically gives multiple directives equal initial influence through a base weight.

B. EVALUATION LOGIC CATEGORIES

The evaluation logic changes according to intent category, covering binary preferences, avoidance penalties, and workload-distribution objectives. These categories let the scorer represent both positive and negative preferences and spreading requirements.

  • Binary preference logic evaluates whether resource or topology attributes satisfy a preference such as prefer_gpu.
  • Avoidance penalty logic subtracts a weighted penalty for negative constraints such as avoid_zones.
  • Distribution and spreading logic computes a least-loaded score using deployment pod counts across a topology domain.

C. NORMALIZATION AND DETERMINISTIC SELECTION

Raw scores are normalized for Kubernetes extender compatibility, then adjusted so exactly one highest-scoring node wins deterministically. The section also catalogs intent classes and their metadata requirements.

  • Normalization: Raw scores are clamped at zero when negative and scaled to the [0,100] range relative to the highest score in the current iteration.
  • Deterministic selection: The highest-scoring node, with alphabetical tie resolution, receives 100 while all other nodes are constrained to [1,99].
  • Intent metadata: Intent classes are documented with descriptions and the metadata fields required for extracting their scheduling parameters.
  • Intent metadata: Examples include preferences for public IPs, network bandwidth, network type, ephemeral storage, SSDs, and TPU cores, alongside administrative no-schedule control.

D. TOPOLOGICAL PROXIMITY QUANTIFICATION

Topological proximity quantification estimates how closely a candidate node is located to existing pods from the target deployment. It combines pod counts across rack, zone, and region with decaying locality weights, then normalizes the result.

  • The proximity function aggregates target-deployment pod counts across rack, zone, and region boundaries.
  • A decaying weight vector prioritizes tighter locality when combining topology-domain counts.
  • The raw proximity score P(n) is computed as the dot product of topology counts and their domain weights.
  • P(n) is normalized against the maximum proximity score among candidate nodes to prevent unbounded growth in large clusters.
  • Hierarchical weighting favors nodes sharing a rack over nodes sharing only a zone or region.

E. TEMPORAL STATE CONSISTENCY

The system uses a hybrid state model to reduce race conditions during burst scheduling by combining confirmed cluster state with short-lived tentative decisions. Affinity and anti-affinity scoring therefore operates on an effective pod set that includes both persisted and pending placements.

  • The hybrid state model addresses latency in asynchronous Kubernetes API updates that can cause race conditions during high-velocity burst scheduling.
  • Confirmed pods from the Kubernetes API and tentative local scheduling decisions are maintained as separate state sets.Tentative decisions are constrained by a 10-second TTL.
  • The effective pod set is the union of confirmed and tentative pods for affinity and anti-affinity calculations.This lets subsequent burst requests account for pending placements before persistence in etcd.
  • Dynamic scoring functions evaluate nodes against the most current approximation of cluster state.

F. MULTI-OBJECTIVE SCALARIZATION

The scheduler models placement as multi-objective optimization and resolves competing preferences through weighted scalarization. It then applies lexicographical ordering among tied optimal candidates to make decisions deterministic and reproducible.

  • The scheduling problem seeks a node that maximizes a vector of objective functions.
  • Conflicting objectives such as spreading and affinity are resolved in real time through linear scalarization.
  • Each dynamic objective weight combines its base weight, confidence, and strength.
  • The global scalarized objective is implemented as the dot product of the weight and objective vectors.
  • Among nodes achieving the maximum score, the scheduler selects the lexicographically smallest candidate.This ordering avoids random tie-breaking and supports reproducible orchestration decisions.

V. SYSTEM EVALUATION

The evaluation measures both the Intent Analyzer’s translation accuracy and the quality and efficiency of the resulting pod placements. These experiments use a prototype testbed to assess the proposed scheduling paradigm.

  • The evaluation uses quantitative experiments on a prototype testbed to assess the semantic scheduler.
  • The experiments measure how accurately natural-language allocation hints become structured intents and metadata.
  • The evaluation also examines the quality and efficiency of the resulting pod placements.

A. INTENT RECOGNITION ACCURACY

The Intent Analyzer was evaluated on a structured ground-truth dataset covering varied natural-language scheduling hints and multiple intent types. Top-tier LLMs achieved very high intent-classification accuracy, while the Regex baseline was substantially less accurate despite near-instantaneous latency; multilingual tests also remained strong.

  • The 314-prompt ground-truth dataset covers all 25 intent types, paraphrases, complex combinations, and invalid inputs.Each prompt is paired with a manually authored expected JSON object.
  • Iterative ground-truth evaluation, failure analysis, and targeted prompt refinement were essential to reaching satisfactory parsing performance.
  • Explicit data-type, naming, default-handling, and negative-constraint instructions improved metadata accuracy, while overly strict rules increased missed values.
  • 29.62% Subset Accuracy was achieved by the Regex Engine, compared with over 95% for the best LLMs.The Regex Engine had sub-millisecond average and P95 latencies but struggled with semantic nuances and list extraction.
  • 97.45% Subset Accuracy was achieved by Amazon Nova Premier, followed by 95.86% for Mistral Pixtral Large and 93.63% for Amazon Nova Pro.These top-tier models also achieved Macro F1-scores of at least 0.95.
  • 93.62% accuracy on Spanish hints and 90.32% on German hints were obtained using Amazon Nova Pro.

B. SCHEDULING EFFICIENCY AND QUALITY

The evaluation compared the intent-driven scheduler with baseline Kubernetes configurations across six scenarios, covering topology, resources, colocation, quantitative preferences, and conflicting intents. Results showed equivalent or improved placement in the tested cases, while exposing limitations in conflict handling.

  • Evaluation Design: Six distinct scenarios compared the prototype’s scheduling decisions against baseline Kubernetes configurations.The scenarios evaluated topology spreading, resource affinity, rapid-burst colocation, quantitative resource preference, and system boundaries.
  • Scenario A - Topology Spreading: The intent-driven scheduler achieved a perfect 3:3 replica split across two zones in the topology-spreading scenario.The baseline scheduler using topologySpreadConstraints achieved the same placement.
  • Scenario D - Rapid Burst Colocation: All 20 replicas were placed on one distinct node by both schedulers during rapid-burst colocation testing.The baseline used a hard podAffinity rule, while the intent-based scheduler validated the recent placements cache during high-velocity creation.
  • Scenario E - Quantitative Resource Preference: 16.7% of pods were placed on the correct high-speed node by the baseline scheduler in the quantitative resource-preference scenario.Only 1 of 6 pods matched the intended node because other Kubernetes plugins outweighed the soft preference.
  • Scenario F - System Limitations: Scenario F explicitly tested the additive scorer’s handling of conflicting intents as a current system boundary.The evaluation therefore assessed both prototype capabilities and limitations rather than only demonstrating successful placements.

VI. CONCLUSIONS

The prototype reduced configuration effort while demonstrating effective intent-based placement, but its LLM-dependent and prototype-scale architecture limits production readiness. Evaluation also exposed parsing, security, latency, and scalability concerns requiring further work.

  • Conclusions: Complex Kubernetes YAML baselines highlighted the prototype’s reduction of cognitive overhead for expressing scheduling preferences.Scenarios A and C required multi-level topology, affinity, and anti-affinity configurations.
  • Conclusions: LLM integration remained sensitive to prompt and schema design, with hallucinated intents, metadata errors, and strength misclassification risking unintended scheduling decisions.A critical directive misclassified from strength 1.5 to 1.0 can allow less important factors to dominate.
  • Conclusions: Production deployment also requires secure extender networking, protection against prompt injection and resource exhaustion, and broader evaluation of compound and conflicting prompts.The current evaluation dataset contains 314 prompts and primarily covers one or two intent objects.
  • Conclusions: The prototype resolved contradictory soft preferences through additive weighted scoring, but a future cost-function optimizer is needed for more principled conflict handling.The current approach schedules a best-fit pod by summing weighted scores.
  • Conclusions: Synchronous LLM calls introduced P95 latency of 0.87s to 5.34s, while the single-threaded Flask server constrained production throughput and scalability.The paper recommends separating intent analysis from scheduling execution for production use.
Loading 2601.09282v2…