Source-linked AI summary
AgentCgroup: Understanding and Controlling OS Resources of AI Agents
Yusheng Zheng, Jiakun Fan, Quanzhi Fu, Yiwei Yang, Wei Zhang, Andi Quinn
TL;DR
AI coding agents create resource-management challenges because tool-driven execution has rapid, unpredictable demands in shared sandboxed infrastructure. This paper characterizes those dynamics across 144 tasks and two models, then presents AgentCgroup, an intent-driven eBPF controller aligned with tool-call boundaries. The characterization identifies OS execution as 55–60% of latency, memory as the concurrency bottleneck, and bursts reaching 15.4× peak-to-average; preliminary evaluation reports 29% lower high-priority P95 latency under multi-tenant memory contention.
Problem
Shared infrastructure needs efficient resource management for AI coding agents because diverse sandboxed tool calls create distinct and rapidly fluctuating resource demands.
Method
AgentCgroup combines tool-call-aligned hierarchical cgroups, in-kernel eBPF enforcement, and runtime-adaptive policies to control agent resources.
Results
Across 144 tasks and two models, OS execution accounts for 55–60% of latency, memory is the primary concurrency bottleneck, memory bursts reach a 15.4× peak-to-average ratio, and demands are highly unpredictable.
Takeaways & Limitations
Agent resource controls must address granularity, responsiveness, and adaptability mismatches through tool-call-level domains, kernel-speed reaction, and adaptation to reconstructed execution strategies.
Takeaways & Limitations
The evaluation is limited to trace replay with a proof-of-concept prototype, while characterization covers one agent framework and one benchmark.
Abstract
from arXiv · showhide
AI agents are increasingly deployed in multi-tenant cloud environments, where they execute diverse tool calls within sandboxed containers, each call with distinct resource demands and rapid fluctuations. We present a systematic characterization of OS-level resource dynamics in sandboxed AI coding agents, analyzing 144 software engineering tasks from the SWE-rebench benchmark across two LLM models. Our measurements reveal that (1) OS-level execution (tool calls, container and agent initialization) accounts for 55-60% of end-to-end task latency; (2) memory, not CPU, is the concurrency bottleneck; (3) memory spikes are tool-call-driven with a up to 15.4x peak-to-average ratio; and (4) resource demands are highly unpredictable across tasks, runs, and models. Comparing these characteristics against serverless, microservice, and batch workloads, we identify three mismatches in existing resource controls: a granularity mismatch (container-level policies vs. tool-call-level dynamics), a responsiveness mismatch (user-space reaction vs. sub-second unpredictable bursts), and an adaptability mismatch (history-based prediction vs. non-deterministic stateful execution). We propose AgentCgroup, an intent-driven eBPF-based resource controller that exploits agents ability to declare resource needs and reconstruct execution strategies, using hierarchical cgroup structures aligned with tool-call boundaries, in-kernel enforcement via sched_ext and memcg_bpf_ops, and runtime-adaptive policies. Preliminary evaluation demonstrates improved multi-tenant isolation and reduced resource waste. AgentCgroup is open-source at https://github.com/eunomia-bpf/agentcgroup
1 Introduction
The paper characterizes OS-level resource dynamics in sandboxed AI coding agents and identifies mismatches between these dynamics and existing resource controls. It then introduces AgentCgroup, an intent-driven eBPF controller designed around tool-call-level adaptation.
- 55–60% of end-to-end task latency comes from OS-level execution, while LLM reasoning accounts for 40–45%.
- Memory, rather than CPU, is the primary bottleneck for multi-tenant concurrency density.
- 15.4× is the reported memory peak-to-average ratio, reflecting tool-call-driven bursts over a stable ∼185 MB framework baseline.
- Resource demands vary 20× across tasks and 1.8× across runs of the same task.
- Existing controls show granularity, responsiveness, and adaptability mismatches for tool-call-level, sub-second, non-deterministic agent execution.
- AgentCgroup combines tool-call-aligned hierarchical cgroups, in-kernel eBPF enforcement, and runtime-adaptive policies.
2 Background
AI coding agents combine language-model reasoning with iterative tool execution inside sandboxed containers. Linux cgroups and eBPF provide the hierarchical governance and programmable kernel enforcement primitives relevant to controlling these workloads.
- AI coding agents iteratively reason, issue structured tool requests, execute tools in sandboxed subprocesses, and return results to the language model.
- Each tool invocation spawns distinct processes with varying resource profiles.
- Linux cgroups organize tasks hierarchically and apply controller-specific accounting and enforcement across the hierarchy.
- memory.high provides soft throttling, while memory.max is a hard limit that triggers OOM when exceeded.
- eBPF supplies dynamically loadable kernel enforcement logic; sched_ext targets CPU scheduling and memcg_bpf_ops targets memory-control hooks.
3 Agent Workload Characterization
Across 144 SWE-rebench tasks, agent workloads are long-running, tool-driven, memory-intensive, and highly variable in both resource use and execution behavior.
- Execution model: 55–60% of end-to-end task latency comes from OS-level execution, including initialization and tool execution.LLM reasoning accounts for 40–45%; initialization contributes 31–48% and tool execution 13–24% over the full lifecycle.
- Execution model: Tool execution spans three orders of magnitude, with Bash dominating GLM tool time and Haiku combining Bash with sub-agent calls.GLM allocates 98.8% of tool time to Bash; Haiku allocates 80.6% to Bash and 17.3% to sub-agent calls.
- Resource dynamics: Memory, rather than CPU, limits multi-tenant concurrency because peaks reach 2–4 GB while average CPU utilization remains low.On the 128 GB platform, peak allocation supports only 32–64 instances, while CPU utilization remains below 36% of total capacity.
- Resource dynamics: Agent memory has a stable ∼185 MB framework baseline plus tool-call bursts, with command semantics determining peak demand more than tool names.Bash calls differ by 13.7× in peak memory; test execution, installation, and other subprocesses drive the bursts.
- Resource dynamics: 98.5% of Haiku memory bursts occur during tool calls, which occupy only 28.5% of sampling time.Bursts can change memory by up to 2.9 GB within one second, while LLM reasoning phases show stable, low usage.
- Resource dynamics: Resource behavior is highly non-deterministic, with 1.8× execution-time variance across repeated runs and up to 20× demand variation across tasks.Retry loops occur in 85%–97% of tasks and can retain memory, causing progressive accumulation of up to 502 MB unreleased.
4 Resource Management Mismatches
AI agent workloads expose granularity, responsiveness, and adaptability mismatches in existing resource controls because tool-call demands are bursty, unpredictable, and stateful.
- Agent resource demands vary at tool-call granularity, while existing controls apply a single policy at the container level.
- Peak demand occurs less than 2% of the time, so peak-based limits waste over 90% of allocated memory while average-based limits trigger OOM kills during bursts.
- 1–2-second bursts with change rates of several GB/s require kernel-speed reaction because their timing is unpredictable.
- User-space PSI reactions take tens of milliseconds, and Kubernetes VPA adjusts resources at restart or minute-level timescales, preventing effective within-execution control.
- History-based prediction fails because demands vary across tasks and runs, while kill-and-restart loses context and incurs 31–48% cold-start overhead.
- Graceful degradation through throttling or freezing is preferred to termination, while agents should adapt behavior in response to resource feedback.
5 AgentCgroup Design and Implementation
AgentCgroup addresses the three mismatches with tool-call-aligned resource domains, in-kernel enforcement, and bidirectional intent-driven adaptation.
- AgentCgroup aligns hierarchical cgroup v2 resource domains with tool-call boundaries while preserving overall workload budgets.
- eBPF enforcement at kernel cgroup points enables microsecond-level reactions without user-kernel round trips.
- AgentCgroup combines sched_ext for CPU scheduling, memcg_bpf_ops for memory control, and runtime-adaptive policies with graceful degradation.
- Agents declare expected resource needs before tool calls, and the wrapper maps hints such as memory:high or memory:low to per-call limits.
- Figure 8 evaluates OOM survival under 1100 MB total memory and HIGH-priority P95 allocation latency under 1300 MB.
6 Preliminary Evaluation
Trace replay shows that BPF enforcement improves survival and high-priority latency under multi-tenant memory contention with low overhead.
- 100% of processes complete under tight memory with BPF, versus 66% baseline survival when one LOW process is OOM-killed.
- HIGH-priority completion incurs only +2.8% overhead while LOW allocations are throttled 239 times.
- 29% lower HIGH-priority P95 allocation latency reduces 70.97 ms to 50.14 ms through reduced memory contention.
- P50 latency increases 0.3%, while total completion time decreases 1.1%.
- Kernel selftests confirm throttling precision within 2.3% relative error for a configured 2000 ms delay.
7 Conclusion and Future Work
The paper characterizes agent resource dynamics and presents AgentCgroup, while identifying trace replay, prototype scope, and limited workload coverage as boundaries for future evaluation.
- OS execution accounts for 55–60% of task latency, and memory reaches a 15.4× peak-to-average ratio.
- AgentCgroup addresses granularity, responsiveness, and adaptability through tool-call-aligned cgroups, microsecond-level enforcement, and resource adaptation.
- The current evaluation uses trace replay with a proof-of-concept prototype rather than live production-scale agent workloads.
- Characterization covers one agent framework and one benchmark, leaving validation across diverse tasks and frameworks for future work.
- The prototype mainly controls CPU and memory, while initialization overhead, large container images, and retry-induced accumulation remain open challenges.