Source-linked AI summary
CORAL: An LLM-Native Harness for Production Recommender Systems
Muhammad Rafay Azhar, Yuhang Zhou, Gilbert Jiang, Yuchen Wang, Rahul Sharma, Matthew DeSousa, Jiayi Liu, Xin Guo, Lizhu Zhang, Xiangjun Fan
TL;DR
Production recommender systems need continual optimization as their environments and operating constraints change, but human experimentation is slow and reactive. CORAL places an LLM in a constrained closed loop around a live recommender, using observations, memory, and tools to update decisions from measured outcomes. Across two large-scale social-platform A/B deployments, it improves engagement or serving efficiency across the engagement–efficiency frontier.
Problem
Human-driven recommender optimization is slow, reactive, and limited by engineering effort, while few LLM systems continually act on live recommenders and learn from measured effects.
Method
CORAL uses an LLM policy with observations, memory, deterministic tools, constrained optimization, and guardrails to reconfigure a live recommender each cycle under a fixed operating budget.
Results
Across two large-scale social platforms evaluated with A/B experiments, CORAL improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other.
Takeaways & Limitations
A single harness generalizes across recommender services and decision types while its decisions improve as the loop iterates.
Takeaways & Limitations
Both studies focus on constrained allocation of recommender resources, and the loop still operates under human supervision.
Abstract
from arXiv · showhide
Production recommender systems shape what billions of people see, and sustaining their performance requires continual optimization: as content, user behavior, and upstream models shift, the choices governing retrieval, ranking, and serving must be revisited. Traditionally, human engineers test such changes through online experiments--a slow, reactive process limited by engineering effort, leaving parts of the system unrevised as conditions change. Although large language models have been applied to ranking, user modeling, and offline model development, few systems place an agent in a continual closed loop that acts on a live recommender and learns from the measured effects of its decisions. We present CORAL (Constraint-Optimized Recommender via an Agentic Loop), an LLM-native harness that closes this loop: each cycle, the agent observes operating signals, reasons over a memory of past decisions and outcomes, and invokes tools--including a numerical optimizer that keeps changes within a fixed operating budget--to reconfigure the recommender, with measured outcomes informing the next cycle. We formulate this as a partially observed, non-stationary, constrained optimization problem in which the policy improves in context, without parameter updates, from its prior actions. Across two large-scale social platforms, evaluated with A/B experiments, the same harness improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other, spanning the engagement-efficiency frontier. Performance improves as the loop iterates, suggesting that a single agentic loop can automate continual optimization work traditionally performed by human algorithm engineers under explicit guardrails.
1 Introduction
CORAL addresses the slow, reactive, and costly human process of revising production recommender systems with an LLM-native harness that continually reasons over measured outcomes and reconfigures a live recommender under constraints. Across two large-scale social-platform deployments, it improves engagement or serving efficiency, including benefits for low-signal and new users.
- Motivation: Human-driven recommender optimization is slow, reactive, conservative, and costly, leaving parts of evolving systems unrevised and low-signal users underserved.These limitations arise from small experimental steps, infrequent revisiting, and aggregate metrics that underrepresent low-signal and new users.
- Approach: CORAL is an LLM-native harness that closes a continual optimization loop around a live recommender.The agent observes operating signals, reasons over memory of prior decisions and measured effects, and invokes tools to produce changes.
- Results: Two large-scale A/B deployments span the engagement–efficiency frontier across different recommender surfaces and decision types.One improves engagement, including for low-signal and new users, while the other improves serving efficiency without degrading engagement.
- Results: The same harness generalizes across surfaces and decision types, and its decisions improve as the loop iterates.The work targets continual optimization that would otherwise require repeated manual engineering effort.
- Approach: The harness formulates agent-driven recommender optimization as a partially observed, non-stationary, constrained problem.Its policy refines decisions from the measured effects of prior actions rather than through parameter updates.
2 Related Work
Prior LLM-agent research develops tool use, memory, user modeling, recommendation, and automation of offline system development. CORAL differs by placing an agent in a continual loop that optimizes a live recommender’s control surface.
- Tool use: Tool-using LLM agents interleave reasoning, tool selection, execution, and observation to access information and specialized capabilities.Research also advances tool discovery, multi-tool planning, API-call generation, and generalization to unseen tools.
- Memory: Memory mechanisms preserve observations, strategies, failures, and reflections across interactions to inform future planning and decision-making.These systems retrieve accumulated experience rather than relying only on parametric knowledge.
- Recommender agents: Some LLM-agent work evaluates recommenders by simulating users, while other systems invoke conventional recommender models independently for each interaction.These lines primarily address evaluation methodology or recommendation generation rather than continual live-system optimization.
- System optimization: More recent approaches use agents to search over models, code, and configurations, but usually operate on offline development pipelines, recommendation models, or user representations.The related work therefore distinguishes system-development automation from optimization of a live recommender control surface.
3 Problem Formulation
The paper models a recommender as a time-varying constrained optimization problem: an agent selects feasible configurations whose engagement and operating cost depend on changing context. Its policy uses aggregate observations and recent memory to keep decisions near a moving optimum.
- Problem setup: A recommender configuration s determines engagement J_t(s) and operating cost c_t(s), with the constraint c_t(s) ≤ B.The operator specifies the objective and bounded resource constraint; the formulation also covers minimizing cost at a target objective.
- Problem setup: The formulation supports either maximizing engagement under a cost budget or minimizing cost while holding engagement at a target level.The two deployments instantiate one form each.
- Configuration: The control surface comprises N units with bounded continuous settings or discrete choices, and each cycle selects a configuration s_t.Examples include retrieval-source budget shares and segment-level serving treatments.
- Moving optimum: Because operating context changes, the oracle-optimal configuration moves across cycles, so unchanged settings become increasingly suboptimal.The agent therefore minimizes cumulative shortfall from the moving best feasible configuration rather than converging to one fixed optimum.
- Agent policy: The LLM policy maps aggregate observations and recent memory to the next configuration, using analysis, retrieval, attribution, and numerical optimization.Memory contains recent observations, chosen configurations, and attributed outcomes; the optimizer projects candidates into the feasible region.
4 The CORAL Harness
The CORAL harness surrounds a general-purpose language model with context, tools, persistent memory, and a fixed-cadence feedback loop for budget-feasible recommender optimization. Measured effects of deployed configurations inform subsequent decisions under human oversight and guardrails.
- 4 The CORAL Harness: The harness combines current recommender context, analytical tools, persistent memory, and a fixed-cadence loop around the language model.The model reasons over current observations and prior decisions, assessments, and outcomes.
- 4.2 Tools: A constrained optimizer projects proposed per-unit adjustments onto the budget-feasible set before deployment.It redistributes changes when a proposal would overspend and guarantees the deployed configuration satisfies the operating budget.
- 4.3 The Optimization Loop: Each cycle applies the accepted configuration, measures its effect with an A/B experiment, and writes the outcome back to memory.This feedback allows the agent to reinforce helpful changes and reverse unhelpful ones.
- 4.3 The Optimization Loop: CORAL uses a three-day cadence and a three-cycle memory horizon as sensible deployment defaults rather than tuned settings.The paper notes that optimal cadence and memory horizon may vary with seasonality and remain an area for exploration.
- 4.3 The Optimization Loop: The loop runs autonomously under human supervision, with feasibility checks, bounded-change limits, and safety constraints supporting deployment.Supervision can be progressively replaced by automated guardrails as confidence in the agent grows.
5 Case Studies
Across two large-scale deployments, CORAL transfers between continuous retrieval-budget allocation and discrete serving-treatment assignment. The studies improve engagement or serving efficiency while preserving the fixed-budget constraint, and the loop improves decisions over successive rounds.
- 5 Case Studies: The two studies use A/B experiments on different services and control problems, spanning engagement and serving efficiency.One targets retrieval-source budgets; the other allocates serving treatments across user segments.
- 5.1 Allocating Retrieval Budget Across Candidate Sources: In the video service, CORAL reallocates a fixed retrieval budget across complementary candidate sources using per-source contribution, conversion, and funnel signals.The agent trims poorly converting sources and redirects budget to sources delivering engaged views efficiently.
- 5.1 Allocating Retrieval Budget Across Candidate Sources: Across three successive rounds, the retrieval-budget policy moved from a small watch-time gain, through a neutral overcorrection, to a converged configuration with stronger session and watch-time gains.The progression was non-monotonic but used measured effects to refine later allocations.
- 5.1 Allocating Retrieval Budget Across Candidate Sources: 0.16% increase in video-viewing sessions and 0.15% increase in total watch time were achieved across all users at no additional serving cost.The experiment spanned millions of users, and the reallocation consolidated rather than expanded the retrieval budget.
- 5.1 Allocating Retrieval Budget Across Candidate Sources: The agent also produced segment-specific retrieval allocations that shifted budget toward content and current-context signals for low-signal and newly joined users.These signals remain reliable when per-user engagement data is sparse.
- 5.2 Allocating Serving Capacity Across User Segments: In the second service, CORAL assigns discrete serving treatments to user segments under a fixed compute budget, increasing treatment where added compute yields more engagement.Treatment choices vary retrieval, ranking, caching, and prefetching intensity.
- 5.2 Allocating Serving Capacity Across User Segments: 44% greater savings were obtained in the second round while engagement remained statistically unchanged.The agent widened a cost-saving allocation to additional segments after evaluating the prior decision.
- 5.3 Discussion: The same harness generalized across continuous and discrete controls, compressing manual tuning from several engineer-weeks to a few autonomous days.The reported deployments improved complementary axes without per-decision engineering effort.
6 Computational Cost
CORAL’s language-model inference cost is charged per control-surface decision cycle rather than per user or request. The deployments use only a handful of calls per cycle, making total inference cost small relative to the reported system outcomes.
- 6 Computational Cost: CORAL incurs language-model cost per decision cycle, not per individual user or request served.Because the agent acts on the control surface, cost depends on cycle-level decisions rather than traffic volume.
- 6 Computational Cost: The total cost rises proportionally when cadence k is shortened, while remaining bounded per cycle by the model’s output-token limit.The number of calls per cycle is determined by the partitioning of the control surface into decision groups.
- 6 Computational Cost: Each deployment used only a handful of calls per cycle and accumulated approximately 10^6 tokens over several cycles.The token quantities were estimated from prompt and payload sizes because the pipelines did not log token usage.
- 6 Computational Cost: The end-to-end inference cost of each deployment was on the order of tens of U.S. dollars under representative frontier-LLM pricing.The paper characterizes this cost as negligible relative to the engagement gains and operating-cost savings reported for the deployments.
7 Conclusion
CORAL places an LLM in a continual closed loop around a live recommender, using memory, tools, and budget-feasible changes to refine decisions from measured outcomes. Across two large-scale social platforms, it improved engagement and serving efficiency, while the authors identify supervision, decision-scope, and evaluation-setting limits.
- CORAL puts an LLM in a continual closed loop around a live recommender rather than optimizing models offline or serving recommendations directly.Each cycle combines operating signals, decision memory, tool-mediated changes, and measured outcomes.
- The harness formulates recommender control as a partially observed, non-stationary, constrained optimization problem solved through in-context refinement without retraining.
- Across two large-scale social platforms, A/B experiments showed improved engagement, including for low-signal and new users, alongside improved serving efficiency.
- The approach still requires human supervision, and both studies cover only constrained resource allocation rather than retrieval or ranking logic.
- The evidence relies on costly, setting-specific A/B experiments, leaving standardized pre-deployment evaluation of agent-driven system optimization open.
A Sample Prompt
Figure 2 presents an abstracted per-cycle decision prompt, while Table 3 describes how the two case studies instantiate its placeholders.
- Figure 2 shows runtime-filled placeholders in the per-cycle prompt, followed by bounded per-unit proposals and constrained optimization before deployment.
- Table 3 maps the abstract prompt template to the concrete configurations used in the two case studies.