Source-linked AI summary
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang
TL;DR
Recommender optimization requires costly, noisy trial-and-error over architecture, objectives, and training strategies, while unconstrained LLM search can allocate limited trials inefficiently. RecHarness routes edit directions with validation-guided bandits and uses an LLM for concrete code changes, consistently improving performance across recommendation scenarios and models.
Problem
Recommender optimization spans diverse design choices, while freely letting LLMs choose exploration directions can make costly, noisy trials inefficient under limited budgets.
Method
RecHarness uses validation-driven Thompson routing to select optimization arms, then has an LLM generate executable edits within the selected direction.
Results
Across two recommendation scenarios and eight models, RecHarness consistently improves performance and transfers across scenarios, model types, and evaluation objectives.
Takeaways & Limitations
RecHarness provides a controlled way to allocate limited optimization trials and shows practical value in industrial recommendation.
Takeaways & Limitations
The framework assumes recommender optimization is conducted under a finite time budget with specified training-validation data and a task-dependent validation metric.
Abstract
from arXiv · showhide
Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.
Introduction
RecSys optimization is a continuous, budget-constrained code-iteration problem in which effective automation must interpret validation logs and accumulate evidence across rounds. RecHarness addresses this challenge with bandit-routed, validation-guided trials that combine posterior updates, Thompson-style arm selection, and LLM-generated optimization edits.
- Motivation: Recommender optimization spans backbone architecture, training objectives, optimization strategies, and regularization across diverse recommendation tasks.The introduction identifies sequential recommendation, ranking and click prediction, and watch-time prediction as representative settings.
- Challenge: Each candidate modification consumes code-generation, execution, training, and validation budget, making trial efficiency a central systems challenge.The optimization process is described as continuous rather than a collection of independent one-shot experiments.
- Challenge: Automated optimizers must interpret logs, trends, and failure signals while accumulating cross-round evidence about effective optimization directions.Scalar validation feedback alone is insufficient for continuous recommender optimization.
- RecHarness: RecHarness organizes optimization into isolated training-validation trials with candidate arms whose posterior states are updated from validation feedback.The harness is presented as a validation-guided system for self-evolving recommender systems.
- RecHarness: Thompson-style bandit routing selects promising optimization arms under a limited trial budget, while the LLM generates edits for the selected arms.The mechanism jointly uses scalar validation scores and textual feedback through bandit routing and LLM reasoning.
Related Work
Prior AutoML methods search predefined hyperparameter or architectural spaces, whereas LLM-based agents can perform code-level, closed-loop recommendation optimization. However, unconstrained exploration makes costly trials inefficient, motivating a harness that combines recommendation priors with posterior exploration probabilities.
- AutoML and NAS: AutoML relies on hyperparameter optimization or neural architecture search within predefined configuration spaces.These methods tune scalar hyperparameters or search fixed architectural templates.
- AutoML and NAS: Code-level changes such as redesigning losses, switching architectural blocks, or altering pooling strategies fall outside enumerable grids.This limitation motivates approaches that can generate and execute broader model edits.
- LLM-based recommendation agents: LLM-based agents support closed-loop recommendation development by generating, implementing, and evaluating model improvements.Recommendation tasks involve iterative adjustments to objectives, optimization strategies, feature interactions, and structural choices.
- Exploration challenge: Unconstrained LLM exploration proposes candidates without principled trial allocation, reducing success rates and optimization efficiency under costly, noisy training-validation trials.The proposed direction is to combine recommendation priors with posterior exploration probabilities to govern where to explore next.
Methodology
RecHarness frames recommender optimization as budgeted, incumbent-based sequential search: a Thompson-sampling bandit routes trials among human-defined edit directions, while an LLM generates and executes hypotheses within selected arms. Scalar validation feedback updates routing, textual Experiment Skill feedback conditions LLM mutations, and a jump mechanism enables structural exploration when local refinement saturates.
- Overall framework: RecHarness uses a three-level control process in which humans define the objective, validation metric, and edit arms; the bandit allocates trials; and the LLM proposes executable hypotheses.Each trial applies a model mutation, trains the result, and evaluates it under a task-dependent validation metric within a finite runtime budget.
- Incumbent-based search: Candidates are generated relative to the best validated incumbent and promoted only when they improve it; otherwise, the incumbent remains unchanged.The resulting reward is incumbent-conditioned rather than a fixed global success probability for each arm.
- Bandit routing: Thompson sampling selects arms from Beta posteriors, balancing exploitation of validation-improving edit dimensions with exploration of uncertain arms under grouped parallel trials.Selected arms specify semantic edit directions rather than exact code patches, and posterior updates use binary validation outcomes.
- Experiment Skill: Experiment Skill records successful edits, rejected directions, failure reasons, and validation trends to condition LLM hypotheses after arm selection without altering scalar posterior updates.Successful trials become reusable lessons, while failed or harmful trials produce avoid rules.
- Basin-aware jumps: RecHarness partitions arms into local refinement and structural-jump directions, accepting a jump only when subsequent local retuning improves the pre-jump incumbent by δjump > 0.The mechanism addresses saturation of incumbent-based search within a local basin.
Experiments
RecHarness is evaluated across sequential recommendation and watch-time/ranking prediction, with gains observed across datasets, model families, and objectives. Ablations indicate that bandit routing improves trial allocation, while a 7-day online A/B test shows improvements in advertising metrics.
- Tasks and datasets: RecHarness evaluates sequential recommendation on four Amazon Reviews datasets and watch-time, watch-ratio, and ranking prediction on KuaiRec.Amazon evaluation uses next-item prediction from user histories; KuaiRec is a dense user-video interaction dataset.
- KuaiRec: Over 26% reductions in both MAE metrics occur on TPM, while RecHarness consistently improves all four KuaiRec metrics on GR and surpasses Ma et al. (2026).The KuaiRec metrics are WT-XAUC, WT-MAE, WR-XAUC, and WR-MAE; lower MAE and higher XAUC are better.
- Ablation studies: 47.92% of RecHarness non-baseline trials refresh the round-start best validation score, compared with 22.45% for TR w/ Random and 21.74% for TR w/ LLM.In the SASRec ablation, RecHarness reaches 0.6125 by Round 2 and 0.6342 by Round 4 from 0.5050, and remains best at every validation checkpoint and in final test metrics.
- Online A/B test: 2.084% ADVV, 0.534% Revenue, and 0.559% Exposure improvements are achieved in a 7-day online A/B test on 10% of traffic.The deployment targets a large-scale short-video advertising ranking system and discovers a deployable structural improvement within a human-defined arm space.
Conclusion
RecHarness is a bandit-routed agentic framework for automated recommender model iteration under limited budgets. It separates validation-driven edit-direction selection from LLM-generated executable code mutations.
- Framework: RecHarness automates recommender model iteration under limited experiment budgets.The framework is presented as a bandit-routed agentic approach.
- Framework: Validation-driven Thompson routing selects structured optimization directions using evidence accumulated across trials.The router operates over structured optimization dimensions.
- Framework: LLM reasoning interprets validation feedback and proposes executable code edits within the selected directions.This design decouples concrete code mutation from edit-direction selection.