Source-linked AI summary

EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery

Xiaopeng Li, Kuo Cai, Bo Chen, Wenlin Zhang, Mengyang Ma, Yingyi Zhang, Zichuan Fu, Yu Yang, Qidong Liu, Yiyu Wang, Ruiming Tang, Wenwu Ou, Jiang Wu, Zhanbo Xu, Xiangyu Zhao

arXiv:2609.34552v1cs.IR

TL;DR

Recommender architecture design relies on expert inductive biases, yet manual design is slow and predefined search spaces constrain automated methods; unconstrained code evolution can also fail to preserve useful innovations. EvoSkillRec evolves typed, reusable skills alongside LLM-invented modules, and improves on the strongest hand-designed baseline across all six benchmarks. Its evaluations do not test cross-task skill transfer.

  • Problem

    Recommender architecture design depends on expert inductive biases, but adapting human-designed models is slow and knowledge is rarely directly reusable across task settings.

    Source: §1 · pp. 1–2
  • Method

    EvoSkillRec decomposes recommenders into executable skills and couples library-based evolution with LLM code invention, promoting validated innovations for reuse.

    Source: §1 · p. 2
  • Results

    EvoSkillRec is the only compared method to improve on the strongest hand-designed baseline on all six benchmarks, and also extends to accuracy–efficiency co-optimization in generative ranking.

    Source: §7 · p. 14
  • Takeaways & Limitations

    The results support combining skill reuse with code-space invention for recommender architecture discovery across the evaluated benchmarks and generative-ranking co-optimization setting.

    Source: §1 · p. 2, §7 · p. 14

    Cross-task transfer remains untested: each experimental library was grown and used within a single task family.

    Source: §6 · p. 14

Abstract

from arXiv · show

Modern recommender systems advance not only by scaling data and parameters, but also by encoding task-specific inductive biases through architecture, including sparse feature interactions for click-through rate (CTR) prediction, temporal attention for sequential recommendation, and expert routing for multi-task learning. However, these biases are typically human expert designed or searched within predefined operator spaces. Although Recent LLM-driven code evolution expands this space, unconstrained edits often produce invalid or ineffective architectures, underuse established architecture design knowledge, and fail to preserve successful innovations for reuse. We introduce EvoSkillRec, a promotion-and-reuse framework for cumulative recommender architecture evolution. It first decomposes recommenders into atomic executable skills and represents architectures as typed skill genomes, with each skill equipped with input--output types, semantic annotations, and implementation code. We then evolve models with different tasks through two coupled spaces: a constrained skill--space that mutates, recombines, specializes, and reuses validated skills, and an open-ended code--space in which LLM planners and synthesizers invent new skill modules using prior evolution traces and accumulated experience. An autoresearch controller evaluates candidates, diagnoses failures, retrieves relevant skills, promotes validated innovations into the skill library, and adaptively allocates the proposal budget between the two spaces. Extensive experiments on CTR prediction, multi-task learning, and multi-domain learning, including resource-constrained co-optimization of predictive quality and model FLOPs utilization in generative ranking models, consistently demonstrate the effectiveness of our proposed EvoSkillRec.

1 Introduction

EvoSkillRec addresses the slow, human-led design of recommender architectures and the limits of fixed search spaces and unstructured code evolution. It combines reusable validated skills with open-ended invention, and reports gains across recommendation benchmarks.

1 Introduction

  • Expert-designed architectures encode useful inductive biases, but adapting them to new tasks or scenarios is slow, manual, and difficult to reuse across model families.Existing AutoML and NAS methods automate some design work but search within spaces specified by humans.Source: §1 · pp. 1–2
  • Unconstrained LLM code evolution can yield many invalid or low-value edits and does not preserve successful ideas as reusable recommender knowledge.A later search may restart from code and feedback rather than validated architectural knowledge.Source: §1 · p. 2
  • EvoSkillRec decomposes models into executable, annotated skills and couples skill-library operations with open-ended LLM invention.Successful inventions are compiled into reusable skill cards; the two search spaces are coupled by an evaluation loop.Source: §1 · p. 2
  • EvoSkillRec outperforms strong hand-designed and automated-design baselines across recommendation benchmarks, with complementary, task-dependent contributions from skill-space and code-space evolution.The framework also decomposes and reuses validated skills while promoting successful new inventions.Source: §1 · p. 2

2 Related Work

Prior automated recommender design searches within predefined spaces, while LLM-driven program evolution proposes code for evaluator-based selection. EvoSkillRec focuses on retaining, validating, and reusing architectural knowledge produced during evolution.

2 Related Work

  • Automated recommender design searches architectures, interactions, embeddings, or objectives, but generally works within a predeclared operator set rather than growing it.EvoSkillRec maintains a growing library of validated skills and can invent new ones when that library is insufficient.Source: §2 · p. 3
  • LLM-driven program evolution uses language models to propose code and external evaluators to select useful variants, including emerging recommender applications.The cited systems retain state in populations or transcripts, but do not extract durable architectural knowledge for reuse.Source: §2 · p. 3
  • EvoSkillRec complements these directions by validating, storing, and reusing architectural knowledge generated during evolution.Its search space can grow through promoted inventions rather than remaining a fixed input to the procedure.Source: §2 · p. 3

3 Problem Setting

The problem setting treats recommendation design as optimizing a model for a dataset, task, and constraints. EvoSkillRec differs from conventional fixed-space search by growing its reachable search space when validated inventions are promoted.

3 Problem Setting

  • A training instance is defined by a dataset, a task with its metrics, and constraints such as latency, parameter budget, or permitted feature types.A model includes code, parameters, data-processing assumptions, and training logic.Source: §3 · p. 3
  • Conventional AutoML and NAS optimize model performance within a search space fixed in advance by human designers.The objective is written as E(M; D, T, C).Source: §3 · p. 3
  • EvoSkillRec searches compositions of existing skills and newly synthesized code, promoting validated improvements so the reachable search space can grow over generations.Its objective includes finding a strong model for the current task and improving the search distribution through accumulated architectural knowledge.Source: §3 · p. 3

4 EvoSkillRec: Skill-Genome Evolution

EvoSkillRec represents recommender architectures as compositions of executable skills and evolves them through constrained recombination and code invention. An AutoResearch loop evaluates candidates, adjusts the exploration budget, and makes validated inventions reusable in later generations.

4 EvoSkillRec: Skill-Genome Evolution

  • EvoSkillRec couples a compiler and two-tier skill library with dual-space evolution and an AutoResearch controller that diagnoses failures and consolidates discoveries.The controller drives both search spaces and records evolutionary memory.Source: §4 · p. 3, Figure 1 · p. 4

4.1.1 The Skill Card Abstraction

  • A skill is an executable module paired with structured metadata, and its typed input and output keys determine how it composes with other skills.Metadata records inductive bias, task applicability, failure signatures, validation, and empirical memory; skills are directly executable and testable.Source: §4.1.1 · p. 4

4.1.2 Model-to-Skill Compiler

  • The model-to-skill compiler extracts modules from established architectures, annotates and rewrites them as independent executable units, then validates them, including by reconstructing seed models.The library is bootstrapped from established recommender architectures and their reusable components.Source: §4.1.2 · p. 5

4.1.3 Two-Tier Skill Library

  • Tier-1 stores validated skills compiled from published architectures, while Tier-2 stores code-space inventions that pass validation and demonstrate improvement when substituted into a genome.Promoted skills become available for subsequent skill-space search.Source: §4.1.3 · pp. 5–6, §4.2.2 · p. 8

4.2.1 Skill-Space Evolution

  • Skill-space evolution retrieves task- and failure-relevant library skills and mutates a genome through adding, replacing, hybridizing, or specializing skill branches.The library can contain both human-designed and previously validated self-discovered skills; retrieval filters for compatibility before ranking candidates.Source: §4.2.1 · p. 6

4.2.2 Code-Space Evolution

  • Code-space uses a structured brief containing the parent genome, diagnosis, evolutionary memory, available tensor keys, and constraints to plan and synthesize new modules.Proposals are screened for wiring and input compatibility, safe executable code, interface and shape consistency, valid gradients, and portability before training decides whether they help.Source: §4.2.2 · p. 7

4.2.3 Adaptive Budget Allocation

  • The candidate budget shifts toward code-space when validated performance plateaus, then relaxes toward skill-space after new skills are promoted.By default, eight candidates are split six to skill-space and two to code-space; code proposals cost an LLM call and a full training run.Source: §4.2.2 · p. 7, §4.2.3 · p. 8

4.3 AutoResearch Loop

  • The AutoResearch loop diagnoses parent models, proposes and screens candidates, trains them under a shared protocol, selects survivors by the objective, and records outcomes and promoted skills.Using the same splits, optimizer, and stopping rule makes candidate scores comparable within a generation.Source: §4.2.2 · pp. 7–8, §4.3 · p. 8

5 Experiments

EvoSkillRec is evaluated across CTR, multi-task, multi-domain, and multi-objective recommendation settings, with experiments examining benchmark performance, evolution dynamics, mechanism contributions, and LLM backbones. It leads the reported test-AUC comparisons across six benchmarks, while skill-space and code-space contributions vary by task and evolved models can improve accuracy and MFU together.

5.1 Experimental Settings

  • Experiments cover CTR on three datasets, MTL on Census-Income, MDL on Amazon and MovieLens, and QK-Video for joint AUC–MFU optimization.Test AUC is reported for CTR, mean test AUC for MTL, and overall test AUC across domains for MDL; held-out test metrics are not used for candidate selection.Source: §5.1 · pp. 8–9

5.2 RQ1: Effectiveness Across Recommendation Tasks

  • EvoSkillRec has the highest reported test AUC on all six benchmarks, gaining 0.0001–0.0085 over each column’s strongest alternative.The comparison covers manual, NASRec, and OpenEvolve baselines under the reported protocols.Source: §5.2 · p. 9, Table 1 · p. 10

5.3 RQ2: How Do Architectures Evolve?

  • Evolution trajectories show complementary, task- and stage-dependent gains from skill-space reuse and code-space synthesis, with neither space universally dominant.The trajectories include code-space inventions and skill-space transitions across CTR and MDL datasets.Source: §5.3 · pp. 10–11, Figure 2 · p. 10

5.4 RQ3: What Does Each Mechanism Contribute?

  • The full model attains the highest test AUC on all six benchmarks, while single-space variants lead on different tasks and some full-model margins are small.Reported full-model margins include +0.0002 on CTR-MovieLens and +0.0005 on CTR-Beauty; the comparisons are from single runs.Source: §5.4 · p. 11, Table 2 · p. 11

5.5 RQ4: Multi-Objective Evolution

  • On QK-Video, many evolved candidates dominate the RankMixer seed on both accuracy and MFU, and P1 is the only Pareto-front candidate reported to retain both test-set improvements.The best validation AUC is 0.9301 and the best MFU is 1.605%; the five Pareto candidates share a deeper RankMixer and an inserted Hadamard cross mixer.Source: §5.5 · p. 12, Table 4 · p. 13, Figure 3 · p. 13

5.6 RQ5: Effect of the LLM Backbone

  • No LLM backbone leads on every task: Claude Opus 4.8 leads CTR, while GLM-5.2 leads MTL and MDL, with the MDL margin over Claude treated as a tie.GLM-5.2 is also reported as a reasonable default for MTL and MDL evolution across these runs.Source: §5.6 · pp. 12–13

6 Discussion and Limitations

The discussion identifies limits in statistical resolution, compute and LLM dependence, transfer scope, and the current reach of autonomous architecture design. It frames cross-task transfer and industrial deployment as open questions.

Statistical resolution

  • Margins around 10^-4 AUC cannot be separated reliably from run-to-run variation in a single evolution run.The authors treat smaller orderings as observations, not established results; several seeds would help, but full evolution runs made that compute unaffordable.Source: §6 · p. 13

Cost and dependence on the LLM

  • Code-space proposals require two LLM calls and full training and validation, and the reported experiments cost roughly $7,000 across backbones and tasks.This cost limits the design space an academic group can explore; results characterize EvoSkillRec together with its driving model, whose choice affects discovered architectures.Source: §6 · p. 13

Scope of transfer

  • Cross-task transfer remains untested because each experimental library is grown and used within one task family; industrial validation rests on QK-Video alone.The experiments use mostly public benchmarks and academic seed architectures, leaving transfer to other tasks and proprietary production settings unresolved.Source: §6 · p. 14

Outlook

  • Engineer-level autonomous architecture design remains out of reach; backbone viability varies, and the surrounding evolution loop still needs careful design.The authors expect model improvements to shift this balance and identify the skill library as the component most likely to carry over.Source: §6 · p. 14

7 Conclusion

The conclusion presents EvoSkillRec as a framework that decomposes recommenders into executable skills, evolves them through skill recombination and code invention, and promotes validated discoveries. The appendix describes the skill cards, tensor compatibility constraints, and curated library that support this approach.

7 Conclusion

  • EvoSkillRec improves on the strongest hand-designed baseline on all six benchmarks and extends to accuracy–efficiency co-optimization in generative ranking.The conclusion also describes the two search spaces as complementary and identifies cross-task skill transfer and industrial deployment as future work.Source: §7 · p. 14

A Skill Library and Skill Cards

  • Each skill card is a machine-readable YAML manifest recording a module’s computation, named tensor inputs and outputs, and composition and retrieval information.Generated Tier-2 cards additionally retain validation and promotion evidence; curated Tier-1 cards record source and extraction information.Source: A Skill Library and Skill Cards · p. 18

A.1 Skill Card Fields and Tensor Interfaces

  • Symbolic dimensions allow skills to be re-instantiated under compatible feature schemas, but trained weights are not assumed to transfer and each concrete schema requires compatibility checks.Actual genome connections require matching instantiated tensor shapes and context keys; listed neighbors are retrieval hints only.Source: §A.1 · p. 19

A.2 Curated Skills by Category

  • The curated Tier-1 library contains 94 skills across 11 functional categories, with each skill counted once even when it serves multiple tasks or model families.The scenario category collects multi-domain components, while generative denotes the modeling paradigm rather than how a skill was obtained.Source: §A.2 · p. 19

A.3 Examples of Curated and Generated Skill Cards

The appendix illustrates curated and generated skill cards, explains typed checks for skill-space operators, and defines the recommendation objectives used in evaluation. The examples show how reusable modules preserve interfaces while carrying distinct provenance and validation evidence.

A.3 Examples of Curated and Generated Skill Cards

  • The curated fm_interaction skill wraps DeepFM’s second-order FM branch and maps field embeddings to an interaction output for CTR or ranking.Its card records the second-order interaction bias, tensor interface, and low-cost use over field embeddings.Source: §A.3 · p. 21
  • The generated macro_field_covariance_shrinkage_v1 module normalizes field embeddings, estimates cross-field correlations, and learns a gated routing update before flattening.It was proposed for a PLE-based multi-task genome to address task-conflicting correlations; its input and output both have shape [B, F, D].Source: §A.3 · pp. 21–22
  • Tier-1 cards record source and extraction details, whereas Tier-2 cards also retain generated implementation, verification results, and promotion provenance.The generated example passed static, unit, shape, and portability checks and was promoted with validation_mean_auc 0.9720098642670008.Source: §A.3 · p. 22

B Skill-Space Operators and Compatibility Checks

  • Skill-space operator preconditions are checked statically against the genome DAG and typed skill interfaces, rejecting invalid proposals before they use training budget.Table 7 lists the four operator categories and their preconditions.Source: B Skill-Space Operators and… · p. 22, Table 7 · p. 23

C.1 Tasks and Evaluation Objectives

  • CTR prediction estimates click probability as binary classification and is evaluated with AUC and LogLoss; the search selects candidates by validation AUC.The task overview groups CTR with single-objective prediction.Source: §C.1 · p. 23
  • MTL shares part of a model’s parameters across related prediction tasks, balancing useful sharing against negative transfer, and reports each task’s AUC.Its overall objective is the weighted sum of per-task BCE losses.Source: §C.1 · p. 23
  • MDL uses one model across domains with shared features and labels but differing data distributions, reporting AUC for each domain and overall.The model must represent both common and domain-specific patterns.Source: §C.1 · p. 23
  • The multi-objective task jointly evaluates AUC and Model FLOPs Utilization, seeking higher AUC without lower MFU or a better AUC–MFU Pareto front.MFU is achieved training FLOP/s divided by theoretical hardware peak FLOP/s.Source: §C.1 · p. 23

C.2 Datasets and Data Splits

The appendix describes datasets for CTR, multi-task, multi-domain, and industrial-scale evaluation, alongside manually designed baselines and generative-ranking seed models. Training and selection settings differ between accuracy-oriented benchmarks and QK-Video co-optimization.

C.2 Datasets and Data Splits

  • Amazon Books is the study’s largest CTR dataset, with over 19 million interactions; the Amazon datasets use a 7:1:2 train-validation-test split.Amazon Beauty and Books are product-review subsets with ratings and item metadata.Source: §C.2 · p. 23
  • Census-Income contains about 300,000 records and 40 demographic and employment attributes, while MovieLens-1M supplies user demographics and movie attributes.MovieLens interactions are divided into three age-group domains: 1–24, 25–34, and 35+.Source: §C.2 · pp. 23–24
  • QK-Video contains about 493 million user–video interactions, providing an industrial-scale setting for measuring training efficiency alongside predictive quality.It is the largest sub-dataset of Tenrec and comes from Tencent’s QQ Kandian video-recommendation platform.Source: §C.2 · p. 24

C.3 Baseline and Seed Models

  • Manual comparison sets include CTR architectures such as Wide&Deep, DeepFM, DCN, and AutoInt, and MTL models including Shared-Bottom, MMoE, PLE, ESMM, and AITM.These are among the manually designed architectures used as task baselines.Source: §C.3 · pp. 24–25
  • MDL baselines include multi-task models adapted by treating domains as tasks, plus domain-oriented systems such as STAR and SAR-Net; 15 UniRank generative-ranking models seed the skill library.RankMixer initializes the reported co-evolution run; the other 14 seed models are source architectures, not separately evaluated AUC–MFU baselines.Source: §C.3 · p. 25

D Training, Search, and Hardware Settings

  • Accuracy-oriented CTR, MTL, and MDL candidates train on one GPU and are selected by validation AUC, whereas QK-Video candidates use multi-GPU data parallelism and joint AUC–efficiency selection.Within each run, settings are fixed and identical across candidates; the two experiment groups differ in data scale, training recipe, and selection rule.Source: D Training, Search, and Hardware… · p. 26

D.1 Model Training and Initial Architectures

EvoSkillRec evaluates task-specific search runs from established seed architectures under shared training settings. Candidate proposals are budgeted across skill and code spaces, validated before training, and selected using validation performance or constrained AUC–MFU criteria.

D.1 Model Training and Initial Architectures

  • CTR starts from DeepFM, MTL from Shared-Bottom, and MDL from SAR-Net; evolution can change their structures and architecture-specific parameters.Candidates train from scratch with Adam at 10^-3; multi-task losses are summed with equal weights.Source: §D.1 · p. 26

D.2 Search Budgets and Survivor Selection

  • The online controller shifts candidate budget toward code space after two generations without a validation-score improvement of at least 10^-3; total budget N stays fixed.When LLM proposals fall short of quota, skill-space candidates fill the remaining slots.Source: §D.2 · p. 27, Table 12 · p. 27
  • On CTR, MTL, and MDL, top-K survivors must score at least the generation mean minus tolerance; QK-Video uses a constrained Pareto selection on validation mean AUC and training MFU.For QK-Video, eligibility also requires AUC within 0.001 of the incumbent best and MFU at least 95% of baseline; test metrics never guide selection.Source: §D.2 · pp. 27–28

D.3 Hardware Setup and MFU Measurement

  • MFU is measured end-to-end, including data loading and training-loop overhead, and accepted only when its coefficient of variation across windows is at most 0.05.The baseline MFU is low for embedding-heavy ranking models, motivating MFU rather than FLOPs alone as the efficiency objective.Source: §D.3 · p. 28, Table 13 · p. 28

D.4 LLM Settings for Planning and Code Generation

  • For code-space proposals, a planner creates architecture sketches from the parent genome, task and tensor dimensions, metrics, failures, budget, and evolution history; a synthesizer turns selected sketches into module proposals.Proposals specify a PyTorch module, tensor signature, genome wiring location, and skill metadata.Source: §D.4 · p. 28, Table 14 · p. 28

D.5 Implementation of the Automated Search Baselines

The automated baselines use the same preprocessing and candidate trainer, while retaining method-specific search spaces and proposal procedures. NASRec screens architectures with a weight-sharing supernet; OpenEvolve proposes standalone model code.

D.5 Implementation of the Automated Search Baselines

  • Both baselines share data splits, random seed, optimizer, and early-stopping settings, but their search spaces, candidate representations, and proposal procedures remain method-specific.Configured budgets are 250 candidates on MovieLens CTR and MDL and 200 on the other four benchmarks; NASRec's supernet training is additional to budget B.Source: §D.5 · pp. 29–31, Table 17 · p. 30
  • NASRec samples 2B architectures from a fixed task-specific space, ranks them by inherited validation AUC after three supernet-training epochs, then independently trains distinct top candidates up to budget B.Its spaces cover CTR interaction branches, MTL expert and tower variants, and MDL sharing, expert, and domain-specific variants.Source: §D.5 · p. 30
  • OpenEvolve proposes ordinary PyTorch modules that pass safety, import, instantiation, forward-shape, and output-contract checks before using the shared standalone trainer.Its search uses a population of 32, an archive of 64, four islands, and two parents per proposal.Source: §D.5 · p. 30, Table 18 · p. 30

E.1 Baseline Comparisons on the Six Accuracy Benchmarks

EvoSkillRec outperforms manual and automated baselines across all six accuracy benchmarks. Its generation traces show contributions from both search spaces, reuse of earlier inventions, and cases where validation gains do not transfer to test scores.

E.1 Baseline Comparisons on the Six Accuracy Benchmarks

  • EvoSkillRec beats every manual and automated baseline on all six datasets, with AUC gains over the strongest manual model of 0.0035, 0.0121, and 0.0179 on CTR, 0.0022 mean AUC on Census, and 0.0021 and 0.0008 on MDL.The full results are reported in Tables 19–22.Source: §E.1 · p. 31, Table 19 · p. 31, Table 20 · p. 31

E.2 Architecture Changes and Scores by Generation

  • Code-space inventions and skill-space steps each produce large validation-AUC jumps across the runs, including the sarnet expert-mixer replacement on Amazon MDL.The trace tables list each generation's best survivor, originating space, mutation, and validation and test scores.Source: §E.2 · p. 33, Table 28 · p. 36
  • On MovieLens, the code-space skill autoint_senet_bridge_branch_r3 is later reused and gives the best test AUC and logloss; a Census skill also recurs in most later skill-space rows.Selection uses validation AUC only, so higher validation scores do not always carry over to test.Source: §E.2 · p. 33

E.3 Generative Ranking on QK-Video

For QK-Video, EvoSkillRec searches for a joint accuracy–efficiency trade-off from a RankMixer backbone using a generative-ranking skill library. The run finds Pareto candidates with distinct AUC and MFU trade-offs, and the low-rank task-basis router paired with Hadamard mixing beats the baseline on both objectives.

E.3 Generative Ranking on QK-Video

  • The 92-skill generative-ranking library has 10 categories, with interaction, embedding, and attention accounting for 63%; it is separate from the classical recommendation library.It contains 54 model-specific skills and 38 shared building blocks.Source: §E.3 · p. 38, Table 30 · p. 39
  • The run's post-search comparison score uses mean validation AUC and measured training MFU, while survivor selection during evolution uses the constrained Pareto criterion.The score's bounds are fixed across the run, and test metrics are not used.Source: §E.3 · p. 38
  • The signed-bucket retokenizer gains +6.7 × 10^-4 validation AUC but loses 8% MFU because bucketing and hashing are memory-bound; it is discarded.Every survivor lies between 1.47% and 1.57% MFU.Source: §E.3 · p. 38, Table 31 · p. 40
  • Search progresses from structural edits that raise MFU, through sequence modules and retokenizers pursuing AUC, to task routing; the low-rank router improves both objectives.Of 180 candidates, 26% survive and 23 skills are promoted to Tier-2; 15 of 33 failures occur in generations 23–29.Source: §E.3 · p. 38, Table 33 · p. 41
  • The Hadamard cross mixer appears in all five Pareto-front candidates; paired with the low-rank task-basis router in P1, it is the only configuration to beat baseline validation AUC, test AUC, and MFU.Table 32 reports the Pareto front over eligible candidates on validation AUC and MFU.Source: §E.3 · p. 38, Table 32 · p. 40
  • Table 32 presents the Pareto front over eligible RankMixer-backbone candidates using validation AUC and MFU; AUC differences are measured in units of 10^-4, and rows progress from highest AUC to highest MFU.Source: Table 32 · p. 40

F Results with Different LLM Backbones

Across CTR, multi-task learning (MTL), and multi-domain learning (MDL), all tested LLM backbones beat the strongest manual baselines on CTR and MTL, while three beat the MDL baseline. No backbone is best everywhere, and their search strategies and calibration outcomes differ.

F Results with Different LLM Backbones

  • All four backbones beat manual CTR (Amazon-Books Wide & Deep, 0.6587 test AUC) and MTL (Census-Income AITM, 0.9726 mean test AUC); three beat MDL SAR-Net (0.7924), while DeepSeek matches.Table 34 reports each backbone’s best candidate and the runs’ starting architectures.Source: Table 34 · p. 42, F Results with Different LLM Backbones · p. 42
  • Claude is best on CTR, while GLM is best on MTL and MDL; no backbone is best across all three settings.On MTL, GLM alone passes the shared plateau, reaching a validation mean AUC of 0.9714.Source: F Results with Different LLM Backbones · p. 42
  • On MDL, GLM, GPT, and Claude independently discover field-level, domain-conditioned modules: a low-rank relation mixer, domain slot memory, and soft field selection, respectively.Source: F Results with Different LLM Backbones · p. 42
  • GPT converges early, Claude and GLM improve into rounds 18–24 mostly through library reuse, and DeepSeek builds wide ensembles or swaps existing templates.DeepSeek’s MDL result only matches the strongest manual model, suggesting new code modules matter when strong manual architectures exist.Source: F Results with Different LLM Backbones · p. 42
  • AUC-only selection can hurt calibration: CTR best models have higher test LogLoss (0.533–0.554) than manual models (0.495–0.507).Each backbone has one run, so AUC differences below about 5 × 10^-4 should be treated as ties.Source: F Results with Different LLM Backbones · pp. 42–43
Loading 2609.34552v1…