PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolving Executable Pipeline Programs for AutoML with Language Models

Sofoklis Kitharidis, Cor J. Veenman, Jan N. van Rijn, Thomas Bäck, Niki van Stein

arXiv:2608.16416v1cs.LGcs.NE

TL;DR

AutoML typically searches predefined pipeline components, limiting its ability to discover structures outside that space. LACE instead evolves complete executable pipeline programs with a language model, matching AutoGluon while outperforming other search-based baselines across 68 OpenML classification tasks and returning editable code.

  • Problem

    Conventional AutoML searches predefined components and hyperparameter ranges, rather than a space capable of producing new pipeline structures.

  • Method

    LACE uses population-based evolution with a language model to generate and revise complete scikit-learn-compatible executable pipeline programs.

  • Results

    Across 68 OpenML classification tasks, LACE is competitive with AutoGluon, outperforms auto-sklearn and H2O, and covers the full benchmark.

  • Takeaways & Limitations

    Executable programs provide a promising AutoML search representation whose task-specific modelling decisions can be inspected, modified, and reused as ordinary code.

  • Takeaways & Limitations

    The conclusions are bounded by one tabular-classification benchmark suite, common-budget evaluation on heterogeneous infrastructure, and results at generator-chosen defaults without external tuning.

Abstract

from arXiv · show

Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space. We present LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator. To our knowledge, LACE is the first to formulate general tabular pipeline AutoML this way, evaluated on standardized OpenML tasks under a leakage-controlled protocol that withholds dataset identity from the generator. Because every candidate is ordinary Python, the returned pipeline and the search that produced it can be inspected and edited directly, rather than only through a framework's model objects. On 68 OpenML classification tasks, LACE with GPT-5.4-mini significantly outperforms auto-sklearn, H2O, and a fixed XGBoost baseline, with no detectable difference against AutoGluon, the strongest search-based system evaluated, while covering the full benchmark. Newer tabular foundation models are more accurate on the subset of tasks they support, but apply a fixed pretrained predictor rather than returning an editable task-specific program. LACE's contribution is therefore not raw accuracy but a search space defined by code: complete coverage, pipelines practitioners can reuse directly, and a component set extended by editing the prompt rather than the framework.

1 Introduction

LACE reframes general tabular AutoML as population-based evolution of complete executable Python pipeline programs rather than search within a fixed component space. It returns task-specific code that can be inspected, edited, and reused, while withholding dataset identity to reduce benchmark-recognition risk.

  • Motivation: Established AutoML systems search curated preprocessors, learners, and hyperparameters within fixed spaces, while evolutionary systems vary structures within fixed operator inventories.Examples include AutoGluon-Tabular, H2O AutoML, auto-sklearn, and TPOT.
  • Contribution: LACE introduces LLM-driven population-based evolution of complete executable pipeline programs for general tabular AutoML.The framework uses generated executable Python and execution feedback to distinguish valid from invalid programs.
  • Evaluation: 68 classification tasks were evaluated with dataset identity and other identifying information withheld from the generator to reduce benchmark-recognition risk.The strongest LACE variant was competitive with AutoGluon, outperformed auto-sklearn and H2O, and covered the full benchmark.
  • Contribution: Generated code replaces a predefined pipeline grammar, allowing programs to choose preprocessing, learners, ensembles, and exposed hyperparameters.This defines the search space through editable code rather than a fixed operator inventory.
  • Contribution: Final LACE programs expose task-specific modelling decisions as ordinary code that can be inspected, modified, and reused outside the search framework.This contrasts with framework-accessed fitted objects and pretrained predictors that are not ordinarily returned as self-contained editable programs.

2 Related Work

Related work spans predefined-space AutoML, LLM systems that generate or configure machine-learning code, and LLM-guided evolutionary program search. These approaches differ in their searched objects and information regimes, limiting direct comparisons with LACE.

  • AutoML and pipeline search: Classical AutoML jointly selects predefined learners and preprocessors while tuning hyperparameters, with systems adding meta-learning, ensembles, or related search capabilities.Auto-WEKA introduced the combined algorithm-selection and hyperparameter-optimization formulation, commonly optimized with engines such as SMAC; Auto-sklearn adds meta-learning and ensemble construction.
  • LLM-based AutoML and ML-engineering agents: Recent LLM-based AutoML systems generate features, evolve feature-transformation programs, configure existing tools, or retrieve prior solutions.These systems construct machine-learning solutions directly as code, but their searched objects and operating procedures differ from LACE.
  • LLM-based AutoML and ML-engineering agents: These LLM agents search whole scripts or trajectories with direct dataset access and arbitrary experimentation, so head-to-head comparisons would confound search method with available information.The paper therefore treats them as related approaches rather than baselines.
  • LLM-guided evolutionary program search: LLM-guided evolutionary program search generates and mutates executable programs that are scored automatically, ranging from fixed-skeleton functions and heuristics to larger codebases.Examples include FunSearch, EoH, ReEvo, and AlphaEvolve.

3 Method

LACE searches over executable scikit-learn-compatible pipeline programs generated and varied by a language model within an evolutionary loop. Candidates are evaluated on inner-validation data under strict leakage controls, then the selected program is refit and assessed once on official test folds.

  • Candidate representation: Each candidate is an executable program with a fixed fit-on-construction, predict-on-call interface, defining the search space implicitly through programs that compile and satisfy the interface.The language model also returns a short description and tunable constructor arguments with defaults.
  • Leakage-controlled evaluation: Search scores candidates only on inner-validation data derived from training partitions, and the official test partitions are never read during evolution.Selection can therefore overfit the finite validation split; invalid or timed-out candidates are included in the candidate budget.
  • Leakage-controlled evaluation: After the search budget is exhausted, LACE refits the best-so-far candidate and evaluates it once on each official test fold under the benchmark’s prescribed protocol.The generator receives only a coarse task summary that omits dataset identity and other identifying information.
  • Candidate representation: The prompt requests restricted libraries, at most three learners, one shared preprocessing stage, and no internal cross-validation or hyperparameter search, while compliance is audited after execution.These constraints are requested rather than enforced by the execution harness.
  • Evolutionary search: LACE uses a (µ+λ) evolutionary loop in which the language model generates the initial population and mutates randomly selected parents using code, feedback, population summaries, and mutation instructions.After offspring evaluation, elitist truncation retains the best µ members of the combined parent and offspring population.

4 Experimental Setup

The experiments cover all 68 OpenML classification tasks across small-data to large, high-dimensional tabular settings, using predefined task splits and controlled inner validation for LACE. Comparisons include mature search-based AutoML systems, pretrained tabular models, and fixed XGBoost under time-matched evaluation, with three language models tested as variation operators on shared infrastructure.

  • Benchmark and protocol: 68 OpenML classification tasks span 100–4,898,431 instances and 4–10,000 features, covering small-data through large, high-dimensional settings.The full task list appears in supplementary Section C.
  • Benchmark and protocol: LACE uses predefined OpenML train/test indices, with each training fold further split into inner-training and inner-validation portions using a 0.2 validation fraction.The protocol uses one repeat of ten folds, and candidate selection uses only the resulting inner data.
  • Compute budget: 100 generated candidates comprise four initial candidates followed by eight generations of twelve, with four-hour heavy-task and one-hour otherwise per-candidate timeouts.20 of 68 tasks qualify as heavy under thresholds of 400,000 samples, 1,000 features, or an instance–feature product of 10,000,000.
  • Compared methods: LACE is compared with AutoGluon-Tabular, auto-sklearn, H2O AutoML, TabPFN v2.5, TabPFN v3, TabICL, and a fixed XGBoost pipeline.The first group represents search-based pipeline construction, while pretrained tabular models provide architecturally distinct references.
  • Compared methods: LACE and search-based AutoML baselines use classical non-neural learners and receive external wall-clock limits matched to corresponding LACE task repetitions.TabPFN and TabICL are evaluated separately as pretrained neural predictors rather than components of the shared search space.
  • Language-model variation: gpt-5.4-mini, deepseek-v4-flash, and locally served qwen3-coder-next operate as variation models under shared infrastructure, so budgets reflect measured runtimes rather than strict compute parity.The comparison is therefore under common infrastructure, not a hardware-normalized benchmark.

5 Main Results

Across OpenML classification tasks, LACE with GPT-5.4-mini combines complete benchmark coverage with performance significantly better than auto-sklearn, H2O, and fixed XGBoost, while showing no significant difference from AutoGluon. Newer pretrained tabular models rank higher on supported tasks, but cover fewer tasks and do not uniformly dominate.

  • Common-task comparison: LACE with GPT-5.4-mini ranks fourth on 45 common tasks, with mean rank 4.71 and normalized deviation 32.8%, ahead of AutoGluon, XGBoost, auto-sklearn, and H2O.TabICL and TabPFN v3 rank first and second, followed by TabPFN v2.5; large standard deviations indicate substantial task-level heterogeneity.
  • Statistical comparisons: LACE with GPT-5.4-mini significantly outperforms auto-sklearn, XGBoost, and H2O, with pHolm < 0.01 in each comparison.The tests are paired Wilcoxon signed-rank tests at α = 0.05 with Holm correction.
  • Full-benchmark coverage: LACE with GPT-5.4-mini covers all 68 tasks and shows no significant difference from AutoGluon, winning 33 and losing 35 shared tasks with median difference −0.001 points and pHolm = 0.66.AutoGluon, auto-sklearn, XGBoost, and H2O also cover all 68 tasks, while TabPFN v2.5, TabPFN v3, and TabICL cover 45, 62, and 67 tasks, respectively.
  • Failure-aware analysis: Under failure-aware penalties, LACE improves from mean rank 4.71 to 4.50, while AutoGluon improves from 5.49 to 4.59, leaving them essentially tied.The top ordering remains unchanged: TabICL has rank 3.28 and TabPFN v3 3.91; TabICL is penalized on a single task.
  • Language-model backbone: GPT-5.4-mini is the strongest LACE backbone, with common-task mean rank 4.71 versus 5.76 for DeepSeek and 5.88 for Qwen3-Coder-Next.All three variants complete the full set of 68 tasks, and tests favor GPT-5.4-mini over both alternatives after Holm correction.

6 Analysis of Generated Pipelines

Analysis of the generated programs shows that evolutionary feedback restores executability, improves already-running candidates, and continues producing diverse structures rather than converging on templates. The executable code representation also makes pipeline structure, edits, and constraint violations directly inspectable.

  • Evolutionary feedback: Final pipelines were all valid, compared with first-shot validity rates of 37.6% for LACE(Qwen3), 66.5% for LACE(DeepSeek), and 79.7% for LACE(GPT-5.4-mini).Thus, runs starting from invalid programs recovered executability within the search budget.
  • Generated structure: Across 70,998 valid candidates, 65,298 (92%) had unique structural signatures, with a median of 68 distinct structures within a run.Signatures ignored numerical values and captured ordered preprocessing steps, learner families, ensemble type, and exposed hyperparameter names.
  • Executable representation: Every returned pipeline was executable Python with inspectable and modifiable preprocessing, learners, and hyperparameters; 89 finals (8.7%, spanning 53 tasks) implemented prediction combination directly in NumPy or ordinary Python.This demonstrates editability beyond relying solely on library ensemble wrappers.
  • Constraint audit: Static auditing found no final pipeline using internal hyperparameter search, nested AutoML, or hyperparameter-driven estimator switching; 22 of 24 flagged programs were confirmed violations.The confirmed violations comprised five candidate-authored out-of-fold constructions, thirteen four-learner ensembles, and four preprocessing refits.
  • Evolutionary progress: Evolution remained active late in search: the median run made its final improvement in generation 7 of 8, had achieved 91% of its eventual gain by candidate 50, and 52.6% of finals appeared in the last two generations.Across valid parent–offspring comparisons, 21.9% of offspring typically improved, with a median gain of 0.23 percentage points on accuracy-scored runs.

7 Discussion and Limitations

LACE demonstrates that evolutionary search over executable pipeline programs is a viable, competitive AutoML approach with complete benchmark coverage, while its conclusions remain bounded by library availability, substantial computational cost, and evaluation limitations.

  • Contributions and scope: LACE covers all 68 tasks and outperforms other search-based baselines while remaining competitive with AutoGluon.Its library-based representation also produced high candidate validity, but the search space remains bounded by available libraries.
  • Computational and operator dependence: LACE inherits the capabilities and computational cost of its language-model variation operator.GPT-5.4-mini significantly outperformed the other two retained backbones under the final configuration, although the margins were small.
  • Computational and operator dependence: 102,000 planned candidate evaluations each faced a one- or four-hour wall-clock limit.All baselines received the same per-task wall-clock budget, so the comparison did not advantage LACE; pretrained models were also subject to that external budget.
  • Evaluation limitations: The evaluation covers only one benchmark suite of tabular classification tasks, with five repetitions per configuration and common-budget infrastructure rather than strict compute parity.Results characterize generator-chosen defaults, while interactions between generated structure and external numerical tuning remain unmeasured.

8 Conclusion

LACE evolves complete executable pipeline programs with a language model as the variation operator. Across 68 OpenML tasks, its strongest variant is competitive with AutoGluon, outperforms other search-based baselines, and produces diverse, revisable, editable task-specific code.

  • Conclusion: Across 68 OpenML tasks, the strongest LACE variant is competitive with AutoGluon and outperforms the remaining search-based baselines.It also covers tasks unsupported by several pretrained tabular models.
  • Conclusion: LACE evolves complete executable pipeline programs using a language model as the variation operator.The framework’s candidates are pipeline programs rather than only framework model objects.
  • Conclusion: The generated pipelines are diverse, improve through revision, and expose task-specific choices as editable code.This makes the resulting pipelines directly inspectable and modifiable.

Supplementary Material … E Exposed Hyperparameter Spaces

The supplementary material documents LACE’s reproducible search procedure, evaluation safeguards, benchmark and baseline settings, and exposed hyperparameter spaces. It shows that returned pipelines include generator-defined spaces that can be tuned outside the framework, while experiments use generator-chosen defaults.

  • Supplementary Material: The supplement provides implementation details, configuration studies, analyses, per-task results, and a Zenodo code artifact supporting inspection of reproducibility.The artifact includes source code, scripts, and representative artifacts.
  • Supplementary Material: The study evaluates LACE on 68 OpenML classification tasks using three language-model backbones and five repetitions per task and backbone.Each run uses B = 100 candidates, µ = 4 parents, λ = 12 offspring, no crossover, and a 20% inner-validation split.
  • B Candidate Evaluation Details: Candidate fitness is computed only from inner-validation data, while official test folds are used after the algorithm returns the selected pipeline.A candidate receives the worst fitness if any fold fails, times out, produces invalid predictions, or returns a non-finite score.
  • B Candidate Evaluation Details: Each candidate runs in an isolated process under a full-evaluation wall-clock timeout of one hour for most tasks and four hours for designated heavy tasks.Successful candidates require every inner-validation fold to return a finite score.
  • A Evolutionary Search Pseudocode: LACE initializes µ candidates, evaluates them, then repeatedly samples parents and mutation instructions to generate offspring until the candidate budget is reached.The algorithm records exposed spaces and uses the LLM with the task, population summary, parent, feedback, mutation instruction, and output format.
  • C OpenML Task Set and Timeout Policy: Timeout tiers change only the per-candidate wall-clock limit; search budget, repetitions, validation fraction, and metric remain fixed.If any split fails or times out, the entire evaluation receives the worst fitness and the first captured error is returned as feedback.
  • D Baseline Configurations: Baselines use the same 68 OpenML tasks, five repetitions, official split instances, task-specific metrics, and per-task repetition wall-clock budget as LACE.The metric is predictive_accuracy for 67 tasks and area_under_roc_curve for task 360114.
  • E Exposed Hyperparameter Spaces: At least 97.5% of candidates emitted a nonempty # Space: block, while experiments evaluated pipelines at generator-chosen default arguments.The returned spaces remain in the artifacts and can be tuned outside the framework.

F Configuration Ablations · G Backbone Screening

Configuration screening favored mutation-only evolution, twelve offspring, random parent selection, and a three-instruction mutation set. Backbone screening retained gpt-5.4-mini, deepseek-v4-flash, and qwen3-coder-next based on performance, validity, cost, and deployment considerations, using overlapping benchmark tasks only for design decisions.

  • F Configuration Ablations: Crossover was disabled because no tested rate showed a consistent advantage over disabling crossover.The comparison used rates 0, 0.25, and 0.75 with µ = 4 parents and λ = 4 offspring.
  • F Configuration Ablations: Increasing offspring from 4 to 12 improved development-set performance, whereas tournament selection did not beat random selection.The adopted settings were λ = 12 with uniform-random parent selection and µ = 4.
  • F Configuration Ablations: The three-instruction mutation set won on all three development tasks and outperformed alternatives removing model-family changes or emphasizing local preprocessing edits.The main instruction set changes estimator family, refines preprocessing, or adjusts a small ensemble.
  • G Backbone Screening: Seven language models were screened as variation operators, and three were retained using screening performance, candidate validity, inference cost, and deployment mode.The screening varied one aspect at a time and preceded the full benchmark runs.
  • G Backbone Screening: Each backbone was evaluated for five repetitions on OpenML tasks 190146, 359970, and 359974 using best inner-validation fitness.These tasks also appear in the 68-task benchmark, so screening was not treated as independent held-out evaluation.
  • G Backbone Screening: Candidate validity was reported separately through each model’s fraction of generated candidates assigned non-finite fitness.The percentages therefore distinguish invalid-candidate rates from the quality of the best valid pipeline.
  • G Backbone Screening: gpt-5.4-mini was strongest or near-strongest with the fewest invalid candidates, qwen3-coder-next led local models, and deepseek-v4-flash offered a competitive lower-cost API alternative.The retained backbones were gpt-5.4-mini, deepseek-v4-flash, and qwen3-coder-next; contextual references were not selection candidates.

H Pipeline Analysis Details · I Pairwise Statistical Comparisons · J Performance by Task Size

The analysis traces LACE’s executable-pipeline search, showing high parser coverage, diverse and inspectable structures, selection toward tree ensembles, and a limited set of confirmed violations. Pairwise comparisons find GPT-5.4-mini significantly better than several baselines but indistinguishable from AutoGluon, while task-size analysis applies failure-aware ranking within row and feature bins.

  • H Pipeline Analysis Details: 1,020 completed runs generated 101,959 candidate records, of which 70,998 (69.6%) received finite fitness and 101,674 programs (99.7%) were parsed.Valid candidates comprised 25,396 for DeepSeek, 29,691 for GPT-5.4-mini, and 15,911 for Qwen3.
  • I Pairwise Statistical Comparisons: Across nine Holm-corrected comparisons, GPT-5.4-mini showed no detectable difference from AutoGluon or TabPFN v2.5, significantly outperformed auto-sklearn, XGBoost, and H2O, and was significantly outperformed by TabPFN v3 and TabICL.The sensitivity analysis changed no coverage figure, fractional-win count, or mean-rank ordering, apart from DeepSeek and AutoGluon exchanging positions under fixed-45 normalized deviation.
  • H Pipeline Analysis Details: 65,298 unique structural signatures arose from 70,998 valid candidates, with a median run producing 68 distinct structures among 100 candidates.The five most frequent signatures together accounted for only 0.1%.
  • H Pipeline Analysis Details: 759 of 1,020 final pipelines constructed ensembles, and 747 finals (73.3%) combined at least two resolved base learners.The ensemble types were 453 voting, 231 stacking, and 75 hand-built combinations.
  • H Pipeline Analysis Details: The audit confirmed 22 violations, while permitted internal cross-fitting occurred in 21.9% of finals through stacking and 35.2% through probability calibration.All thirteen four-learner ensembles and all four cases of fitting during prediction were confirmed violations; the audit is not proof that other noncompliance forms do not exist.
  • I Pairwise Statistical Comparisons: GPT-5.4-mini won 33 of 68 shared tasks against AutoGluon, consistent with a paired test detecting no difference between them.The comparison averages successful repetitions per task and excludes missing tasks rather than counting them as losses.
  • J Performance by Task Size: Task-size comparisons rank all ten methods within row- and feature-count bins using failure-aware ranks that penalize missing results in the bins where methods cannot run.Bins with fewer than ten tasks are marked for cautious interpretation.

K Zero-Shot Validity and Improvement Details

LACE’s improvement analysis distinguishes failure recovery from predictive gains and separates inner-validation optimization from held-out generalization. Across valid first programs, the returned pipeline improves held-out accuracy on most paired runs and retained backbone–task units, including all GPT-5.4-mini units.

  • Definitions: First-shot candidates precede execution feedback and mutation, while final solutions are the highest-fitness returned pipelines; validity requires successful evaluation with correctly shaped predictions.The best initial candidate is the highest-scoring valid pipeline among four initial population members.
  • Held-out replay: 190 of 193 retained backbone–task units improved on held-out accuracy, all 66 retained GPT-5.4-mini units improved, and the two-sided Wilcoxon test gave p < 10−32.The remaining three retained units had negative differences; held-out differences are not sign-constrained by survivor selection.
  • Inner-validation analysis: Failure-aware improvement assigns invalid first-shot programs a score of 0, combining recovery from invalidity with predictive improvement among valid programs.For Qwen3, 212 of 340 first-shot programs are invalid and receive this fixed failure penalty.
  • Inner-validation analysis: The returned pipeline gains 1.58 points over the best valid initial candidate, showing improvement beyond selecting the best of four independent initial generations.The valid-only analysis separates this selection effect from predictive improvement among executable programs.
  • Held-out replay: 3.21 points was the median inner-validation gain versus 2.95 points held out across 611 finite pairs; 609 of 611 improved on the objective and 586 on held-out accuracy.The smaller held-out gain is consistent with mild selection overfitting but remains positive for most runs.

L Search Dynamics Details · M Prompts

The appendix details LACE’s search dynamics, showing declining invalidity under Qwen, rapid early improvement, and generally small but variable mutation gains. It also specifies prompts that generate initial programs and mutate them under explicit pipeline constraints, while noting that auditing is broader than prompt instructions.

  • L Search Dynamics Details: 1,020 complete runs support the dynamics analysis, with 13 runs excluded from normalized progress statistics and 89 lacking generation-0 baselines because of invalid initial candidates.The excluded runs remain included in non-normalized statistics.
  • L Search Dynamics Details: Twelve of 98,062 declared parent pointers were absent from logs and excluded from parent–offspring denominators, while the affected final pipeline remained traceable.All 12 cases came from one DeepSeek run.
  • L Search Dynamics Details: Under qwen3-coder-next, the median invalid share fell from 75% at initialization to 58% in the final generation, remaining highest throughout.The overall series returned to its generation-0 level by the end of a run.
  • L Search Dynamics Details: By candidate 50, the median share of total improvement reached 87.6% for deepseek-v4-flash, 93.7% for GPT-5.4-mini, and 90.7% for qwen3-coder-next.These values measure the median share of each run’s total improvement.
  • L Search Dynamics Details: On accuracy-scored runs, the median valid parent–offspring fitness difference was −0.2 percentage points, while the median improving step was +0.23.The 15 runs on AUC-scored task 360114 showed −0.2 and +0.10 AUC points and were never pooled with accuracy runs.
  • L Search Dynamics Details: Across 335 GPT-5.4-mini runs with valid initial candidates, median inner-validation improvement was 1.7 accuracy percentage points, reaching 10.8 at the ninetieth percentile and 69.5 at maximum.The representative run was selected by a fixed rule and described as typical rather than favourable.
  • M Prompts: Initialization prompts generate μ candidates without parent programs or evaluation feedback, using dataset-derived task summaries and an explicit fit-on-construction, predict-on-call interface.The static prompt text is identical across the three backbones, apart from an adapter-added trailing newline for the locally served model.
  • M Prompts: Mutation prompts combine population summaries, selected-parent code and feedback, and one uniformly sampled instruction covering model-family changes, preprocessing refinement, or small-ensemble edits.The prompt prohibits internal search and tuning loops, limits ensembles, and requires hyperparameters as constructor arguments; auditing tests broader constraints.

N Foundation-Model Ablation · O Per-Task Wall-Clock Time · P Per-Task Performance

The foundation-model ablation shows that prompt disclosure, not mere installation, changes LACE’s generated search space, with performance gains concentrated where TabPFN was already stronger. The accompanying analyses report end-to-end per-task runtimes and full per-task score distributions under explicit validity and coverage rules.

  • N Foundation-Model Ablation: Announcing TabPFN changed generation: 74.5% of candidates used it with DeepSeek and 42.0% with GPT-5.4-mini.When installed but unannounced, TabPFN appeared in none of 5,000 candidates or 50 selected pipelines.
  • N Foundation-Model Ablation: 39 of 50 selected pipelines contained TabPFN, usually as part of an ensemble rather than as the sole estimator.Among selected pipelines, 20 of 25 used TabPFN in the DeepSeek experiment and 19 of 25 in the GPT-5.4-mini experiment.
  • N Foundation-Model Ablation: The five-task ablation supports no significance claims because the smallest attainable two-sided signed-rank p-value is 0.0625.LACE evaluates 100 candidates whereas standalone TabPFN is fitted once, so the ablation is interpreted as evidence about how available components shape the search space.
  • N Foundation-Model Ablation: On task 168350, announcing TabPFN improved accuracy by 1.6 points under DeepSeek and 1.8 points under GPT-5.4-mini.Other tasks were unchanged within run-to-run variation, with the largest observed loss equal to 0.07 points.
  • O Per-Task Wall-Clock Time: Table 17 reports mean total wall-clock hours per repetition for complete 100-candidate LACE runs, averaged over repetitions and backbones on a shared heterogeneous cluster.The end-to-end times are dominated by candidate fitting and evaluation, are not hardware-normalized, and are bounded by one-hour or four-hour per-candidate caps depending on task tier.
  • P Per-Task Performance: Figures 11–28 provide full per-task, per-method distributions of official-test scores underlying the aggregate results.Accuracy tasks are ordered by mean performance across eligible methods; the final figure covers the single ROC-AUC task.
  • P Per-Task Performance: Per-task boxplots appear only when at least three of five repetitions return valid scores, with coverage indicated for three or four valid repetitions.Boxes show interquartile range and median, while points show individual repetitions.
Loading 2608.16416v1…