SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensembling

1. Introduction

• Mainstream AutoML commonly uses preset search spaces and one-shot solving, with little mid-run adaptation when training, features, or implementations fail.Meta-learning typically provides a warm start rather than mid-course behavioral correction.Source: §1 · p. 1
• The framework targets two gaps: AutoML rarely corrects itself during execution, and it often produces model diversity without using it in an evidence-based manner.Fixed or absent ensemble stages can weaken execution reliability and the controlled use of structural diversity.Source: §1 · p. 1
• LLMs can revise prompts or code from error logs while generating structurally distinct learners and synthesizing task-aware integration logic.This links how models are produced with how they are combined.Source: §1 · p. 1
• SAGE-Loop combines feedbackable feature generation, multi-round model generation with execution checks, and adaptive ensemble selection for supervised and unsupervised tasks.For supervised tasks it selects among stacking, bagging, and voting; for unsupervised tasks it fuses heterogeneous clusterings into a consensus.Source: §1 · pp. 1–2
• Across 20 public datasets spanning classification, regression, and clustering, SAGE-Loop consistently improves performance and stability and can repair execution failures for more reliable run completion.Ablations and visual analyses also demonstrate robustness and interpretability benefits.Source: §1 · p. 2
• SAGE-Loop uses multi-round LLM-guided model synthesis to build a diverse, valid, high-quality base-model pool while supporting execution-aware repair.The approach treats the LLM as a structure-search agent.Source: §1 · p. 2

2.1. LLM-Augmented AutoML Systems

• Earlier LLM-augmented AutoML often uses one-shot generation, while iterative or multi-agent efforts focus on code-level recommendations rather than model-structure composition and repair.LLM-generated features also rarely extend to full structural modeling within a coherent loop.Source: §2.1 · p. 2

2.2. Structure-Aware and Adaptive Ensemble Learning

• AutoML platforms commonly use greedy selection or fixed voting, while diversity-aware methods often select or weight models without adapting the ensemble form [47].SAGE-Loop dynamically adapts ensemble forms instead of fixing selection or templates.Source: §2.2 · p. 2

2.3. Unsupervised AutoML and Clustering Fusion Paradigms

• Unsupervised AutoML support remains limited, and automated clustering approaches may stop short of principled fusion; SAGE-Loop fuses diverse clusterings into a stable consensus.This addresses methods that assume homogeneous clusterers and a fixed cluster count.Source: §2.3 · p. 3
• SAGE-Loop emphasizes reliable closed-loop correction and adaptive diversity use across supervised and unsupervised AutoML under a shared feedback cycle.Source: §2.3 · p. 3

3.1. Notation and Problem Setup

• The setup uses tabular data with labels for supervised tasks and clustering outputs for unsupervised tasks, while keeping the test split strictly held out from prompt updates and model or ensemble decisions.Those decisions use training and validation signals only.Source: §3.1 · p. 3

3.2. LLM-based AutoML and Ensemble Learning

• Round memory stores execution status, error logs, and validation outcomes, which inform prompt revision; supervised tasks select among stacking, bagging, and voting using validation evidence.When stacking is selected, its level-2 learner is synthesized to match the current base models and task.Source: §3.1 · p. 3, §3.2 · p. 3

3.3. Clustering Ensemble Background

• For clustering, the co-association matrix captures cross-clustering similarity, and spectral clustering produces a consensus after degenerate or mismatched outputs are filtered.The number of clusters can be estimated by majority vote or the Laplacian eigengap.Source: §3.3 · p. 3

4.1. Overview of SAGE-Loop Framework

• The framework links LLM-driven feature engineering, iterative model synthesis, and task-specific ensemble selection, propagating only executable, validated candidates.Its feedback loop uses execution checks, error logs, validation, and prompt rewriting to support correction during execution.Source: §4.1 · p. 3, Figure 2 · p. 4

4.2. LLM-Driven Feature Engineering and Multi-round Model Synthesis

• Task-aware screening retains a compact top-N feature set, using mutual information and feature importance for classification, F-tests and feature importance for regression, and variance and structural scoring for clustering.Validated features are retained, while ineffective or erroneous transformations can be replaced in later rounds.Source: §4.2 · p. 4
• Multi-round model synthesis generates differentiated architectures, checks executability and validation performance, and revises code using error-driven repair or performance-driven prompts.After each feature/code execution round, error traces and validation feedback can inform the next prompt without additional reinforcement-learning training.Source: §4.2 · pp. 4–5

4.3. AutoML-Oriented Adaptive Ensemble for Supervised Tasks

• For supervised tasks, SAGE-Loop compares stacking, bagging, and weighted or unweighted voting using validation records, with an LLM-generated level-2 learner available for stacking.The selected configuration is the one that generalizes best on the current task.Source: §4.3 · p. 5

4.4. AutoML-Oriented Consensus Ensemble for Unsupervised Tasks

• For unsupervised tasks, SAGE-Loop builds a co-association matrix, estimates cluster count by majority voting or eigengap, then applies spectral clustering after filtering invalid base clusterings.The matrix permits clusterings with different cluster counts to contribute without explicit label alignment because co-association is invariant to label permutations.Source: §4.4 · p. 5

5.2. Comparative Results and Analysis (RQ1)

• SAGE-Loop is best or second-best on AUC and ACC across eight classification datasets and achieves the lowest MAE, RMSE, and RMSLE across four regression datasets.On balance-scale it reaches 100.00% AUC and ACC; on boston, its RMSE is 2.73 versus 2.89 for XGBoost and 2.97 for AutoGluon.Source: Table 1 · p. 6, §5.2 · p. 6, Table 2 · p. 7

5.1. Experimental Setup

• The evaluation compares SAGE-Loop with traditional AutoML systems, tree-based models, and LLM-powered methods across 20 public datasets and three task types.Baselines use default or recommended settings and consistent data splits; SAGE-Loop’s clustering ensemble and regression level-2 learner are refined using prior-round signals.Source: §5.2 · p. 6, §5.1 · p. 6

5.2. Comparative Results and Analysis (RQ1)

• SAGE-Loop has the highest mean in 17 of 20 clustering dataset–metric comparisons; exceptions are ARI on breast and NMI on students and cd2.On glass, it records ARI 26.69 and NMI 41.02, exceeding all compared baselines.Source: §5.2 · pp. 6–7, Table 3 · p. 7

5.3. Completion-Recovery Analysis (RQ2)

• SAGE-Loop achieves 5/5 end-to-end completions on all seven datasets; removing correction reduces completion on multiple datasets.Following failure, it uses error feedback to revise a candidate or generate a replacement; completion counts allow intermediate errors when handled automatically.Source: Figure 3 · p. 7, §5.3 · p. 7

5.4. Ablation Studies (RQ3)

• Removing feature generation or ensemble learning degrades performance in classification, regression, and clustering; removing both produces the highest regression errors.On credit-g, AUC falls from 81.08 to 80.13 without feature generation and to 78.42 without ensemble learning; on boston and concrete, removing ensemble raises RMSE from 2.73 to 3.17 and 4.14 to 4.91, respectively.Source: §5.4 · p. 8

5.5. Mechanism Analysis (RQ4)

• Validation-driven selection favors dataset-matched supervised ensembles, while unsupervised consensus adapts cluster count and filters degenerate clusterings before fusion.Across varying cluster counts, iris and seeds reach stable high-quality regions, glass curves remain smooth, and cd1 avoids extreme collapses.Source: §5.5 · pp. 8–9, Figure 4 · p. 8, Figure 5 · p. 9

5.6. Visualization and Interpretability (RQ5)

• Enhanced features yield tighter residual behavior and smoother, more coherent PCA-projected regression surfaces across boston, california, concrete, and insurance.The residual analysis reports concentration closer to zero and shorter tails on boston and california, with similar reduced spread on concrete and insurance.Source: §5.6 · p. 9, Figure 6 · p. 10, Figure 7 · p. 11

5.7. Prompt Design Case Studies (RQ6)

• Prompt case studies span interpretable feature generation, structurally distinct model generation, score-driven refinement, and exception-driven code correction.The feedback prompt supplies current and best AUC; the error-correction prompt injects the runtime exception and requests revised code.Source: §5.7 · pp. 10–11

• Across 20 public datasets, SAGE-Loop consistently improves performance and stability on classification, regression, and clustering tasks.Source: §6 · p. 11
• SAGE-Loop closes the generation loop through feedbackable feature synthesis and multi-round model synthesis, then uses an adaptive ensemble layer to convert structural diversity into stable gains.Source: §6 · p. 11
• Completion-recovery analysis shows that SAGE-Loop can automatically repair execution failures and achieve reliable end-to-end run completion.Source: §6 · p. 11
• Planned extensions include multimodal settings, theoretical study of convergence and consistency, and efficiency improvements for large-scale deployment.Source: §6 · p. 11