Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance

Computational Models of Kolb’s Experiential Learning

• LLMs support the framework because they handle open-ended inputs and outputs, enabling experiential learning in unstructured and dynamic environments.The intrinsic and extrinsic functions are composable and can be applied iteratively for structured internal reasoning before action.Source: Computational Models of Kolb’s… · p. 4
• With proper abstraction, ReAct-based agents achieve better performance with half the time budget.The passage presents this as a result on data science problem solving.Source: Computational Models of Kolb’s… · p. 4
• ReAct-like prompting is the minimal intrinsic–extrinsic reflect–act cycle, and chaining its steps generalises to repeated reflection, abstraction, and experimentation.The framework’s two phases correspond to outward experimentation and inward conceptualisation.Source: Computational Models of Kolb’s… · p. 4
• The framework composes intrinsic functions to refine internal memory, then uses extrinsic functions to act, gather feedback, and update the agent’s state.The cycle is expressed as Σ′t = I(k)t(Σt), followed by an action from Et and an update Σt+1 = Ut(Σ′t, Ft).Source: Computational Models of Kolb’s… · p. 4, Figure 1 · p. 4

Scaffolded Learning and the Agent’s ZPD

• Agent’s ZPD is the task-complexity frontier where success requires scaffolded support, and competency-gated tasks with feedback build ability incrementally.The agent advances only after meeting explicit success criteria; environmental feedback guides internal adaptation as tasks grow more complex.Source: Scaffolded Learning and the Agent’s ZPD · p. 6

Agent K: Integrating Kolb’s and Vygotsky’s Principles to Master Kaggle Competitions

• Agent K tackles autonomous end-to-end data science on Kaggle, where diverse data, varied domains, leaderboard overfitting risk, and expert competition test generalisation.The described data span tabular, computer-vision, natural-language, and multimodal domains.Source: Agent K: Integrating Kolb’s and… · p. 7

Scaffolded Data Science Environments

• Agent K’s scaffolded environment stages data-science work from task setup to solution building, providing high-level guidance while requiring the agent to devise component implementations.The phases focus first on data abstraction and exploration, then on modelling and strategic optimisation.Source: Scaffolded Data Science Environments · pp. 7–8

Stage I: Workspace Scaffold

• In the workspace stage, Agent K maps heterogeneous inputs and outputs into task-specific representations, revising its strategy when general-purpose tests expose alignment or execution failures.It infers task and output semantics, generates mappings, transformations, and prediction formatting, and advances through feedback-driven correction.Source: Stage I: Workspace Scaffold · p. 8

Stage II: Solution Generation Scaffold

• In the solution-generation stage, Agent K designs models, engineers features, trains and iterates on solutions, adapting reference patterns rather than reusing them as static templates.Training and evaluation feedback support exploration and revision of underperforming approaches.Source: Stage II: Solution Generation Scaffold · p. 8

Experiential Learning in Agent K

• Agent K abstracts scaffolded experience into cognitive traces for open-ended reasoning, bootstrapping pipeline exploration from prior knowledge rather than improvising from scratch, unlike AIDE.In open-ended tasks without external structure, the traces guide hypotheses and debugging; executable pipeline trees are refined or expanded using validation and runtime feedback.Source: Scaffolded Learning and the Agent’s ZPD · pp. 6–7, Experiential Learning in Agent K · p. 9

Quantitative Results

• On 81 competitions spanning tabular, computer vision, natural language, and multimodal tasks, the benchmark evaluated end-to-end data-science generalisation through Kaggle leaderboard submissions.Competitions had high human participation, and comparisons used the same conditions as human Kaggle participants.Source: Quantitative Results · pp. 9–10

Agent K’s Medal Performance

• Agent K earned 4 gold and 4 silver medals in actual medal-awarding competitions, plus 5 gold, 4 silver, and 12 bronze medal equivalents in other tasks.The competitions spanned tabular, computer vision, natural language, and multimodal tasks.Source: Agent K’s Medal Performance · p. 10, Figure 3 · p. 11

Agent K versus Human Data Scientists

• At an Elo-MMR score of 1694, Agent K ranked in the top 18% of 7,311 active participants and slightly exceeded the median score of Kaggle Masters.The human comparison included competitors who participated in at least three of the same competitions.Source: Agent K versus Human Data Scientists · p. 12, Figure 4 · p. 12

Agent K’s Experiential Learning in Competitive Contexts

• Agent K achieved the highest median performance across 69 tasks, with its structured learning cycle outperforming competing reactive agents.Figure 5 reports a median near the 83rd percentile and the strongest medal-equivalent record for full Agent K.Source: Figure 5 · p. 13, Agent K’s Experiential Learning in… · p. 14
• Agent K outperformed all TabPFN-v2 baselines, reaching a median near the 70th percentile versus 30% for the strongest baseline.Across the evaluation set, Agent K earned 3 gold, 4 silver, and 3 bronze medal-equivalent scores; the strongest baseline earned 2 gold, 1 silver, and 2 bronze.Source: Figure 5 · p. 13, Agent K’s Experiential Learning in… · p. 14
• ZPD abstraction improved quantile performance by 5% while requiring 2x less exploration time; after 2 time units, abstract conceptualisation yielded 19% higher quantiles than shallow ReAct.The after-2-time-unit difference was statistically significant (p = 1.67e-4); abstraction also enabled one extra gold and two extra bronze medals across 69 competitions.Source: Agent K’s Experiential Learning in… · p. 15, Figure 6 · p. 15

Contribution

• Agent K is described as the first LLM-based agent to implement Kolb’s theory and Vygotsky’s principle, autonomously completing Kaggle workflows across diverse data science tasks.The workflow spans navigating competition URLs, building models, making submissions, and generating high-score submissions; the paper reports state-of-the-art performance among data science agents.Source: Discussion · p. 16

Technology-release-date bias

• 19 of 29 medals remained after excluding submissions that used models released after the competition deadlines, though this restriction reduced Agent K’s performance.The release-date discrepancy affects comparisons with historical human leaderboards, not baseline-agent or foundation-model comparisons under the same technological conditions.Source: Discussion · p. 16

Risk of solution memorization

• Agent K’s adaptation, multi-step reasoning, and tool integration make direct copying unlikely, and strong results on post-training-cutoff competitions suggest performance is not merely memorized.On competitions ending after the latest LLM cutoff, including playground-series-s4e4 and playground-series-s5e5, it outperformed more than 88% of participants.Source: Discussion · p. 16

Competition difficulty heterogeneity

• Because competition difficulty and participant pools vary, the authors estimate competition level using the average Elo-MMR of users earning at least bronze and examine medal distributions by level.The analysis is reported in Figure 12 in the Appendix to support interpretation across competition types.Source: Discussion · pp. 16–17

Absence of recent featured Kaggle competitions

• Agent K has not been evaluated on newer challenges such as mathematical puzzle solving 1, agent design 2, and ARC-AGI 3, which require stronger scaffolding, reasoning tools, domain knowledge, and potentially multi-agent collaboration.The authors present its results on standard data science and machine learning tasks as a milestone toward expansion into more complex domains.Source: Discussion · p. 17

Stage-Wise Scaffolded Learning with Feedback Control

• Stages increase in complexity or dependency, and the agent advances only after feedback confirms competency, making progression adaptive rather than purely sequential.Tasks are decomposed into supported, interpretable stages with feedback such as unit-test results or output-structure checks.Source: Stage-Wise Scaffolded Learning with… · pp. 19–20

Scaffolded Learning Environment

• From a competition URL, the workspace scaffold detects modalities, builds data maps and transformations under task-agnostic unit tests, then the solution scaffold uses LLM-based feature engineering and blending tools.The general-purpose tests can trigger code revision and retries; feature transformations use task descriptions, feature statistics, and sample data, while blending combines selected model outputs.Source: Scaffolded Learning Environment · pp. 20–21, Figure 7 · p. 22

Experiential Learning in Agent K

• During scaffolded work, prompts use task context and prior attempts; in open-ended generation, scaffold-derived reasoning guides solution drafts, improvements, and debugging until runtime or solution limits are reached.Execution logs and validation feedback are recorded in the internal state to help identify successful solutions and guide further iteration.Source: Experiential Learning in Agent K · pp. 23–26

Experimental Setup, Baselines, Resources

• The evaluation compares two baseline families: ReAct-based agents and foundational tabular prediction models.The methods section introduces these baselines alongside the composition of the Kaggle evaluation set and its performance-quantile computation.Source: Experimental Setup, Baselines, Resources · p. 26

Kaggle Competitions Set

• The benchmark covers 81 Kaggle tasks spanning 2011–2025, varied metrics and task types, and datasets ranging from hundreds to several million labelled inputs.Most tasks are regression or classification, with a few multi-target tasks.Source: Kaggle Competitions Set · p. 26, Figure 10 · p. 27

Performance Quantile Computation

• Performance quantiles are computed after selecting up to the competition’s allowed number of submissions by public score and evaluating their private scores against the leaderboard.The reported medal attribution applies Kaggle’s thresholds even to competitions that did not award actual Kaggle medals.Source: Performance Quantile Computation · pp. 26–27, Table 2 · p. 28

ReAct-based agents

• ReAct baselines use a runtime budget matching Agent K’s combined phases and two attempts; variants include Qwen2.5-72B, Deepseek-R1, and Qwen with Kaggle-based retrieval.The retrieval variant incorporates semantically retrieved competition discussions and notebooks when generating new solution branches.Source: ReAct-based agents · pp. 28–29

Tabular Foundation Models

• Tabular comparisons run TabPFN v2 from Agent K’s automated setup, using K-Means sampling for datasets exceeding its 10,000-sample limit, and also include TabICL for classification tasks.The methods also compare TabPFN extensions; TabICL is designed for datasets up to 100,000 samples on affordable hardware.Source: Tabular Foundation Models · p. 29
• Vanilla TabPFN V2 models were fine-tuned separately for classification and regression on each competition’s training set, then used for test predictions.Because of the 10,000-sample input constraint, training data were split into K-Means-based subsets; training sample size differed due to GPU memory, while inference used 10,000 samples.Source: Tabular Foundation Models · p. 29

Computational Resources

• Experiments ran in isolated Ubuntu 22.04.3 LTS containers with one 32 GB NVIDIA V100 GPU and 9 Intel Xeon CPU cores, using runtime limits for fair comparisons.Table 4 reports the assigned runtime limits across competition types.Source: Computational Resources · p. 30, Table 4 · p. 30

Statistical Tests

• Statistical comparisons used Friedman tests followed, when significant, by pairwise Wilcoxon signed-rank tests on average relative ranks derived from competition leaderboard percentiles.The critical difference diagram shows methods’ relative ranks and uses connecting bars to mark pairs not statistically different.Source: Statistical Tests · p. 30

Discussion on technology timeline

• With competition-era technology, Agent K achieved 19 medals overall, including 7 on CV, 6 on NLP, and 6 on tabular competitions under the stated model-date conditions.The tabular count comes from submissions on competitions held after the base models were available; the CV and NLP counts exclude submissions using architectures or weights unavailable during the competition period.Source: Figure 11 · p. 31, Discussion on technology timeline · p. 31

Data Availability

• The experiments use publicly available Kaggle competition data, and the RAG database is provided as a repository zip file.The MIT-licensed GitHub repository includes Agent K, baseline and TabPFN experiment code, setup documentation, configurations, and hyperparameters to support replication.Source: Data Availability · p. 32, Code Availability · p. 32

A.1 Data-science benchmarking

• General coding benchmarks [71] do not necessarily capture the full range of skills needed for end-to-end data science.The related work describes data-science agent evaluation as spanning different scopes.Source: §A.1 · p. 34
• Benchmark targets localized code infilling from Stack Overflow and common libraries, rather than comprehensive pipeline development.It evaluates a specific aspect of data science rather than the full workflow.Source: §A.1 · p. 34
• Unlike traditional AutoML benchmarks, data-science agent benchmarks assess task understanding and adaptation across flexible problem formulations and solution strategies.Traditional AutoML benchmarks typically use standardized inputs to facilitate solver design.Source: §A.1 · p. 34
• Benchmark evaluates six Kaggle tasks against starter code, but its 10% improvement threshold may use a weak baseline and its predefined scripts guide the approach.A baseline may represent minimal predictive performance, such as random predictions, rather than comparison with established leaderboards.Source: §A.1 · p. 34
• Other evaluations target workflow orchestration and software tools, dataset-query code generation, ML-repository understanding, or dataset characteristics [69], rather than the full data-science workflow.The dataset-characteristics questions cover topics such as imbalance, missing data, and correlations; the competition-focused questions primarily concern tabular data.Source: §A.1 · p. 34
• Benchmarks and MLE-bench use 74 and 75 Kaggle competitions, respectively, but evaluate on custom training-data splits rather than the competitions’ private test sets.For, the competitions are selected for simple raw structures and the custom test split does not support direct Kaggle leaderboard comparison; for MLE-bench, alignment between custom-split rankings and original leaderboard results remains uncertain.Source: §A.1 · p. 34
• The experiments aim to assess agents’ autonomous end-to-end capacity to achieve high performance in data-science tasks.This broader objective goes beyond evaluating isolated data-science skills.Source: §A.1 · p. 34

A.2 Traditional automated data science (Auto-DS)

• Traditional methods automate individual pipeline steps, including data cleaning, feature processing, neural architecture search, hyperparameter tuning, and meta-learning, often with narrow application ranges.The approaches target separate stages rather than automating the entire data-science workflow.Source: §A.2 · p. 35
• Broader Auto-ML libraries combine fixed model collections with rule-based decisions and preset optimizers to automate more of the pipeline.Examples include H2O-AutoML, AutoKeras, Auto-sklearn, and AutoGluon.Source: §A.2 · p. 35
• These tools are limited by predefined models and hyperparameters, require expert problem formatting, and may support only specific data types or structures.For example, AutoKeras may require manual column-type specification, while H2O-AutoML is limited to tabular tasks with specifically structured data.Source: §A.2 · p. 35

• LLM-powered and multi-agent data-science agents represent a shift toward more autonomous and adaptable solutions.Source: §A.3 · p. 35
• Unlike traditional Auto-DS approaches, these agents are designed to perform end-to-end data-science tasks.Source: §A.3 · p. 35
• They leverage advanced reasoning and decision-making capabilities to carry out those tasks.Source: §A.3 · p. 35

A.3.1 Coding Agents

• Coding agents extend LLM capacities with code-writing, execution, debugging, visualization, and machine-learning-library tools for data-science automation.LLM code generation and API interaction provide the groundwork for multi-agent systems that plan, call tools, and act across domains.Source: §A.3.1 · pp. 35–36

A.3.2 Intelligent DS assistants

• Intelligent data-science assistants support human decision-making but require substantial user input and are not designed to autonomously win complex Kaggle competitions.Examples include CleanAgent [43] for user-specified cleaning, LAMBDA [52] for reports, MatPlotAgent [65] for visualization, and WaitGPT [64] for code monitoring and editing.Source: §A.3.2 · p. 36

A.4 End-to-end DS agents

• End-to-end data-science agents target autonomous pipelines, using tools, expert demonstrations, multistep reasoning, and strategies for preprocessing, modeling, and feature selection.Examples include CAAFE, which iteratively generates feature-modification code using accuracy feedback, and EVAPORATE-CODE+ [3], which extracts structured data from semi-structured documents.Source: §A.4 · p. 36
• Prior systems vary in pipeline control, while Agent K combines a modality-dependent setup scaffold with flexibility to select solution steps.DS-Agent, MLCopilot, AIDE, and AIRA-dojo refine scripts as blocks; SEED selects modules, HuggingGPT delegates planning, Data-Interpreter uses hierarchical dynamic plans, and AutoM3L uses predefined modality-specific pipelines.Source: §A.4 · p. 36
• Continual-learning approaches reuse prior experience, but strategic task ordering for progressive learning remains underexplored.DS-Agent retrieves successful offline scripts, MLCopilot infers strategies from offline experiments, and Data-Interpreter creates callable tools from prior experience.Source: §A.4 · pp. 36–37
• Existing systems have coverage limits across modalities and tasks, while online convex optimization assumes well-defined losses unlike the sparse or delayed feedback of real-world competitions.The cited limitations include constrained model or data support, weaker performance on harder tasks, and comprehensive task-specific prompt requirements; Agent K aims to address these gaps across input modalities.Source: §A.4 · p. 37

B Agent K - Pipeline Details

• Agent K’s reported system performs setup in one pipeline and feature engineering, model training, hyperparameter optimization, and submission in the prediction pipeline.The setup pipeline downloads, summarizes, organizes, and cleans data to prepare a workspace and DataLoader for solution generation.Source: B Agent K - Pipeline Details · p. 37

B.1 Setting up Competitions

• The competition-setup pipeline automates preparing a new Kaggle task from its URL through a sequence of stages shown in Figure 7.Source: §B.1 · p. 38

B.1.1 Expected Result - Usable & Interpretable Workspace

• The workspace includes train input maps for tabular, text, and image data: train_tab_input_map.csv, train_txt_input_map.csv, and train_img_input_map.csv.The image map contains paths to image inputs.Source: §B.1.1 · p. 38
• Train target maps cover tabular and text targets plus paths to image targets in train_tab_target_map.csv, train_txt_target_map.csv, and train_img_target_map.csv.Source: §B.1.1 · p. 38
• Agent K’s code for generating CSVs is kept in the workspace, contributing to interpretability.Source: §B.1.1 · p. 38
• The workspace retains code for transforming training targets, implementing the competition metric, and formatting submissions, with an alternative submission formatter as backup.The alternative formatter can be used if the primary function fails or as a backup in the submission pipeline.Source: §B.1.1 · p. 38

Fetch & Scrape Raw Data

• Agent K downloads competition data through the Kaggle API and scrapes the competition page for descriptions of the task, data, and metric.Source: §B.1.2 · p. 39

Summarise

• It summarizes scraped descriptions to remove irrelevant or misleading content and preserve important task information in shorter, reusable prompts.This compression also reduces the number of tokens needed in future prompts.Source: §B.1.2 · p. 39

DetectModalities

• Using summarized descriptions, Agent K classifies competition inputs and targets as vision, language, or tabular, including time-series data.Inputs and targets can have different modalities, such as image inputs with tabular classification targets.Source: §B.1.2 · p. 39

Create Plan & Metadata

• After fetching data, summarizing descriptions, and detecting modalities, Agent K creates a workflow plan that can reprioritize permitted stages, revisit earlier stages, and retry them.Source: §B.1.2 · p. 39

Create Maps & Transforms

• Agent K creates input and target maps across modalities, collates them in a custom DataLoader, and builds target transforms and inverse transforms for model use.For text-encoded classification labels, for example, a transform can produce one-hot encodings and an inverse transform can restore text labels.Source: §B.1.2 · p. 39

Create Submission Format & Metric Function

• Agent K generates code for the competition’s required submission format and metric function, including a mapping from predicted targets to the exact expected test-submission format.Source: §B.1.2 · p. 40

B.1.3 Unit Tests

• Agent K achieved an Elo-MMR score of 1694, but the supplied passages for these sections report no benchmark comparisons or quantitative results.The passages describe pipeline methods and pseudocode, not performance results.Source: §B.1.3 · p. 40
• Manually designed, competition-agnostic unit tests check framework-specific stage outputs and provide error messages that the agent uses to correct faulty code.The tests assess whether generated code or data follows expected formats; they are not tailored to individual competitions.Source: §B.1.3 · p. 40
• When a unit test fails, its error is saved in memory and the stage is retried until the test passes or a budget limit is reached.Meta-unit tests check that multiple stages work together, such as whether a DataLoader can load a batch; failure can send the agent back to earlier stages.Source: §B.1.3 · p. 40
• Task-agnostic tests cannot cover every setup error, so the authors propose generating competition-specific tests at inference time to extend coverage.An agent might pass all current tests yet set up a competition incorrectly.Source: §B.1.3 · pp. 40–41

B.2 Generating Solutions

• Bayesian optimization selects learning rate and optimizer over at most n_trials trials, each capped at max_epochs or max_time, whichever comes first.The scaffold also permits optimizations and additional submission files not directly coded by the agent.Source: §B.2 · p. 41
• After optimization, the system trains on k_folds folds and averages predictions for a cross-validated submission; image tasks also use test-time adaptation and may blend distinct solutions.For image tasks, test-time adaptation applies the training augmentation pipeline for tta_rounds rounds and averages predictions; blending starts after at least blend_after full working solutions are generated.Source: §B.2 · p. 41

B.3 Pipeline Pseudocodes

• The setup pseudocode takes a competition URL through data scraping, task summarization, modality detection, and planning to produce setup files and train/test dataloaders.Each pseudocode retry loop has an upper bound; reaching it ends the episode with an error.Source: §B.3 · pp. 41–42
• The solution-generation pseudocode uses setup outputs to create solutions and leaderboard performance, selecting classical ML for tabular-only tasks and a DNN workflow otherwise.The DNN workflow iteratively builds modality-specific encoders and heads, trains and predicts submissions, and blends them when enough solutions exist.Source: §B.3 · pp. 42–43

Competition difficulty analysis

• The analysis updates participants’ Elo-MMR from private ranks across Kaggle medal-awarding competitions, initializing new participants at 1,500, to estimate participant Elo at each competition’s end.For each benchmark competition, the difficulty measure is the average Elo of participants reaching at least a bronze medal-level score.Source: C Competition difficulty analysis · p. 44
• Four gold and four silver medals by Agent K on medal-awarding competitions are marked against average participant Elo, while medal-awarding competitions show higher average Elo than non-medal-awarding competitions.Figure 12 plots each benchmark competition’s average Elo among participants who reached at least a bronze medal-level score.Source: Figure 12 · p. 44, C Competition difficulty analysis · pp. 44–45
• The analysis motivates further improving Agent K’s performance in competitions involving stronger data scientists.Source: C Competition difficulty analysis · p. 45

D Benchmark details

• Table 6 provides the symbols used for competition types and Kaggle categories in Table 7.Source: Table 6 · p. 45, Table 7 · p. 45
• Table 7 lists the Kaggle competitions included in the benchmark.The prose directs readers to Table 7 for the competition list.Source: Table 7 · p. 45, D Benchmark details · p. 45
• For each competition, Table 7 records its Kaggle ID, type, category, input modalities, retained-submission limit, and number of participating teams.The table caption identifies #Select Subs as the kc value in equation (1).Source: Table 7 · p. 45

E Additional results

• 92.5% was Agent K’s overall setup success rate, and most pipeline stages succeeded at high rates across input modalities.Stage success rates count successes among competitions where that stage was reached, averaged over all competitions.Source: E Additional results · p. 49
• Creating training image input maps was a weaker stage, attributed to the often complicated file structures of Kaggle image-input competitions.Source: E Additional results · p. 49
• Text-map stage percentages were relatively smaller because fewer benchmark competitions required text inputs; stages could also be skipped or not reached depending on task requirements and preceding failures.The pipeline tracks stage success or failure by unit-test outcome, as well as skipped stages and stages not reached after a preceding failure.Source: E Additional results · p. 49

F Details on React-Agent baselines

• AIDE was adapted for open-source models such as Qwen2.5 by requesting JSON outputs, parsing them, and retrying formatting errors up to five times before restarting failed runs.The modifications aligned the ReAct baseline with the experiments’ model and execution constraints.Source: F Details on React-Agent baselines · p. 49
• The LLM prompt was expanded with competition and data descriptions, remaining time, the iteration limit, and completed iterations.Source: F Details on React-Agent baselines · pp. 49–50
• Figure 13 shows success, failure, non-reach, and skip rates for competition setup stages, averaged across competitions.Source: Figure 13 · p. 50
• CV and NLP competitions received four-day budgets and 18 hours per node, while tabular tasks received two days and 9 hours per node; runs allowed 5,000 improvement iterations and retained four top solutions.AIDE also used a maximum debug depth of 3 and a 50% probability of selecting each node for debugging.Source: F Details on React-Agent baselines · pp. 50–51
• The four best submissions were scored on Kaggle’s public and private leaderboards to evaluate them against competition benchmarks.Source: F Details on React-Agent baselines · p. 51
• The ReAct(Qwen)+RAG database contains material from 24 post-2020 Kaggle competitions outside the benchmark: 7 each in CV, NLP, and tabular tasks, and 3 multimodal.The passage notes that Table 8 lists the competitions and the materials used to build the database.Source: F Details on React-Agent baselines · p. 51

G Algorithmic Overview of TabPFN Fine-Tuning

• Algorithm 3 preprocesses train and test data, clusters the test data with KMeans, assigns cluster labels, and samples a training subset to fit the context length.The baseline and fine-tuned TabPFN variants use input datasets generated by Agent K’s setup pipeline.Source: G Algorithmic Overview of TabPFN… · p. 51

H Examples of Prompts and Responses

• The queried LLM has no memory of prior interactions, so each prompt supplies the information needed for its current step; displayed prompts are shortened and code paths anonymized.The appendix notes that full prompts are longer because they include the necessary step-specific context.Source: H Examples of Prompts and Responses · pp. 51, 54
• Table 8 outlines the RAG database inputs by selected competition, competition year, technical-report count, and notebook-summary count.The caption gives these categories but does not provide the table's cell values.Source: Table 8 · p. 52

A Note on Prompts

• Prompts devote most of their text to summaries of the competition, raw data, and relevant past steps, alongside terminal or Python output views and general instructions.Some prompts also include reusable examples from similar competitions selected through RAG.Source: A Note on Prompts · p. 54

H.1 Setup

• For leaf classification, the example uses a multimodal task with tabular features and leaf images, while identifying no text input modality.The tabular data include pre-extracted shape, texture, and margin features.Source: §H.1 · pp. 54–55
• The LLM is first asked for a short plan, then to complete a code template for a training image-input map, after the tabular modality has been handled.The example concerns creating a map that links training IDs to image paths.Source: §H.1 · pp. 54, 57, 60
• The example plan reads IDs from train.csv, constructs image paths, builds a DataFrame with id and image_path columns, and saves train_img_input_map.csv.It also advises checking that IDs match image filenames.Source: §H.1 · pp. 57–58
• The coding instructions require absolute input paths, reuse of existing ID columns rather than resetting the index, and leaving the template's final save step intact.If image filenames serve as IDs, they should be converted to strings.Source: §H.1 · p. 60
• The H.1 code fragments convert ids to strings, construct image paths, create an input-map DataFrame, and save it as a CSV.Source: §H.1 · p. 61

H.2 Setup with RAG Example

• The prompt retrieves a similar past competition case after a unit-test error, so the agent can adapt a previously used fix to its current image-map problem.The prior case came from kaggle-pog-series-s01e03; the current example concerns ml-olympiad-landscape-image-classification.Source: §H.2 · p. 62
• The current task uses image inputs from its train and test directories, with no tabular or text input modality.Source: §H.2 · pp. 63–64
• The requested training image map must contain a unique data-point id and image-path columns, while excluding target columns.Source: §H.2 · p. 64
• The generated CSV reported that it was saved, but its unit test failed because glacier/59804579.jpg could not be found.Source: §H.2 · p. 66
• The retrieved analogous script formed IDs by combining seed_id and view, then built image paths by prepending a base directory to each image value.Source: §H.2 · p. 68
• The RAG example asks for a DataFrame map of image inputs, with an ID column followed by paths to available training images.The map is populated using the data description and files in the raw-data directory.Source: §H.2 · p. 70
• The prompt warns to use absolute paths, retain existing dataframe columns for IDs, and leave the provided save step unchanged.The map is intentionally saved in the current directory; the example traceback includes a FileNotFoundError for an image path.Source: §H.2 · pp. 69, 72
• The example plan reads training data, creates unique IDs by combining labels and image filenames, and saves IDs with image paths as train_img_input_map.csv.The supplied error explanation says to check that paths in train.csv are correct and that the image files exist in the train directory.Source: §H.2 · p. 72

H.3 Solution Generation

• The dogs-versus-cats example shows Agent K beginning a new submission by generating plans and code for image preprocessing and an image-embedding model.The agent had already made one submission, which appears in some prompts.Source: §H.3 · p. 74
• The competition is binary image classification of dogs versus cats, evaluated by log loss, with submissions containing each test image ID and its predicted dog probability.The sample submission uses columns id and label, where label is a probability (1 = dog, 0 = cat).Source: §H.3 · pp. 74, 76
• The dataset has 25,000 training images labeled by cat-or-dog filenames and 12,500 test images named with numeric IDs.A sample submission file is provided for the test images.Source: §H.3 · pp. 75–76
• The prompt fixes the raw-data path and requires directory and file names to remain unchanged.This constraint appears in the solution-generation note.Source: §H.3 · p. 76
• A previous solution is described as combining image preprocessing, a pretrained ResNet50 embedder, a TabularHead classifier, and loss functions for regression and classification.The example identifies this as an already-generated solution for the Dogs vs. Cats task.Source: §H.3 · p. 76
• The transform prompt requests task-appropriate torchvision preprocessing that converts images to tensors, and asks for a plan of at most four steps.The example plan proposes resizing, random augmentation, tensor conversion, and ImageNet normalization.Source: §H.3 · pp. 76–77
• The prompt explains that channel-adjustment transforms should replicate grayscale channels or truncate excess channels so every image has a consistent channel count.It advises applying normalization after channel adjustment and matching normalization vectors to the image dimensions.Source: §H.3 · p. 78
• The transform-generation prompt requires sensible transform parameters, necessary imports, tensor conversion, and a different transform set from the previous attempt.It explicitly says not to repeat all transforms from that attempt.Source: §H.3 · p. 79
• The generated transform composition resizes images to 256×256, randomly crops to 224, applies a horizontal flip with probability 0.5, converts to tensors, and normalizes.The example labels the normalization as using ImageNet mean and standard deviation.Source: §H.3 · p. 81
• The displayed progress state leaves the text-field embedder and logits/regression-target decoder marked TODO.The same status is shown again after the image-embedder planning example.Source: §H.3 · pp. 82–83
• A later prompt asks for a pretrained torchvision image embedder whose forward method returns latent representations, and its plan removes the model’s final fully connected layer.The plan also calls for preprocessing consistent with the pretrained model and an embedder class that returns feature tensors.Source: §H.3 · pp. 82–83
• The example implementation instructions include unfreezing the model’s last few layers for fine-tuning.The example defines an unfreeze method with a default of two final layers.Source: §H.3 · pp. 83–84
• The image-embedder prompt specifies a pre-trained torchvision model that accepts transformed image tensors and returns latent-representation tensors through forward().The accompanying plan calls for preprocessing consistent with the pre-trained model and removing its final fully connected layer for feature extraction.Source: §H.3 · p. 84
• The later embedder prompt requires input shape (batch, input_dim), output shape (batch, latent_dim), and a newer model than the prior attempt without repeating its pre-trained models.It asks the agent to fill in the image embedder in an existing code template.Source: §H.3 · pp. 84–85

H.4 Scaffold with CoT

• The sentiment-analysis example is an Agent K draft generated with a chain-of-thought scaffold from the scaffolding stage.The prompt asks for a brief natural-language solution outline followed by a code block implementing the solution and printing the evaluation metric.Source: §H.4 · pp. 88–89
• The solution-sketch guidelines direct the agent to adapt lessons from past solutions and memory, propose a novel approach and a reasonable metric, and avoid EDA.The requested sketch is 3–5 sentences and should not copy past solutions verbatim.Source: §H.4 · p. 89
• The implementation prompt requires a self-contained script that evaluates on a hold-out set, saves test predictions to submission.csv, and observes runtime and step limits.It also calls for five-fold cross-validation only when appropriate for the task.Source: §H.4 · pp. 89–90
• 66.42573369216967 for Submission 2, 66.00794566192491 for Submission 1, and 65.57734204793029 for Submission 0 are the reported validation-set metric values.For each submission, the reported values are identical for submission.csv and submission_alt.csv.Source: §H.4 · pp. 91–92
• The appendix includes a BERT-based sentiment-classifier code fragment, with a classifier module and BERT tokenizer and model loading.The shown classifier defaults to five classes.Source: §H.4 · pp. 93–94
• Agent K’s five-flower image-classification solution is presented as a draft generated with chain-of-thought guidance from the scaffolding stage.The task has five flower classes—daisy, dandelion, rose, sunflower, and tulip—and is evaluated by categorization accuracy.Source: §H.4 · pp. 96–97
• 0.9627714581178903 (submission 0), 0.9524301964839711 (submission 1), and 0.9503619441571872 (submission 2) are reported validation metrics.The summaries describe image-processing pipelines using EfficientNetV2-S, EfficientNetB0, and ResNet50, respectively.Source: §H.4 · pp. 97–98
• The flower submission summaries describe image pipelines with pretrained feature extractors, augmentation or transformation steps, and classification and regression heads.The summaries mention class-imbalance handling and differing architectures across submissions.Source: §H.4 · pp. 97–98
• The bike-sharing example is another Agent K draft guided by scaffold-stage chain-of-thought, framed as predicting hourly rental counts.The prompt describes training data from the first 19 days of each month and test data from the 20th through month-end.Source: §H.4 · pp. 101–102
• The bike-sharing prompt supplies prior-submission guidance describing an ensemble of models and invites exploration of better models and hyperparameters.It names CatBoost as one of the model sections.Source: §H.4 · p. 103
• The scaffold proposes CatBoost for categorical features and potential overfitting reduction, while retaining Root Mean Squared Logarithmic Error (RMSLE) as its evaluation metric.The prompt says CatBoost may capture complex patterns and potentially improve performance.Source: §H.4 · p. 103
• The CatBoost response parses datetime fields, evaluates validation predictions with RMSLE, and converts test predictions with expm1 for a submission containing datetime and count.The response imports CatBoost and the RMSLE-related metric function, and includes a train-validation split.Source: §H.4 · pp. 103–105

H.5 AIDE with RAG

• AIDE receives a human-expert example from its RAG database as guidance, with the example described as relevant to—but not the same as—the invasive-species task.The displayed retrieved example is introduced as past experience written by a human expert.Source: §H.5 · p. 105
• The retrieved example describes an image-classification pipeline with data loading, transformations, model creation, inference, pretrained models, cross-validation, and test-time augmentation (TTA).It is described as a solution for the Kaggle Cassava Leaf Disease Classification competition.Source: §H.5 · p. 105
• The example names tf_efficientnet_b4_ns, vit_base_patch16_384, and seresnext50_32x4d, each adapted with a classifier output layer for 5 classes.The described models come from the timm library and Facebook Research’s DeiT.Source: §H.5 · pp. 105–106
• The example combines model weighting with cross-validation and test-time augmentation, averaging predictions from transformed images for its final output.Its described configuration also includes per-model weights and a weight for non-augmented inference results.Source: §H.5 · pp. 106–108
• The RAG example’s code defines an invasive-species dataset and includes model training and evaluation functions that collect predictions and labels.The excerpts show dataset item retrieval and separate training and evaluation functions.Source: §H.5 · pp. 109–111
• A response fragment writes a test dataframe with name and invasive columns to a CSV file.The supplied code fragment does not show the complete output path or filename.Source: §H.5 · p. 112