Source-linked AI summary
TuiML: Machine Learning for AI Agents
Nilesh Verma, Nick Lim, Albert Bifet, Bernhard Pfahringer
TL;DR
Language-model agents struggle with programmer-oriented machine-learning libraries because they must recall large interfaces and generate code. TuiML replaces that interaction with a self-describing, schema-driven library whose calls are validated, traced, and locally executed. Across matched TabArena benchmarks, it remains predictively competitive with scikit-learn and Weka, while its alpha status leaves statistical appropriateness to user scrutiny.
Problem
Language-model agents use programmer-oriented machine-learning libraries by recalling interfaces and writing code, despite those libraries being designed for human programmers.
Method
TuiML provides schema-described, discoverable machine-learning actions with validated execution, recorded seeds, replayable notebooks, and local data and model handling.
Results
TuiML achieves accuracy comparable to scikit-learn and Weka while remaining competitive in runtime and memory consumption across matched TabArena experiments.
Takeaways & Limitations
TuiML is designed for agents first, allowing them to discover, extend, and operate machine-learning workflows through a self-describing library.
Takeaways & Limitations
TuiML remains alpha software, and syntactically valid calls can still encode statistically inappropriate experiments.
Abstract
from arXiv · showhide
Machine-learning libraries such as Weka and scikit-learn were designed for human programmers. Language-model agents now use these same libraries by recalling APIs from memory and writing code, an approach that hides what a library offers, delays errors until runtime, and loses experimental state between turns. We present TuiML, a self-contained machine-learning library built for AI agents, with native algorithms across supervised, unsupervised, time-series, data handling, tuning, and evaluation tasks. Every component describes itself through machine-readable metadata and parameter schemas, so an agent can search the library, inspect components, compose validated workflows, and register new ones that become discoverable in turn. Every call is validated, seeded, and traced, and sessions export as runnable notebooks, making experiments reproducible by construction. One specification layer drives the Model Context Protocol (MCP), agent-framework adapters, a Python API, a CLI, and local model serving, while data and models never leave the machine. Benchmarks show TuiML remains predictively competitive with scikit-learn and Weka. While looking like a conventional library to a human user, TuiML is designed for agents first, allowing them to read, extend, and operate machine learning autonomously. TuiML is open source, with documentation at https://tuiml.ai.
1 Introduction
TuiML addresses a mismatch between programmer-oriented machine-learning libraries and language-model agents that must otherwise recall large interfaces and generate code. It instead makes the library self-describing and agent-operable through typed, discoverable actions.
- 1 Introduction: Language-model agents using programmer-oriented libraries face persistent planning, hallucination, and interface-recall difficulties.Reported examples include limited MLE-bench performance, persistent hallucination and long-horizon planning failures, and invented API calls.
- 1 Introduction: A single typed contract separates agent planning from local machine-learning execution and returns metrics, model identifiers, traces, and replayable notebook cells.The forward path uses discovered MCP tools, while the return path provides structured experimental artifacts.
- 1 Introduction: TuiML addresses this mismatch at the library level by exposing machine-learning operations as typed, discoverable actions.Its design principle is that agents need not memorize a library when the library describes itself.
2 Design and Architecture
TuiML uses a shared registry and schema-based contract to let agents discover, validate, execute, and extend machine-learning workflows. The system preserves reproducibility while keeping datasets and fitted models local.
- 2 Design and Architecture: An agent request can resolve into typed calls that discover candidate classifiers, benchmark them, tune a winner, and serve the fitted model.Figure 2 presents this sequence as a representative session.
- 2 Design and Architecture: The registry maps component names to implementations, metadata, and JSON Schemas, making newly registered components immediately searchable and callable.The same registry governs discovery and execution, while invalid names, misplaced parameters, and unsupported estimators produce structured errors.
- 2 Design and Architecture: TuiML validates every workflow argument before fitting, records seeds and model lineage, and exports each session as executable notebook cells.Compact JSON Lines traces retain timestamps, arguments, duration, and status.
- 2 Design and Architecture: Datasets and fitted models remain on the machine, while the agent receives schemas and structured summaries rather than raw data.This local boundary is part of the execution contract.
3 Implementation and Evaluation
TuiML combines native machine-learning functionality with an agent-facing runtime and evaluates it against shared implementations from scikit-learn and Weka. Across matched TabArena experiments, it remains predictively and operationally competitive.
- 3 Implementation and Evaluation: TuiML achieves accuracy comparable to scikit-learn and Weka while remaining competitive in runtime and memory consumption.The comparison covers thirteen shared algorithms on 51 TabArena v0.1 datasets with aligned hyperparameters and ten-fold splits.
- 3 Implementation and Evaluation: The library provides classification, regression, clustering, association mining, anomaly detection, and time-series modeling alongside preprocessing, tuning, metrics, and reporting.Optional scikit-learn and CapyMOA learners are exposed through namespaced wrappers.
- 3 Implementation and Evaluation: Evaluation runs use isolated, single-threaded processes with matched algorithms, aligned hyperparameters, and common ten-fold splits.The benchmark compares implementations shared by TuiML, scikit-learn, and Weka.
4 Conclusion
TuiML concludes as a self-describing, traceable machine-learning library with broad native coverage and cross-platform support. The authors caution that valid calls can still encode statistically inappropriate experiments and identify broader evaluation as future work.
- 4 Conclusion: TuiML combines discoverable components, validated calls, traceable experiments, and performance-oriented implementations across multiple machine-learning tasks.Supported areas include classification, regression, clustering, anomaly detection, association mining, and time-series modeling.
- 4 Conclusion: TuiML remains alpha software, and a syntactically valid call can still encode a statistically inappropriate experiment.The authors state that results therefore warrant the usual scrutiny.
- 4 Conclusion: Future work includes controlled comparisons with free-form code, broader algorithm coverage, and measurement of tool-selection accuracy and token cost.The planned comparison would use matched models and budgets.