Source-linked AI summary
Interactive Training 2: Auditable Control Plane for Live Model Training
Wentao Zhang, Xuanhe Pan, Han Zhou, Yang Lu, Yuntian Deng
TL;DR
Live training intervention remains fragmented and often requires trainer-specific code. Interactive Training 2 introduces a shared, auditable control plane for human and automated steering, demonstrated across five NLP and reinforcement-learning workflows.
Problem
Experiment trackers expose training behavior but lack a general interface for carrying out interventions across trainers and experiments.
Method
Interactive Training 2 uses a shared protocol where applications declare steerable settings and actions, while the training loop validates, applies, and journals requests at safe control points.
Results
The system was demonstrated across five NLP and reinforcement-learning workflows with journals recording executed actions and request-to-result traces.
Takeaways & Limitations
The released system provides a reusable foundation for steering and inspecting interactive training across different frameworks and workflows.
Takeaways & Limitations
The system can block while an LLM responds, does not handle late responses, and cannot guarantee safe or optimal agent actions without task-specific safeguards.
Abstract
from arXiv · showhide
Experiment trackers show how training is progressing, but changing a live run still usually requires trainer-specific code. We present Interactive Training 2, an open-source control plane for steering training through a shared protocol. Training applications declare which settings and actions they expose, humans and automated controllers submit requests through the same interface, and the training loop validates and applies them at safe control points. A customized Aim workspace combines live metrics and controls with a chronological record of requests and outcomes. We demonstrate the system across five NLP and reinforcement-learning workflows. The released code and traces provide a reusable foundation for auditable human- and agent-guided training.
1 Introduction
Interactive Training 2 introduces a reusable, shared control plane that lets humans and automated controllers steer live training through validated, safely timed requests. Its customized Aim workspace records requests and outcomes for auditable review, and the open-source implementation supports five workflows.
- Problem and contribution: Interactive Training 2 replaces trainer-specific callbacks with a shared protocol through which applications expose changeable settings and requestable actions.Humans, scripts, heuristics, and LLM agents submit requests through the same interface.
- Control plane: Training loops validate requests, apply them at designated control points, and record results before continuing.A typed learning_rate request, such as 2 × 10−5, takes effect only when the loop reaches its control point.
- Novelty: Interactive Training 2 generalizes the fixed-command Hugging Face Trainer demonstration into a reusable protocol for diverse applications and training loops.Applications register their own settings and actions, while loops choose their own control points.
- Auditability: The customized Aim workspace combines live metrics and controls with an ordered journal linking decisions, requests, results, evaluations, and checkpoints.The journal supports reconstruction of who requested an action, what it contained, whether it succeeded, and what happened around it.
- Evaluation and release: Five workflows demonstrate the open-source implementation across callbacks, optimizer wrappers, direct PyTorch loops, and reinforcement-learning loops.The protocol supports humans, scripts, and LLM agents.
2 System at a Glance
Interactive Training 2 lets researchers steer live training through a shared protocol: requests are submitted, applied at designated safe control points, and recorded with their outcomes. The system supports human and automated request sources while preserving a chronological, contextual decision record.
- Interactive control: A researcher can inspect live validation, lower BERT’s learning rate from 3 × 10−5 to 2 × 10−5, request evaluation and checkpointing, and continue training.The training loop applies these requests at a designated control point, records whether they succeeded, and proceeds with the updated run.
- Shared protocol: The shared interaction pattern is to observe the run, submit a request, apply it at a safe point, record the outcome, and continue.Humans, scripts, heuristics, and LLM agents all use the same protocol.
- Auditability: The journal records each request with its source, result, and surrounding training context, making decisions and outcomes easier to inspect than scattered console logs.This provides a chronological record of the training interaction.
3 Control-Plane Design
The control plane uses a shared typed protocol: training applications declare settings and actions, while users and automated controllers submit requests that the loop validates, applies, and records at safe control points. An ordered journal connects metrics, requests, results, checkpoints, and controller decisions within rounds, while later rounds receive summarized prior configurations, actions, scores, and reflections.
- Protocol surface: Training code declares typed settings and actions, including learning rates, sampling weights, momentum, and reinforcement-learning curriculum difficulty, without task-specific logic in the training session.Applications implement the corresponding getters, setters, and actions; these controls are exposed through the Python API as knobs.
- Request handling: Human users, scripts, and optional LLM agents submit the same typed requests to one queue, while applications choose control points, validate actions, and can hide dangerous actions from agents.Requests may arrive at any time, but the training loop applies them one at a time at explicit control points before the next training step.
- Auditing: Within each round, metrics, requests, results, checkpoints, and controller decisions share one ordered journal.The journal provides a chronological record linking events within a round.
- Request handling: The session validates, applies, and records declared controls at safe control points, with Aim presenting live metrics alongside the event journal.The shared protocol answers what training exposes, who may submit requests, and when the session applies and records them.
- Cross-round state: Across rounds, demo scripts start a new model and optimizer and pass text summaries of configurations, actions, scores, and reflections into the next plan.Each new round creates a fresh model and optimizer.
4 How the System Works
The system separates training-owned model updates from shared control requests, which are validated and applied at explicit control points. It supports multiple integration paths and records requests, outcomes, metrics, and controller activity in an auditable journal.
- Control plane: Training code retains ownership of every model update, while humans, scripts, and optional LLM agents submit requests through the shared TrainingSession.The session provides the common interface between training loops and controllers.
- Auditability: Typed settings, actions, goals, metrics, plans, calls, results, checkpoints, and reflections enter a shared journal surfaced through a customized Aim workspace.A FastAPI service exposes state, action, event, and WebSocket endpoints, while background transport avoids pausing training for metric writes.
- Control plane: At each session.step(metrics), the system records metrics, runs scheduled controllers, drains one shared request queue, applies validated changes, and records each result.Numerical settings are converted and clamped before training-owned setters run.
- Control plane: Accepted changes are in place before the next training step, while asynchronous HTTP requests return identifiers immediately and later emit action_result events.Clients can retrieve missed events after reconnecting by using a since cursor.
- Integration paths: Three integration paths cover Trainer callbacks, optimizer-step patching, and direct session.step calls for custom pretraining and RLVR loops.The direct path lets applications choose diagnostics and implement checkpoint or evaluation requests.
- Automated control: The LLM agent plans from task and history, acts through registered tools and current metrics, and reflects after each fresh multi-round run.Attached-agent experiments first run a reference round without the agent; subsequent rounds use fresh models and optimizers and retain journal entries.
5 System Validation
System validation shows that one shared protocol spans diverse training controls, records request outcomes, and supports later plans that use earlier reflections. Across five workflows, the released journals and score context document these interactions without serving as controlled optimization comparisons.
- Validation artifacts: The artifact includes five JSONL journals, derived fields, Figure 3 score context, and Table 1 workflow-exposure summaries.Scores contextualize recorded decisions rather than providing controlled comparisons of optimization algorithms.
- Protocol coverage: The protocol spans optimizer, checkpoint, mixture, learning-rate, optimizer-group, and reinforcement-learning difficulty controls across five workflows.Applications publish exposed settings and actions, which controllers read through the same interface.
- Auditable outcomes: A complete request path assigns an identifier, executes at the next control point, and records success or explicit failure in the journal.The illustrated save_checkpoint request records step 200 and evaluation loss 0.2709 two seconds after success.
- Reflection reuse: Three examples show later plans using earlier reflections while every round restarts with a fresh model and optimizer.Sentiment R8 proposes asymmetric class weights after regression; Muon–AdamW lowers momentum from 0.95 to 0.90 before the R5 score of 4.4291; Countdown reacts to stalling at 0.154.
6 Related Work
Interactive Training 2 distinguishes its control plane from experiment trackers, trainer callbacks, and configuration-search systems by centralizing externally submitted changes under a shared protocol with recorded outcomes. LLM agents are optional controllers alongside humans, scripts, and heuristics, while search policies remain separate from the control plane.
- Trackers and callbacks: Interactive Training 2 uses a tracker for monitoring and control, while the training application determines permitted changes and the shared external protocol records each request’s result.This differs from trainer callbacks that make similar changes within a single framework.
- Configuration search: Hyperparameter optimization, population-based training, and dynamic algorithm configuration can act as controllers, while the control plane provides actions, application points, and an audit trail.The search algorithm remains separate from the control plane.
- LLM controllers and predecessor: LLM agents are optional in Interactive Training 2 and use the same protocol as humans, scripts, and heuristics; Interactive Training v1 is its direct predecessor.Table 3 isolates the new shared protocol from the inherited live-intervention demonstration.
7 Conclusion
Interactive Training 2 makes live training intervention reusable by separating application-defined controls and safe application points from shared human and automated requests. Its journal records decisions and outcomes, making runs easier to steer, reuse, and inspect across frameworks and workflows.
- Interactive Training 2 turns live training intervention from experiment-specific logic into a reusable interface.Training applications define what may change and when changes may safely take effect.
- Humans and automated controllers submit requests through a shared protocol, while the journal records the resulting decisions and outcomes.This separation supports steering, reuse, and inspection across different frameworks and workflows.
Limitations · A Minimal Integration Example · B Control and Journal Records
The system’s safeguards constrain but do not guarantee safe live-training interventions, leaving high-stakes use dependent on additional approval, budgeting, rollback, and task-specific checks. Its minimal integration uses a session, registered controls, and safe-point stepping, while the control protocol journals requests, outcomes, events, and permissions.
- Limitations: The training loop blocks while the LLM responds and cannot yet handle responses arriving after training advances.Applications must also check custom actions; bounds, type checks, permissions, and human controls limit agent behavior without guaranteeing safety or optimality.
- Limitations: High-stakes use requires approval gates, resource budgets, rollback policies, and task-specific safety checks.
- A Minimal Integration Example: The direct integration path creates a training session, registers application-owned controls, and invokes one session call at a safe loop point.The example configures a validation-loss goal, an LLM agent queried every 100 steps, and JSONL memory.
- A Minimal Integration Example: The example reports each loss and checks the returned control for a stop command, with a Hugging Face integration helper also shown.The artifact includes a complete 3842 × 1856 unannotated Aim capture preserving journals, controls, metrics, and event-stream context.
- B Control and Journal Records: Each request records its action type, arguments, identifier, timestamp, and source, while each result records success or failure, returned data, and errors.Unknown actions and handler exceptions therefore appear as recorded failures.
- B Control and Journal Records: Increasing sequence and round numbers let reconnecting clients recover missed events, and Aim retains a permanent copy of the event journal.A cross-round journal row identifies whether a round is the Round 0 no-LLM reference.
- B Control and Journal Records: GET /state exposes settings, actions, and session metadata; POST /actions enqueues requests; and GET/WS /events?since= streams later events.Built-ins cover settings, evaluation, checkpoints, execution control, module reset, annotations, context, and agent configuration; only humans may invoke destructive or agent-configuration actions.
C Task Configurations and Controls
The task configurations specify each trace’s model, data, budget, evaluation and controller frequencies, and registered controls. Journals record round-level plans, actions, reflections, scores, and usage, including RLVR workflows.
- Configurations: Table 4 lists each trace’s model, data, budget, evaluation frequency, controller call frequency, and registered controls.These fields define the configuration and available control surface for each trace.
- Journals: Journals provide round-level plans, actions, reflections, scores, and usage for each training trace.The journal captures both control activity and observed training outcomes at round granularity.
- Configurations: RLVR denotes reinforcement learning with verifiable rewards.The abbreviation identifies the reinforcement-learning workflow category used in the traces.
D Released Evidence and Demonstration
The released evidence links workflow rows to source journals, preserves both successful and unsuccessful control rounds, and reports execution volume and configured cost across five sessions. Public demonstrations expose live control outcomes through videos, traces, and a deterministic sandbox.
- Journal evidence: Each Table 5 workflow row links to its source journal through a SHA-256 manifest.The manifest provides auditable linkage between reported workflow evidence and the underlying journal.
- Trace shape: Released plots retain unsuccessful rounds: Sentiment has five before its R9–R10 recovery, while Countdown has four before R6–R7.BERT changes score most in the first LLM-guided round; Layerwise GPT continues new running bests through R8, while Muon–AdamW sets none after R5.
- Execution volume: Across five sessions, 47 LLM-guided rounds record 1,207 summarized successful actions, 3.23M/0.68M input/output tokens, and $36.54 at configured GPT-5.5 prices.These fields quantify exposure and cost, not decision quality.
- Live demonstration: The public demonstrations include a separate five-round Muon video, an 11-round seed-42 paper trace, and a queued tiny-BERT CPU sandbox.The sandbox lets reviewers change settings or request evaluations and observe matching action_result records; privately hosted LLM demos add plan/act/reflect.
- Setup and limitations: All recorded runs use GPT-5.5 with high reasoning effort, while the released journals omit hardware, wall-clock time, and per-step telemetry.The public Muon video is a separate five-round run with 1,000 steps per round.
E Implementation and Reviewer Paths
Interactive Training 2 provides optional LLM control, a zero-install reviewer interface for recorded traces and limited requests, and reproducibility paths for smoke tests and the full Aim workspace.
- LLM control: The optional LLM client supports OpenAI’s Responses API and compatible chat endpoints, with provider, model, reasoning effort, and call frequency configurable at runtime.Recorded runs use GPT-5.5 with high reasoning effort; API keys are write-only and redacted from state and built-in action events.
- Reviewer paths: At interactivetraining.ai/live, reviewers can inspect recorded traces or submit limited setting changes and evaluation requests to a queued CPU sandbox.The reviewer path requires zero installation; the separate Muon video is supplementary.
- Reproduction: Tag v2.0.2 installs without the Aim fork, while tests/run_tests.py exercises the session, action protocol, HTTP interfaces, integrations, journal, and scripted controllers without a GPU or provider key.This core smoke test covers the main interfaces and built-in integrations.
- Reproduction: Reproducing the full Aim interface requires the companion Aim branch and commit specified in demo/aim.lock.json, plus the README’s clone, AIM_SRC, and BERT frontend commands.The pinned Aim source and documented commands define the full-workspace reproduction path.