Source-linked AI summary
A Human-in-the-Loop Autonomous Agent for Industry Time Series Forecasting
Xiaoyu Tao, Mingyue Cheng, Ze Guo, Bokai Pan, Qi Liu, Shijin Wang, Enhong Chen
TL;DR
Real-world forecasting requires more than a model call because practitioners must formulate tasks, connect resources, inspect plausibility, incorporate expertise, and communicate uncertainty. CastClaw provides a human-in-the-loop runtime that coordinates specialized forecasters, tools, user input, validation, and execution records, and reports the lowest point-estimate MSE and MAE in the five-dataset electricity-price setting. Its Nord Pool case demonstrates an inspectable workflow, while provincial-load data provide offline workflow validation without comparative-performance or deployment evidence.
Problem
Reliable time-series forecasting involves task formulation, data and model connection, plausibility checks, domain expertise, and uncertainty communication beyond fixed model pipelines.
Method
CastClaw connects specialized forecasting models, analytical and retrieval tools, natural-language user input, validation checks, and a versioned execution record under explicit stopping conditions.
Results
CastClaw obtains the lowest MSE and MAE in all ten dataset–metric cells across five public electricity-price datasets, with unweighted mean reductions of 14.9% for MSE and 18.6% for MAE.
Takeaways & Limitations
The results support CastClaw as an implemented, inspectable human-in-the-loop forecasting system in the evaluated electricity-price setting.
Takeaways & Limitations
The provincial-load validation demonstrates offline workflow execution only, not comparative accuracy, online deployment, production use, or business improvement.
Abstract
from arXiv · showhide
Real-world time-series forecasting is rarely a one-shot model invocation: practitioners must formulate tasks, connect data and models, incorporate domain expertise, assess prediction plausibility, and communicate uncertainty. Specialized forecasting models provide strong numerical predictions but usually operate in fixed pipelines, while general-purpose large language model (LLM) agents often lack forecasting-specific checks, constraints, and stopping rules. We present CastClaw, a human-in-the-loop autonomous forecasting system built through forecasting-oriented harness engineering. CastClaw connects data, specialized models, analytical tools, user input, and a versioned execution record in one runtime. Users specify the target, horizon, constraints, and hypotheses in natural language. Starting from a supplied or model-generated forecast, CastClaw checks temporal patterns and user constraints; when evidence is missing, it retrieves context, runs an analysis or another model, or asks the user. It then keeps, revises, or escalates the result under explicit stopping conditions. The output contains the final forecast and an execution report recording inputs, evidence, actions, and revisions. In this five-dataset electricity-price setting, CastClaw reports the lowest point-estimate MSE and MAE among 16 baselines. A Nord Pool case demonstrates the inspectable workflow. CastClaw was also validated offline on provincial electricity-load data from North China covering January--June 2026.
I. INTRODUCTION
CastClaw treats reliable forecasting as a system task requiring task formulation, evidence, plausibility checks, domain input, and uncertainty communication. It combines specialized forecasting with autonomous checks, human oversight, versioned records, and explicit stopping decisions.
- Reliable forecasting requires defining targets and horizons, connecting data and models, checking plausibility, incorporating domain knowledge, and communicating uncertainty.
- CastClaw combines task autonomy with human oversight by selecting checks and tool calls while allowing users to correct, constrain, or stop runs.
- The harness connects forecasting models, analysis and retrieval tools, natural-language input, and a versioned execution record.
- When evidence is insufficient, CastClaw retrieves context, runs analysis or another model, or asks the user before keeping, revising, or escalating the forecast.
- The system records inputs, evidence, actions, forecast versions, interventions, and unresolved warnings in an inspectable execution report.
- The paper evaluates CastClaw through electricity-price comparisons, an inspectable Nord Pool run, and offline provincial-load workflow validation.
II. CASTCLAW SYSTEM
CastClaw organizes forecasting as a staged workflow supported by execution, tool, and state-management services. Its task specification covers hourly electricity-price forecasting with explicit data, horizon, split, metrics, and domain context.
- CastClaw’s workflow has five stages from intent understanding to forecast delivery, supported by execution, tool, and state-management services.
- Registered forecasters remain interchangeable while the system maintains a versioned record of requests, data and model versions, constraints, evidence, and actions.
- The Nord Pool task forecasts hourly electricity prices with a 7:1:2 split, a 336-hour lookback, and a 24-hour horizon.
- The task evaluates forecasts using mean squared error (MSE) and mean absolute error (MAE), with lower values preferred.
A. Creating and updating a forecasting task
CastClaw creates forecasting tasks from natural-language specifications and preserves their evolution over time. It retrieves prior context selectively, while treating retrieved items as suggestions subject to current evidence and hard constraints.
- Users specify the target, horizon, granularity, history, constraints, and desired output in natural language.
- CastClaw asks for missing essentials, distinguishes hard requirements from hypotheses, and timestamps corrections to the task.
- Superseded requests mark their forecasts stale while retaining compatible evidence.
- Retrieved rules and past runs are filtered by target, granularity, horizon, and temporal regime before being used as suggestions to check.
- Retrieved items that conflict with current data or a hard constraint are rejected.
C. Checking and revising forecasts
CastClaw checks forecasts against patterns, constraints, context, disagreement, and hypotheses, then selects further actions only when support is insufficient. Revisions must pass validation and hard constraints, while outputs preserve the evidence and stopping decision.
- CastClaw checks candidate forecasts against recent patterns, domain constraints, context changes, model disagreement, and user hypotheses.
- When support is insufficient, it may retrieve context, inspect similar periods, run analysis or another model, test a revision, or ask the user.
- The run stops when the forecast is supported, another action has little expected value, the budget is exhausted, or expert judgment is required.
- A revision replaces the current forecast only after passing the task’s validation metric and hard constraints; rejected candidates remain recorded.
- The result includes final values, warnings, tool outputs, tested forecasts, labeled evidence, and the stopping decision in an execution report.
III. INTERACTIVE DEMONSTRATION
The demonstration presents CastClaw as an interactive forecasting workflow that users can install, launch, and inspect through an end-to-end session.
- Users install CastClaw globally, launch it, and enter a natural-language request to begin forecasting.
- The session covers task setup, forecast checking, and forecast decision, with inspectable inputs, evidence, outputs, and execution reports.
A. Representative Nord Pool run
The Nord Pool run shows CastClaw checking a supplied forecast, retrieving missing context, and rejecting an adjustment that performs worse on validation.
- The run specifies Nord Pool prices, a 168-hour history, a 24-hour horizon, a chronological split, and a user-supplied forecast.
- CastClaw profiles data and residuals, identifies missing contextual evidence, retrieves market information, and constructs one candidate adjustment.
- The candidate performs worse on the validation period, so CastClaw rejects it and retains the supplied forecast as active.
- The decision panel records compared versions, the rejection reason, the stopping decision, and generated artifacts.
B. Human interaction during a run
Human interaction is available before and during execution, while interventions become recorded conditions or corrections governed by validation and consistency checks.
- Users can correct the target or horizon, add a hard constraint or time-limited hypothesis, request another check, or stop the run.
- CastClaw records each intervention with its source, time, and scope; corrections stale affected candidates, while hypotheses become test conditions.
- When interventions require evidence, CastClaw runs the relevant check and shows whether the action changes the active forecast.
- Human input is part of the recorded decision process, and validation and consistency checks govern forecast changes.
IV. EVALUATION AND CASE STUDY
CastClaw is evaluated against 16 forecasting baselines on five public electricity-price datasets, using chronological splits and test-set MSE and MAE.
- The evaluation uses BE, DE, FR, NP, and PJM electricity-price datasets, each split chronologically into training, validation, and held-out test periods at 7:1:2.
- The comparison includes 16 baselines spanning statistical methods, neural models, foundation models, LLM-based forecasters, and forecasting agents.
- CastClaw obtains the lowest MSE and MAE in all ten dataset–metric cells.
- 14.9% and 18.6% are the unweighted mean reductions across five datasets for MSE and MAE, respectively.
- Table I reports point estimates from the complete validated system, so repeated controlled runs are needed to assess significance and attribute gains to specific components.
C. Case study and offline workflow validation
The paper demonstrates CastClaw through an inspectable Nord Pool session and offline workflow validation on provincial electricity-load data. The provincial validation was conducted under confidentiality constraints and does not support comparative-accuracy or deployment claims.
- Nord Pool case: The Nord Pool session records the request, supplied forecast, residual checks, retrieved context, tested adjustment, validation comparison, and stopping decision.The comparison explains why the tested revision was rejected.
- Offline workflow validation: CastClaw underwent offline workflow validation on provincial electricity-load data from North China covering January–June 2026.The validation used the same task specification, interfaces, user-input mechanism, checks, and execution report with load-specific resources.
- Nord Pool case: Fig. 2 shows a user-supplied 24-hour forecast, residual and context checks, and a tested adjustment whose retained result is recorded.
- Offline workflow validation: The provincial-load data were used only for offline workflow validation under the provider’s authorization and confidentiality requirements.They were excluded from comparative accuracy, deployment, production-use, business-impact, user-study, and component-level causal claims.
V. CONCLUSION
CastClaw is presented as an implemented human-in-the-loop forecasting system that coordinates specialized predictors with tools, targeted checks, user input, and inspectable execution records. It reports the lowest point-estimate MSE and MAE in the five-dataset electricity-price setting, while ablations and user studies remain future work.
- CastClaw connects specialized models, analytical tools, user input, targeted checks, and an execution report without replacing the predictors.
- CastClaw reports the lowest point-estimate MSE and MAE in this five-dataset electricity-price setting.
- The Nord Pool case records rejection of an unsupported revision, while the provincial-load validation exercises the same workflow offline without comparative-performance or deployment evidence.
- Ablations and user studies remain future work.