Source-linked AI summary
A Human-in-the-Loop Autonomous Agent for Industry Time Series Forecasting
Xiaoyu Tao, Mingyue Cheng, Ze Guo, Bokai Pan, Qi Liu, Shijin Wang, Enhong Chen
TL;DR
Real-world forecasting requires more than a model call, while specialized forecasters and general-purpose agents leave parts of the forecasting workflow unsupported. CastClaw combines specialized models, analytical tools, user input, and an inspectable execution record in a human-in-the-loop system. Across five electricity-price datasets, it reports the lowest MSE and MAE in all ten dataset–metric comparisons.
Problem
Specialized forecasters use fixed pipelines, while general-purpose agents often lack forecasting-specific checks, constraints, and stopping rules, despite forecasting requiring a broader system-level workflow.
Source: §I · p. 1Method
CastClaw coordinates specialized forecasters, analytical and retrieval tools, user input, and a versioned execution record, checking forecasts and keeping, revising, or escalating them under explicit stopping conditions.
Source: §I · p. 1Results
Across five datasets, mean reductions versus the best competing results were 14.9% for MSE and 18.6% for MAE; CastClaw had the lowest values in all ten dataset–metric cells.
Source: B. Main table analysis · p. 3Takeaways & Limitations
The results support CastClaw as an implemented forecasting system, with the Nord Pool case recording rejection of an unsupported revision.
Source: A. Representative Nord Pool run · p. 3, §V · p. 4The provincial-load validation exercises the workflow offline and provides neither comparative-performance nor deployment evidence.
Source: §V · p. 4
Abstract
from arXiv · showhide
Real-world time-series forecasting is rarely a one-shot model invocation: practitioners must formulate tasks, connect data and models, incorporate domain expertise, assess prediction plausibility, and communicate uncertainty. Specialized forecasting models provide strong numerical predictions but usually operate in fixed pipelines, while general-purpose large language model (LLM) agents often lack forecasting-specific checks, constraints, and stopping rules. We present CastClaw, a human-in-the-loop autonomous forecasting system built through forecasting-oriented harness engineering. CastClaw connects data, specialized models, analytical tools, user input, and a versioned execution record in one runtime. Users specify the target, horizon, constraints, and hypotheses in natural language. Starting from a supplied or model-generated forecast, CastClaw checks temporal patterns and user constraints; when evidence is missing, it retrieves context, runs an analysis or another model, or asks the user. It then keeps, revises, or escalates the result under explicit stopping conditions. The output contains the final forecast and an execution report recording inputs, evidence, actions, and revisions. In this five-dataset electricity-price setting, CastClaw reports the lowest point-estimate MSE and MAE among 16 baselines. A Nord Pool case demonstrates the inspectable workflow. CastClaw was also validated offline on provincial electricity-load data from North China covering January--June 2026.
I. INTRODUCTION
CastClaw treats forecasting as a system task, combining specialized forecasters, tools, user input, validation, and a versioned execution record. It supports inspectable workflows with explicit stopping decisions, alongside quantitative comparisons and workflow demonstrations.
I. INTRODUCTION
- Conventional models use fixed input–output pipelines, while general-purpose agents and LLM forecasters add reasoning, tools, or numerical prediction –; related work includes AlphaCast and TimeSeriesScientist [9], [10], [22].CastClaw combines greater task autonomy with human oversight, recording evidence and interventions, gating revisions, and making stopping decisions explicit.Source: §I · p. 1
- The harness connects registered forecasters, analysis and retrieval tools, natural-language requests, and a versioned execution record to produce forecasts and inspectable reports.Users specify the target, horizon, constraints, and optional hypotheses.Source: §I · p. 1
- CastClaw seeks additional support only when checks identify a gap, and replaces a forecast only after validation and constraint checks.It may retrieve context, run another analysis or model, or ask the user; users can correct, constrain, request checks, or stop the run without bypassing validation.Source: §I · p. 1
- The execution record preserves the request, data and model versions, tool outputs, user input, forecast versions, and stopping decision.The live demonstration exposes the inputs, checks, actions, and decisions behind the forecast.Source: §I · p. 1
- CastClaw reports quantitative comparisons on five public electricity-price datasets, an inspectable Nord Pool run, and offline workflow validation on provincial electricity-load data from January–June 2026.The provincial-load dataset supports offline workflow execution, while public datasets support quantitative forecast comparisons.Source: §I · p. 1
II. CASTCLAW SYSTEM
CastClaw coordinates forecasting from intent understanding through forecast delivery using interchangeable forecasters and supporting execution, tool, and state-management services. Its versioned record tracks requests, inputs, constraints, evidence, candidates, and actions while protecting user-supplied values and hard constraints from silent changes.
State management
- CastClaw maintains a versioned record of the request, data and model versions, constraints, evidence, candidates, and actions.Source: §II · p. 1
- User-supplied values and hard constraints cannot be overwritten silently.Source: §II · p. 1
System workflow
- CastClaw’s five-stage workflow runs from intent understanding to forecast delivery, supported by execution, tool, and state-management services.Registered forecasters remain interchangeable.Source: §II · p. 1, Figure 1 · p. 2
A. Creating and updating a forecasting task · B. Reusing domain rules and past runs
CastClaw formalizes forecasting tasks through explicit user requirements and reuses scoped rules and past runs only as suggestions checked against current data and hard constraints.
A. Creating and updating a forecasting task
- Users specify the target, horizon, granularity, history, constraints, and desired output; CastClaw asks for missing essentials, distinguishes requirements from hypotheses, and timestamps corrections.Superseded requests make their forecasts stale while retaining compatible evidence.Source: A. Creating and updating a forecasting… · p. 2
B. Reusing domain rules and past runs
- Before prediction, CastClaw retrieves rules and past runs by target, granularity, horizon, and temporal regime, treating them as scoped suggestions and rejecting conflicts with current data or hard constraints.Each retrieved item retains its source and scope.Source: B. Reusing domain rules and past runs · p. 2
C. Checking and revising forecasts · D. Returning the forecast and execution report
CastClaw checks candidate forecasts against patterns, constraints, context, disagreement, and user hypotheses, then gathers more evidence or stops under explicit conditions. It returns final values and warnings with a versioned execution report documenting the workflow.
C. Checking and revising forecasts
- Candidate forecasts are checked against recent patterns, domain constraints, context changes, model disagreement, and user hypotheses.CastClaw can load a user forecast or run registered models.Source: C. Checking and revising forecasts · p. 2
- When support is insufficient, CastClaw retrieves context, inspects a similar period, runs analysis or another model, tests a revision, or asks the user.The run stops when support is sufficient, another action has little expected value, the budget is exhausted, or expert judgment is required.Source: C. Checking and revising forecasts · p. 2
- Every call and affected forecast version is recorded, and an action is repeated only when evidence changes.Rejected candidates remain in the versioned execution record.Source: C. Checking and revising forecasts · p. 2
- A revision replaces the current forecast only if it passes the task’s validation metric and hard constraints; otherwise, the current forecast is retained.The rejected candidate is also retained in the versioned execution record.Source: C. Checking and revising forecasts · p. 2
D. Returning the forecast and execution report
- The result includes final values, applicable warnings, and an execution report covering inputs, versions, evidence, tool outputs, tested forecasts, and the stopping decision.The report records the execution process alongside the forecast.Source: D. Returning the forecast and… · p. 2
- Report items are labeled as user input, model output, retrieved context, or derived analysis.Public experiments measure forecast accuracy, whereas provincial-load validation demonstrates execution of the offline forecasting workflow; both use the same interfaces with different resources.Source: D. Returning the forecast and… · p. 2
III. INTERACTIVE DEMONSTRATION
CastClaw’s demonstration presents an end-to-end forecasting session where users can inspect inputs, evidence, tool outputs, and the forecast decision, and correct or stop the run. Users can also install and launch CastClaw to begin a workflow with a natural-language request.
INTERACTIVE DEMONSTRATION
- Install CastClaw with npm install -g castclaw, launch castclaw, and enter a natural-language request to start an interactive forecasting workflow.Source: §III · p. 2
- The demonstration covers task setup, forecast checking, and decision, exposing the request, supplied forecast, run evidence, tool outputs, final forecast, and execution report for inspection.Users can explicitly correct or stop the run.Source: §III · p. 2
A. Representative Nord Pool run
In a representative Nord Pool run, CastClaw evaluated a candidate adjustment to a user-supplied forecast and rejected it, retaining the original with an auditable decision record.
A. Representative Nord Pool run
- The request specified Nord Pool prices, a 168-hour history, a 24-hour horizon, a chronological split, and an existing 24-hour forecast, which CastClaw attached to the versioned run record.During setup, CastClaw parsed the request and confirmed the data configuration.Source: A. Representative Nord Pool run · p. 2
- After profiling data and residuals, CastClaw retrieved Nord Pool market context, constructed one candidate adjustment, and evaluated it against the stated metric and constraints.Source: A. Representative Nord Pool run · p. 2
- The candidate adjustment performed worse on validation, so CastClaw rejected it and kept the supplied forecast active, recording the reason and stopping decision.The decision panel also records the compared versions and generated artifacts.Source: A. Representative Nord Pool run · pp. 2–3
B. Human interaction during a run
CastClaw supports user interventions before and during a run, recording them as part of the decision process. Corrections invalidate affected candidates, while hypotheses are tested as conditions rather than imposed as forecast changes.
B. Human interaction during a run
- Users can correct the target or horizon, add a hard constraint or time-limited hypothesis, request another check, or stop execution; CastClaw records each intervention’s source, time, and scope.A correction marks affected candidates stale, whereas a hypothesis becomes a condition to test rather than a forced forecast change.Source: B. Human interaction during a run · p. 3
- When an intervention requires new evidence, CastClaw runs the relevant check and shows whether it changes the active forecast, with evidence, warnings, and unresolved questions inspectable before and after.Validation and consistency checks govern forecast changes, making human input part of the recorded decision process rather than post-hoc feedback.Source: B. Human interaction during a run · p. 3
IV. EVALUATION AND CASE STUDY · A. Experimental settings · B. Main table analysis
Across five public electricity-price datasets, CastClaw achieves the lowest MSE and MAE in all ten dataset–metric cells against 16 baselines. Its reported row is the complete system’s forecast after validation-based candidate checks, not a new forecasting backbone.
A. Experimental settings
- The evaluation uses BE, DE, FR, NP, and PJM datasets, chronological 7:1:2 train-validation-test splits, and 16 baselines spanning five method categories.The categories are statistical methods, specialized neural models, foundation models, LLM-based forecasters, and forecasting agents.Source: A. Experimental settings · p. 3
- The CastClaw row is the complete system’s forecast after validation-based checks of supplied and system-generated candidates, not a new forecasting backbone.Source: A. Experimental settings · p. 3
B. Main table analysis
- CastClaw records the lowest MSE and MAE in all ten dataset–metric cells, with mean reductions of 14.9% and 18.6%, respectively.Source: B. Main table analysis · p. 3
- CastClaw’s MSE reductions versus the best competing result are 5.5% on BE, 30.1% on DE, 11.4% on FR, 17.7% on NP, and 9.7% on PJM.Source: B. Main table analysis · p. 3
- CastClaw’s MAE reductions versus the best competing result are 13.9% on BE, 20.9% on DE, 23.5% on FR, 23.5% on NP, and 11.3% on PJM.Source: B. Main table analysis · p. 3
- The strongest competitor varies by dataset; the largest MSE gain is on DE, while positive gains on BE and PJM show results are not driven by one dataset.Source: B. Main table analysis · p. 3
- MSE emphasizes large deviations and MAE typical absolute error, but point estimates alone establish neither significance nor causal effects of system components.Repeated, controlled runs are required to test significance and attribute gains to checks, tool calls, user interventions, or stopping rules.Source: B. Main table analysis · p. 3
C. Case study and offline workflow validation
The Nord Pool case documents an inspectable workflow, while provincial-load data from North China validate offline execution only. The load data support no claims of comparative accuracy, deployment, business impact, or causal effects.
C. Case study and offline workflow validation
- The Nord Pool run records the request, supplied forecast, residual checks, retrieved context, tested adjustment, validation comparison, and stopping decision.The comparison explains why the adjustment was rejected; Fig. 2 depicts the supplied 24-hour forecast, checks, tested adjustment, and retained forecast.Source: C. Case study and offline workflow… · p. 3, Figure 2 · p. 4
- Provincial electricity-load data from North China covering January–June 2026 validate offline workflow execution, not comparative accuracy, deployment, production use, or measured business improvement.Validation uses the same task specification, model and tool interfaces, user-input mechanism, checks, and execution report with load-specific resources; the data are excluded from Table I and support no component-level causal or user-study claims.Source: C. Case study and offline workflow… · pp. 3–4
- Aggregate provincial-load measurements contain no personal data or human-subject records and were used only for offline workflow validation under provider authorization and confidentiality requirements.They support no claim of business impact.Source: C. Case study and offline workflow… · pp. 3–4
V. CONCLUSION
CastClaw connects specialized forecasting models, analytical tools, user input, targeted checks, and an execution report without replacing the predictors. It reports the lowest point-estimate MSE and MAE in a five-dataset electricity-price setting, while its provincial-load validation provides no comparative-performance or deployment evidence.
V. CONCLUSION
- CastClaw reports the lowest point-estimate MSE and MAE in the five-dataset electricity-price setting.The conclusion presents these results as support for CastClaw as an implemented system.Source: §V · p. 4
- Its harness connects specialized models, analytical tools, user input, targeted checks, and an execution report without replacing the predictors.The Nord Pool case records rejection of an unsupported revision.Source: §V · p. 4
- Offline provincial-load validation for January–June 2026 exercises the workflow but provides neither comparative-performance nor deployment evidence.Ablations and user studies remain future work.Source: §V · p. 4