PaperlayerView exact revision on arXiv

Source-linked AI summary

Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital

T. J. Barton, Chris Constantakis, Patti Hauseman, Annie Mous, Alaska Hoffman, Brian Bergeron, Hunter Goodreau

arXiv:2604.26091v1cs.AIcs.CEcs.MA

TL;DR

Capital-managing language-model agents lack evaluation evidence under persistent real-capital exposure and irreversible settlement. This paper studies the operating layer connecting user mandates to validated onchain actions and finds that instrumentation, structured intent, validation, and tracing made agents more measurable and correctable while iterative harness changes reduced observed failures.

  • Problem

    Most agent evaluations use backtests, simulations, or synthetic environments without persistent real-capital exposure, repeated fees, shared market feedback, or irreversible settlement.

  • Method

    The paper analyzes a live-capital operating layer that compiles user configuration into model context, validates actions before settlement, and traces decisions through long autonomous horizons.

  • Results

    The live deployment showed that failures could be found, attributed, and iteratively reduced through operating-layer prompt and harness interventions.

  • Takeaways & Limitations

    Capital-managing agents should be measured across the full chain from user mandate and model context through policy validation, execution, and settlement.

  • Takeaways & Limitations

    Evidence is scoped to one venue, market structure, base-model family, bounded action set, and 21-day tournament.

Abstract

from arXiv · show

We study reliability in autonomous language-model agents that translate user mandates into validated tool actions under real capital. The setting is DX Terminal Pro, a 21-day deployment in which 3,505 user-funded agents traded real ETH in a bounded onchain market. Users configured vaults through structured controls and natural-language strategies, but only agents could choose normal buy/sell trades. The system produced 7.5M agent invocations, roughly 300K onchain actions, about $20M in volume, more than 5,000 ETH deployed, roughly 70B inference tokens, and 99.9% settlement success for policy-valid submitted transactions. Long-running agents accumulated thousands of sequential decisions, including 6,000+ prompt-state-action cycles for continuously active agents, yielding a large-scale trace from user mandate to rendered prompt, reasoning, validation, portfolio state, and settlement. Reliability did not come from the base model alone; it emerged from the operating layer around the model: prompt compilation, typed controls, policy validation, execution guards, memory design, and trace-level observability. Pre-launch testing exposed failures that text-only benchmarks rarely measure, including fabricated trading rules, fee paralysis, numeric anchoring, cadence trading, and misread tokenomics. Targeted harness changes reduced fabricated sell rules from 57% to 3%, reduced fee-led observations from 32.5% to below 10%, and increased capital deployment from 42.9% to 78.0% in an affected test population. We show that capital-managing agents should be evaluated across the full path from user mandate to prompt, validated action, and settlement.

1 Introduction

The introduction frames autonomous capital management as an operating-layer problem rather than a single-prompt problem. It motivates studying live systems that translate mandates into bounded, validated, and traceable onchain outcomes under persistent real-capital exposure.

  • Moving funds requires a surrounding control loop that translates user intent into bounded actions, exposes market state, rejects invalid trades, and preserves execution evidence.A model response is only one component of the control loop.
  • Most existing evaluations omit persistent real-capital exposure, repeated fees, shared market feedback, and irreversible settlement.Financial systems are commonly tested in backtests or simulations, while general agent benchmarks use software, web, or synthetic environments.
  • DX Terminal Pro created a narrower but more demanding setting in which users funded vaults with real ETH and agents operated those vaults during normal market operation.Humans could configure strategy and risk preferences but could not directly choose individual token buys or sells.
  • The research question is how a live system can make capital-managing agent behavior measurable, attributable, and correctable before capital moves.The operating layer spans user controls, prompt compilation, model calls, parsing, policy checks, execution, vault contracts, market indexing, and trace logs.
  • The paper contributes a live-capital architecture, measured failure modes with prompt and harness interventions, and an analysis of production behavior after the final harness.Its empirical target is the bounded Terminal Pro deployment; cross-asset and cross-venue transfer remain follow-on work.

2 System Setting and Runtime

DX Terminal Pro was a real-capital deployment in which users configured and funded vaults while agents alone selected trades within a bounded onchain environment. Reliability depended on separating prompt-level guidance from hard execution controls and preserving an instruction-to-settlement trace.

  • Runtime environment: Users funded one vault per wallet, while agents alone traded 12 genesis-launched memecoins through Uniswap V4 pools.Each swap incurred a 2.0% protocol fee and a 0.3% LP fee, totaling 2.3%; agents were polled roughly 12–15 times per hour.
  • Observability: Onchain configuration and per-inference logging created an authoritative instruction-to-settlement trace from user mandate through portfolio state, validation, and chain outcome.The runtime read the latest committed slider and strategy state before each inference call and preserved the compiled prompt, response, reasoning, tool call, portfolio snapshot, validation result, and chain outcome.
  • Runtime environment: The runtime held core experimental settings constant, while user variation entered through funding, sliders, strategies, and owner-level controls.The controlled components included the kernel, hardware allocation, model-serving path, model version, sampling settings, prompt template, and execution policy.
  • Control architecture: Structured 1–5 controls and natural-language strategies shaped prompts, but backend policies independently enforced trade size, slippage, balances, allowlists, and other constraints.Maximum trade size and slippage tolerance were hard backend constraints, distinct from the Trade Size slider’s prompt-level guidance.
  • Execution safeguards: 99.9% of production settlements were valid parsed model actions that passed policy checks and were submitted, while malformed or rejected outputs were counted separately.The operator had least privilege, and the offchain policy layer checked token validity, balances, slippage bounds, position limits, malformed calls, and impossible parameters before submission.

3 Control-Loop Method

The control loop evaluated prompt and harness variants through repeated, controlled multi-turn replays rather than one-turn instruction tests. Trace classification and macro metrics then localized failures, enabling narrow interventions that were remeasured on the same scenarios.

  • Controlled evaluation: 3,000 sampled scenarios were replayed under controlled prompt comparisons, preserving market snapshot, portfolio state, strategy context, and slider settings.Each candidate prompt was evaluated across the slider grid at 60 samples per slider level.
  • Trace diagnosis: 4,900 sampled reasoning traces were classified across trade, observation, and sizing drivers, then checked against buy-sell ratio, trade rate, capital deployment, fee-cited observation rate, and slider gradients.The labels diagnosed failure-mode incidence but were not used to execute trades.
  • Diagnostic calibration: 2.3% fee conditions required calibrated fee awareness rather than universal fee citation, because opportunity should sometimes outweigh cost.Fee citation was treated as a trace diagnostic, not an optimization target.
  • Iterative intervention: The control loop ran a variant, inspected trace and metric deltas, attributed failures to compiled-brief or runtime features, applied a narrow intervention, and remeasured on the same scenario population.Interventions included moving a sentence, removing a number, adding a skip gate, shortening memory, or injecting a slider-specific conditional.

4 Failure Modes and Fixes

The paper identifies failure modes as interpretation failures arising from prompt text, memory, fees, and repeated operation rather than generic bad trades. Fixes therefore target the operating harness—especially reading order, numeric phrasing, structured tokenomics context, precedence, source of truth, memory semantics, and domain payoffs.

  • Failure-mode framing: Failure modes were interpretation failures produced by interactions among prompt text, memory, fees, and repeated operation.The paper distinguishes these failures from generic bad trades.
  • Reading order: 74% of reasoning traces cited the fee sentence after it moved from paragraph 8 to paragraph 1, versus 3% before, without changing the model, wording, or market.This demonstrates that the model overweighted reading order.
  • Numbers as rules: At v2.12, TA=5 agents traded at 8.3%, below TA=3 at 10.7% and TA=4 at 12.9%, because explicit OBSERVE floors were read as targets.Replacing floors with comparative language restored the intended gradient; “allocate up to 33%” similarly became an equal-weight target for some agents.
  • Structured tokenomics context: The reap mechanic made visible price action conflict with payoff: DOGPANTS crashed on the chart, while holding through the event could be better than selling into the crash.The prompt initially foregrounded the crash, contributing to the tokenomics interpretation problem.
  • Structured tokenomics context: Agents could reason over arbitrary tokenomics when mechanics were inserted as structured context with payoff order, source of truth, and state variables.The successful intervention inserted whitepaper mechanics alongside portfolio and market state rather than imposing a hard-coded trade directive.
  • Harness over model: A larger model may reduce malformed outputs, but it does not remove the need to specify precedence, source of truth, memory semantics, and domain payoffs.These failures provide the core empirical reason to study the harness rather than the model alone.

5 Trace Reuse and Harness Transfer

Trace-level records support diagnosis beyond post-hoc explanation by distinguishing model, mandate, memory, validation, and execution failures. Harness changes also transferred across model versions, while the same traces can seed testing and future training.

  • Trace reuse: Each action trace links user settings, strategy text, compiled prompt, reasoning, tool arguments, validation, and settlement, separating failure classes that look identical in aggregate.The trace can distinguish model confusion from contradictory strategies, impossible risk settings, stale memory, or correctly rejected execution requests.
  • Trace reuse: Traditional open-ended memory and RAG-style retrieval were not obviously helpful in this deployment.Portfolio state, transaction history, strategy status, and cooldown state were treated as more relevant context than unrestricted retrieval, according to the supplied passage.
  • Harness transfer: 99.9% aligned successful EVM swap construction followed harness optimization, up from 96% for Claude 4.6 and 87% for Claude 4.The separate evaluation covered an average Ethereum DEX buy/sell task; the values were internal evaluation summaries, not production open-market P&L.
  • Future training: Live adversarial execution traces can seed synthetic scenarios, offline policy tests, and reinforcement-learning reward definitions.The paired records contain user intent, state, model interpretation, validation outcome, and chain result; MEMEbench reconstructed trading scenarios across 18,560 inference calls.
  • Proactive diagnosis: Trace-level labels can be extended with reasoning classifiers and activation-level signals to detect model confusion before final reasoning text.Ongoing mechanistic-interpretability work found structured internal market representations and causal handles for some market signals.

6 Production Behavior Under a Frozen Harness

Under a frozen 21-day harness, production agents preserved ordered slider behavior while exhibiting attention cascades and heterogeneous flow from shared state, settings, positions, and strategies. Concrete exit-oriented controls were associated with better observed profitability, while language-cohort differences were non-random and confounded.

  • Control behavior: All five sliders produced directionally correct behavior, with Trading Activity spanning 2.8%–16.8% of invocations and Trade Size spanning about 2%–95% of available ETH.Holding Style and Diversification remained ordered but were compressed by overlapping safety mechanisms.
  • Attention cascades: 1,544 of 3,454 active vaults bought FEET within one hour, creating an attention cascade without direct communication.Each purchase increased the volume and momentum visible to later invocations.
  • Attention cascades: 3,878 sell cascades occurred across the tournament, including a POOPCOIN cascade compressing 438 sells into a 9.5-second median inter-agent gap.A sell cascade required at least 10 vaults to sell the same token within 10 minutes.
  • Flow heterogeneity: 92.9% of trades occurred in two-sided five-minute token-windows, with heterogeneity arising from slider settings, inherited positions, and user strategies rather than model diversity.A two-sided window contained at least one buy and one sell for the token.
  • Observed outcomes: 4.2× as often, users specifying exit conditions or parameter changes achieved profitability compared with users asking agents to outperform or pick winners.Among 87 owners who never used chat but actively configured sliders and strategy UI, 41% closed in profit.
  • Language cohorts: Chinese-language strategy cohorts had higher observed end-of-event profitability than predominantly English-language cohorts, but the comparison was non-random and confounded by activity and strategy specificity.Roughly one quarter of user strategy and chat activity was Chinese-language or Chinese-led.

7 Related Work

Prior financial LLM and agent benchmarks often stop at backtests, simulations, or task success, whereas this work evaluates a linked instruction-to-settlement trace under real execution. The trace and operating-layer perspective expose failures involving prompt sensitivity, fabricated rules, persistence, resource depletion, and multi-agent feedback.

  • Financial LLM evaluation: Financial LLM systems are useful starting points, but evaluation that stops before execution is especially fragile in finance.Prior systems including FinGPT, BloombergGPT, TradingGPT, and FinMem evaluate language modeling, memory, and trading behavior in backtests or simulations.
  • Trace-based evaluation: The contribution is a linked instruction-to-settlement trace spanning user configuration, prompt, reasoning, tool call, validation result, and chain outcome.This trace distinguishes user-mandate errors from model confusion, prompt-order effects from market effects, and invalid action construction from valid but poorly performing trades.
  • Operational failure modes: Repeated financial operation turns prompt sensitivity and ordering effects into failures such as fee paralysis and slider inversion.The paper connects these effects to prior studies of prompt sensitivity in static tasks and reports cross-model harness and ticker-name tests suggesting model-family tendencies in trading-agent failures.
  • Agent benchmarks: Trading-agent failures require persistence, resource depletion, and multi-agent feedback beyond the task-success measures used by general agent benchmarks.Fabricating an allocation rule can cause a trading agent to pay fees every tick while reinvesting, unlike a coding agent whose fabricated function name fails a test.
  • Operating layer: Finance-native sequence models can serve as situational predictors, tools, and synthetic-data components, but they do not replace the operating layer.The paper frames language-model agents’ distinctive market value as following arbitrary and creative user guidance while simultaneously r

8 Limitations and Ethics

The study’s claims are bounded by a single venue, market structure, model family, action set, and 21-day tournament, while its strongest causal evidence comes from controlled pre-launch comparisons. The deployment also involved real user funds and publicly inspectable onchain activity and configuration, with specified models used for agents and trace classification.

  • Scope: The study covers one venue, one market structure, one base-model family, one bounded action set, and one 21-day tournament with a fixed token universe and 2.3% per-swap fees.Its claim is limited to operating and diagnosing capital-managing agents under a constrained but real settlement loop.
  • Causal status: Controlled pre-launch prompt comparisons on replayed scenarios and live-like agents provide the strongest causal evidence, while production profitability, cascade incidence, and herding remain observational.The live tournament extended controlled findings through real execution, user mandates, public settlement, market feedback, and long-horizon autonomous operation under a frozen harness.
  • Risk and privacy: The deployment exposed real user funds, public Base transactions, and publicly inspectable vault activity, settings, active strategies, agent transactions, and related API records.Participants were informed that the event was experimental and that real capital was at risk; inferences were associated with prompt identities or hashes.
  • LLM use: The deployed agents used Qwen/Qwen3-235B-A22B-Thinking-2507, while Claude Sonnet 4.5 classified sampled reasoning traces for aggregate analysis.Claude Opus 4.7 and OpenAI Codex GPT-5.5 supported manuscript editing and figure or script preparation, with authors retaining responsibility for claims, plots, and citations.

9 Conclusion

Live onchain capital-managing agents became more reliable through operating-layer controls that structured user intent, validated execution, and traced outcomes. The study also shows that iterative testing can reduce failures and that full user-to-agent-to-execution traces expose broader research opportunities.

  • Operating-layer reliability: Operating-layer controls made live agents more measurable and correctable by structuring user intent, compiling per-agent context, validating actions, and tracing settlement.The main reliability gains came from the system around the model rather than the model alone.
  • Failure reduction: Fabricated rules, fee paralysis, and tokenomics misreads fell when reasoning, costs, and protocol mechanics were represented in more appropriate context.The interventions demoted prior reasoning from precedent, placed costs beside expected moves, and structured relevant tokenomics mechanics.
  • Long-horizon evaluation: User-to-agent-to-execution traces over autonomous market horizons expose opportunities in instruction following, strategy consistency, tooling, synthetic data, model evaluation, and future training loops.In markets, agents act through changing states with user capital, execution constraints, and other agents reacting to the same state.

A Metric and Figure Data … C. Market snapshot loop

The paper documents a reproducible figure-data pipeline and a late-production prompt harness that assembled agent-specific context from configuration, market, portfolio, strategy, memory, and runtime state. Operating rules prioritized constraints and actionable strategies, while the market loop supplied conditional price, volume, flow, and holder snapshots.

  • A Metric and Figure Data: Figures were regenerated from CSV files by figures/generate rewrite figures.py, with aggregate values drawn from analysis artifacts and public activity inspectable through Dune.User settings and strategies were public onchain configuration, while inference records linked prompt identities or hashes through the system API.
  • B Prompt Excerpts: Each production invocation combined fixed prompts with public configuration, indexed market data, portfolio state, strategy records, recent memory, and runtime clock fields.Go-template conditionals and helper functions inserted only the relevant market, portfolio, strategy, reap, slider, and price-impact context for each agent.
  • B Prompt Excerpts: The selected final-day prompt slices expose the control logic, template variables, and conditional insertions that generated per-agent context from the production Go template.The slices were condensed for page fit but were not synthetic examples.
  • A. Operating rules and strategy router: The operating loop polled agents approximately every 5 minutes for days, treating each tick as an independent opportunity to act.This rule framed repeated decision-making as separate evaluations rather than one continuous action.
  • A. Operating rules and strategy router: The decision hierarchy placed hard constraints and tool schema first, followed by actionable active strategies, medium strategies, sliders, and low-priority suggestions.Immediate-action and Triggered-action directives were executed when feasible or fired; compliant restrictions and holds produced record_observation.
  • C. Market snapshot loop: The market snapshot loop rendered ETH/USD price, token supply, current-token listings, token prices, conditional ages, and multiple price-change windows.The template included 5m, 1h, and 24h changes when available, alongside broader conditional change fields.

B. Strategy lifecycle protocol … M. Invalid motives explicitly blocked

The protocol turns user strategies into explicitly classified, status-tracked directives and constrains every action through literal interpretation, execution limits, memory contracts, and sell safeguards. It also blocks invented motives and requires observation whenever an action lacks exact prompt or active-strategy support.

  • B. Strategy lifecycle protocol: Only strategies in the active-strategies section are binding, with directives classified as immediate actions, triggered actions, restrictions, or hold rules.Each class has distinct lifecycle states for execution, monitoring, compliance, violation, or completion.
  • D. Reaps / tokenomics context: Reap outcomes depend on market cap at reap time, while holding through a reap preserves eligibility for pro-rata compensation and selling early can incur round-trip fees.Trading activity before the reap can change which token graduates.
  • E. Slider values and conditional rules / G. Cooldowns and churn veto: High trading activity is framed as setup-finding rather than churning, with repeated same-token actions lacking meaningful new evidence converted to observations.Risk and holding sliders condition candidate selection and active-patient behavior.
  • G. Cooldowns and churn veto: Cooldown and pre-trade checks reject actions that violate restrictions, lack a valid sell reason, create an unjustified round trip, or lack exact strategy or trigger support.Immediate-action and triggered-action trades are the only stated exceptions to cooldowns.
  • I. Anti-fabrication rules: Agents must follow only explicit prompt rules, interpret constraints literally, treat prior decisions as context, and avoid converting infrastructure polling into fixed trading cadences.The anti-fabrication rules directly prohibit invented thresholds, named rules, and formulas.
  • F. Portfolio and execution constraints: Execution remains bounded by maximum trade amounts, price-impact limits, slippage, balances, and token-pair checks, including when high-priority strategies override pacing guidance.Immediate-action strategies may override slider pacing or churn guidance but not hard execution caps.
  • H. Memory and tool-output contract: Memory records provide context rather than binding precedent, each action represents approximately five minutes, and tool calls must name the strategy or slider and key market signal.The available actions are BUY, SELL, and OBSERVE.
  • J. Reasoning status labels / K. Sell rules and rotation / M. Invalid motives explicitly blocked: Selling requires a specific risk or strategy reason, while rotation additionally requires a new opportunity, the weakest thesis, and no recent rotation; fabricated motives are explicitly blocked.Invalid motives include zero-balance requirements, mandatory redeployment, ETH generation, buffer rebuilding, and idle-ETH urgency.

L. Launches and new-coin caps · N. Output schema discipline

The operating layer surfaces upcoming launches and risk-gated buying guidance while imposing progressively increasing purchase caps on recently launched coins. It also strictly constrains each tick to one validated tool action with controlled token selection, strategy labeling, and concise reasoning notes.

  • L. Launches and new-coin caps: Upcoming launches are presented through a dedicated section with a countdown and, when available, the token’s display name.The launch block conditionally renders launch information, including the next-launch countdown and display name.
  • L. Launches and new-coin caps: At risk preference level 2 or higher, new launches are explicitly treated as valid buying candidates.The buying guidance is conditional on AssetRiskPreference being at least 2.
  • L. Launches and new-coin caps: 0.01 ETH per BUY is the initial cap for recently launched coins, increasing by 0.01 ETH every 5 minutes and becoming uncapped after 50 minutes.These hard purchase limits apply specifically to recently launched coins.
  • N. Output schema discipline: Allowed actions are limited to BUY, SELL, and OBSERVE.The output schema enumerates the permitted action values.
  • N. Output schema discipline: BUY and SELL calls must select a token from the MARKET SNAPSHOT.Token selection is constrained to the current market snapshot rather than arbitrary output.
  • N. Output schema discipline: The optional strategy field is included only when the action directly executes an ACTIVE STRATEGY label such as strategy1.Strategy metadata is therefore conditional rather than required on every action.
  • N. Output schema discipline: Each tick MUST produce exactly one tool call and no non-tool text.The schema enforces one executable output per tick.
  • N. Output schema discipline: Reasoning notes must be 1-2 conversational but specific lines naming the strategy or slider and the key market signal.The note format combines brevity with explicit references to the decision control and market evidence.
Loading 2604.26091v1…