LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets

1 Introduction

• Because external conditions evolve and earlier actions persist, observed performance reflects agents, available mechanisms, accumulated state, and environmental variation.Evaluating capability in these settings therefore requires examining the process that produces outcomes, not only the outcomes themselves.Source: §1 · p. 2
• Live markets continually elicit information gathering, experience reuse, constraint following, and coordination as prices and information change under external forces.The benchmark takes a complementary perspective to work focused primarily on trading outcomes, using evolving interactions to diagnose capability.Source: §1 · p. 2
• LiveMACEBench evaluates five frontier LLMs in live cryptocurrency and U.S. equity markets using matched capability configurations and complete decision traces.It combines task outcomes, controlled mechanism effects, and process diagnostics to distinguish mechanism access, effective use, and realized benefit.Source: §1 · p. 2
• Similar outcomes can arise from different mechanism-use patterns, while trace diagnostics reveal capability-specific bottlenecks and an outcome–capability gap.Observed bottlenecks include redundant tool interactions, ineffective memory reuse, unauditable rule execution, and limited evidence integration in multi-agent systems.Source: §1 · p. 2
• The benchmark matches Tool Use, Persistent Memory, Rule Following, and Multi-Agent Collaboration configurations and uses live markets for persistent evaluation.Its contributions include controlled diagnosis of mechanisms and an evolving testbed where external conditions change continuously and agent actions have persistent consequences.Source: §1 · p. 2

2 LiveMACEBENCH: Benchmark Design

• The benchmark pairs a shared live-market interaction protocol with controlled agent variants on a common ReAct scaffold, assessing outcomes and mechanism-specific diagnostics.It is designed to expose all agents to the same evolving market conditions.Source: §2 · p. 3

2.1 Live and Open-Ended Evaluation Environment

• Agents trade in live cryptocurrency and U.S. equity markets, where independent market changes and non-resettable account states make each action consequential for later decisions.Decision rounds occur every four hours; each trace includes model responses, tool calls and outputs, and actions, while account-equity checkpoints measure outcomes between rounds.Source: §2.1 · p. 3

2.2 Controlled Agent Configurations

• Five configurations share a ReAct scaffold: Base and variants targeting Tool Use, Persistent Memory, Rule Following, and Multi-Agent Collaboration.The variants add, respectively, dynamic tool routing, reusable experience across rounds, hierarchical trading rules with explicit compliance reasoning, and manager-led specialist delegation followed by execution.Source: §2.2 · pp. 3–4

2.3 Capability-Specific Evaluation Dimensions

• Evaluation combines cumulative return with configuration-specific diagnostics for tool use, memory, rule following, and collaboration.The dimensions include tool relevance and call reliability; memory quality and downstream drawdown and tail-loss changes; rule compliance and auditability; and role diversity and evidence integration.Source: §2.3 · p. 4

3 Evaluation

• Five frontier LLMs were evaluated over about 30 days across six cryptocurrencies and seventeen U.S. equities.The evaluation ran from April 13 to May 13, 2026.Source: §3 · pp. 4–5

3.1 Live Markets Reveal an Outcome–Capability Gap

• Return rankings show 21, 19, 19, and 29 reversals across configurations, versus 0, 5, 8, and 2 for capability rankings.Capability leaders also differ: GPT-5.4 leads tool use, Grok-4.20 memory utilization, Gemini-3.1-Pro rule following, and DeepSeek-V3.2 collaboration.Source: §3.1 · p. 5, Figure 2 · p. 5

3.2.1 Tool Use

• All models achieve adequate evidence coverage, yet tool efficiency, discovery, and execution show distinct bottlenecks.Information Coverage exceeds 0.8 (0.809–0.911), while Invocation Efficiency is weakest (0.581–0.731); Qwen3-Max has RQ 0.736 but VCR 0.978.Source: §3.2.1 · p. 6, Figure 3 · p. 6

3.2.2 Persistent Memory

• Memory accumulation, quality, use, and downstream effects diverge rather than forming one capability.Gemini stores 93 entries but has the lowest MUQ (0.497), while Grok stores 12 and has the highest MUQ (0.855); all five memory-augmented agents reduce tail loss.Source: §3.2.2 · pp. 6–7, Table 1 · p. 7

3.2.3 Rule Following

• Rule execution and auditability separate: DeepSeek-V3.2 scores RSS 0.847 but RAS 0.623, while Gemini-3.1-Pro scores 0.872 and 0.882.Trace analysis finds DeepSeek decisions with little explicit rule checking or grounded conflict reasoning, and exposure-estimation errors contribute to concentration-limit violations.Source: §3.2.3 · pp. 7–8, Figure 4 · p. 8
• Nearly equal final returns (+5.765% vs. +5.738%) accompany different HPR (0.877 vs. 0.682) and RSS (0.847 vs. 0.634).Gemini-3.1-Pro returns +3.503% but achieves the highest HPR, RSS, and RAS among evaluated models.Source: Table 2 · p. 7, §3.2.3 · p. 8

3.2.4 Multi-agent Collaboration

• Multi-Agent raises returns for three backbones but lowers Qwen3-Max’s by 4.39 percentage points, while all four incur greater downside risk.All four models record more sharp gains alongside larger maximum drawdowns and more negative average tail losses.Source: §3.2.4 · p. 8, Figure 5 · p. 8, Table 3 · p. 9
• A shared multi-agent harness yields different collaboration patterns, and Observable Collaboration Score does not monotonically track return gains.DeepSeek-V3.2 has RD 0.760 and EI 0.852; Gemini-3.1-Pro pairs RD 0.492 with EI 0.984, while GPT-5.4 nearly collapses to one role (RD 0.012).Source: §3.2.4 · p. 9, Figure 6 · p. 9

• Agent evaluation has expanded from isolated responses to multi-step interaction, while analytical benchmarks argue that final success alone provides limited diagnostic information [4] [5] [6] [1] [2].Source: §4 · p. 9
• LiveMACEBench studies persistent trajectories in markets that evolve independently of agents while earlier actions alter future state, unlike task-scripted environments [7] [19] [20] [22].Source: §4 · p. 9
• Rather than prioritizing trading performance, LiveMACEBench uses shared live-market trajectories, matched mechanism variants, and trace-derived metrics to distinguish mechanism access, effective use, and realized benefit [8].Source: §4 · p. 9

5 Conclusions

• Across five frontier LLMs and approximately 30 days, realized performance varied substantially and did not reliably reflect how effectively agents used available mechanisms.The benchmark combines persistent, non-resettable trajectories with controlled configurations and trace-derived diagnostics.Source: §5 · p. 10
• Process diagnostics showed that memory accumulation need not yield effective reuse, rule execution can diverge from auditability, and multi-agent gains can occur without strong observable collaboration.Similar outcomes can mask different capabilities, and mechanism access need not produce effective use or improved performance.Source: §5 · p. 10
• Evaluating agents in evolving environments requires measuring the mechanisms behind outcomes, with longer horizons, broader regimes, and other continuously evolving settings identified as next steps.LiveMACEBench provides one instantiation of this process-aware approach in financial markets.Source: §5 · p. 10

A.1.2 Rule-Following Agent Prompt

• The Rule-Following agent prompt directs agents to gather information, pre-check rules, and trade for profit within hard-rule boundaries.It treats soft rules as optimization targets and requires a final <TRADE_DONE> token.Source: §A.1.2 · p. 19

A.1.3 Rule-Following Auditor Prompt

• The Rule-Following auditor scores written rule awareness and conflict handling separately from the mechanical checker’s compliance results.It uses the mechanical check as ground truth, evaluates rule coverage and the R0 > R1 > R2 priority hierarchy, and normalizes the two 1–10 scores by dividing their sum by 20.Source: §A.1.3 · pp. 19–20

A.2 Persistent Memory

• Persistent Memory evaluation averages normalized 0–5 rubric scores across three judge models to produce content and usage scores.The judges are GPT-5.4, DeepSeek-V3.2-Thinking, and Qwen3-Max.Source: §A.2 · p. 20

A.2.1 Content-Quality Judge Prompt

• The content-quality rubric scores sampled memories on generalizability, information density, and diversity contribution.Memories are expected to encode reusable trading rules without specific dates or prices and to include a condition–observation–rule causal chain.Source: §A.2.1 · p. 20

A.2.2 Usage-Quality Judge Prompt

• The usage-quality rubric rates whether memory searches are timely and whether retrieved memories influence subsequent reasoning and trade actions.Judges assess sampled memory-search trace fragments using market context, the query, returned memories, and subsequent steps.Source: §A.2.2 · p. 21

A.3.1 Base Configuration System Prompt

• The Base ReAct prompt requires planned, tool-grounded analysis, while the Tool-Augmented ReAct prompt adds dynamic routing through select_tools.Both prompts require evidence before trading and prohibit guessing market or account information.Source: §A.3.1 · pp. 21–22, §A.3.2 · p. 23

A.3.3 Tool Use LLM-Judge Prompt

• The Tool Use judge evaluates trace-level tool relevance, timing and budgeting, information coverage, and synthesis faithfulness.Each evaluation instance supplies the complete decision trace, including reasoning, tool calls, and outputs.Source: §A.3.3 · p. 24

A.4 Multi-Agent Collaboration System Prompts

• Runtime-specific values are passed in separate user messages to the manager, specialists, and execution agent; tool schemas come through the model interface.The manager receives the objective, portfolio, prices, timestamps, tradable universe, evidence, conflicts, and collaboration state; other agents receive role-relevant inputs.Source: §A.4 · p. 25

A.4.1 Manager System Prompt

• The manager selectively calls specialists by weighing information benefit against coordination cost and can turn multiple valid recommendations into an ordered execution plan.The prompt discourages repeated checks without new evidence and reserves a single-step plan for cases where only one recommendation survives risk, timing, or market-status constraints.Source: §A.4.1 · p. 25

A.4.2 TradingAgent System Prompt

• The TradingAgent reviews the full tradable universe and normally returns 2–4 distinct recommendations ordered by execution priority; if no trade is actionable, it returns a single hold.For any US-stock idea, it checks market_status before recommending execution and recommends hold/defer when the stock is not currently tradable.Source: §A.4.2 · pp. 26–27

A.4.3 NewsAgent System Prompt

• The NewsAgent focuses on recent, high-relevance catalysts, using concise searches and reporting uncertainty rather than stretching weak or stale results.It reports events, dates, sentiment, risks, implications, and confidence in a structured response.Source: §A.4.3 · p. 28

A.4.4–A.4.5 Specialist System Prompts

• The CoderAgent performs targeted calculations to inform trading decisions, while the AnalystAgent synthesizes existing evidence and makes conflicts explicit.The CoderAgent returns methods, results, limitations, and recommendation impact; the AnalystAgent reports consensus, conflicts, hidden risks, and a proposed resolution.Source: §A.4.4 · p. 28, §A.4.5 · p. 29

A.4.6 CriticAgent System Prompt

• The CriticAgent challenges the current thesis and surfaces downside scenarios, risk controls, and veto conditions before execution.Its structured review includes challenged assumptions and a final warning.Source: §A.4.6 · p. 29

A.4.7 Execution Agent System Prompt

• The execution agent follows the approved plan in order, makes an execute_trade call for each executable item, and finishes with <TRADE_DONE>.An empty plan ends immediately with the completion marker.Source: §A.4.7 · p. 29

B.1 Multi-Granularity Evaluation Protocol and Trading Outcome Metrics

• Account checkpoints at 15-minute, 1-hour, 4-hour, and daily intervals convert cash plus holdings into returns, with one-hour sequences underpinning trajectory metrics.PnL is the change in account equity between checkpoints; the account can change in value without a new trade as existing investments move with the market. Trajectory metrics include return, drawdown, win rate, Sharpe and Calmar ratios, tail loss, and rolling volatility.Source: §B.1 · pp. 29–30

B.2 Tradable Universe and Baseline Details

• The evaluation restricts agents to 23 cryptocurrency and U.S. equity assets and includes a reproducible grid-trading baseline with five levels on either side of a reference price.Grid levels use multiplicative 1% spacing, and the strategy allocates up to 80% of account equity to grid trading.Source: §B.2 · p. 31

B.3 Runtime and Agent Configuration Details

• The runtime uses a shared decision-round structure for its configurations, with trace recording, tool routing, persistent memory, rule-aware interaction, and hierarchical multi-agent workflows.Capability metrics are defined separately from these runtime implementation details.Source: §B.3 · p. 31

B.3.1 Base Runtime and Decision-Round Execution.

• Each Base decision round is a ReAct interaction over fixed tools; account state persists across rounds, and traces record responses, calls, outputs, and actions.The agent may gather evidence and execute multiple trades before ending the round or exhausting its execution budget.Source: §B.3.1 · pp. 31–32

B.3.2 Tool Use Runtime

• Tool Use preserves the Base interaction but adds dynamic discovery and activation from a 305-tool candidate space, while five execution-critical Base tools remain directly available.The expanded catalog includes 295 additional interfaces derived from public API specifications; routing requests and tool results are retained in decision traces.Source: §B.3.2 · pp. 32–33

B.3.3 Persistent Memory Runtime

• Persistent Memory adds market-scoped search and storage of reusable condition–observation–rule experience, retrieved by relevance and recency and retained across decision rounds.Near-duplicate entries with at least 0.9 embedding cosine similarity to an existing entry in the same market are not added.Source: §B.3.3 · pp. 33–34

B.3.4 Hierarchical Rule-Following Runtime

• Rule Following applies a three-level hierarchy: R0 and R1 are hard boundaries, while R2 preferences may be traded off if higher-priority requirements remain satisfied.Before any trade or hold decision, the agent supplies a structured compliance rationale and audit; traces are evaluated by a deterministic checker and an independent LLM auditor.Source: §B.3.4 · pp. 34–35

B.3.5 Multi-Agent Runtime

• Multi-Agent replaces the single-agent decision process with a manager, five specialists, and a separate ExecutionAgent, using fixed role permissions and source-attributed evidence.The manager produces an ordered execution plan from specialist input, which ExecutionAgent processes; traces retain the evidence and subsequent actions.Source: §B.3.5 · pp. 35–36

B.4 Capability-Specific Evaluation Details

• Capability-specific measures evaluate mechanisms using decision trajectories and records, complementing task outcomes with diagnostics defined in this appendix.The judging prompts and scoring rubrics are provided separately.Source: §B.4 · p. 36

B.4.1 Tool-Use

• Tool-use judging scores relevance, invocation efficiency, information coverage, and evidence faithfulness in the decision trace.The dimensions assess whether tools address information needs, calls avoid waste, evidence is sufficient, and reasoning is supported by tool outputs.Source: §B.4.1 · p. 36
• Objective tool-use diagnostics assess routing relevance, valid-call rate, and hallucination-free rate; execution-cost statistics also include calls per assistant step and token consumption.Routing Quality is normalized to [0, 1], and traces without scored routing requests receive zero.Source: §B.4.1 · pp. 36–37

B.4.2 Persistent Memory

• Memory Content Quality assesses whether stored experience is generalizable, information-dense, and diverse, while Memory Usage Quality assesses timely retrieval and subsequent adoption.Both are judged and normalized to [0, 1].Source: §B.4.2 · p. 37
• Memory’s downstream risk effects are measured against a matched memory-free configuration using changes in maximum drawdown and TailLoss; Memory Score equally averages four memory metrics.The normalized TailLoss change is larger when losses in the worst-performing intervals are smaller.Source: §B.4.2 · pp. 37–38

B.4.3 Rule Following

• Rule execution combines hard-rule pass rate and soft-rule satisfaction, with hard-rule compliance gating the contribution of lower-priority preferences.RSS requires both reliable hard-constraint compliance and strong satisfaction of soft preferences; HPR is reported separately to expose hard-rule failures.Source: §B.4.3 · p. 38
• Rule reasoning is audited for coverage and conflict handling, while Rule Auditability Score summarizes the two audited dimensions on a normalized scale.The audit checks correct identification and interpretation of applicable rules and respect for the R0 > R1 > R2 hierarchy.Source: §B.4.3 · pp. 38–39

B.4.4 Multi-Agent Collaboration

• Multi-agent collaboration measures specialist-role breadth and balance alongside evidence coverage and integration; Observable Collaboration Score summarizes these patterns, not semantic accuracy.Role Diversity penalizes concentration only when one role exceeds 80% of scored-role calls; Evidence Integration rewards using and combining evidence from sources.Source: §B.4.4 · p. 39

C.1 Market Conditions During the Evaluation Period

• Cryptocurrency and U.S. equity markets experienced both positive and negative stages, while the buy-and-hold reference remained positive over the full evaluation period.The changing conditions provide a non-stationary setting for comparing agent behavior and realized trading outcomes.Source: Figure 7 · p. 40, §C.1 · p. 40

C.2 Trading Outcome Summary

• Tool Use has the highest return for DeepSeek-V3.2 and Grok-4.20, Base for Qwen3-Max, and Multi-Agent Collaboration for Gemini-3.1-Pro; no configuration leads across all models.Table 7 marks the Gemini–Tool Use and Grok–Multi-Agent Collaboration runs as unavailable.Source: Table 7 · p. 40, §C.2 · p. 40

C.3 Detailed Tool-Use Results

• DeepSeek-V4-Pro assigns judge means 0.0481 points higher on average, but both judges preserve the same model ordering across tool-use scores.The evaluated models rank Qwen3-Max, GPT-5.4, DeepSeek-V3.2, then Grok-4.20.Source: §C.3 · p. 41
• Information Coverage agreement is 95.6% and Tool Relevance agreement is 91.0%, compared with 69.4% for Invocation Efficiency and 80.7% for Evidence Faithfulness.Invocation Efficiency has MAE 0.1123; Evidence Faithfulness has MAE 0.1022. The latter dimensions require more interpretive judgments about redundancy and the value of long tool traces.Source: §C.3 · pp. 41–42, Figure 9 · p. 41
• All four models have HFR of 1 and zero dynamic-schema misses, while Routing Quality and Valid Call Rate vary substantially.The diagnostics distinguish tool discovery and invocation reliability from evidence use; calls per assistant step and token consumption additionally describe execution cost.Source: Table 9 · p. 41, §C.3 · p. 42
• The Tool-Augmented Gemini-3.1-Pro run is excluded from tool-use and trading-performance comparisons after an environment consistency defect corrupted its trajectory metrics.Lagging state observations led the agent to repeat trades after execution; the platform defect was subsequently fixed.Source: §C.3 · p. 42

C.4.1 Full-Period Performance.

• All five memory-augmented accounts achieve lower Tail Loss than matched memory-free accounts, while four of five also improve Maximum Drawdown and consecutive-loss streaks.GPT is the exception for Maximum Drawdown.Source: §C.4.1 · p. 42
• Memory improves full-period PnL for DeepSeek (+1.78% vs. -0.12%) and Grok (+0.84% vs. -2.07%), while memory-free accounts return more for GPT, Gemini, and Qwen.The results support separating downside-risk effects from raw return when assessing memory effectiveness.Source: §C.4.1 · p. 42

C.4.2 Memory Construction and Stagnation

• GPT-5.4 makes 332 memory_search calls but only one memory_add, leaving one stored entry; Gemini stores 93 entries, while DeepSeek, Grok, and Qwen retain 41, 12, and 11.These account statistics reveal sharply different memory-construction dynamics despite an identical interface.Source: §C.4.2 · p. 42
• In 88% of analyzed decisions, GPT concludes that no additional reusable rule needs storing, illustrating active retrieval alongside a memory store that stops expanding.Its cryptocurrency rule refers to market conditions relative to recent highs, so the rule remains applicable as price levels change.Source: §C.4.2 · pp. 42–43
• DeepSeek and Qwen generate candidate memories that deduplication removes, while Gemini continues accumulating a substantially larger set of entries.The shared interface therefore accompanies qualitatively different experience-construction processes across backbones.Source: §C.4.2 · p. 43

C.4.3 Temporal Effects

• DeepSeek’s memory return differential rises from +1.90 pp full-period to +2.14 pp late-phase, while Grok’s rises from +2.91 to +3.18 pp.Gemini shifts from -1.32 pp to +0.53 pp; GPT shifts from -1.46 pp to -1.80 pp, and Qwen’s disadvantage narrows from -6.50 to -4.85 pp. Downside-risk differences persist for most models.Source: §C.4.3 · p. 43

C.4.4 Judge Robustness

• Memory Score rankings are identical across three judges (pairwise Kendall’s τ=1.0), while Memory Usage Quality rankings have mean pairwise Kendall’s τ of 0.73.Population standard deviation of Memory Score across judges ranges from approximately 0.014 to 0.038.Source: §C.4.4 · p. 43
• All three judges rank Grok highest and Gemini lowest on Memory Usage Quality, but each assigns Gemini higher Memory Content Quality than Grok.The relative ordering of GPT, DeepSeek, and Qwen on Memory Usage Quality varies across judges.Source: §C.4.4 · p. 43

C.5 Additional Rule Following Analysis

• Gemini-3.1-Pro receives the highest scores from two auditors and remains near the highest under GPT-5.5, while Qwen3-Max ranks last for all three.Average pairwise Spearman rank correlation is 0.83; DeepSeek-V3.2 consistently has lower RAS despite strong HPR and RSS.Source: §C.5 · pp. 44–45, Table 13 · p. 46
• The dominant hard-rule bottleneck is R1-02: GPT-5.4 triggers it at 31 events per 100 decisions and Qwen3-Max at 36.Trace analysis attributes failures to inconsistent accounting for existing positions, incremental orders, and price movements, and sometimes treating the 15% limit as a target.Source: §C.5 · p. 45
• GPT-5.4 and Grok-4.20 trigger R2-02 cash-utilization deviations at 81 and 51 events per 100 decisions, while Grok and GPT trigger R2-06 at 64 and 46.R2-02 reflects cash ratios outside the target range; R2-06 reflects periods of insufficient trading activity.Source: §C.5 · p. 45
• DeepSeek-V3.2’s frequent rebalancing keeps variables such as cash ratio within ranges rewarded by the checker, but its lower RAS indicates less stable, well-grounded rule explanations.The broader comparison separates rule execution from auditability.Source: §C.5 · p. 45
• Gemini handles hard-constraint and soft-preference trade-offs with the highest rule-satisfaction and auditability scores despite not achieving the highest return.Grok follows a more conservative strategy and opens fewer positions, while GPT-5.4 and Qwen3-Max frequently incur single-asset concentration violations when increasing positions.Source: §C.5 · p. 46

C.6 Additional Multi-Agent Collaboration Analysis

• The Grok-4.20 Multi-Agent trajectory was incomplete after repetitive tool-call batches led to timeouts and failure to complete the trading protocol, so its partial trades and collaboration events were excluded.Figure 12 describes rule-level event rates and final return versus hard-rule failure rate, with color encoding RAS and marker size encoding MDD.Source: §C.6 · pp. 46–47, Figure 12 · p. 47
• The role-selection sensitivity changed OCS values but preserved the model ordering, and varying the dominance threshold also preserved that ordering.The role change affected 176 of 669 trajectories; threshold changes shifted model-level OCS by at most 0.016.Source: §C.6 · pp. 47–48, Table 14 · p. 48

C.7 Cumulative Ranking Protocol

• Tool-use capability rankings had 0 reversals versus 21 return reversals; memory had 5 versus 19, rule following 8 versus 19, and collaboration 2 versus 29.The counts cover 180 comparable tool-use pairs, 280 memory pairs, 300 rule-following pairs, and 180 collaboration pairs.Source: §C.7 · p. 48

D Case Studies

• The case studies section presents representative traces and case studies to illustrate behaviors, strengths, and failure modes across evaluated agent architectures.Figure 13 compares cumulative returns and capability rankings over time, with rank 1 highest and annotations counting pairwise ranking reversals.Source: D Case Studies · p. 48, Figure 13 · p. 49

D.1 Tool-Augmented ReAct

• GPT-5.4’s Tool Agent finished at $10,387.46 versus $10,165.62 for Base ReAct, a $221.84 gap after entering NVDA earlier.The Tool Agent’s 62 calls were unique and valid, and routing quality was 0.958; Figure 14 aligns the account trajectories with NVDA’s price path.Source: §D.1 · pp. 48–49

Case 1: Memory-driven volume confirmation

• The Persistent Memory account ended $185.05 ahead of Base after holding cash during simultaneous low-volume readings and fear-zone conditions.It kept $9,330 idle across two rounds while Base carried long BTC, SOL, and BNB positions and adjusted exposure before the anticipated rebound failed.Source: §D.2 · p. 50

Case 2: Memory-driven position restraint

• The Persistent Memory account ended $221.93 ahead of Base after maintaining an approximately 20% BTC position near resistance while Base expanded its exposure.The retrieved rules advised controlling position size and waiting for a breakout or the FOMC reaction.Source: §D.2 · pp. 50–51

Case 3: Return suppression in a trending market

• +0.33% return for Qwen3-Max Persistent Memory versus +6.83% for Base illustrates how retrieved rules suppressed upside during a SOL advance.The memory account held ETH rather than joining the SOL rally, despite 15 memory_add calls and 11 distinct retained entries.Source: §D.2 · p. 51

Synthesis

• Persistent memory’s risk-control rules protected capital in range-bound or declining regimes but could cap upside in a one-directional advance.The authors distinguish Memory Content Quality, Memory Usage Quality, and downstream risk effects measured by changes in maximum drawdown and tail loss.Source: §D.2 · p. 51

Case 1: Silent compliance

• DeepSeek-V3.2 achieved RSS 1.0 with no hard-rule violation, yet its RAS was 0.1 because none of 30 turns cited a rule.Fourteen of its 163 evaluated rounds had no rule citation, and all fourteen had RAS below 0.5.Source: §D.3 · pp. 51–52

Case 2: Hallucinated violations and repair trades

• DeepSeek-V3.2’s RAS was 0.2 after it fabricated R1-02 violations and repair trades despite the checker recording zero R0/R1 violations and full R2 satisfaction.The phantom repair nevertheless ended mechanically compliant.Source: §D.3 · p. 52

Case 3: Boundary targeting and increment blindness

• Qwen3-Max and GPT-5.4 treated the 15% concentration ceiling as a target, failing to combine existing exposure, new orders, and price movement into a post-trade estimate.The traces identify R1-02 as the dominant hard-rule failure for these two models.Source: §D.3 · p. 52

Case 4: Auditable violation repair

• DeepSeek-V3.2 correctly identified a 9.92% cash reserve against the 10% minimum and 15.06% ETH concentration, then repaired both with one $60 trade.The auditor assigned RAS 1.0 to the quantified, state-grounded explanation.Source: §D.3 · p. 52

Synthesis

• Rule following requires both policy-consistent execution and state-grounded, auditable reasoning; strength in either dimension alone does not guarantee strength in the other.The paper summarizes these perspectives with HPR and RSS for execution, and RC, CH, and RAS for auditable reasoning.Source: §D.3 · p. 53

Case 1: Selective delegation with an effective decision

• GPT-5.4 stopped after one TradingAgent call and gained 3.63% over 24 hours, compared with 1.28% for paired ReAct.Only TradingAgent was invoked, yielding OCS 0.25; the case shows a focused delegation can resolve a localized information need.Source: §D.4 · p. 53

Case 2: Conflict-driven plan revision

• Gemini-3.1-Pro used conflicting specialist evidence to reject a BTC addition and reduce ETH exposure, with all four scored roles and evidence items reflected in the plan.The round obtained R_t=1, E_t=1, and OCS=1.Source: §D.4 · p. 53

Case 3: Surfaced risks without corresponding restraint

• Qwen3-Max referenced five evidence items and scored OCS=1, yet opened leveraged ETH and SOL positions without adopting specialists’ proposed risk reductions.The Multi-Agent account fell 0.08% over the next 24 hours while paired ReAct rose 0.64%.Source: §D.4 · p. 53

Synthesis

• The cases distinguish selective delegation, conflict-driven plan revision, and broad evidence integration that leaves an aggressive plan intact; observable collaboration need not move monotonically with realized outcome.OCS measures how roles and evidence enter the process, while OI records Multi-Agent performance relative to ReAct over the evaluation period.Source: §D.4 · p. 54