PaperlayerView exact revision on arXiv

Source-linked AI summary

E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation

Wei Fan, Xinjie Shen, Xudong Guo, Jianhong Tu, Yang Su, Yinger Zhang, Lianghao Deng, Fengyu Wang, Baohua Dong, Yangqiu Song, Dayiheng Liu

arXiv:2608.30730v1cs.LGcs.CL

TL;DR

Long-horizon agentic tasks require models to make sequential decisions, adapt from experience, and remain coherent across thousands of steps. E-Commerce Bench evaluates these abilities in a deterministic, real-data-derived, year-long retail operation with repeated negotiation and dynamic events. Across 18 models and seven dimensions, no single model dominates, with strong differences between earning, fraud avoidance, efficiency, and learning.

  • Problem

    Existing agentic benchmarks rarely test sustained competence across thousands of steps with sequential adaptation in evolving environments.

  • Method

    E-Commerce Bench runs agents through a deterministic 365-day retail business using real e-commerce-derived products and suppliers, repeated supplier negotiation, dynamic events, and 18 merchant tools.

  • Results

    No single model dominates across seven dimensions; GPT-5.6 Sol earns the most, while Qwen3.8-Max-Preview leads open-weight models and shows the strongest learning over the horizon.

  • Takeaways & Limitations

    Multi-dimensional evaluation reveals capability trade-offs that total assets alone would conceal, including differences in bargaining, fraud avoidance, operational efficiency, and experiential learning.

  • Takeaways & Limitations

    Negotiation metrics include only honest pairs, weight each reorder equally regardless of order size, and exclude episodes ending early.

Abstract

from arXiv · show

Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM agent concurrently runs multiple online stores, researching the market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.

1 Introduction

E-Commerce Bench targets long-horizon business operation by combining repeated negotiation, dynamic events, and concurrent store management in a realistic, deterministic environment. Its 18-model evaluation shows sharply divergent capability profiles rather than a single universally strongest model.

  • Long-horizon agentic tasks require sequential decisions, adaptation to past experience, and coherence across thousands of steps.
  • 6,886 products and 576 suppliers are calibrated from empirical platform logs, while promotions, natural disasters, and supply-chain shocks shift market demand.Baseline product costs and exact consumer demand remain hidden from the agent.
  • 18 models are evaluated on seven dimensions, including total assets, negotiation quality, fraud avoidance, solvency, operational efficiency, execution, and learning.The evaluation is designed to reveal weaknesses that a total-assets leaderboard alone could hide.
  • GPT-5.6 Sol grows capital roughly 14-fold but ranks 16th of 18 on fraud avoidance, while Claude Opus 4.7 leads bargaining and fraud avoidance from mid-pack on profit.Among ten open-weight entries, Qwen3.8-Max-Preview ranks first at 4.2 times the initial balance.
  • E-Commerce Bench combines deterministic supplier negotiation, customer sales modeling, dynamic events, and concurrent multi-store management over a year-long horizon.The benchmark is grounded in Taobao & Tmall data and supports up to four online stores concurrently.
  • The market is deterministic: fixed demand governs purchases and returns, and a negotiation kernel sets pricing, concessions, and accept/reject decisions while an LLM renders dialogue.

2 Related Work

Related benchmarks distinguish episodic tasks from continuing tasks and include business-operation simulations judged by accumulated balance. E-Commerce Bench is positioned within the continuing business-operation setting and adds a layered architecture for that environment.

  • Episodic benchmarks set a goal and reward multi-step execution that reaches a terminal state, whereas continuing benchmarks accumulate score without one.
  • Business-operation simulations evaluate an agent running a going concern by its accumulated balance.Vending-Bench established this benchmark line with one vending machine.
  • E-Commerce Bench uses an agent loop, tool layer, deterministic environment, and data layer to support long-horizon retail operation.The environment contains a sales-and-economy engine alongside a two-layer negotiation engine, operating over real e-commerce-derived data.

3 E-Commerce Bench

E-Commerce Bench evaluates autonomous merchant agents over a simulated year, combining store operations, inventory procurement, sales, fulfillment, cash flow, and adversarial supplier interactions. Its environment imposes long-horizon constraints through deterministic market dynamics, timed tools, deferred settlement, context eviction, persistent memory, and explicit scope limits.

  • Benchmark task: E-Commerce Bench gives an agent ¥100,000 and one simulated year to operate up to four online stores and maximize year-end assets.The agent researches markets, negotiates suppliers, buys inventory, prices goods, ships orders, handles returns, and withdraws settled revenue.
  • Agent loop: The turn loop batches ordered tool calls, advances a simulated clock, and triggers daily settlement when the clock reaches 08:00.Settlement posts fees, sales, and returns while appending news and notices to tool responses.
  • Operational constraints: Tool use consumes a 600-minute day budget, making information gathering and action planning competing operational choices.A balance check costs 10 minutes, opening a store costs 60 minutes, and only chatbox can create purchase orders through negotiate blocks.
  • Context management: Context eviction begins after 120,000 tokens, removes whole turns oldest-first toward a 60,000-token release target, and preserves the system message, first user turn, and two newest groups.Persistent memory remains outside eviction but has at most 20 entries and re-entered contents become evictable context.
  • Order execution: Fulfillment decisions trade freight cost against delivery speed and returns, while missed two-day shipment deadlines cancel orders and reduce reputation.Returns arrive days after shipping, extending the financial consequences of fulfillment choices.
  • Supplier negotiation: Supplier evaluation combines honest and fraudulent counterparts, with hidden labels and scripted scam overlays requiring both price bargaining and fraud avoidance.Of 576 suppliers, 152 are fraudulent; no single observation reveals fraud, while concession behavior differentiates the populations.
  • Evaluation scope: The benchmark’s negotiation metrics score the fraction of bargaining range captured, while its scope excludes fraudulent pairs, weights re-orders equally by count, and omits early-terminated episodes.The resulting metrics emphasize per-unit discipline rather than its direct cash consequence.

4 Experimental Results

Across 90 deterministic episodes, models diverged sharply in assets and capability profiles: the strongest earner was not consistently the best negotiator, fraud avoider, or operator. Results also exposed recurring weaknesses in solvency, efficiency, execution, learning, and error coordination.

  • Experimental setup: 90 episodes covered 18 models, with five independent runs per model under one fixed world and bankruptcy episodes included in the average.All models shared the same tools, context window, and memory setup; the longest episode used 3,599 of 4,000 allowed turns.
  • Year-end assets: A factor of 1,264 separated GPT-5.6 Sol at the top from Qwen3.5-Plus at the bottom, while Qwen3.8-Max-Preview led the open-weight tier and ranked fifth overall.The environment is deterministic given the action sequence, so the observed range reflects agent action sequences rather than resampled demand.
  • Negotiation quality: Claude Opus 4.7 retained the most bargaining range at 0.811, while every model remained above the 0.50 score associated with accepting the opening quote.The field settled at least 96% of concluded honest sessions, so negotiation ranking primarily reflected captured deal value rather than agreement frequency.
  • Fraud avoidance: Every model spent money on fraudulent suppliers, and the cleanest buyer finished mid-pack on assets while the profit leader ranked 16th of 18 on BadSpend%.Fraudulent spending varied by more than 160-fold, and defective lots accounted for 53.4% of everything recovered in the reported loss breakdown.
  • Operational efficiency: Fable5 produced ¥479 profit per tool call versus GPT-5.6 Sol’s ¥363, despite ranking second on total assets and using 59.9% fewer calls.Shipping, day advancement, withdrawals, and inventory publication consumed 71.8% of calls, whereas pricing revisions represented only 0.66%.
  • Operations execution: Controllable return rates ranged from −0.05 for Qwen3.7-Max to 5.88 for Qwen3.5-Plus, while GPT-5.6 Sol combined 4.46 points with the year’s largest refund bill of ¥1.62M.The metric measures pricing-added percentage points above the natural return rate of the selected goods.

5 Conclusion

E-Commerce Bench evaluates year-long autonomous retail operation through a deterministic environment, real-platform-derived economy, and persistent agent loop. Across 18 models, no system was strong on all seven measured capabilities, underscoring the gap between total assets and a complete capability profile.

  • Benchmark: E-Commerce Bench runs an agent-operated online retail business for a simulated 365-day year using an agent loop, persistent memory, 18 merchant tools, and a deterministic market.The environment covers research, supplier chat, pricing, shipping, and cash withdrawal, while purchases, returns, and negotiations are fixed by models and kernels.
  • Evaluation: 18 frontier and open-weight models each ran the year five times and were scored on end-of-year total assets plus six capabilities behind that number.The setup also specifies how tool results and notices are delivered within the context window.
  • Conclusion: No model was strong on all seven capabilities: GPT-5.6 Sol earned the most but ranked 16th of 18 on fraud avoidance and earned less per tool call than Fable5.The conclusion frames the benchmark as revealing uneven capability profiles rather than a single dominant system.
  • Information flow: The morning pass exposes at most three of 17 computed fields—date, day index, and news—and the 90 archived episodes carried 8,301 channel items.Most of those items were daily aggregated deliveries.

A.2 Loop Control and Context Editing

The benchmark controls long-horizon execution through token-aware group eviction while exposing a frozen, platform-grounded economy and calendar. Its seasonality, promotions, events, supplier mix, and hidden margin parameters create a deterministic but dynamically shifting operating environment.

  • Context accounting: Token counts include text, tool calls, and reasoning traces, while excluding encrypted reasoning artifacts and schemas outside the budget.The system prompt specifies a 120,000-token threshold, a 60,000-token release floor, and two protected groups.
  • Context editing: When usage exceeds the ceiling, the editor scans tool-call groups, protects the two newest uncleared groups, and clears older complete groups while preserving protocol validity.The system message and first user turn remain permanently protected, and eviction occurs only under the stated group conditions.
  • Context editing: The editor’s firing frequency depends on how each model uses the context window, with Figure 12 counting eviction passes across all 90 episodes.A model that never reaches the ceiling performs no eviction.
  • Released environment: The released economy contains 6,886 SKUs, 60 categories, 576 suppliers, and 12 store types, with the calendar fixing 10 market events and 8 promotions for every run.The world is frozen across models and episodes, while supplier identities include honest or fraudulent status, behavior templates, and scam attributes.
  • Seasonality: Seasonality differs mainly in amplitude: nine of 12 store types peak in November, Food & Beverage peaks in February, and four types stay within ±0.2 of neutral.Figure 14 displays monthly multipliers by store type and difficulty tier, including trough, peak, and peak-to-trough ratio.
  • Economy parameters: Agents observe store-type difficulty, operating cost, seasonal range, and category structure, but must discover achievable wholesale costs because wholesale ratios remain hidden.Reference price and natural return rate vary by SKU, while category rows define return bands and elasticity families.
  • Event and promotion calendar: Promotions are opt-in and announced up to seven days ahead, whereas events provide no lead time and post news only on their first day.Promotions can be joined up to 30 days ahead, allowing inventory positioning that is unavailable for unexpected events.
  • Event and promotion calendar: Promotion windows include demand multipliers from ×1.8 to ×3 and elasticity boosts from ×1.5 to ×2, according to the released calendar.The calendar lists each promotion’s dates, phase, maximum demand, and elasticity boost.

B.5 Properties a Reader Should Check

The released environment contains several inspectable data and rule properties that affect how merchants perceive returns, suppliers, costs, demand, reputation, and settlement. Its deterministic, ordered simulation also hides key quantities and introduces timing and inventory constraints.

  • Measurement-relevant properties: Two of the four checked file properties affect measurement, while two are minor and do not change measurements.The return-rate wording and supplier-email identifier can alter what an agent observes or exploits; the rent-column and unused shortage-cost discrepancies do not change measurements.
  • Measurement-relevant properties: 25.5% of all SKUs lie in categories whose numeric return bands reach 20% to 35% despite market_search wording them low or very low.A merchant pricing from the wording underestimates returns and can detect the discrepancy only through trace_return_sources on owned stock.
  • Measurement-relevant properties: Supplier identifiers can expose fraudulent counterparts because honest suppliers consistently have lower numeric suffixes than fraudulent suppliers within every category.This signal permits screening without sending a message, despite the returned fields otherwise lacking an honest-versus-fraudulent ranking.
  • Settlement and timing: The simulation advances thirteen ordered settlement steps at each 08:00 boundary, charging operating costs before revenue and testing solvency at the day’s low-water mark.Figure 16 maps each step’s reads and writes, while the daily trigger makes the order non-configurable.
  • Demand and reputation: Demand is shaped by price, promotions, seasonality, events, reputation, category and store capacities, with stockouts truncating demand without signaling the agent.The model uses a closed-form price factor, saturation terms, seeded rounding, and on-shelf inventory caps.
  • Returns and reputation: Returns are decided at shipment, combine natural, quality, price and shipping channels, and use pooled SKU inventory that makes mixed-lot quality effects untraceable.Reputation is rebuilt from shipped, returned, cancelled and sold-unit counters, with cancellation weighted more heavily than a return.

C.6 Unit Profit and Break-Even

Unit profitability depends strongly on procurement cost, return behavior, shipping and storage, while break-even volume converts fixed operating costs into a daily retained-unit requirement. Negotiation rules constrain how prices move and how long sessions run.

  • Break-even: Each retained unit funds 1/(1 −θj) shipping attempts and corresponding storage cycles because returned units re-enter stock.A storage cycle covers the warehouse waiting time before sale.
  • Break-even: Break-even volume n⋆ counts the retained units needed daily to cover the store tier’s operating cost.Figure 18a decomposes retail price into five shares that sum to one; Figure 18b and 18c show profit and required volume relationships.
  • Unit profit: Procurement is the largest cost line, and the negotiated closing price affects unit profit more than pricing choices do.The section compares a high-return fashion SKU with a thin-margin snacks SKU to illustrate the economic leverage of sourcing.
  • Unit profit: 49.2% natural returns leave the Women’s Fashion SKU earning ¥87.29 per retained unit at the floor price, requiring 2 units daily to cover operations.The example assumes standard shipping and Th = 14 days in a medium-tier store.
  • Unit profit: Closing at the floor rather than accepting the opening quote multiplies unit profit by 2.4 for Women’s Fashion and 2.8 for Snacks & Nuts.The Snacks & Nuts comparison requires 13 retained units per day to cover its hard-tier operating cost.
  • Negotiation: Counter-offers move toward the supplier floor, with clipping enforcing a monotone non-increasing quote path that never falls below the floor.The kernel accepts, walks away, or counters in that order; an honest supplier is almost never the party causing impasse when offers exceed the floor.
  • Negotiation: 66.0% of concluded sessions close in 3 message turns, and no kernel session passes round 7.The deadline scale shapes decision terms but does not truncate sessions while both parties continue answering.

D.2 Data Grounding and Calibration

The negotiation economy grounds supplier behavior in fixed SKU and category ratios, with honest and fraudulent floors derived deterministically from cost, wholesale and scam-cap parameters. Fraud timing changes whether deception appears in price or fulfillment, while session records and memory asymmetry shape evaluation.

  • Calibration: The data layer fixes each SKU’s cost floor, wholesale quote and scam cap through rounded category ratios.Honest suppliers use the category cost floor, while fraudulent suppliers can raise the reservation price through the scam rule.
  • Calibration: 1.218 is the average overpayment multiple, while accepting the opening quote scores exactly 0.5 against the released baseline before rounding.The baseline follows wrj = round(0.5 + 0.5 cfrj, 2) across all 60 categories.
  • Calibration: Urgency and stance alter concession and acceptance behavior without moving the first quote because opening harshness is solved to match the nominal wholesale price.The first-quote calibration uses pmax = 1.5 wj and cancels harshness from the relevant product.
  • Templates: The six honest and five scam templates use lower urgency for scams, making them slower to accept and stingier with counters.Cue policies differ across templates: some render state-dependent posture and sentiment, while others collapse to fixed cues.
  • Templates: The supplier list fixes template assignment, but the per-supplier urgency field never reaches the kernel, so suppliers sharing a template bargain identically.Mean turns barely vary across honest templates, indicating that harder templates do not induce extra rounds.
  • Session protocol: Every successful order appends both sides of the exchange to the supplier’s deal log, whereas the agent retains only information surviving context eviction or persistent memory.The anchoring metrics evaluate this asymmetry across repeated contacts.
  • Session protocol: 192 supplier closures across 90 archived episodes limit repeated sourcing after suppliers reach private bankruptcy thresholds of 10 to 20 successful orders.Concentrating purchases on favorable counterparts can therefore lose those suppliers later in the year.

D.5 Fraud Catalog

The fraud catalog separates scams by when damage occurs: pre-deal scams raise the reservation price, whereas post-deal scams defect during fulfillment. Price-only observation therefore cannot reliably reveal every fraud class, and one scam is too rare for stable measurement.

  • Fraud timing: Pre-deal scams monetize negotiation through elevated reservation prices, while post-deal scams keep honest pricing and defect by short-shipping or delivering defective stock.The five scams comprise 152 of 576 suppliers and are split according to when the damage lands.
  • Fraud timing: 0.79% to 1.66% of reference price per counter-offer characterizes pre-deal scam concessions, versus 3.94% to 9.93% for honest templates.The two post-deal scams concede 4.37% and 4.44%, inside the honest band.
  • Detectability: Price-only observation reveals overpayment from pre-deal scams but not fulfillment defection from post-deal scams.The two fraud classes therefore separate in bargaining behavior, not as five individually distinguishable scams.
  • Measurement boundary: The vip_fee pay-first scam is too rare to measure because its failed purchase order triggers rollback, leaving only three payments totaling ¥3,000 across two models and 90 episodes.Table 8 consequently reports single draws for this scam.

E Metric Definitions

The evaluation defines year-end assets as the primary score and derives dimension metrics from completed, settled bargaining sessions. Metrics are averaged across five episodes, with honest-counterpart filtering for value-extraction measures.

  • Primary score: The primary score sums bank balance, platform wallet, and unsettled escrow after year-end finalization.Escrow is drained into the wallet after cancellations and seeded returns are processed.
  • Primary score: The asset multiplier is end-of-year total assets divided by the ¥100,000 opening stake.A = b + w + e and M = A/A0.
  • Session records: Each bargaining cycle over a supplier–SKU pair produces one session record used by negotiation, fraud, and learning metrics.Sessions close with either agreement or disagreement.
  • Admission rules: Only terminal outcomes enter aggregation, only settled agreements remain, and value-extraction metrics restrict sessions to honest suppliers.Open records and agreements whose purchase orders failed are excluded.
  • Aggregation: Each statistic scores one episode, and each model’s reported value is the unweighted mean over five episodes.Units sold and orders sold accompany the asset score as volume measures.

E.1 Negotiation Quality

Negotiation quality is measured from completed session records using surplus, agreement, oracle-normalized, and deal-count statistics, while fraud metrics track cleared spending. The measures expose a trade-off between extracting surplus and preserving agreements, with important filtering and recording caveats.

  • Session admission: 13,223 sessions pass the terminal and settlement rules, including 12,060 honest and 1,163 fraudulent sessions.The rules discard 1,610 records left open when episodes stopped.
  • Surplus efficiency: SE+ = AGR+ · CSE+, but agreement rates are saturated because every model settles at least 96% of concluded honest sessions.CSE+ therefore carries the ranking among the evaluated models, while AGR+ guards against selective dealing.
  • Surplus efficiency: The strongest surplus-per-deal agents also occasionally let sessions collapse, creating a trade-off between surplus extraction and agreement.Surplus is unclamped, so agreements above the reference price contribute negative terms.
  • Surplus efficiency: CSE+ weights sessions by bargaining-range width, whereas its companion average uses unweighted per-session ratios.The two statistics therefore order sessions differently by construction.
  • Deal counts: Deals closed totaled 13,128 across 90 episodes, while rounds to deal counts exchanges plus the closing accept and includes fraudulent and honest counterparts.These counts are diagnostics rather than scored rates.
  • Fraud avoidance: Fraud avoidance measures money that left the bank account, not offers refused, across five scam types.Fraudulent-spend accounting includes cleared order charges and membership fees.
  • Fraud avoidance: Refusals are not recorded, so an uncompleted approach cannot distinguish deliberate screening from simply stopping communication.The resulting ρ measure is reported as pass-through rather than screening.

E.5 Operations Execution

Operations execution measures controllable return expectations, fulfillment reliability, shipping choices, and store activity. Its interpretation is bounded by dispatch-time accrual, cancellation handling, and rate-normalization conventions.

  • Return control: The execution axis ranks controllable return-rate points added by the model’s pricing, with lower values preferred.Diagnostics include realized returns, uncontrollable return channels, return costs, on-time share, and dispatch cancellations.
  • Return channels: All four return channels are booked when parcels are dispatched, carrying earlier layers under the shipment’s tier factor and ceiling.This aligns the expected-return accounting with the tier selected at fulfillment.
  • Return channels: The pricing channel becomes the controllable return-rate component because the tier channel is zero below the 0.95 ceiling in 89 of 90 episodes.The tier channel records −0.0001 in the remaining episode.
  • Fulfillment: Orders must be dispatched within two days or they are canceled, returning units to inventory and forfeiting revenue.On-time share is one minus the canceled share of orders sold.
  • Shipping mix: Fast shipping doubles freight cost, standard is the default, and slow shipping halves freight cost.The three dispatch tallies count shipments rather than units.
  • Store activity: Opening a store costs ¥500, while reopening a type resets its reputation to the floor instead of inheriting the closed store’s goodwill.Opened and Reopens count activity against four concurrent slots and one store per type.
  • Measurement limits: Return rates are divided by shipped units floored at one, so episodes shipping nothing record zero rather than no value.Twenty-six of 90 episodes have negative pricing contributions because prices fall below reference.

E.6 Learning over the Horizon

E.6 measures whether repeated supplier–SKU purchases improve on earlier prices and whether performance changes across the year. The analysis uses filtered repeat pairs, permutation nulls, and time-ordered half-year comparisons, with explicit limits on interpretation.

  • Sequential price discipline: AnchorRatio measures repeat-order overpayment against the cheapest price previously paid for the same supplier–SKU pair.The anchoring block retains honest pairs transacted at least twice and scores later orders after the first.
  • Permutation analysis: The permutation null preserves each pair’s positions while reshuffling their order to assess whether observed repeat-order behavior exceeds chance ordering.Regret subtracts the observed statistic from the null because lower regret is better, while new-low statistics reverse that direction.
  • Sequential price discipline: 2,230 repeat supplier–SKU pairs remained after filtering fraudulent, unresolved, non-degenerate, and single-transaction pairs.Episodes without qualifying repeat pairs contribute no anchoring value to the model mean.
  • Half-year comparisons: Half-year comparisons split honest sessions by conclusion day, requiring at least four sessions and assigning an odd extra session to the later half.The fraud half-lift instead uses all concluded sessions and its own median day.
  • Interpretation: Anchoring means can exclude episodes without repeat pairs, while truncated trends and unscaled half-lifts limit cross-model interpretation.GPT-5.5 contributes three rather than five episodes, and one bankrupt run can determine a model’s mean slope.
  • Interpretation: AnchorRatio distinguishes more model pairs than the surplus half-lift, separating 53 of 153 pairs by over two standard errors versus 15.The half-lift is therefore reported beside the ranking axis as a diagnostic rather than used to rank models.

F.2 Negotiation Quality

Negotiation quality is evaluated alongside supplier difficulty, fraud exposure, operational behavior, and broader capability profiles. Model policy differences generally exceed counterpart-template effects, while asset leadership does not imply uniformly strong performance.

  • Negotiation quality: Counterpart-template variation is 0.051 field-wide, below the 0.093 median within-model span and the 0.215 best-to-worst per-model CSE+ gap.This comparison indicates that model policy differences are larger than the easiest-to-hardest preset difference in aggregate.
  • Fraud avoidance: 22.0% is the median model’s share of contacted fraudulent suppliers that became suppliers it ordered from.Figure 26 separates fraudulent outreach from the subsequent ordering stage.
  • Supplier outreach: 14.3% versus 2.4% bounds the widest and narrowest honest-supplier search as a share of 424 honest names.Screening therefore operates over a narrow, model-selected slice of the supplier population.
  • Operations execution: Only GPT-5.6 Sol, GLM 5.2 (high), and Qwen3.8-Max-Preview average more than 3.5 stores trading per day.Most models still open stores cumulatively beyond the four-store cap because new openings replace existing stores.
  • Capability profile: 10.2 of 18 places is the average span between a model’s best and worst ranks across seven axes; no model reaches the top five on all seven.Operations execution shifts models furthest from asset rank, averaging 4.2 places, while operational efficiency shifts it 0.7.
  • Fraud avoidance: GPT-5.5 declined every membership-fee pitch, whereas almost no model paid one.The advertised member price did not affect the supplier’s charge in the described scam transcript.

G.3 Defective Lot

The defective-lot case shows why delayed return signals can make apparently clean reorder decisions unsafe. The supplier’s deliveries were defective, but the conversation and initial quotes did not reveal the problem.

  • Defective lot: Every unit from the defective-lot supplier was flagged defective, raising the SKU’s return rate to the defective blend.Returns arrived days after shipment, so a reorder could precede the first visible evidence of trouble.
  • Defective lot: ¥740.00 was accepted for the reorder after the agent observed zero returns on twelve delivered units.Four sold units were still within the three-to-seven-day return lag, and half the lot later came back.
  • Defective lot: 35 orders made defective stock the episode’s most costly scam over the year.The case transcript follows one reorder made while the preceding lot still appeared clean.

G.4 One Collapse, Day by Day

The traced collapse combines aggressive early procurement, fraudulent sourcing, insufficient sales, and a cash reserve too small to survive the resulting negative balance streak.

  • Initial procurement: Three fifths of the ¥100,000 stake became inventory after four stores opened on day one.The episode bought stock intended for a full year before sales materialized.
  • Sales and procurement: 586 orders cleared, selling 1,428 units in 130 days against inventory bought for a full year.Twelve of the episode’s 24 orders went to fraudulent suppliers at the raised pre-deal floor.
  • Cash-flow collapse: ¥1,597.98 was the wallet’s maximum balance before the ten-day negative-close bankruptcy counter ran out.Liquidating three stores one day before expiry recovered too little, and withdrawal could not have saved the episode.

G.5 Detection Rules Behind the Incidence Matrix

The incidence matrix evaluates ten failure modes using one mechanical test per episode, with explicit rules defining what each flag does and does not establish. These rules distinguish realized failures from fragility and expose measurement boundaries in fraud, bargaining, and behavior metrics.

  • Detection framework: Ten failure modes are scored by one mechanical test evaluated once per episode.The rules are summarized with pooled counts and caveats for interpreting the incidence matrix.
  • Financial failure rules: Bankruptcy records realized ruin, not fragility, because episodes surviving until one day before the limit are not flagged.
  • Financial failure rules: Wallet overdraft flags an avoidable close under a counterfactual that ignores escrow timing, not a guaranteed rescue.
  • Fraud rules: Fraud flags cover both pre-deal and post-deal fraudulent suppliers, but one flag cannot distinguish undelivered-unit losses from refund losses.The pre-deal rule records money handed over, while the post-deal rule combines different loss locations.
  • Fraud rules: Reordering from a fraudulent supplier detects concentration rather than stubbornness because orders before and after return or delivery signals count identically.
  • Behavioral rules: Accepting an unimproved opening quote is partly definitional: 636 of 11,987 honest agreements began near the supplier’s own floor.
  • Behavioral rules: Verbatim catalog-query repetition cannot distinguish deliberate refreshing after supplier bankruptcy from eviction or inattention.
Loading 2608.30730v1…