PaperlayerView exact revision on arXiv

Source-linked AI summary

Reinforcement Learning for Long-Horizon Interactive LLM Agents

Kevin Chen, Marco Cusumano-Towner, Brody Huval, Aleksei Petrenko, Jackson Hamburger, Vladlen Koltun, Philipp Krähenbühl

arXiv:2502.01600v3cs.LGcs.AI

TL;DR

Long-horizon interactive LLM agents remain limited on sophisticated multi-app tasks. This paper trains them with LOOP, an efficient reinforcement-learning method, and reports that a 32-billion-parameter agent outperforms OpenAI o1 by 9 percentage points in AppWorld.

  • Problem

    Long-horizon IDA tasks require multi-app action chains, replanning, and extensive context management, while prior models succeed on fewer than half of sophisticated AppWorld tasks.

    Source: §1 · p. 1

  • Method

    The paper formulates IDA interaction as a POMDP and introduces LOOP, a PPO-based method with leave-one-out baselines and per-token clipping.

    Source: §1 · pp. 1–2, §4.1 · p. 4

  • Results

    LOOP achieves state-of-the-art performance on both AppWorld test splits and outperforms the much larger OpenAI o1 agent by 9 percentage points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Reinforcement learning substantially improves IDAs after training on only a small number of scenarios and elicits behaviors such as consulting API documentation and recovering from setbacks.

    Source: §1 · pp. 1–2, §6 · p. 8

    Even the best agents succeed on only about 7 out of 10 AppWorld tasks, and AppWorld lacks several features of everyday environments.

    Source: §6 · p. 8

Abstract

from arXiv · show

Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests. While IDAs powered by instruction-tuned large language models (LLMs) can react to feedback from interface invocations in multi-step exchanges, they have not been trained in their respective digital environments. Prior methods accomplish less than half of tasks in sophisticated benchmarks such as AppWorld. We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments. We formalize this training as a partially observable Markov decision process and derive LOOP, a data- and memory-efficient variant of proximal policy optimization. LOOP uses no value network and maintains exactly one copy of the underlying LLM in memory, making its implementation straightforward and as memory-efficient as fine-tuning a single LLM. A 32-billion-parameter agent trained with LOOP in the AppWorld environment outperforms the much larger OpenAI o1 agent by 9 percentage points (15% relative). To our knowledge, this is the first reported application of RL to IDAs that interact with a stateful, multi-domain, multi-app environment via direct API calls. Our analysis sheds light on the effectiveness of RL in this area, showing that the agent learns to consult the API documentation, avoid unwarranted assumptions, minimize confabulation, and recover from setbacks.

1. Introduction

The paper addresses long-horizon interactive digital-agent tasks, where success requires multi-step information gathering, state changes, replanning, and careful context management. It presents LOOP, a memory- and data-efficient RL method that achieves state-of-the-art AppWorld performance and induces more reliable agent behaviors.

Problem setting

  • AppWorld tasks can require up to 40 REPL interactions and 32K tokens while spanning up to 30M environment-state tokens, challenging existing agents.Best open-weight models achieve below 40% success, while top reasoning models succeed barely above half the time.Source: §1 · p. 1

Contributions and results

  • RL trains long-horizon interactive agents from a simple task-completion reward on only 24 training scenarios, without ground-truth action sequences or large scenario datasets.The learned behaviors generalize to diverse held-out tasks resembling the motivating multi-app task.Source: §1 · p. 1

Method

  • LOOP combines PPO with leave-one-out baselines and per-token clipping to reuse off-policy samples, store one backbone LLM, and avoid a value network.These choices make LOOP sample-efficient, memory-efficient, and implementation-efficient.Source: §1 · pp. 1–2

Contributions and results

  • LOOP achieves state-of-the-art performance on both AppWorld test splits, improving over open-weight SOTA by 47 pp, GPT-4o by 23 pp, and OpenAI o1 by 9 pp.The 32-billion-parameter agent also outperforms all evaluated fine-tuned and non-fine-tuned models, including the much larger OpenAI o1 agent.Source: §1 · p. 2

Analysis

  • Training leads agents to avoid unnecessary batching and unwarranted assumptions, consult API documentation before invoking functions, avoid placeholder confabulation, and recover more often after failed API calls.Unnecessary batching decreases by ∼6x, API documentation queries increase by ∼60%, assumption-related words decrease by ∼30x, “dummy” decreases by ∼6x, and capitulations decrease by ∼3x.Source: §1 · p. 2

2. Related Work

Prior work spans API-based LLM agents, supervised trajectory learning, and reinforcement learning for language models and interactive environments. LOOP extends leave-one-out policy optimization to off-policy updates and targets the substantially more complex AppWorld setting, where it outperforms prior baselines.

LLM agents

  • LLM agents have used structured API calls, reasoning-acting-reflection workflows, code interpreters, and supervised fine-tuning on agent trajectories to interact with software environments.These approaches cover information retrieval, stateful environments, and trajectory datasets.Source: §2 · p. 2

Reinforcement learning for LLMs

  • LOOP generalizes RLOO by permitting off-policy policy updates within PPO’s trust region, enabling rollout reuse and looser coordination between collection and updates.RLOO avoids reference and critic LLMs by using on-policy updates and multiple rollouts for sampling-based advantage estimates.Source: §2 · p. 2
  • For reasoning and static code generation, RL methods include GRPO’s sample-based baselines, VinePPO’s branched rollouts, and filtered-rollout supervised-training alternatives.The paper compares against these methods and reports that combining PPO with a leave-one-out estimate performs significantly better for training IDAs.Source: §2 · p. 2

Reinforcement learning for LLM agents

  • LOOP outperforms REINFORCE- and DPO-based baselines in AppWorld, which is substantially more complex than WebShop.WebShop has eight actions and at most one parameter per turn, whereas AppWorld spans nine apps, 457 API endpoints, and up to 17 parameters with nontrivial logic.Source: §2 · pp. 2–3
  • RL has trained language-model agents in text games, web shopping and navigation, mobile control, and embodied environments, including several WebShop-specific approaches.Related WebShop methods include REINFORCE with a learned value baseline, ArCHer’s hierarchical off-policy/on-policy training, and AgentQ’s tree search with DPO.Source: §2 · p. 2

3. Preliminaries

This section formalizes autoregressive language generation as sequential token sampling and connects it to reinforcement-learning optimization. It introduces REINFORCE, RLOO, PPO, and the leave-one-out PPO variant LOOP as foundations for subsequent training.

Autoregressive language models

  • Autoregressive language models generate sequences token by token from next-token distributions conditioned on the context and preceding tokens, terminating at a stop token ⊥.Conditional generation samples x from pθ(x1:n|c).Source: §3 · p. 3

Language modeling as a Markov decision process

  • Language generation can be modeled as an MDP whose states append generated tokens, while terminal rewards and deterministic transitions reduce it to a contextual bandit over whole generations.The bandit formulation treats an entire generation as one action.Source: §3 · p. 3

REINFORCE and RLOO

  • RLOO estimates advantages with K independent samples by subtracting the mean return of the other K−1 samples, producing an unbiased, low-variance estimate but requiring on-policy rollouts.On-policy sampling must come from the current policy, limiting reuse of expensive rollouts across gradient updates.Source: §3 · p. 3

Proximal policy optimization

  • PPO permits controlled off-policy drift through trust-region clipping and per-token importance weights, motivating LOOP as PPO with a leave-one-out estimator instead of a value network.Value networks can be slow, memory intensive, and inaccurate for language models.Source: §3 · p. 3

4. LOOP

LOOP formulates interactive digital-agent tasks as POMDPs and trains the agent with a leave-one-out variant of PPO. The method supports multiple importance-weight granularities and recovers RLOO and GRPO under specific settings.

4.2. Leave-One-Out Proximal Policy Optimization

  • LOOP supports per-token, per-trajectory, or per-turn importance weights within PPO, as illustrated in Figure 2.These alternatives determine how policy updates weight agent-generated tokens, complete trajectories, or interaction turns.Source: Figure 2 · p. 4, §4.2 · p. 4

4.1. POMDPs for interactive digital agents

  • Interactive digital-agent tasks are modeled as POMDPs whose state combines the initial environment, task context, generated tokens, and tokenized responses triggered by code execution.The environment is stateful, comprising both an AppWorld relational database and the REPL state, while the LLM observes generated text and environment responses across turns.Source: §4.1 · p. 4

4.2. Leave-One-Out Proximal Policy Optimization

  • LOOP trains agents by collecting K rollouts per task, estimating leave-one-out advantages, and applying PPO updates over shuffled mini-batches for N_epoch epochs.The procedure has separate rollout-collection and policy-update phases and uses the PPO objective for policy optimization.Source: §4.2 · p. 4
  • LOOP reduces to RLOO when run purely on-policy, while its relation to GRPO differs through return normalization by standard deviation, which experiments found beneficial to omit.RLOO corresponds to N_epoch = 1 with no mini-batches; GRPO’s normalization disproportionately favors low-standard-deviation trajectories.Source: §4 · p. 4, §4.2 · p. 5

5. Evaluation

LOOP achieves the strongest AppWorld performance, outperforming competing methods and larger baselines across normal and challenge tests. Reinforcement learning improves behavior through documentation use, fewer assumptions and placeholders, error recovery, and diverse rollout strategies.

5.1. AppWorld

  • AppWorld evaluates stateful API interaction across nine simulated consumer apps using task and scenario goal completion, with challenge tasks requiring complex sequences and unseen apps.The benchmark contains 250 scenarios, 750 tasks, and separate normal and challenge test sets.Source: §5.1 · p. 5

5.2. Implementation

  • Training uses Qwen2.5-32B-Instruct with LoRA, 6 rollouts per task, up to 40 training interactions, and up to 50 evaluation interactions.The reward is the fraction of unit tests passed for each task.Source: §5.2 · p. 5

5.3. Evaluated Methods

  • The evaluation compares zero-shot open- and closed-source models with supervised fine-tuning, preference optimization, PPO, RLOO, GRPO, and LOOP variants.Fine-tuned methods use their highest-performing dev-set checkpoint; LOOP variants apply importance weighting to trajectories, turns, or tokens.Source: §5.3 · p. 5

5.4. Results

  • 71.3 TGC on Test-N and 45.7 TGC on Test-C make LOOP (token) outperform all other methods.It improves over Qwen2.5-32B-Instruct by 81% on Test-N and 117% on Test-C, and over OpenAI o1 by 15% and 24%, respectively.Source: Table 1 · p. 6, §5.4 · p. 6
  • Token actions outperform turn and trajectory actions because the latter produce unstable training and frequently clipped importance weights.All Monte Carlo-baseline RL methods outperform PPO with a learned value function, while reward normalization causes a 9 pp drop for LOOP (token).Source: §5.4 · pp. 6–7

5.5. What does the policy learn?

  • RL reduces open-loop control, increases API-documentation lookup by ∼1.6x, and reduces assumptions, placeholders, and giving up after errors.Multiple code cells per turn decrease by ∼6x, ‘assuming’ words by ∼30x, ‘dummy’ by ∼6x, and giving-up frequency by ∼3x.Source: §5.5 · pp. 7–8
  • LOOP-trained agents complete tasks through diverse API-call sequences rather than a single behavior mode, with 98 of 100 rollouts succeeding and 94 using unique sequences.The four observed strategies include direct contact search, social-feed browsing, relationship queries, and combinations of these approaches.Source: §5.5 · p. 7, Figure 4 · p. 8

5.6. Why does RL on small data work?

  • Across 24 scenarios and 72 tasks, all reinforcement-learning-like methods work much better than supervised fine-tuning methods or prompt engineering, despite the small training-data amount.The model maintains multiple distinct solution “phenotypes”; rollout variety fosters exploration early and prevents collapse onto a single solution late, fostering generalization.Source: §5.6 · p. 8

6. Discussion · A. Emergent behavior after training

The discussion concludes that LOOP-based reinforcement learning meaningfully improves interactive digital agents after training on few scenarios, while current agents remain insufficiently robust for broad autonomy. AppWorld demonstrates RL’s promise but omits several features of everyday environments.

6. Discussion

  • LOOP substantially improves interactive digital-agent performance after reinforcement-learning training.Source: §6 · p. 8

A. Emergent behavior after training

  • Training on only a small number of scenarios can produce meaningfully better interactive digital agents.Source: §6 · p. 8

6. Discussion

  • Even the best agents succeed on ∼7 out of 10 AppWorld tasks, below the robustness required for broader autonomy.Source: §6 · p. 8
  • This performance may be acceptable when agents receive close human supervision.Source: §6 · p. 8
  • AppWorld remains the literature’s most advanced multi-turn interactive-agent benchmark, but it lacks non-determinism, transient failures, and unsolvable or ambiguous tasks.Source: §6 · p. 8
  • AppWorld also lacks adversarial scenarios, user clarification and confirmation steps, and interactive counterparties such as customer service representatives.Source: §6 · p. 8

A.1. Quantitative comparison on training tasks

On the 72 reinforcement-learning training tasks, fine-tuning reproduced the emergent behaviors seen on development tasks: greater interactivity, persistence, documentation checking, and fewer assumptions or dummy variables.

A.1. Quantitative comparison on training tasks

  • Fine-tuning made the agent more interactive, persistent, careful, and serious on the training tasks, with fewer multi-cell turns, lower give-up rates, more documentation checks, and fewer assumptions or dummy variables.These changes were aggregated across all 72 training tasks using three independent rollouts per task.Source: Figure 5 · p. 11, §A.1 · p. 11
  • The reported metrics were computed from agent turns and environment responses, including extracted Python code blocks and counted occurrences of relevant strings or API-documentation calls.Number of turns counts agent submissions followed by environment responses, while code-cell and text-frequency measures are normalized by analyzed rollouts or turns as specified.Source: §A.1 · pp. 11–12
  • The analysis quantified behavior changes using normalized measures of multi-cell turns, execution errors, failed-API give-up rates, documentation calls, and assumption or dummy-variable mentions.The give-up rate compares unrecovered with overall failed API calls and is not normalized by turns.Source: §A.1 · pp. 11–12

A.2. Qualitative comparison (early vs. late in training) via summarized rollouts · B. PPO in a POMDP

Comparing rollouts early and late in LOOP training shows that the policy reduces poor assumptions, dummy values, and erroneous API calls while learning to consult documentation. The method section then formulates the equivalent PPO objective for the POMDP.

A.2. Qualitative comparison (early vs. late in training) via summarized rollouts

  • After training, the agent significantly reduces poor assumptions, dummy values, and erroneous API calls across multiple independent rollouts.Early rollouts frequently exhibit all three behaviors, whereas late rollouts correct them consistently.Source: §A.2 · p. 12
  • The comparison uses three i.i.d. rollouts early in training and three i.i.d. rollouts late in LOOP training.The analyzed task is shown near step 38 early and step 530 late.Source: §A.2 · pp. 12–13, Figure 6 · p. 13
  • Early-training behavior is characterized by poor assumptions, dummy values, and erroneous API calls.These behaviors occur frequently without extensive training.Source: §A.2 · p. 12
  • Late in training, the policy consistently looks up API documentation before making API calls and avoids specified anti-patterns.The rollouts contain API calls within Python code blocks that may include loops, branching, and filtering logic.Source: Figure 6 · p. 13
  • Figure 6 summarizes the same task across early and late checkpoints, with full first episodes provided in the corresponding appendices.The early condition is step 38, and the late condition is step 530 near the model’s best checkpoint.Source: Figure 6 · p. 13

B. PPO in a POMDP

  • Following the POMDP definition and objective, the paper derives an equivalent PPO objective for the earlier formulation.This derivation is stated with reference to Eq. 6, Eq. 7, and Eq. 4.Source: B. PPO in a POMDP · p. 14

C. Evaluated Methods: Details … C.5. PPO with a Learned Critic

The paper evaluates supervised fine-tuning, rejection sampling, expert iteration, DPO-MCTS, and PPO variants for AppWorld agents, finding that RL methods generally outperform simpler fine-tuning approaches while PPO with a learned critic underperforms Monte Carlo-based methods. These methods differ in data construction, off-policy handling, search, memory use, and advantage estimation.

C.1. Ground Truth Supervised Fine-Tuning (SFT-GT)

  • Ground-truth supervised fine-tuning recovered near-perfect train performance but failed to meaningfully improve dev and test performance, consistent with memorization replacing environment interaction.The dataset contains one ReAct-style example per train task derived from successful programmatic solutions, some of which require unavailable environment state.Source: §C.1 · p. 14

C.2. Rejection-sampling Fine-Tuning (RFT)

  • Multi-iteration RFT solved 88 rather than 87 training tasks during rejection sampling, but further fine-tuning did not significantly improve model performance.The first RFT model was selected at epoch 30 based on the highest mean dev performance before later overfitting.Source: §C.2 · pp. 14–15
  • One-pass RFT brought a 32B model to approximately GPT-4o’s previous state-of-the-art performance, although RL methods retained a meaningful performance advantage.Rejection sampling produced 1,613 successful rollouts covering 87 of 90 training tasks, or 96.7%.Source: §C.2 · pp. 14–15

C.3. Expert Iteration (EI)

  • Expert Iteration remained stable and matched certain RL variants such as GRPO, including under stale asynchronous experience, but overfit training data more than RL approaches.Its training retains only rollouts with return 1.0 and replaces policy-gradient loss with cross-entropy fine-tuning.Source: §C.3 · p. 15

C.4. DPO-MCTS

  • DPO-MCTS collects action-preference pairs through UCB-guided search and stores them in an offline replay buffer because AppWorld’s long horizons make online rollouts expensive.The simplified Agent Q-style implementation runs up to 30 iterations without a separate LLM critic heuristic.Source: §C.4 · p. 15

C.5. PPO with a Learned Critic

  • PPO replaces LOOP’s rollout-wise Monte Carlo baseline estimates with per-token value estimates from a learned critic and uses Generalized Advantage Estimation (GAE) to compute per-token advantages.The implementation uses non-discounted returns (γ = 1.0); runs with λGAE ∈ {0.95, 0.99, 0.999} quickly diverged, whereas λGAE = 1.0 was most stable.Source: §C.5 · p. 15
  • PPO with a learned critic underperformed Monte Carlo-based methods because biased advantages, instability, and hyperparameter sensitivity outweighed its potential token-level credit-assignment benefit.The critic enables token-level advantages, but value estimation remained difficult and training was sensitive to GAE settings.Source: §C.5 · p. 16

D. Additional training details · D.1. Training Tasks

Training uses Qwen2.5-32B-Instruct with LoRA and unit-test-based rewards, while LOOP trains on 24 AppWorld scenarios excluding difficulty-3 tasks. Each iteration collects 240 rollouts, and episodes are capped at 40 interactions during training and 50 during evaluation.

D. Additional training details

  • The agent uses Qwen2.5-32B-Instruct with LoRA applied to attention and MLP modules, while Cut Cross-Entropy reduces training memory use.LoRA uses rank r = 16 and α = 32; agents share a ReAct prompt with one successful-task example and face 1500 output-token and 3K API-response limits.Source: D. Additional training details · p. 16
  • Unit-test rewards equal the fraction of successfully passed tests while penalizing undesired state changes.Source: D. Additional training details · p. 16
  • Training runs on two NVIDIA H100 8-GPU nodes, separating rollout generation from learning and recomputing generating-policy log-probabilities for PPO gradients.Source: D. Additional training details · p. 16
  • LOOP uses a constant learning rate of 5 × 10−5, clips gradient norms at 1, and removes rollouts with |Â_(i,j)| < 0.01 before gradient computation.Source: D. Additional training details · p. 16
  • Each iteration generates 240 rollouts from 40 randomly sampled tasks using K = 6 rollouts per task at temperature 1.0.Rollout collection uses at least 4 rollouts per task and stops after 90% of the target total; only difficulty-1 and difficulty-2 tasks are trained, because difficulty-3 inclusion hurt performance.Source: D. Additional training details · p. 16
  • Episodes allow up to 40 agent-environment interactions during training and 50 during evaluation, with unfinished episodes counted as failures.Source: D. Additional training details · p. 16

D.1. Training Tasks

  • The training subset excludes difficulty-3 AppWorld tasks and contains 24 scenarios with three minor task variations per scenario.The complete training scenario set is enumerated in the paper.Source: §D.1 · p. 17
  • The appendix lists the complete training set through 24 scenario identifiers, each corresponding to the selected task scenarios.Source: §D.1 · p. 17

E. Evaluation over multiple training runs for each method … G. Full Rollouts (LOOP)

Across three training runs, LOOP’s relative performance remained largely consistent despite lower aggregate scores and instability in the bandit variant. Rollout examples illustrate both successful API-guided execution and failure from overly broad action selection, followed by full trajectory printouts.

E. Evaluation over multiple training runs for each method

  • Across three training runs, performance was lower than the best-run results but relative algorithm performance remained largely consistent, while LOOP (bandit) showed high variance from unstable training and clipped importance weights.Each run used its best dev-set checkpoint evaluated five times, totaling 15 evaluation runs per algorithm; LOOP (bandit) could diverge before reaching peak performance.Source: E. Evaluation over multiple training… · p. 17

F. Additional rollout examples

F.1. Example rollout: success

  • The successful rollout followed a planned sequence of documentation lookups, authentication, contact and message searches, payment creation, and confirmation messaging.Source: §F.1 · p. 18

F.2. Example rollout: failure

  • In a difficulty-1 dev task, LOOP passed 4 of 6 tests but failed to download only the liked songs in the user’s library, downloading additional liked songs instead.The trajectory shows it gathered liked songs, invoked the download API for all of them, then checked already downloaded songs too late.Source: §F.2 · p. 19

G. Full Rollouts (LOOP)

G.1. Rollout 0 of task 2a163ab 1early in training

At gradient step 38, the early rollout decomposed the Venmo task into login, roommate and feed retrieval, filtering, and transaction liking. After an unavailable credential API caused failure, the agent recovered with an alternative API and completed the task.

Rollout setting

  • The rollout occurred near the beginning of training at gradient step 38 and used Venmo APIs to execute the task through interactive environment calls.Source: §G.1 · p. 20

Task procedure

  • Its planned procedure was to log in, obtain roommates as Venmo friends, retrieve the social feed, filter by date and roommate involvement, and like matching transactions.Source: §G.1 · pp. 20, 23

Error recovery

  • The agent recovered from the missing `show_account_credentials` API by using `show_account_passwords` instead, then logged in successfully.The environment reported that no API named `show_account_credentials` existed.Source: §G.1 · pp. 21–22

Task outcome

  • The rollout successfully liked all transactions from that day involving the agent’s roommates and marked the task complete.Source: §G.1 · pp. 24–25

G.2. Rollout 1 of task 2a163ab 1early in training

Near the beginning of training, the agent completed this Venmo task after recovering from invalid login credentials, but it could not access the phone app and proceeded using assumed roommate emails.

G.2. Rollout 1 of task 2a163ab 1early in training

  • The agent consulted the API list, used Venmo’s social feed to identify transactions, and filtered them by date and roommate involvement before liking them.Its planned workflow required login, social-feed retrieval, roommate identification, filtering, and repeated like_transaction calls.Source: §G.2 · pp. 25, 26, 28, 29
  • An initial login failed with a 401 Invalid credentials error, after which the agent retrieved the actual password and logged in successfully.It then obtained an access token and retrieved the social feed.Source: §G.2 · pp. 26–27
  • Accessing the phone app then failed with a 401 authorization error, so the agent assumed dummy roommate emails instead of retrieving the contacts.This assumption was used to continue filtering the social-feed transactions.Source: §G.2 · pp. 28–29
  • The rollout ultimately completed the task by filtering today’s roommate transactions and liking each matching transaction.The agent then called the supervisor completion API.Source: §G.2 · pp. 29–30

G.3. Rollout 0 of training task 2a163ab 1 after training · G.4. Rollout of dev task 530b157 1 after training · G.5. Rollout of dev task 3ab5b8b 2 after training. Fails to download only the liked songs in the users library, and instead downloads all liked songs including those not in the user’s library.

These post-training rollouts illustrate the agent’s execution of multi-step tasks across AppWorld applications, including consulting APIs, authenticating, retrieving information, filtering records, and completing requested actions. They also capture recovery from errors, alongside a Spotify case where the agent downloaded more liked songs than requested.

G.3. Rollout 0 of training task 2a163ab 1 after training

  • The agent successfully identifies roommates, retrieves same-day Venmo transactions, and likes each relevant transaction after checking the required API specifications.It logs into the phone and Venmo apps, searches contacts by roommate relationship, filters today’s transactions involving roommates, and receives repeated “Liked transaction.” confirmations.Source: §G.3 · pp. 31, 34, 35, 36

G.4. Rollout of dev task 530b157 1 after training

  • For dev task 530b157 1, the agent plans to search phone text messages for Kristin’s payment information and consult the supervisor app for payment-card details.The supplied excerpt ends during API inspection and does not report whether the payment or follow-up text message was completed.Source: §G.4 · pp. 37–38
  • A lookup using phone_number Kristin failed because no such user existed, although Kristin White appeared as a contact with phone number 6017026518 and a message saying “sounds good.”Source: §G.4 · pp. 40, 42
  • The cited passages describe using search_users to find Kristin’s Venmo username and create_transaction to send her $54, with a response reporting transaction_id 8216.Source: §G.4 · pp. 43–45
  • 35/35, the task was complete, and a text message was successfully sent to Kristin; the environment reported “Text message sent.”Source: §G.4 · p. 46

G.5. Rollout of dev task 3ab5b8b 2 after training. Fails to download only the liked songs in the users library, and instead downloads all liked songs including those not in the user’s library.

  • The Spotify rollout initially failed because a song was already downloaded, returning HTTP 422.The agent diagnosed the error and consulted the downloaded-songs API before retrying.Source: §G.5 · p. 51
  • After checking downloaded song IDs, the agent downloaded the liked songs and marked the task complete, but the rollout downloaded all liked songs rather than only those in the user’s library.The agent iterated over the liked-song list, skipped already downloaded songs, and received repeated “Song downloaded” responses.Source: §G.5 · p. 52

Related papers

Loading 2502.01600v3…