PaperlayerView exact revision on arXiv

Source-linked AI summary

Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance

Yaxi Lu, Shenzhi Yang, Cheng Qian, Guirong Chen, Qinyu Luo, Yesai Wu, Huadong Wang, Xin Cong, Zhong Zhang, Yankai Lin, Weiwen Liu, Yasheng Wang, Zhiyuan Liu, Fangming Liu, Maosong Sun

arXiv:2410.12361v3cs.AIcs.CL

TL;DR

Most LLM agents remain reactive, requiring explicit instructions rather than anticipating tasks from users’ activities and context. This paper builds ProactiveBench and a human-judgment reward model, then fine-tunes LLM agents for proactive assistance. The fine-tuned Qwen2-7B-Instruct achieves a 66.47% F1-Score, outperforming existing open- and closed-source LLMs.

  • Problem

    Most existing LLM agents require explicit human instructions, leaving proactive task anticipation and assistance without a direct request as an open challenge.

    Source: §1 · p. 2

  • Method

    The authors build ProactiveBench from real-world activities and generated events, train a reward model on human judgments, and fine-tune LLM agents on the dataset.

    Source: §1 · p. 2

  • Results

    66.47% F1-Score: fine-tuned Qwen2-7B-Instruct outperforms all existing open-source and closed-source LLMs on ProactiveBench.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The experiments report improved proactive-agent performance on ProactiveBench, while inappropriate task proposals and contextually relevant predictions remain challenges.

    Source: §5 · p. 9

    The explored environments are limited, and models still have a relatively high false-alarm ratio that can lead to unnecessary or incorrect actions.

    Source: Limitations · pp. 9–10

Abstract

from arXiv · show

Agents powered by large language models have shown remarkable abilities in solving complex tasks. However, most agent systems remain reactive, limiting their effectiveness in scenarios requiring foresight and autonomous decision-making. In this paper, we tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions. We propose a novel data-driven approach for this problem. Firstly, we collect real-world human activities to generate proactive task predictions. These predictions are then labeled by human annotators as either accepted or rejected. The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents. Building on this, we develop a comprehensive data generation pipeline to create a diverse dataset, ProactiveBench, containing 6,790 events. Finally, we demonstrate that fine-tuning models with the proposed ProactiveBench can significantly elicit the proactiveness of LLM agents. Experimental results show that our fine-tuned model achieves an F1-Score of 66.47% in proactively offering assistance, outperforming all open-source and close-source models. These results highlight the potential of our method in creating more proactive and effective agent systems, paving the way for future advancements in human-agent collaboration.

1 Introduction

Most LLM agents wait for explicit instructions, while proactive agents infer possible tasks from their environment and offer assistance. The paper introduces a data-driven approach and ProactiveBench, then reports improved proactive-assistance performance after fine-tuning.

1 Introduction

  • Most existing LLM agents require explicit instructions, whereas proactive agents infer tasks from environmental observations and can offer assistance without a direct request.The paper frames this capability as context awareness that can reduce users’ cognitive burden and identify latent needs.Source: §1 · p. 2
  • ProactiveBench combines 6,790 training events with a 233-event test set; the agent-model and reward-model subsets have separate data statistics.The cited table gives 1,640 annotated reward-model training labels and 120 test labels.Source: §1 · p. 2, Table 1 · p. 2
  • The data-driven approach collects real-world activity across coding, writing, and daily life, then uses an LLM-driven gym to generate events and proactive tasks for fine-tuning.The paper fine-tunes LLaMA-3.1-8B-Instruct and Qwen2-7B-Instruct on the generated training data.Source: §1 · p. 2
  • 66.47% F1-Score: fine-tuned Qwen2-7B-Instruct outperforms all existing open-source and closed-source LLMs on ProactiveBench.The reward model used for evaluation achieves up to 91.80% consistency with human judgments in F1-Score.Source: §1 · p. 2

2 Related Works

Prior approaches have explored proactive planning, support in text-to-SQL, and multi-turn clarification, but still require an initial user query. This work instead centers on anticipating tasks from monitored activity and environmental state.

2 Related Works

  • Earlier approaches include proactive planning, text-to-SQL support, and multi-turn clarification, but still require an initial user query.The cited discussion notes that multi-turn clarification can further increase user cognitive load.Source: §2 · p. 2
  • This approach monitors user activities and environmental states to anticipate potential tasks and initialize interaction without a query.Source: §2 · p. 3
  • Prior work also uses “Proactive Agent” for dialogue systems that improve response helpfulness or quality, unlike this paper’s focus on task anticipation and initiation.Source: §2 · p. 3

3 Methodology

The method defines proactive task prediction and user acceptance, then uses a simulated environment, agent, and user to generate events, predictions, and judgments. Human annotations also support a reward model that simulates user judgment.

3.1 Task Definition

  • The agent predicts a possible task from user activities, environmental events, and environment state, with no task as an option when none is needed.The framework aims to maximize user acceptance, including correctly remaining silent when assistance is not needed.Source: §3.1 · p. 3

3.2 Pipeline Overview

  • The data-generation pipeline simulates activities and responses across scenarios using an environment gym, proactive agent, and user agent.The gym generates events and maintains state; the agent predicts and acts; the user agent simulates activity and accepts or rejects proposals.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 Environment Gym

  • The environment gym derives scenarios from seed jobs and collected events, generates events in context, and updates entity states for subsequent event generation.Collected activity includes computer interactions, which are organized into coherent segments and rendered as natural-language descriptions.Source: §3.3 · pp. 4–5

3.4 Proactive Agent

  • The proactive agent updates memory with each new event, uses user-agent feedback on draft predictions, and either proposes a task or stays silent.Accepted tasks are executed in the gym through multi-turn tool interaction, generating further events.Source: §3.4 · p. 5

3.5 User Agent

  • The user agent simulates user activity and acceptance decisions, while human-annotated predictions train a reward model to simulate those judgments.The annotation process uses three annotators and majority voting; the cited table reports the fine-tuned LLaMA-3.1-Instruct-8B reward model’s best F1-Score of 91.80%.Source: §3.5 · pp. 5–6, Table 2 · p. 6

4 Experiments

The experiments assess the reward model and proactive agents across benchmark evaluations, prediction settings, and case studies. Fine-tuning improves proactive-agent results, while unnecessary or mistimed proposals remain a challenge.

4.1 Reward Model Assessment

  • The reward model achieves 100% agreement on false alarms and 91.80% F1-Score across scenarios, and is selected for ProactiveBench analysis.Source: §4.1 · p. 6

4.2 Proactive Agent Evaluation

  • 66.47% F1-Score: fine-tuned Qwen2-7B achieves the best result among open-source models on ProactiveBench; GPT-4o exceeds 64.60% among close-sourced models.Source: Table 3 · p. 7
  • Closed-source models often assist when needed but produce false alarms; GPT-4o-mini may propose abstract help early, while Claude-3-Sonnet can miss user needs.Source: §4.2 · p. 7
  • LLaMA-3.1-8B F1-Score rises from 44.78% to 61.74% after fine-tuning, while fine-tuned Qwen2-7B reaches 66.07%; over-assistance remains a pattern.Source: §4.2 · pp. 7–8

4.3 Performance Analysis

  • All models improve across metrics from pred@1 to pred@3; GPT-4o’s false alarm ratio falls from 51.85% to 36.44%.Source: Table 4 · p. 8, §4.3 · p. 8
  • Reward-model feedback generally lowers false alarms and improves accuracy but sharply reduces recall; combining it with pred@3 yields broader gains.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Case Study

  • Figure 4 contrasts two errors: GPT-4o misses needed payment-API guidance, while GPT-4o-mini proposes an action when no need is apparent.The discussion links mistimed proposals to unintended events in human activities.Source: Figure 4 · p. 9, §4.4 · p. 9

5 Conclusion

The conclusion presents ProactiveBench and a data-generation approach for improving proactive agent behavior, while identifying inappropriate and contextually inaccurate proposals as ongoing challenges.

5 Conclusion

  • ProactiveBench comprises 6,790 events and is designed to refine proactive behavior and provide an automatic benchmark for assessing LLM-agent proactiveness.Source: §5 · p. 9
  • Iteratively generated events in synthesized scenarios create training data that enhances the proactive capabilities of the models.Source: §5 · p. 9
  • The conclusion identifies minimizing inappropriate task proposals and improving contextual accuracy as ongoing challenges for proactive interaction.Source: §5 · p. 9

Limitations

The study is limited to a narrow set of environments, and models still produce relatively many false alarms. Broader settings and context-sensitive adjustment remain areas for further work.

Limitations

  • The explored environment settings are limited, so broader scenarios are needed to assess the agent’s generalizability and robustness.The paper also calls for investigation of application areas where proactive prediction could improve user experience or operational efficiency.Source: Limitations · pp. 9–10
  • Models still have a relatively high false-alarm ratio, which can lead to unnecessary or incorrect actions and reduce user trust and system efficiency.The paper identifies reducing false alarms through better understanding of context and user behavior as a future research direction.Source: Limitations · pp. 9–10
  • The agent’s proactiveness should be dynamically adjusted to the specific context to avoid bothering users.Source: Limitations · p. 10
  • The paper calls for extensive user-centric evaluation using user feedback and behavioral data to refine prediction algorithms and improve intuitiveness and reliability.Source: Limitations · p. 10
  • Because proactive prediction uses environment information, ethical and privacy concerns require responsible data handling and transparent operation.The paper frames these practices as important for user trust and acceptance.Source: Limitations · p. 10

A Reward Model Training Setting

The reward model is trained on a dataset of about 1,640 examples, while its prompt directs it to judge proposed tasks from the user’s perspective and minimize interruptions.

A Reward Model Training Setting

  • The reward model uses Llama-3.1-Instruct-8B, trains for three epochs on about 1,640 examples, and uses batch size 32 and learning rate 1e−5.Training uses Adam with a 0.01 warm-up ratio and eight A100 GPUs for approximately 1.5 hours.Source: A Reward Model Training Setting · p. 14

Prompt Template

  • The evaluator prompt asks the model to assess the proposed task as the user, considering current needs and accepting only valuable tasks to minimize interruptions.Source: Prompt Template · p. 14
  • The evaluator returns its thoughts and a judgment labeled accepted or rejected in JSON format.Source: Prompt Template · p. 14

B Agent Model Training Setting

The agent model is fine-tuned on about 6,790 examples, and its prompt directs it to infer user needs from events before proposing feasible, timely assistance or no task.

B Agent Model Training Setting

  • Agent training uses Llama-3-Instruct 8B and Qwen2-Instruct-7B as base models on approximately 6,790 examples, with 3 epochs used to prevent overfitting.Training uses batch size 32, learning rate 1e−5, an Adam optimizer with a 0.01 warm-up ratio, and 8 A100 GPUs on one node for approximately 2 hours.Source: B Agent Model Training Setting · p. 14

Prompt Template

  • The agent prompt requires understanding the user’s actions and anticipating needs from events, proposing assistance only when understood and feasible.Source: Prompt Template · p. 14
  • The response format records the user’s last-action purpose, the agent’s thoughts, a proposed task or null, and a response to the user.Source: Prompt Template · pp. 14–15
  • Prompt rules ask the agent to consider event relevance, current needs, timing, and event history, and to set the task to null when help is unnecessary.Source: Prompt Template · p. 15

C Prompt Template for Environment Gym

The appendix prompt templates specify environment and user simulation as well as task evaluation, while the metric section defines prediction labels and explains what recall, precision, accuracy, and F1 capture.

Prompt Template

  • The environment simulator generates and refines settings and events consistent with activity history, including occasional failures and checking whether operations are feasible.Source: Prompt Template · p. 15
  • The scenario-generation prompt asks for realistic, detailed assistance scenarios and iterative additions that remain consistent while increasing diversity.Source: Prompt Template · p. 16
  • The user simulator generates human-like activities with distinct identities and responds to environment events while accounting for possible inaccuracies and changing conditions.Source: Prompt Template · p. 16
  • The evaluator judges proposed tasks against the user’s current situation and requirements, accepting null when no task is needed and otherwise favoring valuable assistance.Source: Prompt Template · pp. 16–17
  • The evaluator returns its thoughts and an accepted-or-rejected judgment in the specified JSON format.Source: Prompt Template · p. 17

C.5 Metrics Calculation

  • True positives are proposed tasks accepted by the user; false positives are rejected proposals, true negatives are appropriate non-proposals, and false negatives miss needed assistance.Source: §C.5 · p. 17
  • Recall measures how often the agent identifies situations needing help, while precision reflects proposing good tasks without bothering the user too much.Source: §C.5 · p. 17
  • Accuracy measures relevance and correctness across predictions, while a high F1-Score indicates a balance between helpfulness and proactiveness.Source: §C.5 · p. 17

Collected Raw Data

The example presents raw activity snippets, processed events, and proactive suggestions, then asks for a judgment on a browsing sequence with no proposed task. The judgment accepts the decision not to interrupt the user.

Collected Raw Data

  • The raw activity snippets record timestamps, durations, mouse and keyboard input, and web-app status.The snippets include a search phrase and an Enter key press.Source: Collected Raw Data · pp. 17–18

Processed Event

  • The processed events describe opening Google in a new tab, switching to Code.exe, and remaining there without actions.The sequence spans three events.Source: Processed Event · p. 18

Example 1

  • For opening Google in a new tab, the assistant suggests reviewing search history and offering personalized recommendations.Its response asks whether the user wants help finding what they are looking for.Source: Example 1 · p. 19

Example 2

  • After the user switches to Code.exe, the assistant offers to create a terminal or command prompt for coding tasks.The response frames this as an offer and asks whether to set it up.Source: Example 2 · p. 19

Example 3

  • While the user remains inactive in Code.exe, the assistant offers coding help, including resources, tutorials, or writing code together.The response asks whether the user wants those forms of assistance.Source: Example 3 · p. 20

Input Prompt

  • The input prompt supplies a chronological Bing-search sequence and asks for a first-person judgment, with no proposed task.The sequence includes searching for remote-working software, opening and switching tabs, and exploring results.Source: Input Prompt · pp. 20–21

Ouput Judgment

  • The judgment accepts not proposing a task during the user’s Bing search, reasoning that an interruption could disrupt their workflow.The prompt describes searching for remote-working software and exploring Bing results; the proposed task is null.Source: Input Prompt · pp. 20–21, Ouput Judgment · p. 21

Related papers

Loading 2410.12361v3…