PaperlayerView exact revision on arXiv

Source-linked AI summary

Fara-7B: An Efficient Agentic Model for Computer Use

Ahmed Awadallah, Yash Lara, Raghav Magazine, Hussein Mozannar, Akshay Nambi, Yash Pandya, Aravind Rajeswaran, Corby Rosset, Alexey Taymanov, Vibhav Vineet, Spencer Whitehead, Andrew Zhao

arXiv:2511.19663v1cs.AIcs.CLcs.CV

TL;DR

Computer-use agents lack large, high-quality interaction datasets comparable to the text corpora used for language models. The paper introduces FaraGen to generate and verify synthetic web trajectories, then uses them to train screenshot-driven Fara-7B. Fara-7B performs strongly for its size, competing with much larger models, while its compact design supports on-device use.

  • Problem

    Computer-use-agent training lacks large, high-quality datasets capturing how people perceive and act through multi-step computer interactions.

  • Method

    FaraGen uses task proposal, collaborative solving, and multiple LLM verifiers to generate high-fidelity web trajectories, which train the screenshot-based, coordinate-predicting Fara-7B model.

  • Results

    Fara-7B achieves state-of-the-art performance among models of its size and remains competitive with significantly larger models on challenging web tasks.

  • Takeaways & Limitations

    Scalable synthetic data can support capable small computer-use agents with potential for on-device agency.

  • Takeaways & Limitations

    The authors do not recommend using Fara-7B in commercial or real-world applications without further testing and development.

Abstract

from arXiv · show

Progress in computer use agents (CUAs) has been constrained by the absence of large and high-quality datasets that capture how humans interact with a computer. While LLMs have thrived on abundant textual data, no comparable corpus exists for CUA trajectories. To address these gaps, we introduce FaraGen, a novel synthetic data generation system for multi-step web tasks. FaraGen can propose diverse tasks from frequently used websites, generate multiple solution attempts, and filter successful trajectories using multiple verifiers. It achieves high throughput, yield, and diversity for multi-step web tasks, producing verified trajectories at approximately $1 each. We use this data to train Fara-7B, a native CUA model that perceives the computer using only screenshots, executes actions via predicted coordinates, and is small enough to run on-device. We find that Fara-7B outperforms other CUA models of comparable size on benchmarks like WebVoyager, Online-Mind2Web, and WebTailBench -- our novel benchmark that better captures under-represented web tasks in pre-existing benchmarks. Furthermore, Fara-7B is competitive with much larger frontier models, illustrating key benefits of scalable data generation systems in advancing small efficient agentic models. We are making Fara-7B open-weight on Microsoft Foundry and HuggingFace, and we are releasing WebTailBench.

1 Introduction

FaraGen addresses the scarcity of high-quality computer-use interaction data with a scalable pipeline for generating and verifying web trajectories, which supports training the compact Fara-7B model. Fara-7B achieves strong performance for its size while offering on-device and cost advantages.

  • Motivation: FaraGen targets the CUA data bottleneck by replacing expensive, slow human annotation with scalable synthetic data generation from real websites and task prompts.CUA training requires demonstrations of visual interpretation, clicking, recovery, and multi-step task completion, for which no comparable corpus exists to web text.
  • FaraGen: FaraGen automates task proposal, collaborative solving, and trajectory verification to generate diverse, high-fidelity web trajectories at roughly $1 per completed task.Its verifiers filter hallucinations and execution errors, while the resulting data covers modern layouts, realistic intents, dynamic content, and multi-turn reasoning.
  • Fara-7B: Fara-7B uses a compact native CUA formulation that perceives webpages through screenshots and predicts actions directly as coordinates.The model is trained from trajectories containing screenshots, reasoning text, and coordinate-grounded actions.
  • Fara-7B: Fara-7B achieves state-of-the-art results among models of its size and remains competitive with much larger frontier models across web-based CUA benchmarks.The evaluation includes existing benchmarks and WebTailBench, which targets under-represented real-world web tasks.
  • Deployment advantages: Fara-7B’s small footprint enables local inference with lower latency and greater privacy, while averaging a few cents per task.The paper reports a more favorable performance-to-cost tradeoff than UI-TARS and substantially lower cost than frontier-model-based systems.
  • Evaluation: WebTailBench addresses gaps in existing evaluations by covering under-represented real-world scenarios on live websites to measure agent generalization.The benchmark is introduced alongside Fara-7B as a contribution of the work.

2 FaraGen– A Synthetic Data Engine for CUA

FaraGen addresses CUA data scarcity with a scalable pipeline that proposes realistic web tasks, generates solution trajectories, and verifies them before training. It produces diverse data at roughly $1 per task while supporting benchmark construction and large-scale model training.

  • 2 FaraGen– A Synthetic Data Engine for CUA: FaraGen combines task proposal, multi-agent solving, and trajectory verification to generate realistic, diverse web-task demonstrations without manual annotation.Tasks are sourced from live websites and refined into verifiable user goals; solving agents generate candidate trajectories, while LLM verifiers filter errors and hallucinations.
  • 2.1 Task Proposal: ClueWeb22 is preferred over Tranco for seed URLs because it contains fewer corporate landing pages and therefore offers a broader scope of actionable tasks.
  • 2.1 Task Proposal: WebTailBench releases 609 tasks across 11 underrepresented task segments to evaluate agent generalization on realistic live-web scenarios.
  • 2.2 Task Solving: Trajectory diagnostics remove up to 25% of completed multi-item shopping trajectories for repeated loops, compared with 7.5% for single-item shopping.The orchestrator uses loop and action-success flags to re-plan, retry, or stop when trajectories fail, reach critical points, or complete.
  • 2.2 Task Solving: Browserbase raises successful trajectory yield from 9% to 35% for shopping and from 3% to 11% for flights, with additional gains from domain-specific instructions.
  • 2.3 Trajectory Verification: FaraGen yields 145K trajectories across 70K unique domains for roughly $1 per task, making large-scale CUA data generation economically feasible.Verifier predictions agree with human judgments on 83.3% of cases, while the resulting data supports a cost-effective end-to-end approach with Fara-7B.

3 Fara-7B – An Efficient CUA Model

Fara-7B is a single native computer-use model distilled from multi-agent trajectories, operating from screenshots and browser metadata while predicting grounded actions. Training combines trajectory data with auxiliary grounding, refusal, and screenshot-understanding tasks.

  • 3 Fara-7B – An Efficient CUA Model: Fara-7B distills multi-step reasoning and recovery behaviors from multi-agent trajectories into a single native CUA model.The model is trained on trajectories generated by the multi-agent pipeline rather than deploying that pipeline at inference time.
  • 3 Fara-7B – An Efficient CUA Model: The model predicts thoughts and one tool action per step from the current observation and interaction history, continuing until it outputs a stop action.Supported actions include clicking, typing, scrolling, navigation, searching, waiting, memorizing, and other browser operations.
  • 3 Fara-7B – An Efficient CUA Model: Fara-7B operates without accessibility trees at inference time, using screenshots and browser metadata to predict click coordinates and other grounded actions.This removes runtime reliance on scaffolding that can vary across websites and UI implementations.
  • 3 Fara-7B – An Efficient CUA Model: Fara-7B retains recent screenshot observations while preserving prior thoughts and actions to balance contextual information with computational cost.The implementation sets the retained screenshot history to N = 3 because screenshots can consume thousands of tokens.
  • 3 Fara-7B – An Efficient CUA Model: Training uses trajectory screenshots, reasoning, and actions alongside auxiliary grounding, refusal, and screenshot question-answering or captioning data.The full mixture contains 1.8 million training samples and uses supervised fine-tuning on Qwen2.5-VL-7B.

4 WebTailBench

WebTailBench complements existing CUA benchmarks with broader task coverage, greater complexity, and evaluations designed around realistic, goal-oriented web activities. It also includes refusal scenarios and verification intended to align more closely with human judgment.

  • WebTailBench expands CUA evaluation with eight underrepresented task subcategories, measuring both individual skills and aggregate performance.The benchmark is designed to increase task diversity and coverage while also increasing task complexity.
  • Its coverage includes realistic activities such as real-estate, job applications, multi-item shopping, and comparison shopping that are often missing from existing benchmarks.The tasks are drawn from high-traffic webpages to reflect actual human information needs.
  • WebTailBench emphasizes breadth and depth through multiple subcategories, sufficient task counts, and varied complexity levels across domains and platforms.This addresses sparse coverage such as flight-booking tasks and single-platform representation in existing benchmarks.
  • Its goal-oriented tasks award full credit when models correctly report uncontrollable outcomes, avoiding penalties for failures such as sold-out bookings.This contrasts with benchmarks containing underspecified instructions that mainly test navigation rather than task completion.
  • WebTailBench adds 111 refusal tasks spanning seven harmful-task categories and uses verification designed to match human assessments more closely.The benchmark also includes time-sensitive tasks that may require periodic refreshing after November 2025.

5 Experiments

Fara-7B delivers strong web-agent performance across benchmarks, grounding, scaling, and safety while remaining substantially more token- and cost-efficient than larger proprietary agents.

  • 5.1 Main Results: 73.5% WebVoyager success surpasses SoM GPT-4o (65.1), GLM-4.1V-9B-Thinking (66.8), and OpenAI computer-use (70.9), while 38.4 WebTailBench leads comparable models.Fara-7B is also comparable to GPT-4o on Online-Mind2Web and scores 26.2 on DeepShop versus 16.0 for GPT-4o and 11.6 for UI-TARS-1.5-7B.
  • 5.1 Main Results: $0.025 average cost per WebVoyager task is roughly twelve times below proprietary baselines, with about 1.1k output tokens versus GPT-5’s 13k.Fara-7B completes tasks in 16.5 ± 21.1 actions on average, comparable to GPT-4o and GPT-5.
  • 5.1.3 WebTailBench Results: Fara-7B stays within 3 points of o3 on WebTailBench flight and hotel categories despite fewer than 4k training tasks in each category.The paper connects this result to the effectiveness of modest amounts of high-quality training data.
  • 5.2 Grounding: Fara-7B reaches 89% on ScreenSpot-V2, improving over base Qwen2.5-VL and showing strong localization across grounding segments.The authors attribute particularly strong performance on text elements to their prevalence among interactive web elements.
  • 5.3 Scaling Trends: Fara-7B improves substantially as training data grows from 18K to 180K to 1.8M action steps, and also benefits from larger inference-step budgets.Its inference-step scaling is similar to UI-TARS-1.5-7B on WebVoyager and Online-Mind2Web.
  • 5.4 Safety - Refusals and Critical Points: Fara-7B refuses 94.2% of harmful AgentHarm-Chat tasks and avoided harmful behavior in 9 of 13 adversarial Magentic-UI tasks.The four adversarial failures involved links to local or cached files and were stopped by browser sandboxing.

6 Related Work

Related work spans tool-calling, multimodal screen understanding, structured and pixel-based computer-use agents, and increasingly realistic benchmarks. Across these areas, the paper emphasizes that real-world web interaction remains difficult because websites are dynamic and high-quality trajectories are scarce.

  • Tool-Calling LLMs: Tool-calling LLMs support reasoning with structured external tools, but typically operate in environments more structured than real websites.CUA systems must additionally handle pixel-level grounding, action modeling, and long-horizon planning.
  • Multimodality and screen understanding: Multimodal screen-understanding work improves perception and GUI localization, supplying capabilities needed for agents that act directly on screens.Representative efforts include ScreenSpot, OmniParser, GUI-Actor, and ScreenQA.
  • Agentic CUA models: Structured CUA benchmarks expose agents to DOM or accessibility representations, whereas pixel-in, action-out systems consume screenshots and produce low-level actions such as clicks and scrolls.The latter formulation is intended to better approximate human computer use, but remains constrained by scarce diverse trajectories.
  • Agentic CUA models: Existing trajectory sources are often manually collected, sandbox-constrained, limited to few websites, or mined from videos with privacy and annotation problems.The paper positions synthetic data generation as an approach to overcoming these quantity and quality limitations.
  • Benchmarks: CUA evaluation is difficult because websites change over time and agents must be assessed beyond task performance, including safety and privacy.Benchmarks span atomic visual and grounding capabilities as well as multi-step browser interactions.

7 Discussion

Fara-7B combines targeted long-horizon training data with native screenshot-based computer use to make a small model competitive on web tasks. The authors emphasize broader evaluation coverage, cost-conscious native action prediction, and unresolved robustness and safety boundaries.

  • Potential for Agentic SLMs: Fara-7B achieves state-of-the-art performance among its size class while remaining competitive with significantly larger models.It operates directly on browser GUIs using screenshots, without accessibility trees or complex scaffolding.
  • Overcoming agentic data scarcity: FaraGen addresses agentic data scarcity by automating task proposal, execution, and verification to generate high-fidelity trajectories for less than $1 per task.The training approach uses carefully targeted long-horizon trajectories from multi-agent runs.
  • Comparing SoM Agents to Native CUA: Native coordinate prediction reduces token use and avoids failure modes caused by noisy, incomplete, or hallucinated accessibility-tree elements.The authors contrast harmless mis-clicks with more serious interactions involving nonexistent or misleading elements.
  • Evaluation of CUA models: WebTailBench broadens CUA evaluation with under-represented real-world tasks, including real estate, jobs, shopping comparisons, and activity planning.The benchmark is intended to provide more comprehensive coverage of agentic capabilities.
  • Limitations: Fara-7B remains limited on complex tasks, instruction following, environmental changes, hallucinations, and unsupported action types such as native drag-and-drop.The paper also identifies broader human-agent collaboration as an open challenge.
  • Guidelines for Safe Use: The authors recommend sandboxing, human monitoring, restricted access to sensitive resources, and avoiding sensitive data, high-stakes domains, and untested real-world use.They specifically warn that harmful websites may expose the model to prompt injection.

B Data Visualization

The training data includes solved web trajectories that pair user tasks with screenshots, reasoning, and browser actions. Additional examples target fine-grained UI grounding and screenshot interpretation across varied interfaces.

  • Trajectory examples: Solved trajectories demonstrate multi-step web tasks by pairing goals with navigation, visual reasoning, memorized facts, and actions.One example stops before authentication after reaching the subscription flow.
  • Grounding data: The grounding dataset pipeline specializes the broader seed-to-verification recipe for fine-grained UI element understanding from screenshots and selected elements.It maps natural-language queries to concrete atomic actions involving UI elements.
  • UI interaction examples: The examples include quantity changes, size selection, code resending, and open-ended descriptions of interface screenshots.These prompts connect natural-language instructions with concrete UI interactions or visual understanding.
  • Screenshot understanding: Screenshot descriptions cover interface elements such as branding, page structure, dates, locations, and other visible content.The examples span loan and conference websites.

B.3 Screenshot Question Answering Data

The screenshot question-answering data teaches the model to answer questions using only the current interface image. Generated examples cover visually grounded facts such as book labels, age recommendations, and ticket-delivery timing.

  • Data generation: Screenshot QA prompts require answers based only on visible image content, without additional information.The generation process asks for challenging or rare questions that require direct visual or textual understanding.
  • Question-answering examples: Examples ask about webpage-specific facts, including which Agatha Christie book is marked “Teachers’ pick” and its recommended age group.The supplied answer identifies And Then There Were None and ages 16 years and up.
  • Question-answering examples: Another example asks when tickets will be ready for delivery, with the answer giving March 28, 2025.The answer cites the page’s Additional Notes.

B.4 Refusal Data

The refusal data trains Fara-7B to decline harmful browser tasks across seven policy categories. Examples cover deceptive reporting, targeted harassment, illicit drug purchasing, and smear campaigns.

  • Refusal training: Fara-7B is trained to refuse harmful browser tasks using synthetic examples across seven categories, supplemented by WildGuard refusal data.The categories are summarized in Table 15.
  • Refusal examples: A refusal example rejects falsifying emissions data and deceptive regulatory reports while offering legitimate carbon-footprint and compliance assistance.The response redirects toward calculating actual emissions or reducing them legally.
  • Refusal examples: Additional examples target privacy-invasive workplace identification, illicit drug procurement, and fabricated smear campaigns.These tasks combine online research, communication, purchasing, or content-generation actions.

C Model Training Details

Fara-7B’s training mixture is dominated by trajectory data, supplemented with grounding, UI VQA, captioning, and limited safety data to support native computer use and reduce hallucinations.

  • C Model Training Details: Grounding data is included to help train Fara-7B’s native pixel-in, action-out computer-use behavior.
  • C Model Training Details: Trajectory data forms the majority of Fara-7B’s 1.8 million training samples, with grounding as the second-largest component.Some data is upsampled, especially longer and less frequent compositional-task trajectories.
  • C Model Training Details: UI VQA and captioning data supplement trajectories because qualitative analysis found hallucination failures.
  • C Model Training Details: Excess refusal data can make the model refuse benign tasks, so only a small amount of safety data is used.
  • C Model Training Details: Training uses AdamW, a 10% learning-rate warmup, 2 epochs, batch size 128, and 64 H100 GPUs.

D Safety Evaluation Details

The safety evaluation covers seven harmful-task categories using handcrafted examples collected for WebTailBench-Refusals.

  • D Safety Evaluation Details: The evaluation uses 111 handcrafted harmful tasks spanning the categories defined in Table 15.
  • D Safety Evaluation Details: WebTailBench-Refusals provides example tasks for each of seven harmful-task categories.

D.2 Critical Point Evaluation

Critical-point evaluation tests whether Fara-7B completes benign task setup but stops before sensitive or irreversible actions requiring user confirmation.

  • D.2 Critical Point Evaluation: The critical-point evaluation uses 23 tasks to assess whether Fara-7B stops before critical actions.
  • D.2 Critical Point Evaluation: Fara-7B completes hotel search and date selection, then stops when personal-information entry appears.This behavior treats entering personal details as a critical step requiring explicit user input.
  • D.2 Critical Point Evaluation: In restaurant reservations, Fara-7B configures cuisine, location, date, and time but stops before clicking “Reserve.”The final click would place a real booking and trigger collection of user details.

D.3 Web Task-Solving Performance

Across four web benchmarks, Fara-7B achieves the best average performance among 7B-scale computer-use agents with low variability, while its cost-accuracy trade-off remains favorable.

  • D.3 Web Task-Solving Performance: Fara-7B achieves the best average performance among 7B-scale computer-use agents across four web benchmarks.Table 19 reports mean success rates with standard deviations for WebVoyager, Online-Mind2Web, DeepShop, and WebTailBench.
  • D.3 Web Task-Solving Performance: Fara-7B’s standard deviation is around 1.0 on WebVoyager and below 2.0 on another reported benchmark.The passage contrasts this stability with higher fluctuations from competing systems.
  • D.3 Web Task-Solving Performance: The model’s favorable cost-accuracy trade-off supports a cost-effective end-to-end approach.
  • D.3 Web Task-Solving Performance: Fara-7B’s gains are stable across repeated evaluations rather than strong only in expectation.
Loading 2511.19663v1…