PaperlayerView exact revision on arXiv

Source-linked AI summary

NorthCUA Technical Report: Turbocharging Computer-Use Agents through Experience at Scale

Chaoyun Zhang, Bingxuan Li, Pinxue Guo, Yangfan Gao, Zikun Cai, Tianci Xie, Hui Li, Yuqi Huang, Yifei Cao, Bo Mao, Tian Tang, Qichang Zheng, Tongtong Liu, Xiaoxing Wang, Jinxiu Liu, Lizhi Lin, Zhou Shao, Maosen Zhou, Miao Zheng, Shuo Zhang, Yangji He, Kepeng Lei, Ning Zhao, Wei Shen, Zhenjiang Jin, Xinyue Zhang, Peng Sun, Zhenhua Han, Rui Zheng, Hang Yan

arXiv:2611.02102v1cs.LG

TL;DR

Computer-use agents need experience that connects decisions across long workflows to complete task outcomes. NorthCUA combines executable-task construction, a Harness for sustained execution and assessment, and staged learning from curated trajectories and fresh policy attempts. The resulting model family reports state-of-the-art open-source results across GUI grounding, desktop execution, and Web interaction under the reported evaluation settings.

  • Problem

    Computer-use agents need experience that develops interface skills while connecting successive decisions to complete outcomes, including feedback from their own attempts.

    Source: §1 · p. 2

  • Method

    NorthCUA co-designs executable-task construction, Harness execution and assessment, trajectory curation, staged training, and fresh policy interaction in a shared experience loop.

    Source: §1 · pp. 2, 4

  • Results

    NorthCUA achieves state-of-the-art results among open-source models across GUI grounding, desktop execution, and Web interaction.

    Source: §1 · p. 4

  • Takeaways & Limitations

    The Harness provides a reusable foundation for continued learning across desktop and web environments by returning updated policies to collect further experience.

    Source: §1 · p. 4

    Feedback quality depends on executable-predicate coverage and the calibration of model judgments for open-ended content.

    Source: §8 · p. 76

Abstract

from arXiv · show

Scaling computer-use agents from individual actions to complex software work requires experience that connects decisions to complete outcomes. We present NorthCUA, an integrated system for turbocharging computer-use agents through experience at scale. Its Experience Flywheel supplies diverse executable tasks to the NorthCUA Harness, which sustains long interactions, records decisions, and checks task outcomes. The Flywheel uses these records to curate verified trajectories. For unsolved tasks, recursive self-improvement evolves reusable skills and preserves verified milestones so later attempts build on earlier progress. Staged Learning combines this experience with grounding data from the Flywheel to develop GUI competence through mid-training and long-horizon supervised fine-tuning. The policy then learns from fresh attempts in the same Harness through joint desktop-web reinforcement learning, closing the loop between experience production and policy improvement. The resulting Nano, Mini, and Pro family achieves state-of-the-art results among open-source models across GUI grounding, desktop execution, and Web interaction under the reported evaluation settings. For more responsive Web workflows, NorthCUA-Ultrafast pairs direct action selection with open-ended generation on pre-RL Nano, offering a measured quality-latency tradeoff. Together, these advances establish NorthCUA as a high-performing model family and a reusable foundation for agents that carry complex software work from intent to completion.

1 Introduction

NorthCUA co-designs task construction, execution, and learning so computer-use agents can connect successive decisions to complete outcomes. Its integrated experience loop combines curated tasks and trajectories, staged training, and fresh policy attempts; the resulting family reports strong open-source results across GUI grounding, desktop execution, and Web interaction.

1 Introduction

  • The system co-designs experience production and use, linking executable tasks, assessed trajectories, demonstrations, grounding examples, and fresh policy attempts.The updated policy returns to the same Harness, where new attempts can be evaluated and curated into subsequent training experience.Source: §1 · pp. 2, 4
  • The Experience Flywheel creates grounding examples and executable tasks, then curates assessed trajectories; unsolved tasks can use reusable skills and verified milestones to guide further attempts.The hard-task collection branch uses skill-guided exploration and milestone-guided search.Source: §1 · pp. 3–4, Figure 2 · p. 3
  • Staged Learning progresses from broad GUI mid-training to verified long-horizon SFT and joint desktop–web reinforcement learning from fresh successes and failures.The policy’s own attempts provide outcome feedback in the same Harness.Source: §1 · p. 4
  • NorthCUA establishes state-of-the-art results among open-source models in GUI grounding, desktop execution, and Web interaction; Pro also outperforms larger models on several benchmarks.Pro (397B) outperforms trillion-parameter Qwen3.8-Max and Kimi-K3 on both grounding benchmarks, OSWorld-Verified, and WebArena-Verified.Source: §1 · p. 4
  • NorthCUA-Ultrafast pairs direct finite-action selection with open-ended generation on pre-RL Nano, and its Head + LoRA variant completes matched successful WebArena-Verified tasks 2.3× faster on average than full generation.The report releases the three models, the full Harness code, and the technical report.Source: §1 · p. 4

2 Scope and System Overview

NorthCUA evaluates single-step control localization separately from end-to-end software tasks, where agents act through successive observations and feedback. Its lifecycle turns task specifications into assessed trajectories for learning, then uses staged training and fresh policy attempts to continue improvement.

2.1 Task Scope and Characteristics

  • Single-step grounding predicts a target UI element’s coordinates from a fixed screenshot and natural-language query, while end-to-end tasks require successive actions and completion checks against resulting artifacts or software state.Web observations also include DOM information.Source: §2.1 · pp. 5–6, Figure 3 · p. 5
  • Desktop agents use screenshots and mouse or keyboard actions across applications and files, while web agents use page screenshots, DOM information, and browser tools.Both settings share an observation, action, and feedback loop.Source: §2.1 · p. 6

2.2 System Organization and Experience Lifecycle

  • The Experience Flywheel packages requests, materials, environment setup, and assessment requirements into CaseSpecs; the Harness runs and records attempts for task-specific checks or evaluator inspection.Assessed outcomes and execution quality support demonstration selection, while valid unsolved tasks can enter hard-task collection.Source: §2.2 · p. 6
  • Staged Learning combines broad trajectories with separate screenshot–query–target examples, then trains on verified complete workflows before joint desktop–web reinforcement learning updates one policy from grouped attempts.The updated policy returns to the Harness under the same task and outcome interfaces.Source: §2.2 · pp. 6–7

3 NorthCUA Harness: Full-Lifecycle Infrastructure for Computer Use

The NorthCUA Harness keeps execution, workspace state, records, and assessment connected through each task lifecycle. Shared desktop–web execution, context management, and evidence-based evaluation support long workflows and turn attempts into inspectable experience for learning.

3 NorthCUA Harness: Full-Lifecycle Infrastructure for Computer Use

  • The Harness retains the workspace through assessment and preserves execution records and artifacts, connecting decisions, outputs, and judgments from the same attempt.The evaluator checks declared requirements using the trajectory and live workspace before resources are released.Source: §3 · p. 7, Figure 4 · p. 8

3.1.1 Shared Agent Runtime

  • A shared runtime assembles requests, observations, and relevant history, executes policy tool calls, and feeds returned observations and results into subsequent decisions.The runtime separates the common interaction loop from platform-specific observation and action tools.Source: §3.1.1 · pp. 7–8, §3.1 · p. 7

3.1.2 Tool Integration and Dispatch

  • The common tool interface dispatches calls to local functions or MCP tools, while serial execution of multiple calls in one response can share one model invocation.Post-action observation hooks attach fresh interface views to state-changing results.Source: §3.1.2 · p. 9

3.1.3 Desktop and Web Instantiations

  • Desktop and web adapters supply different observations and controls—desktop screenshots and input tools versus web screenshots, DOM information, and browser tools—while reusing runtime and trajectory-recording services.The same separation provides a path for adding platforms around the shared runtime.Source: §3.1.3 · p. 9, Table 1 · p. 10

3.2 Sustaining Long-Horizon Interaction

  • Long workflows require controlling context growth while keeping current state, earlier progress, and task-relevant knowledge available for later decisions.The system separates the decision context from the complete execution record retained for inspection and learning.Source: §3.2 · p. 10, Figure 5 · p. 11

3.2.1 Action-Aware Visual Context

  • Action-aware visual context overlays recent operations on historical frames while keeping the latest clean screenshot current; textual action and result history can persist after image eviction.The paired views help connect a prior click, drag, scroll, or text entry to the resulting screen change.Source: §3.2.1 · pp. 11–12, Figure 6 · p. 12

3.2.2 Task-Preserving Context Compaction; 3.2.3 Structured Working Memory

  • Task-preserving compaction protects recent exchanges, summarizes goals, constraints, progress, and open work, and omits stale GUI locators; structured notes let agents proactively retain facts, decisions, corrections, or checkpoints.Notes remain separate from the summary, can be revised, and are supplied in later decision contexts.Source: §3.2.2 · p. 13, §3.2.3 · pp. 13–14

3.3 Agentic Outcome Evaluation; 3.3.1 Artifact and Live-State Inspection

  • The adaptive evaluator selectively retrieves records and artifacts and inspects the live workspace to assess requirements that may lack dedicated executable checks.Evidence-linked findings distinguish task fulfillment from execution quality and connect criterion-level judgments to evidence and uncertainty.Source: §3.3 · p. 14, §3.3.1 · pp. 14–15, Figure 7 · p. 15

Appendix H.2.3 lists the evaluator’s retrieval tools, authorized live interfaces, and report schema.

The evaluator links task-grounded outcome checks with process assessment, while the rollout service coordinates isolated attempts and delivers complete, inspectable records. These mechanisms turn individual attempts into assessed experience for selection and training.

Appendix H.2.3 lists the evaluator’s retrieval tools, authorized live interfaces, and report schema

  • The evaluator uses task-specific judgment to combine recorded evidence with authorized live inspection when available observations leave an outcome unclear.For document exports, recorded actions, file checks, and application inspection address distinct parts of acceptance; judgments are linked to the submitted work and observations.Source: Appendix H.2.3 · p. 15

3.3.2 Task-Grounded, Multidimensional Rubrics

  • The evaluator combines requirement-linked judgments of task fulfillment with assessments of process quality, efficiency, side effects, failure diagnosis, and uncertainty.Findings link to relevant requirements and recorded steps, supporting review of the judgment scope.Source: §3.3.2 · p. 16

3.3.3 From Evaluation to Experience Selection

  • Separating fulfillment from execution quality lets reports support selection of complete demonstrations, local segments, and recovery behavior for different learning goals.Thresholds can favor concise successes or retain useful corrections and exploration in successful attempts.Source: §3.3.2 · p. 16, §3.3.3 · pp. 16–17

3.4 Reliable Batch Rollout at Scale

  • The Batch Rollout Service coordinates parallel task lifecycles because preparation, execution, evaluation, and recording consume different time and resources.Simply launching more agents can create bottlenecks elsewhere, and interruptions after execution can leave useful work without a complete record.Source: §3.4 · p. 17

3.4.1 Remote Execution with Centralized Agent Control

  • Centralized control separates changing prompts and agent logic from device workers, which execute tool calls in task-specific sandboxes and capture observations.Controller-side trajectories persist after sandbox release, and batch configuration supports resumed collection.Source: §3.4.1 · pp. 17–18

3.4.2 Isolated Environments with Pipelined Preparation

  • Fresh isolated sandboxes preserve task-specific starting conditions, while a task-bound queue prepares upcoming environments during other attempts’ execution or evaluation.Ready workspaces remain available through outcome checks and output capture before release.Source: §3.4.2 · pp. 18–19

3.4.3 Completion-Oriented Scheduling and Recovery

  • Stage-specific concurrency limits and controller-pressure feedback regulate admission as preparation, execution, and evaluation progress at different rates.After failures, completed execution records can support retries of evaluation or artifact processing; rerun tasks start in fresh environments.Source: §3.4.3 · p. 19

4 Experience Flywheel: Scaling Grounding and Agentic Experience

This section describes how the Experience Flywheel builds grounded examples and executable, assessed tasks across desktop and Web environments, and curates trajectories from their execution. It also covers recursive methods for carrying forward validated skills and progress, and preserving learning fidelity as Flywheel experience supports staged policy learning.

4.1 Single-Step UI Grounding

  • Single-step grounding trains control localization by pairing a screenshot and natural-language query with the requested control’s coordinate.The examples target visible controls, such as buttons or toolbar icons, in the original image.Source: §4.1 · p. 20

4.1.1 Broad Visual and Structural Coverage

  • Broad visual and structural coverage varies interfaces, states, control descriptions, and query wording while keeping each target’s original location fixed.Screenshots come from public UI datasets, web-collected GUIs, and agent trajectories; visual parsing and available accessibility metadata help identify targets.Source: §4.1.1 · pp. 20–21

4.1.2 Documentation-Grounded Functional Localization

  • Documentation-grounded localization resolves ambiguous control functions by retrieving documented candidates and matching them against the application and surrounding GUI context.The accepted function description then guides query variants, all retaining the control’s original screenshot and coordinate.Source: §4.1.2 · pp. 22–23

4.1.3 Quality Control and Coverage

  • Grounding candidates pass correctness checks for target geometry and query-control fit, then coverage checks remove duplicates while retaining varied interfaces, states, controls, and query styles.Unresolved cases are excluded; repairable geometry or identity failures return to construction.Source: §4.1.3 · p. 23

4.2 Scaling Intents, Environments, and Verifiers

  • Task construction scales intents, environments, and verifiers through complementary routes that generate requests, prepare execution settings, or adapt tasks with executable checks.Each route produces a request, workspace initialization, and outcome assessment; assessment may use an executable verifier or evaluator inspection.Source: §4.2 · pp. 24–25

4.2.1 Scaling Intents

  • Desktop base tasks ground requests in a supported software operation and compatible file, then bind the request to a prepared workspace and inspectable completion criteria.Function and file-content summaries guide request generation; startup instructions establish the initial application state, and the requested change determines the completion criteria.Source: §4.2.1 · pp. 26–27
  • Long-horizon tasks connect multiple work stages into a shared workflow, with a dependency graph specifying prerequisite information, artifacts, and execution order.Blueprints define candidate stages and prerequisites; prepared materials make each stage executable and its required outcome specific.Source: §4.2.1 · pp. 27–29
  • Scaling Intents grounds desktop and Web requests in available materials or site capabilities, tying goals to initial states and outcome checks in a CaseSpec.For multi-stage desktop workflows, the shared task plan aligns the goal, workspace, and assessment as tasks expand across applications.Source: §4.2.1 · p. 30

4.2.2 Scaling Environments

  • Scaling Environments builds executable workspaces from sourced requests, reference websites, and runnable software projects, and reconstructed sites can also support new task generation.For desktop tasks, EnvAgent prepares missing files and startup conditions; local Web environments enable controlled initialization and repeated tasks. These environments support experience collection.Source: §4.2.2 · pp. 31, 36
  • For issue-driven frontend testing, a web agent replays reported interactions against candidate repairs and returns action traces, screenshots, and verdicts to guide revisions under a fixed expected behavior.In a homepage case study, retesting passed three selected checks, resolving two reported defects.Source: §4.2.2 · pp. 34, 36

4.2.3 Scaling Verifiers

  • Scaling Verifiers adapts existing task–verifier pairs by decomposing, varying, or composing subgoals, changing the request, initial state, and success checks together.The described decomposition example makes record selection a standalone task with checks for the returned selection and preservation of all records.Source: §4.2.3 · pp. 37–38
  • Task expansion checks both whether an agent can execute each rewritten request and whether its verifier judges expected outcomes correctly, routing failures back for revision and retesting.Verifier probes distinguish complete outcomes from missing exports, incomplete updates, protected-record changes, and allowed alternatives.Source: §4.2.3 · p. 39

4.3 Recursive Self-Improvement for Hard-Task Collection

  • Recursive self-improvement uses skill-guided exploration to accumulate tested procedures and milestone-guided search to preserve completed work, targeting tasks unsolved within ordinary collection.The original acceptance conditions remain fixed; completed assisted tasks are checked before admission as full-task demonstrations, while verified milestone traces can also be retained from unfinished attempts.Source: §4.3 · p. 40

4.3.1 Skill-Guided Recursive Exploration

  • Skill-guided exploration converts trial-and-error into reusable instructions that a separate agent validates before a new attempt reuses them and contributes further discoveries.Exploration observations are consolidated into actionable guidance; failed validation returns the candidate for revision, and accepted full-task attempts retain the supplied guidance in context.Source: §4.3.1 · pp. 40–42

4.3.2 Milestone-Guided Trajectory Search

  • Milestone-guided search decomposes a task into checked goals, saves workspace and agent context at passing milestones, and explores continuations before the original verifier checks completion.Passed milestone traces from attempts that do not finish the full sequence can be retained as subtask-level demonstrations with their original decision context.Source: §4.3.2 · pp. 42, 44

Attempt 1 · Skill v1 1 Wrong orientation 2 Wrong colors

  • The TERRA dashboard attempt using revised skill v2 passed all 26 final checks, while the first attempt failed the palette check after correcting chart orientation.Skill v2 specified Vertical=False, explicit series-color assignment, and verification; the fresh attempt also passed input-change tests of live formulas and charts.Source: §4.3.3 · p. 44, Attempt 1 · Skill v1 1 Wrong… · p. 45, Figure 22 · p. 45

ORBIT BOTANICA

  • In ORBIT BOTANICA, retained equipped checkpoints let later branches address rendering; two clear-glass branches passed full-task checks, producing a complete demonstration along a passing path.The opaque-glass continuation failed, while the accepted path retained its interaction prefix and successful continuations; failed siblings remained search records.Source: Figure 23 · p. 46, ORBIT BOTANICA · p. 47

4.3.3 Case Studies of Knowledge and Progress Accumulation

  • The two case studies illustrate complementary forms of accumulated experience: TERRA reuses learned procedures, while ORBIT BOTANICA reuses completed work to target unresolved requirements.Both routes retain verified task trajectories for learning.Source: ORBIT BOTANICA · p. 47

ORBIT BOTANICA

  • The Experience Flywheel supplies grounding examples, complete-workflow demonstrations, and executable tasks for fresh rollouts, while recursive self-improvement expands attempts on difficult tasks.The collected records support curated verified trajectories and preserve milestones so later attempts can build on earlier progress.Source: ORBIT BOTANICA · p. 47

5 Staged Learning from Scalable GUI Experience

  • The three-stage pipeline moves from broad GUI mid-training to verified long-horizon supervised fine-tuning and joint desktop–web reinforcement learning.Each stage starts from the preceding policy while shifting emphasis from operational breadth to sustained completion and outcome-driven improvement.Source: §5 · p. 47

5.1 Broad GUI Mid-Training

  • Broad mid-training uses over 100,000 trajectories, over 4 million GUI interaction steps, and approximately 40,000 grounding examples across dozens of applications and hundreds of operation families.The mixture also includes web-specific workflows and sustained multi-operation tasks; outcome-based review selects completed work with useful execution behavior.Source: §5.1 · pp. 48–49

5.2 Verified Long-Horizon SFT

  • Verified long-horizon SFT continues from the mid-trained checkpoint with approximately 3,000 trajectories and 310,000 GUI interaction steps whose complete outcomes pass executable verifiers.Selection prioritizes difficult, dependent workflows and retains coherent recovery; responses are trained in decision-time context to strengthen sustained execution.Source: §5.2 · pp. 50–51

5.3 Joint Desktop–Web Agentic Reinforcement Learning

  • Joint desktop–web reinforcement learning uses fresh policy decisions and task-specific outcome feedback to improve one shared policy through the NorthCUA Harness.The manager prepares executable cases and verifiers, workers record attempts, and the learner uses task-grouped samples with GRPO before returning updated weights to workers.Source: §5.3 · p. 51

5.3.1 Selecting Tasks with Outcome Variation

  • RL task selection collects at least five SFT-checkpoint rollouts per candidate and selects cases whose valid verifier scores vary across attempts.This targets tasks where outcome feedback can distinguish alternative executions; environment and verifier errors are tracked separately.Source: §5.3.1 · pp. 51–52

5.3.2 Joint Rollout and Task-Relative Feedback

  • For each task group, isolated sandboxes give attempts the same request, initial state, verifier, and execution budget, and valid verifier outcomes supply terminal rewards.The Experience Adapter binds outcomes to recorded model calls and preserves task and attempt identity so GRPO compares attempts within the correct group.Source: §5.3.2 · pp. 52–53

5.3.3 Preserving Executed Decisions for Learning

  • The Experience Adapter preserves each response’s recorded decision-time input, starting a new segment when context history changes, and retains the complete attempt’s reward and learning weight.GRPO assigns one relative advantage per complete attempt, shared across its segments, while trajectory-mean loss aggregation prevents extra segments from increasing an attempt’s weight.Source: §5.3.3 · pp. 53–55
  • All segments of an attempt share one optimizer step, so segmentation does not increase the attempt’s policy-loss weight or create extra reward comparisons.The loss denominator covers the complete attempt across microbatches, and decision-time inputs are preserved to keep learning tied to executed decisions.Source: §5.3.3 · p. 55

5.3.4 Continuous Rollout and Policy Updates

  • Visual-delivery mechanisms reduce repeated preprocessing and tensor-transfer costs while retaining the complete visual input and preserving sample contents.For supported processors, unchanged screenshots reuse preprocessing tensors; lossless compression is retained only when smaller, and the learner decodes each microbatch to the original values, shape, and type.Source: §5.3.4 · p. 55
  • Asynchronous updates let completed groups supply the learner while other workspaces stay active, pausing new calls and waiting for in-progress calls before publishing weights.Ongoing tasks keep their workspace and completed work, allowing later decisions to use the updated policy; when attempts span policy versions, ratios use the corresponding recorded behavior probabilities, but the clipped objective is an off-policy approximation affected by policy lag.Source: §5.3.4 · pp. 55–56

6 Experiments and Analysis

This section examines NorthCUA’s experiments and analysis, tracing training-stage progress and evaluating performance across GUI grounding, desktop workflows, and Web tasks alongside reliability, recovery, and execution costs. It also describes the infrastructure, evaluation protocols, limitations, safeguards, related work, and release context surrounding the reported results.

6.1 Evaluation Setup

  • The evaluation uses SFT checkpoints for main grounding results and final RL checkpoints for desktop, WebArena, and WebTestBench results; Ultrafast starts from pre-RL Nano.The suite covers six benchmarks spanning grounding, desktop workflows, Web user tasks, and defect detection; protocols include 361 OSWorld-Verified tasks, all 108 OSWorld 2.0 tasks, and all 812 WebArena-Verified tasks.Source: §6.1 · pp. 56–57

6.2 GUI Grounding Results

  • 93.3% on SSP and 91.0% on OSWorld-G make NorthCUA-Pro the top grounding system among those compared, exceeding Claude Opus 5.5 by 5.2 and 1.1 points.Mini scores 92.4% and 87.9%, while Nano scores 72.9% and 83.0%, on SSP and OSWorld-G, respectively; the main results use long-horizon SFT checkpoints and direct SSP inference.Source: §6.2 · pp. 58–59, Table 6 · p. 59
  • For narrow grounding targets under 16 pixels, accuracy rises from Nano’s 53.4% to Mini’s 85.2% and Pro’s 92.1%; Pro also scores 90.2% on SSP icon targets.Pro’s gain over Mini is larger on the narrowest targets (6.8 points) than on targets at least 32 pixels wide and tall (1.2 points). Across three OSWorld-G draws, Pro averages 91.7% on visible-target items and a 75.9% correct-refusal rate on absent-target items; it is correct on all three draws for 489 of 564 items, compared with 469 for Mini and 445 for Nano.Source: §6.2 · pp. 58–60, Table 7 · p. 60

6.3 Desktop Agent Results

  • NorthCUA-Pro reaches 88.6% on OSWorld-Verified, the highest score among compared systems, while Mini scores 85.3% and Nano 78.9%.Pro exceeds Qwen-CUA-Max’s reported 87.6% by 1.0 points and Qwen3.8-Max’s 86.1% by 2.5 points. On OSWorld 2.0, Pro reaches 30.6% binary completion and 66.8% partial credit across 108 tasks, exceeding GPT-5.6 Sol by 3.2 and 4.1 points, respectively.Source: §6.3 · pp. 60–62, Table 8 · p. 61, Table 9 · p. 62

6.4 Web Agent Results

  • 73.9% on WebArena-Verified makes NorthCUA-Pro the highest-scoring system in Table 10; Mini reaches 70.4% and Nano 67.2%.Pro is 2.3 points above the joint-highest reference score of 71.6%, while Mini exceeds Qwen3.8-Max’s reported 66.8% and Nex-N2.5-Pro’s 67.6%.Source: §6.4 · pp. 62–63, Table 10 · p. 63
  • Pro exceeds 70% success in each of five single-site website groups, but completes only 17 of 48 cross-site tasks (35.4%), versus 583 of 764 single-site tasks (76.3%).Pro completes 274 of 374 state-changing tasks (73.3%). Its successful WebArena attempts have a median of ten decisions, while unsuccessful attempts have a median of 15.5; the report also illustrates recovery from an unavailable option while preserving a date constraint.Source: §6.4 · pp. 63–64
  • 60.3% F1 gives NorthCUA-Pro the highest WebTestBench defect-detection score, 2.1 points above GPT-6 Astra; Mini’s 54.4% is 5.8 points above same-size Nex-N2.5-mini.With the gold checklist supplied, Mini improves both precision and recall by 6.3 points over Nex-N2.5-mini; Pro has the highest precision at 67.8%, while GPT-6 Astra has the highest recall at 74.6%.Source: §6.4 · pp. 64–65, Table 11 · p. 65

6.5 Progress Across Training Stages

  • Long-horizon SFT raises desktop success by 7.4, 15.0, and 10.0 points for Nano, Mini, and Pro, while Web gains are largest for Nano at 9.2 points.On OSWorld 2.0, SFT increases partial scores by 16.7, 20.7, and 33.0 points for Nano, Mini, and Pro, respectively, and binary completion increases for every tier; Pro rises from 9.3% to 27.3%.Source: §6.5 · p. 65, Table 12 · p. 66
  • In two OSWorld 2.0 runs, long-horizon SFT shifted a source-paper task from 10% partial credit after 243 decisions to full credit after 124 decisions.The later run checked question-specific criteria, rejected a mismatched Q5 paper, and recovered the matching original from Trash.Source: §6.5 · p. 67

6.6 Joint Desktop–Web Reinforcement Learning

  • After joint RL, OSWorld-Verified gains were 2.9, 2.0, and 2.8 points and WebArena-Verified gains were 3.0, 2.5, and 4.3 for Nano, Mini, and Pro; Pro reached 88.6% and 73.9%.Desktop gains were similar across tiers, while Pro had the largest reported Web increase.Source: §6.6 · p. 67, Table 13 · p. 68
  • On OSWorld 2.0, partial credit rose 4.7, 3.8, and 3.2 points for Nano, Mini, and Pro; Pro reached 30.6% binary completion and 66.8% partial credit.Mini’s binary score remained 18.5% as its partial score increased from 46.3% to 50.1%.Source: §6.6 · p. 68
  • Lossless encoding reduced the summed image-tensor payload by 97.2%, leaving 2.8% of its original size while preserving screenshots and their values.Across 28 batches, median raw and encoded sizes were 110.9 and 2.7 GiB; synchronization timing includes a nested in-flight wait and is non-additive.Source: §6.6 · p. 68, Figure 29 · p. 69

6.7 An Evaluated Browser Recovery Case

  • After an unavailable Quarter option timed out, the Pro browser agent used Month with January 1–March 31 bounds, found Sprite, and earned evaluator success in 13 decisions.It inspected the dropdown, ran the report, and checked the remaining rows before answering.Source: §6.7 · pp. 70–71, Figure 31 · p. 71

6.8 NorthCUA-Ultrafast

  • Head + LoRA completes 492 WebArena-Verified tasks (60.6%), 11 more than Head-only, and retains 94.4% of Full generation’s 64.2% success rate.The 492-task result is 1.4 percentage points above Head-only; Head + LoRA uses about 0.5% trainable parameters and is 3.6 points below Full generation.Source: §6.8 · p. 73, Table 14 · p. 73
  • On the fixed 320-task timing subset, Head + LoRA reduced mean task time from 616.8 s to 266.8 s, a factor of 2.3, despite longer trajectories.Head-only and Head + LoRA bypassed generation for 40.2% and 31.1% of actions, respectively; the choice head handles finite decisions while generation remains available for text and other outputs. Head-only test route-and-target accuracy was 85.4%, versus 85.0% for Full generation.Source: §6.8 · pp. 71–73, Table 14 · p. 73

7 Related Work

  • Related work spans GUI grounding and policy learning, executable task and experience construction, reusable skills and search, and harnesses or infrastructure for execution and learning.NorthCUA is distinguished by integrating broad GUI mid-training, verified long-horizon SFT, joint desktop–Web RL, executable CaseSpecs, validated skills and workspace checkpoints, and decision-time input preservation with complete-attempt weighting.Source: §7 · p. 75
  • NorthCUA connects stateful visual execution and assessment to policy training through a shared desktop/web runtime, live outcome inspection, parallel lifecycle service, and an Experience Adapter that preserves recorded decisions.The same task and outcome interfaces link experience collection, assessment, and training samples.Source: §7 · p. 76

8 Limitations and Responsible Use

  • NorthCUA’s coverage remains challenged by unfamiliar or changing software, while its feedback depends on verifier predicate coverage and calibrated judgments for open-ended content.Documentation-guided grounding and skill- and milestone-guided collection broaden coverage; controlled verifier probes and evidence-linked judgments check feedback, while intermediate feedback may improve credit assignment and reduce unnecessary actions.Source: §8 · p. 76
  • Deployment safeguards include isolated workspaces, explicit tool profiles, and retained execution records, alongside authorized access, least-privilege credentials, and confirmation for consequential actions.The report also calls for application-appropriate adversarial testing against indirect prompt injections and licensing, privacy, and credential review of released materials.Source: §8 · p. 76

9 Conclusion

  • NorthCUA combines verified demonstrations and fresh policy interaction, while recursive self-improvement preserves procedures and milestones so later attempts can advance difficult workflows.The conclusion says improved policies can return to the same infrastructure to collect further experience, and that the three models, full Harness code, and report are released as an open foundation for future adaptation and development.Source: §9 · pp. 76–77
  • The passage mentions gains in task completion and says NorthCUA-Ultrafast explores a complementary route to faster decision-making.Source: §9 · p. 77
  • NorthCUA-Ultrafast explores a complementary route to faster decision-making alongside task-completion gains.Source: §9 · p. 77

A Implementation and Reproducibility

The implementation links each executable task, its workspace, recorded interaction, assessment, and learning samples through a shared attempt identity. Information and permissions are separated across construction, execution, and evaluation.

A Implementation and Reproducibility

  • A shared attempt identity connects the task, workspace, recorded interaction, assessment, and resulting learning samples.CaseSpec binds the request, materials, initial state, and execution settings; the Harness records interaction, and the verifier or evaluator establishes the outcome.Source: A Implementation and Reproducibility · p. 83

A.1 Information and Permission Boundaries

  • Information access is separated by role: construction metadata and host-only acceptance criteria are withheld from the acting policy.The policy receives the request and permitted task inputs, while tool checks constrain operations and extraction rules constrain sample information.Source: §A.1 · p. 83
  • Actor records preserve submitted results and decision-time inputs, while imitation excludes privileged evaluation calls.Evaluator demonstrations used in mid-training are separate judgment tasks with their own inspection context.Source: §A.1 · p. 83

B Grounding Data Details

The appendices specify how grounding labels and executable tasks are constructed, checked, and bound to environments. They also describe controls for valid task dependencies and reconstructed Web workflows.

B Grounding Data Details

  • Grounding labels link an unannotated screenshot and visible control to target geometry and one or more queries, with documentation supplying functional evidence.The GUI image supplies the target location; the annotation records are described as tracing labels from source evidence to the original screenshot.Source: B Grounding Data Details · p. 83, Table 15 · p. 83
  • Accepted grounding examples require actionable, visible targets, exclude queries needing hidden state, and keep diagnostic annotations separate from the unannotated input.Point targets must lie inside the actionable region without occlusion, and menu items must be visible in the captured menu state.Source: B Grounding Data Details · pp. 83–84
  • Documentation matches are accepted only when appearance, nearby controls, visible state, and manual applicability align; ambiguous or version-incompatible matches are excluded.Label checks distinguish a wrong function from a spatial miss, and paraphrase variants are grouped with their original target.Source: B Grounding Data Details · p. 84

C Task Construction Details

  • Executable tasks bind requests, materials, initial workspaces, and acceptance conditions, while dependency checks reject invalid graphs and keep host-only evaluation artifacts with evaluators.Tasks distinguish read-only inputs, distractors, mutable targets, and required outputs; actors retain freedom to choose GUI actions within prerequisites.Source: C Task Construction Details · p. 84

C.1 Environment Construction and Binding

  • The desktop builder filters materials by format and application needs, ranks them by content, and refines requests without changing their source goals or constraints.A request-material mismatch returns the task to construction; finalized workspaces retain paths, checksums, configuration, and startup actions.Source: §C.1 · p. 84
  • Dependency plans distinguish artifact data flow from required workflow order, so a task may require an independent result beyond its designated terminal artifact.In the example, C-to-D imposes order without making C's record a source of PDF content.Source: §C.1 · p. 84
  • Reference-based Web environments specify both visible pages and supporting state, then check rendering, navigation, persistence, initialization, and feasibility.Descriptions record the reference source, supported operations, observed versus generated content, backend schema and seed state, reset procedure, and external dependencies.Source: §C.1 · pp. 84–85

C.2 Hard-Task Collection Protocol

Hard-task collection preserves validated guidance and recoverable progress from unfinished tasks, while controlled verifier tests check whether acceptance judgments respond to task obligations rather than presentation alone.

C.2 Hard-Task Collection Protocol

  • Hard-task collection begins after ordinary attempts reveal useful obstacles, and candidate skills or milestones evolve while the task's final acceptance conditions stay fixed.Invalid environments, ambiguous requirements, and unreliable verifiers are repaired or excluded before further exploration.Source: §C.2 · p. 85
  • Skill records capture applicability, guidance, versions, source attempts, validation cases, and known failures, with validation separated from the attempts that produced the candidate.Task-specific assistance, reuse within a task family, and transfer to a new application are treated as distinct uses.Source: §C.2 · p. 85
  • Milestone search checkpoints preserve the actor's environment, action history, guidance, configuration, and remaining budget, while hidden verifier constants remain evaluator-only.Restoration re-establishes device bindings and obtains fresh observations; external service state requires separate isolation and restoration.Source: §C.2 · p. 85
  • Search budgets include failed attempts and validation costs, and the original task verifier—not local milestone checks—determines full-task acceptance.Selected recoverable failures retain their unresolved milestone label until continuation repairs that requirement.Source: §C.2 · p. 85
  • Training samples follow the selected actor path and retain the guidance that conditioned decisions; assisted paths, checkpoint continuations, and verified subtasks remain distinct sample types.Unassisted completion is evaluated in a fresh-start run using the original task inputs.Source: §C.2 · p. 85

C.3 Controlled Verifier States

  • Verifier tests vary submitted states while holding obligations fixed, using negative and positive controls, near misses, and valid alternatives to test decision sensitivity.The report example requires complete regional content, template compliance, and unchanged protected files; both a complete table and an equivalent chart are accepted, while missing output or regional detail is rejected.Source: §C.3 · pp. 85–86, Figure 34 · p. 86

D Training Data Sources and Scale

Sections D–E describe how supervised data are represented and selected, how grouped terminal feedback informs clipped policy learning, and how visual and sequence-handling mechanisms preserve experience integrity. The Ultrafast section converts WebAgent decisions into finite-choice predictions while retaining native generation for other outputs.

D Training Data Sources and Scale

  • The data inventory separates source materials, constructed tasks, attempts, accepted trajectories, extracted samples, and training consumption; counts are rounded, with grounding targets kept separate from trajectory interactions.Some SFT domain totals and the Web mean are estimated from rounded counts.Source: D Training Data Sources and Scale · p. 86, Table 16 · p. 87

E Model Identities and Learning Details

  • The model-identity table covers family identities and language-model parameter counts, while the training stages distinguish a NorthCUA Mid checkpoint from an upstream pretrained release.The supplied passages do not give the table’s individual parameter values.Source: Table 17 · p. 87, E Model Identities and Learning Details · p. 87

E.1 Supervised Representation and Sampling

  • Supervised learning maps grounding examples, task trajectories, and evaluator episodes into prediction units, with distinct source use and sampling weights across mid-training and long-horizon SFT.Selection requires completed-task judgments for broad trajectories; long-horizon trajectories also require full-task script acceptance and execution-quality and dependency checks.Source: §E.1 · p. 88

E.2 Task Groups and Terminal Feedback

  • RL groups attempts on equivalent task states under a shared verifier and budget, computes terminal advantages within each task group, and shares each attempt’s advantage across its segments.Invalid evaluations are separated from valid negative outcomes; identical group rewards yield zero terminal advantages.Source: §E.2 · p. 89

E.3 Clipped Policy Objective

  • The clipped policy update uses each segment’s captured behavior version, but asynchronous attempts spanning versions make the update an off-policy approximation under policy lag and changing state distributions.The recipe uses terminal feedback; local process rewards and distillation are separate extensions.Source: §E.3 · p. 89

E.4 Visual Experience Delivery; E.5 Sequence Integrity and Update Boundaries

  • Visual delivery reduces transfer volume without changing learner inputs, while sequence checks preserve response identity, media order, and segment boundaries when inputs or behavior versions change.Preprocessing reuse depends on processor compatibility, and equivalence checks compare captured inputs and likelihoods.Source: §E.4 · p. 90, §E.5 · p. 90

E.6 Sampling Likelihood Convention

  • The sampling-likelihood convention uses captured behavior support and recorded token identities and offsets for replay, rather than rebuilding a nucleus from current learner logits.Temperature, support capture, and replay settings are part of the run configuration.Source: §E.6 · p. 91

E.7 NorthCUA-Ultrafast Adaptation and Evaluation; E.7.1 Multimodal Inputs and Action Routing

  • Ultrafast trains on decision states from WebAgent trajectories: a finite-choice prompt retains task history and candidates, while native generation keeps the original messages, tool schemas, and template.The retained candidate set excludes disabled elements and is capped at 150 per observation; only the selected operation’s target is used for execution.Source: §E.7 · p. 91, §E.7.1 · p. 91

E.7.2 Readout and Training

Ultrafast adds a shared readout head that scores operation and target candidates from four representations. Its training objective averages supervised choice questions within each decision state, with dataset and adaptation settings specified for the reported variants.

E.7.2 Readout and Training

  • A shared head combines four candidate-specific representations through role and channel mixing, then scores candidates and applies softmax within each question.The representations capture context, question instruction, candidate, and decision marker; the head projects them to width 512 and uses two Mixer blocks.Source: §E.7.2 · p. 92
  • The choice objective averages supervised questions within each decision state, so states with both operation and target labels receive equal weight to operation-only states.Other target questions and original output tokens receive no generation loss.Source: §E.7.2 · p. 92
  • The dataset contains 225,710 training, 5,203 validation, and 6,087 test states from 25 trajectory collections, with fixed task-group holdouts and filtering of stale targets and invalid actions.Semantically related paraphrases and task templates may still occur across groups.Source: §E.7.2 · p. 92
  • Head + LoRA adds 43.278M adapter parameters to the 4.214M head, with adapters left unmerged so generation can disable them.The two variants share data, logical batch size, and seed, but use different completed budgets.Source: §E.7.2 · pp. 92–93, Table 19 · p. 93

E.7.3 Offline Decision and Action Results

In the offline test, Head-only slightly exceeds projected full generation on route-and-target accuracy, while complete-action matches favor full generation. The comparison separates route prediction from matching the full tool action or response.

E.7.3 Offline Decision and Action Results

  • 5,196 test states have correct Head-only route-and-target predictions versus 5,172 for projected full generation, but complete-action matches are 3,897 versus 3,933, respectively.The paired comparison has 3,757 states matching under both, 140 only under Head-only, and 176 only under full generation.Source: §E.7.3 · p. 93, Table 20 · p. 93
  • The hybrid gains 46 exact matches among 2,576 CLICK states but loses 25 among 906 FILL states and 30 among 2,087 NATIVE states.Losses on generation-dependent FILL and NATIVE actions exceed the CLICK gains.Source: §E.7.3 · p. 93
  • Among 1,403 states with a correct Head-only route but incorrect hybrid action, 935 have same-tool parameter mismatches and 359 have a tool or final-output kind mismatch.The remaining cases include 18 final-JSON-field differences and 91 route or target consistency rejections; these are reference matches, not environment success.Source: §E.7.3 · pp. 93–94
  • At completed epoch 13, Head + LoRA and Head-only achieve joint accuracies of 86.9% and 85.5%, respectively; the reported results use one training seed and states can share a task.Validation-selected checkpoints are epoch 1 for Head + LoRA and epoch 22 for Head-only.Source: §E.7.3 · p. 94

E.7.4 Online Task Completion and Timing

The online comparison reports task outcomes and timing for Ultrafast variants and full generation, followed by a browser-evidence protocol and software case studies. The cases illustrate sustained GUI work and preserve editable deliverables and acceptance records.

E.7.4 Online Task Completion and Timing

  • Of 812 tasks, evaluators assign success to 521 for Full generation, 481 for Head-only, and 492 for Head + LoRA; failures, timeouts, and evaluation errors remain in the denominator.Timing uses a fixed matched subset of 320 jointly successful tasks from the timing snapshot.Source: §E.7.4 · p. 94
  • On the 320-task matched timing set, Full generation has a measured mean task time of 616.8 s; the set contains 3,025, 3,260, and 3,386 steps for Full generation, Head-only, and Head + LoRA.Task time includes the execution loop through terminal observation and result persistence, but excludes provisioning, teardown, and evaluator scoring.Source: §E.7.4 · pp. 94–95
  • Direct choice-head decoding bypasses generation for 40.2% of Head-only dispatched actions and 31.1% of Head + LoRA dispatched actions.These fractions use available actions from successful and failed tasks, not the 320-task timing subset.Source: §E.7.4 · p. 94

F.1 Qualifying Browser Evidence

  • Browser repair evidence supports a product-defect candidate only after the relevant test actions complete; unresolved execution failures instead call for diagnosis or bounded retesting.Final acceptance uses fixed checks from a protected clean state outside the repair loop.Source: §F.1 · p. 95, Figure 35 · p. 96

G NorthCUA Case Studies

  • The case studies trace complex GUI tasks across geometry, data analysis, visual composition, circuit simulation, and gameplay, using external controllers and preserving traces and native outputs.These demonstrations are separate from checkpoint evaluations.Source: G NorthCUA Case Studies · p. 96

G.1 Ruby Heart

  • The GeoGebra Ruby Heart workflow checks 18 combinations of depth, slice position, and pulse phase and ends with a saved editable construction after 440 recorded steps.The saved construction supports continued interaction with the geometry established during modeling.Source: §G.1 · pp. 96–97

G.2 Silicon Atlas

  • The Silicon Atlas classifies 1,071 operating groups from 2,142 sweep rows and retains 573 passing, 489 failing, and nine unknown groups in its audit.After 928 steps, the workbook and plotting project preserve calculations and editable plots alongside PDF and PNG outputs.Source: §G.2 · p. 97, Figure 37 · p. 99

G.3 NOVA Presentation

  • The NOVA presentation workflow checks transferred chart data by reading it back, finding the same mean of 17.6268 at 5 ms in both views.The 691-step task ends with an editable presentation, calculation workbook, and matching PDF; NOVA uses illustrative fictional-product data.Source: §G.3 · p. 97, Figure 38 · p. 100

G.4 CMOS Inverter Analysis

This section documents the workflows, resources, and protocols used to produce and preserve inspectable outcomes, admit examples to training cohorts, and evaluate interactions. It covers execution and admission procedures alongside the evaluation settings and scoring conventions underlying reported results.

G.4 CMOS Inverter Analysis

  • At a nominal 0.5 V input, the CMOS inverter’s device reports showed drain currents of +5.778 µA and −5.778 µA, and its transfer-curve crossing was estimated near 0.498 V.The crossing is interpolated from the 5 mV sweep; the agent corrected an uncommitted bias edit by entering 0.5 V, pressing Enter, and rerunning the analysis.Source: Figure 39 · p. 101, §G.4 · p. 102

G.5 TIDAL-01 Modeling and Animation

  • Across 1,805 recorded steps, the TIDAL-01 workflow produced a packed Blender scene and a 12-second, 288-frame animation at 24 fps, with no missing files on reopening.The workflow moved from an Inkscape sketch to editable geometry, revised glass materials, an articulated iris, and a tracked camera arc.Source: §G.5 · p. 102, Figure 40 · p. 103

G.6 Zuma Gameplay

  • After 199 GUI actions, the agent completed the six-color Zuma task and reached native STATS with 9,070 points.The run progressed through a six-shot chain bonus and cleared the remaining groups after navigating the board’s bend and bottom lane.Source: §G.6 · p. 104, Figure 41 · p. 104

H.1 Runtime Configuration

  • Runtime configuration controls which interaction history enters model calls, while separating actor actions, evaluator inspection, and terminal evidence for attribution.Profiles govern visual history, text windows, and compaction; failed refreshes require state inspection before retry, and potentially mutating evaluator checks require state-preserving procedures or remain unresolved.Source: §H.1 · pp. 105–106

H.2.1 Desktop Agent

  • Desktop tools act on the current screen and focused application, with coordinate mapping governed by the run’s declared policy and distinct schemas for actions such as keypress, wait, and screenshot.A step may contain multiple tool calls, so model decisions, tool calls, and GUI interactions are separate counts.Source: §H.2.1 · p. 106, Table 21 · p. 106

H.2.2 Web Agent

  • Web tools target elements from the latest page observation or viewport-relative CSS coordinates, while element references are refreshed with page state and tab references identify browser tabs.The web action space includes reference-based and point-based interactions, page waits, screenshots, and tab operations.Source: §H.2.2 · p. 107, Table 22 · p. 107

H.2.3 Evaluator

  • Evaluator interfaces retrieve recorded evidence and, when authorized, inspect the evaluated workspace; state-changing inspection requires explicit permission and follows the attribution protocol.Evaluation concludes with a final report, with web reports additionally recording the final URL and title, verification flags, blocker category, and critical evidence references.Source: §H.2.3 · pp. 108, 110, Table 23 · p. 109, Table 24 · p. 110

H.3 Batch Collection and Resource Ownership

  • Task preparation binds each CaseSpec to its inputs, initialization, application and tool profile, budgets, and optional checks; resumed work retains the original task and configuration identity.Supported context-managed settings are retained in batch-local snapshots, while external dependencies keep separate version records.Source: §H.3 · pp. 110–111
  • Collection separates scheduler claims from device leases, checks provider lifetime against task timeouts, and retains resources until live evaluation and output capture finish.After sandbox destruction, replay is limited to retained artifacts; unconfirmed cleanup remains unresolved and continues to incur resource cost.Source: §H.3 · p. 111

H.4 Cohort Admission

  • Completion demonstrations require actor-only extraction, aligned actions and context, required artifacts, full-task acceptance, and cohort quality checks; online RL retains valid negative outcomes within task groups.Broad judge-filtered mid-training uses a separate selector, and evaluator demonstrations are separate judgment-task samples.Source: §H.4 · p. 111, Table 26 · p. 112

I Benchmark Evaluation Details

  • Interactive evaluation uses benchmark scorers independently of collection-harness evaluation, with OSWorld-Verified using 361 supported tasks and OSWorld 2.0 using all 108 tasks.Grounding records specify the OSWorld-G instruction variant and absent-target convention, and map crop/resize predictions back to annotation coordinates.Source: I Benchmark Evaluation Details · p. 112
  • WebTestBench uses a supplied gold checklist and task-level macro-averaged precision, recall, and F1, with Fail as the positive class; reported NorthCUA tiers use final post-joint-RL checkpoints.The task-level F1 average generally differs from the harmonic mean of macro precision and recall.Source: I Benchmark Evaluation Details · p. 112
  • Published-baseline notes record task scope, permissions, budgets, and scoring conventions, while NorthCUA stage comparisons use declared benchmark tasks, tools, observations, and execution budgets.Parameter counts distinguish total from activated language-model parameters.Source: I Benchmark Evaluation Details · p. 112
  • Repeated-prediction reporting covers run-to-run variation and pass@3, while SSP tables break accuracy down by application form and annotated target extent or type.The SSP target-extent/type table’s six disjoint rows together cover the full evaluation.Source: I Benchmark Evaluation Details · pp. 112–113, Table 27 · p. 113, Table 28 · p. 113

I.1 Grounding Scores and Repeated Sampling

NorthCUA Pro leads on grounding, with repeated sampling and item-level analyses showing both improved accuracy and a smaller but persistent set of errors. Remaining errors cluster around certain icon sizes and vary across target geometries and application groups.

I.1 Grounding Scores and Repeated Sampling

  • Mini and Pro share many successes but retain distinct wins: Pro is correct on 102 items Mini misses, while Mini is correct on 88 items Pro misses.The paired comparison also records 18 items missed by both and a net Pro gain of 14 correct items.Source: §I.1 · p. 113
  • Pro’s 106 remaining errors are valid coordinates outside target boxes; icons account for 59, with the 16–<32 pixel band contributing 43.4% of all Pro errors.Pro reaches 93.0% accuracy on icons smaller than 16 pixels; its largest gains over Mini are in CAD (3.1 points) and scientific software (2.4 points), while development software is 1.0 point lower.Source: §I.1 · p. 114
  • 95.6% SSP pass@3 is 2.3 percentage points above Pro’s single-run result, while Pro raises single-run accuracy by 0.9 points over Mini.Table 29 reports grounding summaries, including SSP results.Source: §I.1 · p. 114, Table 29 · p. 114
  • Pro persistently misses 34 items, accounting for 102 of its 166 incorrect predictions (61.5%); 16 items remain persistently incorrect across all three tiers.Mini and Pro share 17 persistent misses, while Pro succeeds on 489 items in all three draws.Source: §I.1 · p. 114
  • On rectangular targets, Pro averages 91.1% versus Mini’s 87.9%; on polygonal regions, Pro reaches 98.3% versus Mini’s 97.5%.The geometry groups cover all 564 items; their labels denote scoring regions, not semantic task categories.Source: §I.1 · pp. 114–115, Table 31 · p. 115

I.2 Interactive Results and Trajectory Breakdowns

Final NorthCUA RL checkpoints complete Web and desktop tasks at progressively higher rates across model tiers. The trajectory analyses show that outcomes vary with task scope and execution length, while successful long runs are part of the training signal.

I.2 Interactive Results and Trajectory Breakdowns

  • 73.9% of WebArena-Verified tasks are completed by Pro, versus 70.4% for Mini and 67.2% for Nano, on final RL checkpoints.Pro’s score covers one complete evaluation of all 812 tasks.Source: §I.2 · p. 115
  • Earlier SFT evaluations report WebArena-Verified pass@3 of 69.3% for Nano and 73.9% for Mini.Figure 27 includes Mini’s SFT website-level breakdown alongside Nano SFT and Pro RL.Source: §I.2 · p. 115
  • Pro completes 75.4% of retrieval tasks, 73.3% of state-change tasks, and 71.7% of navigation tasks; cross-site completion is 17 of 48 tasks.Mini has 13 more state-change successes and 19 more retrieval successes than Nano, with nearly identical navigation counts.Source: §I.2 · p. 116
  • For Mini, cross-site tasks have a 25.0% completion rate and a median of 27.5 decisions, versus 70.7% and 10 decisions for single-site tasks.Even successful cross-site tasks have a median of 17 decisions, compared with nine for single-site tasks.Source: §I.2 · p. 116
  • Failed WebArena tasks have longer median decision counts than successful ones for Nano, Mini, and Pro, and account for 58.5%, 50.6%, and 38.4% of their recorded decisions, respectively.The learning pipeline retains successful long workflows for supervision, while RL also uses unsuccessful attempts for negative feedback.Source: §I.2 · pp. 116–117
  • On OSWorld-Verified, final RL scores are 78.9% for Nano, 85.3% for Mini, and 88.6% for Pro.On OSWorld 2.0, Pro’s Binary/Partial scores are 30.6/66.8, compared with Mini’s 18.5/50.1 and Nano’s 13.0/45.6.Source: §I.2 · p. 117

I.3 External Comparison Settings

The comparison appendix documents the scope and settings behind published benchmark scores, a controlled Web repair protocol, and an execution case spanning different OSWorld 2.0 revisions.

I.3 External Comparison Settings

  • Additional OSWorld-Verified references span model generations, including 86.1% for Qwen3.8-Max and 83.2% for GPT-5.6 Sol.The later Opus 5.5 results use updated September task assets and a changed harness configuration, so they are kept separate from the OSWorld 2.0 main table.Source: §I.3 · pp. 117, 119
  • WebArena-Verified references distinguish published scores from the report’s own evaluations, while WebArena scores from other sources may use different settings.The cited published scores include 71.6% for Kimi-K3 and 62.3% for DeepSeek-V4-Flash-Vision; starred entries identify the report’s evaluations.Source: §I.3 · pp. 118, 120
  • Published comparisons retain task scopes, metrics, and aggregation rules, with OSWorld 2.0 main-table references matched to the full 108-task setting where available.For example, GPT-5.6 Sol averages 27.3% binary and 62.7% partial over two runs, while Claude Opus 5 averages 31.4% and 68.3% over seven runs.Source: §I.3 · pp. 117–118, Table 34 · p. 119

I.4 Web Repair Evaluation Protocol

  • The Web repair comparison fixes the issue set, starting patch, agents, tests, adjudication, budgets, and acceptance conditions, then evaluates the final patch from a protected clean state.Acceptance uses the assigned issue set as its denominator, retains tester noncompletion and unresolved issues, and withholds final-run outputs from further revision.Source: §I.4 · p. 120

J OSWorld 2.0 Execution Cases

  • The paired execution case uses identical requests across different dataset revisions, with the earlier run lacking OCR and the later run using it.The case follows source selection and verification in each execution.Source: J OSWorld 2.0 Execution Cases · p. 121

J.1 Preserving a Delivery Constraint Across Long Contexts

Two execution cases show NorthCUA preserving task constraints through long context and recovering from an application export failure by scripting through the GUI-accessible terminal.

J.1 Preserving a Delivery Constraint Across Long Contexts

  • After payment matching, the agent preserves Emily’s $250 balance and preview-only rule through compaction, sends the watermarked preview, verifies the reminder, and earns full credit in 243 decisions.The task distinguishes Emily’s partial payment from Sarah’s fully paid invoice; the final image inspection confirms the watermark.Source: §J.1 · p. 121, Figure 42 · p. 122

J.2 Using a Terminal Through the GUI

  • When Blender’s direct export rejects WEBM and FFMPEG settings, the agent renders 120 PNG frames and encodes them with terminal-based ffmpeg as a 24-fps, 960 × 540 VP9 video.GUI playback verifies the result, which receives full credit after 147 decisions and 253 tool calls.Source: §J.2 · pp. 122–123, Figure 43 · p. 123
Loading 2611.02102v1…