Sage-A1.5: From Long-Horizon Execution to Scientific Exploration with Capability Composition

• Figure 1 presents benchmark performance for Sage-A1.5, but the supplied text does not specify its benchmark labels or explain the listed values.Source: Figure 1 · p. 1, SWE-Pro · p. 1

1. Introduction

• Longer execution can still involve repeated unproductive searches, unsupported assumptions, or premature conclusions, motivating learned delegation, evidence review, and strategy revision.Source: §1 · p. 3
• Sage-A1.5 outperforms larger models on four named benchmarks, remains competitive on two others, and matches or surpasses same-sized models across twelve benchmarks.Source: §1 · p. 3
• The model adds Agentic Orchestration for delegating parallel subtasks and combining findings, and Agentic Verification for checking evidence and adjusting actions within its own reasoning process.Source: §1 · p. 3
• The contributions include Adaptive Agentic Execution, unified multi-domain training, and scientific-discovery demonstrations through benchmarks and case studies.Source: §1 · p. 3
• The report also presents a three-stage training pipeline that builds multi-domain teachers and unifies them through outcome-process post-training.Source: Figure 2 · p. 4

2.1. Adaptive Agentic Execution: From Capability Coverage to Capability Composition

• Capability composition concerns how problem-solving capabilities are organized and adapted during execution, complementing domain-specific capability coverage.Source: §2.1 · p. 4, Figure 3 · p. 5

2.1.1. Agentic Orchestration

• Orchestration makes execution structure an explicit training target, complementing learning to solve individual tasks.Source: §2.1.1 · p. 4
• The model can explore alternative approaches to one question or delegate complementary subproblems, then integrate outputs as evidence informs decomposition and follow-up.Source: §2.1.1 · p. 5
• Orchestration tasks test parallel investigation and research synthesis, including whether findings cover the objective and remain consistent before report integration.Source: §2.1.1 · pp. 5–6
• For long-horizon search, relation chains around a shared answer yield complementary constraints that agents investigate in parallel.Source: Figure 4 · p. 6, §2.1.1 · p. 6
• Subagents return results with self-assessments, supporting evidence, and remaining issues, helping the main agent identify gaps or conflicts.Source: §2.1.1 · p. 6

2.1.2. Agentic Verification

• Verification checks evidence and prior judgments when progress stalls or an important conclusion needs confirmation, then adjusts subsequent actions within the model’s reasoning.Source: §2.1.2 · p. 7
• Training trajectories combine intermediate reflections with follow-up actions; external supervision constructs the data, while the model performs verification independently at inference.Source: §2.1.2 · pp. 7–8

2.2.1. MLE Tasks

• Kaggle-MLE Factory converts historical competitions into executable tasks with public workspaces, hidden tests, and a unified evaluation interface, followed by human review.Source: §2.2.1 · p. 8
• The MLE interaction harness tracks candidate solutions and execution results in an experiment tree, supporting branch comparison, backtracking, and diagnosis by an independent-context subagent.Source: §2.2.1 · p. 8
• MLE trajectories are selected using execution results, with degrading tails truncated near the best submission when performance drops beyond a threshold.Source: §2.2.1 · p. 8
• Windowed Interaction Reinforcement Learning groups interactions between experiment results into windows that share outcome-based rewards and permit policy updates before task completion.Source: §2.2.1 · p. 9

2.2.2. SWE Tasks

• SWE tasks pair repository snapshots with issue descriptions and evaluate patches on held-out tests, requiring designated failures to pass while preserving previously passing tests.Source: §2.2.2 · p. 9
• SWE task construction combines public and in-house instances, prepares repository environments, and validates candidate tasks with reference-patch and no-op checks.Source: §2.2.2 · p. 9
• Rollouts use mini-swe-agent or Claude Code in isolated containers, and zero-reward patches are replayed once to recover successes affected by transient failures.Source: §2.2.2 · p. 10
• SWE collection restricts access to reference fixes and held-out tests during agent execution, restoring required Git objects only after the agent exits.Source: §2.2.2 · p. 10

2.3. Scientific Research

• The three-stage pipeline selects scientific problems, optimizes research tools, and distills and trains on filtered long-horizon trajectories.The stages integrate problem selection, robust tool interaction, and model training.Source: §2.3 · p. 11
• Problems are prioritized when they are medium-to-high difficulty, reasoning- or research-oriented, and likely to benefit from external tools.Selection assesses difficulty, task type, and potential tool benefit across search, visit, code, and scholar.Source: §2.3 · p. 11
• The research harness combines web search, page and document inspection, code execution, and academic literature retrieval.An isolated sandbox restricts host-local access, while replaying recorded retrieval results reduces redundant calls and supports stability and reproducibility.Source: §2.3 · p. 11
• Filtered demonstrations retain correct, deep, effective tool-use trajectories, including recoverable failures followed by successful strategy changes.The selected trajectories are used for token-level supervised fine-tuning on Qwen3.5-35B-A3B with an extended multi-epoch schedule.Source: §2.3 · pp. 11–12

2.4. Recursive Task Evolution for Long-Context Reasoning

• Question rewriting can leave evidence dependencies and solution structure unchanged, so recursive evolution targets changes in evidence organization and reasoning structure.Reference-model feedback exposes shortcuts that guide subsequent revisions.Source: §2.4 · p. 12
• Tasks draw on LongBlocks documents of 16K–128K tokens and 24 categories, with fixed objectives and explicit evidence scopes and answer formats.Categories include cross-document integration, timeline reconstruction, and version-impact analysis.Source: §2.4 · p. 12
• Difficulty enhancement composes atomic capabilities across ten axes while keeping the source document and task objective fixed.Axes include search-space size, distractor strength, evidence non-locality, state-transition depth, and shortest-derivation depth.Source: §2.4 · p. 12
• The process yields 3,512 hard QA pairs, on which Qwen3.5-35B-A3B has a mean per-question success rate of 31.57%.Candidates undergo evidence alignment, two-stage review, and difficulty assessment; eligible hard-pool candidates have empirical success rates of at most 0.75.Source: §2.4 · p. 12
• 1,949 LongBlocks trajectories are retained for supervised fine-tuning after joint quality review and sequence-length constraints.Three DeepSeek-V4-Pro trajectories are sampled per task; reviewers assess evidence coverage, retrieval, verification, consistency, and redundancy, repeating assessment with candidate order reversed.Source: §2.4 · p. 13

2.5.1. Search Agents

• Open-source search datasets emphasize short answers with single verifiable answers, providing limited supervision for broad exploration and evidence-grounded synthesis.The section adds open-ended research questions paired with explicit evaluation criteria.Source: §2.5.1 · p. 13
• Research questions are collected or generated through web exploration, with criteria grounded in discovered evidence and specifying aspects a satisfactory answer should cover.Collected trajectories use multi-turn search and reading to produce reports with supporting sources; criteria-based evaluation filters incomplete or unsupported reports for supervised fine-tuning.Source: §2.5.1 · pp. 13–14

2.5.2. Economically Valuable and Real-World Tasks

• Agents’ Last Exam evaluates long-horizon tasks across 55 subdomains and 13 industry clusters using real inputs, hidden references, and deterministic grading.ALE-CLI is the subset used to evaluate general agentic capability.Source: §2.5.2 · p. 14
• A manually seeded pool grows to more than one thousand professional workflow environments through controlled evolution and separate builder, reviewer, and judger roles.Hidden references are executed on final inputs as a solvability check, and isolated environments reproduce the assumed software and separate solver execution from evaluation.Source: §2.5.2 · p. 14
• Teacher trajectories are collected in per-task sandboxes, and the highest-scoring attempt is selected while a small share of substantive failed attempts is retained.Long runs may continue from compressed interaction histories; scoring uses a separate sandbox, and the final set balances tasks and domains.Source: §2.5.2 · pp. 14–15

2.6. Unified Training across Domains, Modes, and Harnesses

• Multi-domain SFT combines task-oriented demonstrations with protocol-focused cold-start examples from compatible harnesses.Harnesses include Claude Code, Codex, and internal configurations; protocol-focused examples emphasize tool use, subagent dispatch, and message routing.Source: §2.6 · p. 15
• Tool outputs, environment feedback, and harness-injected text remain in context but are excluded from the token-normalized SFT loss.The loss targets model-generated tokens, with preceding context including observations and harness metadata.Source: §2.6 · p. 15
• The SFT objective mixes task-oriented and protocol-focused examples, with ρ denoting the total sampling mass assigned to protocol-focused examples.The subsets differ in demonstration content while sharing the same loss and token mask.Source: §2.6 · p. 15
• Unified post-training combines outcome-based RL and multi-teacher on-policy distillation on student-generated trajectories, using outcome-only, process-only, or mixed supervision.Signal weights reflect each sample’s interaction profile and signal availability and cost; process supervision is token-level teacher feedback rather than a separately trained reward model.Source: §2.6 · pp. 15–16
• Outcome checks use task scores and penalties, while process-only supervision applies teacher feedback to the student’s visited context.Outcome scores may reflect constraint satisfaction, reference agreement, hidden-test performance, or a native environment metric; these checks do not establish correctness of every intermediate reasoning step.Source: §2.6 · pp. 15–16
• Student rollouts use compatible harnesses and shared trajectory formats, with outcome checks on executed states and teachers scoring student prefixes under matching tool semantics.Asynchronous generation, adaptive sampling, and cross-domain evaluation address varied rollout durations and monitor interference and harness sensitivity.Source: §2.6 · p. 16

2.7. Agent Training Infrastructure

• Agentic orchestration and verification require many evolving execution contexts, but their compute demand is intermittent.pVisor provides default sandbox environments for agentic rollouts, with file and network interactions visible to the host for access control and resource management.Source: §2.7 · p. 16
• pVisor loads shared dependencies as needed and retains fixed environment versions, workspace code, and experimental outputs for continued exploration.Access policies govern file and network operations, and workspace changes remain available for review.Source: §2.7 · pp. 16–17
• pVisor can pause execution, offload memory, and resume from the same point while preserving the agent’s tools and control flow.An experimental integration advances model requests during waits, separating rollout lifetime from continuous residency of the full environment.Source: §2.7 · p. 17
• Approximately 80% lower resident memory followed offload in preliminary static tests, with one run per condition.The comparison used the same application working set; actual savings depend on workload and the costs of managing requests and restoring execution.Source: §2.7 · p. 17

3.1. Evaluation Setting

• Evaluation spans search, repository engineering, and scientific reasoning and research, using benchmark-specific protocols and graders.The comparisons include open 35B models and T-level models; missing results are evaluated with the same protocol as Sage-A1.5.Source: §3.1 · pp. 17–18, Table 1 · p. 19
• General and scientific-agentic evaluations use strict or task-specific scoring across long-context, instruction-following, real-world agent, chemistry, materials, and document-parsing tasks.ALE-CLI runs independently in clean sandboxes; MatTools reports success across 138 subtasks, and OmniDocBench combines text, table, and formula scores.Source: §3.1 · pp. 18–20

3.2. Results and Observations

• The science-enhanced teacher beats Qwen3.5-35B-A3B across all six scientific benchmarks, with especially large gains on research-level tasks.Scores rise from 2.5 to 70.0 on FS-R, 20.0 to 84.0 on CMT, and 0.9 to 25.7 on CritPt. FS-O improves from 64.5 to 89.0, HiPhO from 37.0 to 49.7, and HLE with tools from 47.4 to 56.8.Source: Table 2 · p. 20, §3.2 · p. 20
• The search-enhanced teacher outperforms Qwen3.5-35B-A3B across all four search benchmarks.Accuracy improves from 59.8 to 96.1 on GAIA and from 61.0 to 85.5 on BrowseComp. XBench-DS-2510 rises from 77.0 to 89.0, and DeepResearch Bench II from 42.3 to 48.2.Source: Table 3 · p. 20, §3.2 · p. 20
• Agentic Orchestration improves Sage-A1.5 on BrowseComp and DeepResearch Bench II, while the teacher achieves the highest report-generation score.Sage-A1.5 rises from 81.7 to 82.8 accuracy on BrowseComp and from 43.7 to 47.5 on DeepResearch Bench II; the teacher scores 53.6 on DeepResearch Bench II.Source: §3.2 · pp. 20–21, Table 4 · p. 21
• Agentic Verification raises BrowseComp accuracy to 85.5, a 4.5-percentage-point gain and the highest accuracy among compared configurations.In paired attempts, average output tokens per task fall from 23,245 to 21,962 and average interaction turns from 84.81 to 82.14.Source: Table 5 · p. 21, §3.2 · p. 21
• Sage-A1.5 achieves a 19.0% strict pass rate on ALE-CLI, ranking fourth overall and first among evaluated 35B-class models.It scores 90.0 overall on another reported general benchmark, ranking first among evaluated systems and ahead of Kimi-K3, GLM-5.3, and GPT-5.6-sol.Source: §3.2 · pp. 21–22
• The MLE teacher achieves a 72.7% medal rate on MLE-Bench Lite, versus 37.9% for its base model, a gain of 34.8 percentage points.Its gold medal rate reaches 50.0%, more than twice that of the base model.Source: Table 6 · p. 21, §3.2 · p. 21

4.1.1. Agentic Orchestration

• The main agent dynamically followed findings across 50 channel investigations, using parallel subagents and follow-ups to distinguish commercial partnerships from ownership and correct the StarPlus operator.Six batches of subagents gathered evidence independently; the main agent incorporated the corrected Disney Star / JioStar operator into a structured report.Source: §4.1.1 · p. 23

4.1.2. Agentic Verification

• Sage-A1.5 self-initiated verification at turn 20, shifted to targeted searches, and returned the accepted answer variant Madhumati; the baseline returned incorrect Jacqueline after 48 turns.The agent identified unsupported assumptions and evidence gaps, while the baseline selected Tom Selleck despite unresolved evidence and used a television role for the film-role clue.Source: §4.1.2 · pp. 24–25

4.1.3. Local Automated Deployment

• FP8 and 4-bit quantized variants support deployment across heterogeneous local hardware, while local runtimes and tools enable on-premise workflows without external cloud inference.The report describes reduced memory requirements, local inference APIs, and local code execution, file access, and document retrieval for privacy-sensitive, enterprise, or offline use.Source: §4.1.3 · p. 26

4.2. Scientific Information Integration and Long-Horizon Reasoning

• Ten numerical tests reduced the quantum-optics calculation’s maximum residual from discrepancies up to 24.16 to approximately 3.55 × 10−15, with consistency checks across three physical limits.Sage-A1.5 found an incorrect complex conjugation in the correlation cross-term and corrected its numerical implementation; Claude Opus 4.8 retained incorrect extremum expressions on the same task.Source: §4.2 · p. 27

4.3. From Scientific Co-working to End-to-end Auto-Research

• Across ten professional workflows and three scientific domains, Sage-A1.5 produced inspectable deliverables, including verified engineering outputs and analysis workspaces with code, results, figures, and reports.The hydrogen analysis estimated a cost crossover near EUR 5.50/kg at approximately 7.3% cost of capital; gravitational-wave median differences increased 7.6-fold from ℓ=2 to ℓ=8, and atomistic adsorption fits had slopes of 0.637 and 0.676.Source: §4.3 · pp. 27–29

4.4. Solving Scientific Problems aligned by Scientists

• The neuroscience reanalysis reproduced shared brain-wide decoding structure, while iterative spot-detection revisions raised aggregate_improvement from −0.0469 to 0.1158.The PC1 explained 63.2% of standardized decoding-metric variance versus a 24.0% permutation mean; movement explained approximately 60% of across-region variance in choice decoding. The detection pipeline combined U-Net predictions with a Laplacian-of-Gaussian fallback and was refined using evaluation feedback.Source: §4.4 · pp. 29, 30, 33, 34
• A scientist-guided quantum-chemistry workflow completed in approximately 12 hours, while crystal refinement accepted six updates and reduced Rwp from 88.3091% to 86.1092%.The chemistry workflow recovered failed calculations and found stronger-than-predicted 2-methylpyridine binding; the refinement used independent geometric and residual checks, though substantial mismatch limited structural-accuracy conclusions.Source: §4.4 · pp. 30–32
• In a replay of historical electrochemistry data, feedback revised mechanistic hypotheses and identified EXP-038 as the highest recorded condition at 89% yield, without establishing causal effects.The five batches had cumulative best yields of 32%, 32%, 43%, 89%, and 89%; the proposed HFIP effects remained hypotheses rather than demonstrated causal relationships.Source: §4.4 · p. 33
• A retained-run DNA-FISH overlay compares predicted spots with reference annotations, showing matched detections, unmatched predictions, and missed spots alongside benchmark results.The case illustrates how Sage-A1.5 uses evaluation feedback to revise its image-analysis pipeline and progressively improve detection performance.Source: §4.4 · p. 34

5. Limitation and Future Work

• Sage-A1.5 is presented as a practical route for a compact open model to tackle complex, multi-stage engineering and scientific problems.The authors also plan to extend orchestration to broader engineering workflows.Source: §5 · p. 34
• Current orchestration and verification training centers on search and research synthesis, while ultra-long trajectories can accumulate heavy context overhead.Future work proposes compressing completed exploration history into concise stage records at intermediate verification points.Source: §5 · p. 34
• The unified model does not yet fully retain every domain teacher’s peak performance after joint post-training.The authors propose reducing cross-domain distillation loss.Source: §5 · p. 34
• Scientific showcases remain limited to computational simulations and historical data replays rather than active physical experiments.The authors plan to work with scientists on real-world experimental validation.Source: §5 · p. 34