Source-linked AI summary
Iris: Climbing to the Search Frontier
Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang, Xu Chu, Shaowei Chen, Yuantao Gu, Mu Chuan
TL;DR
Training search agents calls for questions that require evidence gathered across sources, yet naturally occurring questions rarely meet that need and hand-written questions are costly to scale. Iris builds and trains agents on reverse-constructed web tasks, and Iris-pro leads or matches the best open-source result across four benchmarks in the ∼400B parameter range.
Problem
Naturally occurring web questions rarely both exceed parametric memory and require composing evidence across sources, while hand-written questions are expensive and hard to scale.
Source: §2 · p. 3Method
Results
88.6 on BrowseComp: Iris-pro records the highest score in the ∼400B range, 3.8 points above XYZ-Aquila-pro, and leads or matches the best on all four benchmarks.
Source: §4.2 · p. 7Takeaways & Limitations
Search data and search-specialized models transferred positively to untargeted tool-use and cowork domains, suggesting search behaviors may be reusable beyond web browsing.
Source: §5 · p. 9Retry can produce higher scores but carries substantial inference cost, so it is not the primary configuration for reported model performance.
Source: §4.3 · p. 9
Abstract
from arXiv · showhide
We present Iris-mini and Iris-pro, two search agents trained at the 35B-A3B and 397B-A17B scales, together with the data pipeline and training recipe behind them. Tasks are reverse-constructed from the hyperlink structure of a web corpus: we author multi-hop chains over an entity graph distilled from a seed page and its out-links, rewrite every non-answer entity into a descriptive reference so that no clue can be resolved by string matching, and admit only questions that a reference model fails closed-book yet solves once the supporting evidence is supplied. These questions are then turned into trajectories, which are filtered at both the trajectory and the turn level before SFT. The policy is then optimized by RL against live search, with the reward judge and the observation summarizer served inside the training cluster, and with over-long rollouts interrupted at the request level and resumed from their committed prefix at the next step. We alternate the two stages in a procedure we call SFT-RL climbing, returning the hardest solved and most efficient rollouts of each RL round to the next supervised pass. Because inference-time context management is worth more on these benchmarks than most reported differences between systems, we evaluate every benchmark both with and without it, holding the tool set, the context limit, and the judge fixed. All results come from a single ReAct agent, with no sub-agents and no test-time verification. With management enabled, on BrowseComp, BrowseComp-ZH, DeepSearchQA, and HLE the two models reach $82.2/84.8/86.9/52.3$ and $88.6/85.1/92.9/56.4$, the strongest overall results among open-source search agents in their respective parameter ranges. We plan to release the model weights together with the complete recipe for data construction, training, and evaluation.
1 Introduction
The introduction presents Iris as an end-to-end recipe for training and evaluating search agents, emphasizing that context management can materially affect reported performance. It reports strong results for Iris-mini and Iris-pro across four challenging benchmarks.
1 Introduction
- Context management can substantially affect long-horizon search performance, so managed-context results alone can obscure the policy’s contribution.Its benefit can diminish with larger context windows or fewer interaction steps.Source: §1 · p. 2
- The training pipeline constructs and verifies challenging multi-hop tasks from web structure, then filters teacher trajectories at both trajectory and turn levels.Entity rewriting removes easily searchable anchors, and questions are retained only when sufficiently difficult and objectively verifiable.Source: §1 · p. 2
- Iris combines filtered supervised fine-tuning with live-web reinforcement learning, then returns successful RL trajectories to later supervised rounds through iterative SFT–RL climbing.The live-search training uses an in-cluster judge and observation summarizer.Source: §1 · p. 2
- Iris-mini and Iris-pro achieve the best overall performance among open-source search agents across BrowseComp [22], BrowseComp-ZH [30], DeepSearchQA [7], and HLE.The evaluation compares inference with and without context management while holding the tool interface, context budget, and judging procedure fixed.Source: §1 · p. 2
- The authors plan to release model weights and key data-construction, training, and evaluation pipeline components to support reproduction and further research.Source: §1 · p. 2
2 Data Pipeline
The data pipeline reverse-constructs questions from a web corpus’s hyperlink structure to create tasks that require evidence across sources and cannot be answered from parametric memory alone. It comprises web-graph construction, task synthesis, and dual-criteria verification.
2.1 Web-Graph Construction
- The corpus is modeled as a directed page-and-hyperlink graph, and answer-anchored seed pages are expanded along their out-links into local subgraphs.The answer-anchored sampling mode fixes a target answer entity and retrieves pages describing it.Source: §2.1 · p. 3
2.2 Task Synthesis
- The pipeline distills local subgraphs into entity graphs, authors multi-hop questions requiring coupled relations, and replaces non-answer entities with descriptive references to prevent string matching.The entity graph retains salient entities and relations on paths toward the seed theme; the reasoning structure and target answer are preserved after abstraction.Source: §2.2 · p. 3
2.3 Dual-Criteria Verification
- Questions are retained only when a reference model fails to answer correctly closed-book but gives the target answer from the entity graph, verifying difficulty and solvability.A semantic-matching judge checks answer equality; accepted questions have verified unique answers and are used for trajectory generation.Source: §2.3 · p. 4
3 Training Recipe
The training recipe alternates supervised fine-tuning with reinforcement learning, using filtered search trajectories and recycling selected successful rollouts into later supervised passes.
3.1 Supervised Fine-Tuning
- Teacher-generated ReAct trajectories [26] interleave reasoning, SEARCH/SCRAPE calls, and document summaries, then pass trajectory-level correctness, pathology, and depth filters.Compression and auxiliary loop detectors remove degenerate behavior, while exact-sequence deduplication drops byte-identical trajectories but retains distinct rollouts for the same question.Source: §3.1 · p. 4
- A rubric-guided LLM judge labels individual turns KEEP or MASK, masking at most 10% per trajectory while retaining masked turns in context.Masked turns are excluded from the training loss, and the judge uses the question, reference answer, and a fixed local window of surrounding turns.Source: §3.1 · p. 5
3.2 Reinforcement Learning
- Request-level partial rollouts abort over-sampled sessions after enough trajectories complete, then resume from committed prefixes with truncated importance sampling.An in-cluster Qwen3.5-397B-A17B model serves as both binary reward judge and page summarizer; page digests are the only context-reduction mechanism, with roughly 2× over-sampling as headroom.Source: §3.2 · p. 5
3.3 Iterative Climbing
- Each climb alternates RL exploration with SFT on at most one selected successful rollout per query, targeting queries with pass rates above zero and at most one-half.The selected rollout must meet a minimum tool-call depth and be the shortest valid trajectory; this defines a self-paced curriculum that shifts toward harder cases as the policy improves.Source: §3.3 · pp. 5–6
4 Evaluation
Iris achieves strong results on four search benchmarks, with performance measured both with and without context management (CM). The evaluation also shows how CM’s gains vary by model scale and benchmark, while the no-CM results demonstrate the agents’ underlying search capability.
4.1 Setup
- Evaluation used one rollout per question, benchmark-specific LLM judging, and a consistent tool set, context limit, and turn budget.BrowseComp, BrowseComp-ZH, DeepSearchQA, and text-only HLE were evaluated; DeepSearchQA uses F1 and the others accuracy. Benchmark-hosting Hugging Face dataset and Space pages were blocked from results, scraping, and tool-manager requests.Source: §4.1 · p. 6
4.2 Experimental Results
- Iris-pro scores 88.6 on BrowseComp, 85.1 on BrowseComp-ZH, 92.9 on DeepSearchQA, and 56.4 on HLE, leading or tying all four benchmarks.It exceeds XYZ-Aquila-pro by 3.8 points on BrowseComp and 3.1 points on HLE, and ties it on BrowseComp-ZH. In standard ReAct, it is comparable to or better than MiroThinker-H1 and Apodex-1.0-H under heavy-compute configurations, though a gap remains to frontier systems.Source: Table 1 · p. 7, §4.2 · p. 7
- Iris-mini scores 82.2 on BrowseComp, 84.8 on BrowseComp-ZH, and 52.3 on HLE, the best in its parameter range, but its DeepSearchQA F1 of 86.9 trails XYZ-Aquila-mini’s 89.5.On BrowseComp, Iris-mini beats XYZ-Aquila-mini by 3.4 points (82.2 vs. 78.8); it approaches the BrowseComp performance of 1T-scale models such as Kimi-K2.6 and DeepSeek-V4-Pro.Source: Table 1 · p. 7, §4.2 · p. 7
4.3 Analysis
- CM improves performance across evaluated settings, with larger gains for Iris-mini because it takes more steps and more often exhausts its context.CM is most valuable when effective search behavior is already learned but the context limit prevents the model from using it fully.Source: Table 2 · p. 8, §4.3 · pp. 8–9
- Without CM, Iris-mini scores 64.7 on BrowseComp and 72.3 on BrowseComp-ZH, ahead of FORT-Searcher, OpenSeeker-v2, and REDSearcher; Iris-pro adds 7.9 and 4.5 points, respectively.The advantage persists with CM, indicating that gains are not solely attributable to the inference-time strategy.Source: §4.3 · p. 8, Table 2 · p. 8
- Iris-mini’s CM gains reach 21.2 points on BrowseComp, versus at most 12.8 on BrowseComp-ZH, 9.2 on HLE, and 8.9 on DeepSearchQA.The larger BrowseComp gains track its long-horizon evidence gathering and frequent context exhaustion; HLE relies more on expert knowledge and reasoning, so extending the search horizon recovers less.Source: §4.3 · p. 9
- On BrowseComp-ZH, Iris-mini with discard-all and retry and Iris-pro with either discard-all setting all score 85.1, or 246 of 289 answers correct.The shared score suggests that, on this benchmark, remaining performance may be constrained by factors beyond model capacity.Source: §4.3 · p. 9
- Discard-all with retry gives the strongest result in most settings and pushes Iris-pro above 90 on BrowseComp, but each retry incurs another full search attempt.The paper therefore reports discard-all as the primary configuration; it excludes additional inference-time verification or re-answering, and notes that argues compute cost should be accounted for in CM benchmarking.Source: §4.3 · pp. 8–9
5 Conclusion: Beyond Search
The report presents an end-to-end recipe for training Iris-mini and Iris-pro, which achieve the best overall performance in their respective parameter ranges on the benchmarks studied. It also reports positive transfer of search data and models to other agentic domains, suggesting search may be a reusable capability beyond web browsing.
5 Conclusion: Beyond Search
- The models achieve the best overall performance within their respective parameter ranges on widely recognized search benchmarks.The report frames this evaluation as an initial view of the capabilities enabled by its recipe, while noting that broader agentic scenarios remain to be explored.Source: §5 · p. 9
- The end-to-end recipe reverse-constructs questions from web links, filters questions and trajectories, trains on demonstrations, and alternates supervised training with live-search optimization.Questions are retained when a reference model fails closed-book but solves them with supporting evidence; filtering occurs at both trajectory and turn levels.Source: §5 · p. 9
- Search data and search-specialized teachers transferred positively to General Tool Use and Cowork, suggesting search behaviors may be reusable beyond web browsing.The reported transfer covered BFCL and τ-bench for General Tool Use, and OfficeQA and APEX for Cowork; the authors present broader reuse as a direction to pursue if the observation generalizes.Source: §5 · p. 9
A A Case of Ground-Truth Inconsistency
BrowseComp-ZH Question 85 illustrates a possible ground-truth inconsistency: the agent’s answer, Bolton, is supported by Sansa Stark’s second formal marriage, although the official answer is Lannister. The case motivates higher-quality annotations and broader, more reliable search benchmarks.
A A Case of Ground-Truth Inconsistency
- In BrowseComp-ZH Question 85, the agent answered Bolton while the official ground truth was Lannister, so it received no credit despite support from the series’ events.The paper presents this as a potential inconsistency between the benchmark annotation and its underlying source material.Source: A A Case of Ground-Truth Inconsistency · p. 12, Figure 2 · p. 12
- The clues identify Sansa Stark, whose first formal marriage was to Tyrion Lannister and second was to Ramsay Bolton, making Bolton the answer to the question.Her marriage to Tyrion was arranged by Tywin Lannister; Petyr Baelish arranged her later match with Ramsay Bolton.Source: A A Case of Ground-Truth Inconsistency · p. 12
- The case motivates developing search benchmarks with higher-quality annotations and broader, more reliable capability coverage.The paper frames the discrepancy as a potential benchmark annotation inconsistency.Source: A A Case of Ground-Truth Inconsistency · p. 12