PaperlayerView exact revision on arXiv

Source-linked AI summary

Masking Stale Observations Helps Search Agents -- Until It Doesn't: A Regime Map and Its Mechanism

Haoxiang Zhang, Qixin Xu, Zhuofeng Li, Lei Zhang, Pengcheng Jiang, Yu Zhang, Julian McAuley

arXiv:2606.00408v1cs.CLcs.AIcs.IR

TL;DR

Observation masking can improve long-horizon search, but when it helps and why remain unclear. This paper maps masking across agent regimes and finds its gains peak for strong retrieval paired with moderately capable models before collapsing for saturated systems.

  • Problem

    It remains unclear when lightweight observation masking helps agentic search and how its effects interact with stale evidence, model capability, and retrieval.

  • Method

    The paper treats minimal turn-based observation masking as a diagnostic instrument for analyzing context use across agentic search regimes.

  • Results

    +11.7 pts is the peak masking gain, with a low plateau under weak retrieval and collapse at model saturation.

  • Takeaways & Limitations

    Observation masking is regime-dependent and should be calibrated jointly with retriever recall and model capacity rather than treated as a default.

  • Takeaways & Limitations

    The study evaluates a minimal masking policy and open-weight 4B–284B backbones, omitting frontier proprietary models.

Abstract

from arXiv · show

Long-horizon search agents accumulate large amounts of retrieved content across many tool calls, making context-budget efficiency increasingly important. A minimal intervention is to mask stale observations from the context as the trajectory progresses, but it remains unclear when this form of context management helps and why. We study observation masking through a systematic sweep over various agent backbones (4B to 284B parameters) and three retrievers on offline and live-web agentic search benchmarks. We find that the accuracy gain from masking follows an asymmetric inverted-U shape when plotted against the model's accuracy without context management: a plateau under weak retrievers, a peak when a strong retriever meets a mid-capacity model, and a sharp collapse when the model is saturated. This pattern reflects the interaction between retriever recall and the model's implicit filtering capacity, rather than either factor in isolation. Mechanistically, masking implements a token-for-turn trade-off: it removes observations the model has largely stopped attending to and pages the agent rarely re-opens. The added turns help when they convert failures into successes, but they fail when masking removes evidence the model would otherwise have used. We therefore reframe context management as a regime-dependent intervention and provide a holistic perspective for analyzing context use in agentic deep search. We release our scaffold and trajectories here (https://github.com/i-DeepSearch/observation-masking) to support future research.

1 INTRODUCTION

Long-horizon search agents accumulate observation-heavy contexts, motivating lightweight observation masking, yet its benefits depend on the interaction between backbone capability and retriever quality. The study maps this regime dependence across model sizes, retrievers, offline benchmarks, and live-web environments, identifying plateaus, a sweet spot, and collapse.

  • Motivation: Search agents accumulate tens of thousands of context tokens dominated by retrieved snippets, page contents, tool errors, and intermediate reasoning over extended action horizons.Each turn appends new environment observations to the trajectory, creating the context-budget problem addressed by context management.
  • Motivation: Observation masking replaces old tool outputs with placeholders while preserving surrounding reasoning and tool-call structure, making it a lightweight context-management intervention.The method is described as widely adopted in deployed agent systems.
  • Experimental scope: The study varies backbone models from 4B to 284B parameters and retrievers from sparse BM25 to dense agentic-search retrievers while fixing the scaffold, masking rule, prompts, and evaluation protocol.Configurations are evaluated on offline benchmarks with varying retrievers and live-web agentic search environments using a unified parallel-tool-call scaffold.
  • Contributions: Observation masking gains follow a non-monotonic regime map: retriever-bottleneck systems gain +6–7 pts, the CM sweet spot gains +11.7 pts, and model-saturated systems gain ≤0 pts.This pattern is established across the offline suite and the right-hand collapse also appears in live-web environments.
  • Observation masking: Masked observations are removed only from the model’s context: the underlying page remains reachable through the page pool, cursor, id, or URL, while reasoning chains and tool calls remain visible.The most recent K observations are retained, and earlier observations are replaced with a placeholder.

2 METHODOLOGY

The methodology represents search as a multi-turn trajectory whose growing observation-heavy context can be selectively masked. A decoupled scaffold preserves pages externally, enabling historical re-reference while supporting parallel search, reading, and localization.

  • Trajectory and context: Search proceeds as a multi-turn trajectory of reasoning, parallel tool calls, and returned observations, with the final action containing the answer.The context is the rendered trajectory prefix, and its length grows linearly with turns, dominated by observation tokens.
  • Observation masking: Masking replaces observations outside a K-turn retention window with a placeholder while retaining tool-call errors and structural call information.Operational staleness denotes eligibility for masking, not guaranteed irrelevance, because old observations may later become useful.
  • Observation masking: Masking is a minimal diagnostic intervention that adds neither tokens nor model calls, isolating when stale-observation content matters to performance.It is intended to study context-use conditions rather than compete with more sophisticated context-management methods.
  • Search scaffold: The scaffold decouples persistent page state from volatile context, so masked observations do not prevent re-referencing previously fetched pages through stable cursors.The append-only page pool deduplicates URLs and preserves page records independently of context rendering.
  • Search scaffold: Three factored tools implement the search loop: search retrieves snippets, open reads scoped page content, and find localizes exact matches within stored pages.Parallel tool execution supports efficient long-horizon search without requiring the agent to regenerate cumbersome URLs.

3 EXPERIMENT SETUP

The experiments span four agentic search benchmarks, diverse open-weight agents and retrievers, and both offline and live-web retrieval settings. The scaffold also establishes a stronger No-CM baseline, making subsequent masking gains more conservative.

  • Benchmarks: Evaluation covers four benchmarks across offline and live-web settings: BrowseComp-Plus, GAIA, xBench-DeepSearch, and BrowseComp-ZH.BrowseComp-Plus uses a fixed corpus, while the other three are live-web benchmarks spanning multiple languages.
  • Models: The study evaluates open-weight tool-calling agents ranging from 4B to 284B parameters, including general-purpose and specialized agentic-search backbones.The listed families include Qwen, GPT-OSS, OpenResearcher, DeepSeek-V4-Flash-Max, and Tongyi-DeepResearch.
  • Retrievers: BrowseComp-Plus uses sparse BM25 and two dense retrievers, whereas the live-web benchmarks use Serper API.The dense retrievers are Qwen3(-Emb)-8B and reasoning-tuned AgentIR-4B.
  • Evaluation protocol: All experiments use a 500-turn limit, retain K = 5 observations, and evaluate with LLM-as-Judge using GPT-5-mini.Additional evaluation details and settings are provided in Appendices §B, §C, and §F.5.
  • Higher baseline accuracy: Our scaffold consistently exceeds the highest publicly reported matched model–retriever accuracies on BrowseComp-Plus, with gains of 11.2–12.6 points across three reported comparisons.The reported improvements are +11.2 points for GPT-OSS-20B, +12.4 for GPT-OSS-120B, and +12.6 for Tongyi-DeepResearch-30B-A3B.
  • Motivation: The stronger No-CM baseline makes the regime map measure masking’s conservative contribution above an already stronger scaffold rather than gains caused by fixing scaffold deficiencies.The authors argue that weaker baselines can exaggerate the apparent value of context pruning.

4 MAIN RESULTS

Observation masking follows a three-regime, asymmetric pattern on BrowseComp-Plus: it helps most when retrieval is strong but model accuracy remains moderate, and can harm saturated models. The regime depends on retriever recall and model filtering behavior, with collapse becoming sharper on noisier live-web benchmarks.

  • Three regimes: +11.7 points is the peak gain for Qwen3.5-35B-A3B+AgentIR at 62.9% No-CM accuracy, whereas gains plateau under BM25 and collapse below zero once No-CM accuracy exceeds 70%.Under BM25, masking gains remain between +6.2 and +6.6 because recall never exceeds 0.55; Tongyi-DeepResearch-30B-A3B reaches −1.1 points despite 80.7% No-CM accuracy and 0.93 recall.
  • Mismatch, not size: Model scale alone does not determine masking’s regime: Qwen3.5- and Qwen3.6-35B-A3B differ sharply with AgentIR, gaining +11.7 versus +3.7 points.The paper attributes this mismatch to attention dynamics shaped by training inherence, examined through localization behavior in §5.3.2.
  • Live web amplifies the collapse: On live-web benchmarks, the collapse sharpens: GPT-OSS-120B shifts from +0.1 points on BrowseComp-Plus to −4.8 points on GAIA, while gaining +8.0 points on xBench.The paper links the sharper collapse to noisier, less controllable live-web context, where masking can remove signal a capable reader would otherwise use.

5 RESEARCH FINDINGS

Context masking produces asymmetric query-level effects: it fixes more trajectories with fewer tokens, but broken trajectories incur costly re-search. Its benefits arise from removing stale, rarely attended observations while preserving useful reasoning, and can be anticipated from trajectory evidence density and complexity.

  • Query-level transitions: +11.7 points for Qwen3.5-35B-A3B versus +0.1 points for saturated GPT-OSS-120B, with masking’s token growth driven mainly by broken trajectories.Fixes consume fewer rolling tokens, whereas correct→wrong trajectories require additional inference turns to re-search and reopen pages.
  • Predictive diagnostics: A two-coordinate probe uses No-CM prefix complexity and fitted evidence-density SNR to distinguish masking-rescued from non-rescued trajectories, with xBench using cited-answer lines as a proxy.The proxy is diagnostic rather than a certificate of factual correctness, and the evidence-density curves show AgentIR’s clearer initial evidence diluting as pages accumulate while BM25 remains retrieval-limited.
  • Mechanism: Reasoning receives 53.7% of per-step attention versus 25.6% for tool observations, while middle-trajectory observations receive little attention and are rarely reopened.Observation attention is concentrated on the most recent turn and collapses after it, whereas reasoning also receives renewed attention on the earliest turns.
  • Mechanism: The strongest masking gains coincide with stronger reasoning attention and increased reopening of the first page, whereas higher recent-observation attention limits masking’s benefit.Across settings, 9B-AgentIR shows the strongest reasoning attention and largest gain, while 3.6-35B-AgentIR preserves the observations it attends to most.
  • Scaffold ablations: Removing error retention raises open error rates from 18.6% to 22.6% on 4B and from 20.4% to 24.6% on 9B, while blurred titles raise them to 20.8% and 26.2%.Retaining tool-call errors preserves feedback for recovering from malformed calls, and exact URLs avoid forcing the agent to infer targets.

6 RELATED WORK

Prior work frames agentic search as long-horizon, tool-augmented information seeking that requires maintaining state across many turns. Context-management research builds on working-memory ideas and explores truncation, eviction, compression, and adaptive approaches for handling long histories.

  • Long-horizon Agentic Search: Long-horizon agentic-search benchmarks require agents to maintain state across dozens of tool-use turns.Examples include BrowseComp, BrowseComp-Plus, GAIA, xBench-DeepSearch, HLE, and BrowseComp-ZH.
  • Context Management for Autonomous Agents: Context-management work studies static truncation, heuristic eviction, and online compression to manage long agent histories without disrupting task logic.This line of work builds on working-memory ideas inspired by MemGPT (Packer et al., 2023).
  • Context Management for Autonomous Agents: Adaptive methods such as AgentFold (Ye et al., 2025) and ReSum (Wu et al., 2025) add flexibility while introducing additional costs.

7 CONCLUSION

Observation masking is a regime-dependent tool rather than a default intervention, with utility governed by baseline capability. It helps most when enhanced retrieval recall outpaces the model’s intrinsic noisefiltering capacity.

  • 7 CONCLUSION: Observation masking is a regime-dependent tool, not a default intervention.Its utility is strictly governed by the system’s baseline capability.
  • 7 CONCLUSION: Masking thrives in an intermediate sweet spot where enhanced retrieval recall outpaces the model’s intrinsic noisefiltering capacity.
  • 7 CONCLUSION: Outside this mismatched regime, masking either lacks evidence to expose with weak retrievers or risks discarding information capable models could otherwise exploit.

LIMITATIONS

Although the findings are robust across benchmarks, the study is bounded by its deliberately minimal masking policy and non-exhaustive model coverage.

  • Policy scope: The study isolates a minimal turn-based observation-masking policy, leaving learned, attention-guided, and semantic-adaptive alternatives for future work.Such policies might preserve gains in the model–retriever mismatch regime while avoiding collapse in model-saturated settings.
  • Model coverage: Model coverage is broad but not exhaustive because of computation, inference, and attention-analysis constraints.

ETHICAL CONSIDERATIONS … B EVALUATION DETAILS

The paper situates context management within long-horizon agentic search while highlighting ethical risks from cheaper automation and the suppression of critical context. It also reviews related agent settings and existing stale-context pruning strategies.

  • ETHICAL CONSIDERATIONS: Context management can lower the cost of complex web research while enabling scalable production of low-quality or misleading content.Adaptive history compression may democratize advanced research capabilities but also reduce the cost of harmful automated content generation.
  • ETHICAL CONSIDERATIONS: Suppressing subtle context, including ethical disclaimers and edge-case safety warnings, may harm downstream outputs.The passage identifies inadvertent removal of critical safety-relevant information as a specific risk of context management.
  • A MORE RELATED WORK: Long-horizon agentic interaction extends language-model use from single-turn generation to complex tasks requiring sustained interaction.Representative applications include literature surveys, computer-use tasks, autonomous software engineering, and multimodal tool use with adaptive self-refinement.
  • A MORE RELATED WORK: Deep agentic search repeatedly combines search, reading, and localization actions to gather external evidence before answering complex questions.Early systems use search as the primary external tool, while newer deep-research agents organize the process into structured tool-calling loops.
  • A MORE RELATED WORK: Existing context-management systems use diverse pruning policies, including discarding stale observations, compacting trajectories, and retaining only recent tool results.The related work cites DeepSeek-V3.2, Claude Code, and Kimi Researcher as examples of boundary-pruning approaches.
  • A MORE RELATED WORK: Other systems evict intermediate reasoning traces at milestones or route among multiple context-management strategies.GLM-5 and AgentSwing illustrate approaches that differ from simple boundary pruning.

B.1 BENCHMARKS · B.2 RETRIEVERS • · B.3 COMPARED BASELINES •

The study evaluates deep-research agents across four offline and live-web benchmarks, three retrieval approaches, and diverse open-weight baseline models. The setup combines controlled corpus-based evaluation with sustained web-search tasks and agent-oriented model comparisons.

  • B.1 BENCHMARKS: BrowseComp-Plus provides controlled offline evaluation using a fixed corpus, human-verified documents, mined hard negatives, and 830 multi-document research questions (Chen et al., 2025c; Wei et al., 2025).Its BM25 and Qwen3-Embedding-8B FAISS indexes eliminate reliance on live web access while testing deep retrieval and multi-hop reasoning.
  • B.1 BENCHMARKS: The benchmark suite also covers Chinese multi-hop browsing with 289 BrowseComp-ZH questions, text-only GAIA with 103 examples, and 100 open-web xbench-DeepSearch examples (Zhou et al., 2025; Mialon et al., 2024; Chen et al., 2025a).These tasks span fragmented Chinese-web retrieval, real-world reasoning and tool use, and sustained multi-turn evidence gathering.
  • B.2 RETRIEVERS •: Retrieval baselines comprise lexical BM25, dense Qwen3-Embedding-8B, and reasoning-aware AgentIR-4B, which jointly embeds search queries with the agent’s internal reasoning traces (Zhang et al., 2025b; Chen et al., 2026).BM25 and Qwen3-Embedding-8B are used for BrowseComp-Plus retrieval agents, while AgentIR is designed to exploit multi-turn problem-solving context.
  • B.3 COMPARED BASELINES •: The compared Qwen3.5/3.6 models are open-weight Alibaba Cloud foundation models combining Gated Delta Networks with sparse mixture-of-experts routing and supporting up to 1M input tokens (Team, 2026; QwenTeam, 2026b).The evaluated family emphasizes agentic capability, multimodal understanding, multilingual coverage, and long-context input.
  • B.3 COMPARED BASELINES •: GPT-OSS-120B and GPT-OSS-20B provide open-weight reasoning baselines with 128K-token context, adjustable reasoning effort, instruction following, and browser-tool use (Agarwal et al., 2025).They are evaluated through the same browser-tool interface.
  • B.3 COMPARED BASELINES •: DeepSeek-V4 supplies Pro and Flash open-weight mixture-of-experts baselines with 1M-token context, long-context support, and agent-oriented post-training.The evaluation uses parallel browser-tool access for extended deep-research trajectories.
  • B.3 COMPARED BASELINES •: OpenResearcher-30B-A3B is an open long-horizon deep-research model trained with a fully open trajectory-synthesis pipeline and built on Nemotron-3-Nano-30B-A3B (Li et al., 2026a; Blakeman et al., 2025).It is included among the compared agentic baselines.

C EXPERIMENT SETTINGS

Experiments use consistent multi-GPU serving, model-native tool-call formatting, and fixed decoding settings across model–retriever configurations. A 15% human audit of LLM-judged results achieved over 99.9% agreement.

  • Model serving: Models are served on eight NVIDIA H100 GPUs with tensor parallelism scaled from TP=1 for 4B–9B models to TP=8 for DeepSeek-V4-Flash-Max.The allocation uses TP=2 for 20B–35B models and TP=4 for the 120B model, allowing larger models to fit while keeping inference consistent.
  • Tool-call formatting: Each model uses its native chat and tool-calling template, with tool specifications and observations injected through the corresponding model-specific format.This avoids penalizing models for template mismatch and better reflects intended deployment settings.
  • Generation configuration: Unless otherwise specified, evaluation fixes maximum generation length at 8192 output tokens per turn and sampling temperature at 1.0 across model–retriever configurations.Keeping decoding hyperparameters fixed isolates differences associated with backbone capability, retrieval quality, and context management.
  • Human Audit Evaluation: Human annotators verified a random 15% sample of LLM-judged experimental results, achieving over 99.9% agreement.This audit was used to support evaluation reliability.

D REGRESSION PROBE: EXPERIMENTAL SETUP … D.6 LIVE-WEB XBENCH PROXY

The regression probe analyzes BrowseComp-Plus trajectories through sampled prefixes, trace-shape features, grouped ridge regression, and two-dimensional separability probes. Because xBench lacks gold-document qrels, it uses a weaker citation-overlap proxy with the same downstream procedure.

  • D.1 BROWSECOMP-PLUS TRACES AND LABELS: The probe uses five released BrowseComp-Plus model–retriever settings and computes features and SNR targets exclusively from No-CM trajectories.Gold URLs come from BrowseComp-Plus qrels using the recall evaluator’s exact-match URL normalization.
  • D.2 PREFIX SAMPLING: Each trajectory contributes at most eight prefixes, sampled uniformly before the final turn, while always retaining the final evidence state.This yields roughly 6.5k prefix examples per 830-query configuration.
  • D.3 INPUT FEATURES: Features describe trace shape rather than hidden states, including normalized position, URL counts and growth, observation lengths, tool-call indicators, and budget variables.Cumulative and turn-local counts are log-transformed where specified.
  • D.4 REGRESSION MODEL AND TWO-DIMENSIONAL PROJECTION: For each model–retriever configuration, standardized features are expanded with second-degree interactions and fitted using crossvalidated ridge regression with query-grouped folds.Predictions are clipped to [0, 1], and the vertical coordinate is the min–max normalized fitted SNR.
  • D.4 REGRESSION MODEL AND TWO-DIMENSIONAL PROJECTION: The first principal component correlates strongly with trace scale, with positive loadings on cumulative turns and observation characters, providing an unsupervised proxy for input complexity.The displayed representation uses the fitted SNR as its vertical coordinate.
  • D.5 AUC COMPUTATION: Scatter-plot AUC measures linear separability in the displayed two-dimensional representation, using a balanced query-grouped logistic probe rather than the SNR-regression objective.The reported score is the macroaverage of held-out-fold Mann–Whitney ROC-AUC values.
  • D.6 LIVE-WEB XBENCH PROXY: For CM rescue, positives are No-CM-wrong →CM-correct queries; BrowseComp-Plus uses all non-rescued prefixes as negatives, while xBench excludes degraded trajectories.The xBench live-web proxy replaces unavailable gold-document SNR with the fraction of cumulative observed browser lines cited by the final No-CM answer, then applies the same grouped regression and separability procedure.

E ATTENTION ANALYSIS: EXPERIMENTAL SETUP … G MORE EXPERIMENT RESULTS

The attention analysis instruments full-attention layers across three browsing-agent settings, computes role- and turn-level attention from reasoning steps, and visualizes both trajectory-specific patterns and aggregated positional decay. The setup uses matched 150-trajectory samples, causal single-pass query/key capture, and role-aware aggregation across interaction histories.

  • E ATTENTION ANALYSIS: EXPERIMENTAL SETUP: The procedure records query/key tensors from every full-attention module, covering 8 layers in the 4B and 9B models and 10 layers in the 35B model.All models use hybrid blocks with one full-attention layer every four transformer layers; each hooked layer has 16 query heads and head dimension 256.
  • E.1 MODELS AND TRAJECTORIES: The analysis compares 4B-BM25, 9B-AgentIR, and 3.6-35B-AgentIR browsing agents using the same randomly sampled 150 trajectories per setting.The models pair different Qwen backbones with BM25 or AgentIR retrieval and parallel tool calls, enabling like-for-like cross-model comparison.
  • E.2 CAPTURING QUERY/KEY TENSORS WITH HOOKED VLLM: A single chunked-prefill pass under hooked vLLM captures per-layer tensors at every token position, while causal masking supports attention computation at each reasoning-start token.The 35B run uses tensor parallelism across two H100 GPUs and concatenates head shards offline.
  • E.3 PER-TURN ATTENTION SCORES: The analysis labels tokens as system, user, reasoning, tool call, or observation, then groups reasoning spans with their following tool-call/observation pairs into interaction turns.The first turn also includes the leading system and user segments.
  • E.3 PER-TURN ATTENTION SCORES: For each reasoning step, layer- and head-averaged attention from the first post-<think> query is summed over tokens by interaction turn and role.This produces a per-step matrix of role-specific attention scores over the interaction history.
  • E.4 PER-TRAJECTORY HEATMAP: Trajectory heatmaps render reasoning attention in the lower-left triangle and observation attention in the upper-right triangle, masking the diagonal and excluding self-attention from color limits.The illustrative example is a 50-turn 9B-AgentIR trajectory.
  • E.5 AGGREGATED DECAY CURVES: Aggregated decay curves bin past turns by normalized relative position into 20 equal-width bins, restricting analysis to reasoning steps with at least 5 past turns.The curves report both mean role-specific attention and cumulative attention share, whose saturation below 100% reflects attention allocated to system, user, and current-self tokens.
  • F PROMPT TEMPLATES: The prompt templates specify web-search behavior, citation formatting, parallel tool-call instructions, and an exact-answer LLM judge format for the broader experiments.These supplied templates include generic, Qwen-style, and DeepSeek-style parallel-call instructions and define extracted answers, correctness reasoning, and confidence output fields.

G.1 MAIN RESULT ANALYSIS · H CASE STUDY

Observation masking follows three regimes: a retriever-bottleneck plateau, a mismatch-driven gain peak, and saturation where gains collapse. It improves recall by trading context for additional tool calls, with regime boundaries transferring across benchmarks.

  • G.1 MAIN RESULT ANALYSIS: Sparse BM25 produces a near-constant low masking-gain plateau: +6.3, +6.6, and +6.2 for Qwen3.5-4B, Qwen3.5-9B, and GPT-OSS-20B.This retriever-bottleneck regime reflects recall below 0.54 and an accuracy ceiling below 39%.
  • G.1 MAIN RESULT ANALYSIS: +11.7 is the largest masking gain, achieved by Qwen3.5-35B-A3B+AgentIR in the mismatch regime.The peak occurs when a capable retriever surfaces signal that the model cannot itself separate from noise.
  • G.1 MAIN RESULT ANALYSIS: Retriever–model mismatch, rather than parameter scale alone, selects the regime: same-scale Qwen3.5 variants gain +11.7 versus +3.7 under different training recipes.The comparison holds architecture and parameter count roughly fixed while contrasting mismatch and saturation.
  • G.1 MAIN RESULT ANALYSIS: Masking lengthens trajectories: every populated configuration adds tool calls, ranging from 20.3 to 91.6, while stale-observation removal enables higher recall.The extra calls arise because agents requery and reread after masked observations are cleared.
  • G.1 MAIN RESULT ANALYSIS: The regime ordering transfers across benchmarks: Qwen3.5-9B gains +4.8 on GAIA-text and +7.0 on xBench, while DS-V4-Flash-Max gains +0.0, +1.0, and +0.0.The weaker backbone remains in the mismatch regime, whereas the strongest backbone is saturated across the additional benchmarks.
  • G.1 MAIN RESULT ANALYSIS: Qwen3.5-4B gains only +1.9 on GAIA-text versus +6–+11 across BrowseComp-Plus retrievers because live-web context is noisier and harder to act on.With recall effectively bounded by an unconstrained web, this configuration behaves like the retriever-bottleneck regime.

H.1 CASE: PARALLEL TOOL USE ON BROWSECOMP-PLUS … H.5 CASE: BROWSECOMP-PLUS — CM IMPROVES (CM OFF WRONG →CM ON CORRECT)

The case studies show observation masking supporting successful long-horizon investigations, including parallel-search traces and citation-heavy reasoning, but also causing failure when it archives evidence needed for final synthesis. Across BrowseComp-Plus examples, masking can either recover from an incorrect trajectory or turn a correct trajectory into an incorrect answer.

  • H.1 CASE: PARALLEL TOOL USE ON BROWSECOMP-PLUS: Qwen3.5-9B used parallel search, open, and find calls to connect the May 2001 archaeology book, Otago Conservancy, William Larnach, and 404 ha of consolidated land.The agent retried quoted searches, opened promising documents, used browser.find to locate suicide and consolidation evidence, and verified the final biographical context.
  • H.2 CASE: TONGYI-DEEPRESEARCH-30B-A3B — PARALLEL SEARCH IN ARRAY QUERY: DeepResearch-30B-A3B correctly identified the film as Brazuca after a 38-call parallel-search turn and subsequent verification of its director, editor, and production-design clues.The final answer identifies Faidon Gkretsikos, born in 1988 and trained in Public Administration, Yorgos Georgopoulos, and Eleni Kariori as matching the clues.
  • H.3 CASE: XBENCH-DEEPSEARCH — GPT-OSS-120B: GPT-OSS-120B correctly answered that Wei Zheng had four sons after identifying the Zhao Huaiman summer field contract, its 643 CE date, and the chancellor who died that year.Although the model had the answer by the middle of the trajectory, it spent many later rounds gathering exact line-numbered citations before producing the final answer.
  • H.4 CASE: BROWSECOMP-PLUS — CM IMPAIRS (OFF CORRECT →CM ON WRONG): On BrowseComp-Plus qid 171, masking changed a correct June 2023 answer into an incorrect August 2023 answer by archiving the NTV evidence distinguishing the relationship’s end from its public announcement.Without masking, the agent reopened the relevant pages and answered June 2023; with masking, the live context retained the Citizen Digital publication and announcement date and led to August 2023.
  • H.5 CASE: BROWSECOMP-PLUS — CM IMPROVES (CM OFF WRONG →CM ON CORRECT): In another BrowseComp-Plus run, masking recovered the correct Sankomota and Frank Leepa answer, whereas the unmasked baseline fixated on the incorrect band Nass el Ghiwane.The CM-off trajectory repeatedly followed wrong leads and search-engine-biased 2003-death results; the CM-on trajectory continued exploring across non-English sources and reached the correct answer.
Loading 2606.00408v1…