Harness-G: A Graph-Structured Harness for Search Agents

Introduction

• Existing RL search agents optimize multi-turn free-form query trajectories with final-answer rewards, while later methods add process rewards or finer-grained credit without changing that retrieval interface.The introduction frames retrieval-interface design, alongside reward signals, as shaping search-agent optimization.Source: Introduction · pp. 1–2
• By training step 30, rollout groups spanning multiple retrieval-equivalence classes fall from approximately 86% to below 10%, even as query-form diversity remains high.Retrieval outcomes are clustered by overlap among accumulated evidence sets; the authors call the divergence retrieval-equivalence collapse.Source: Introduction · p. 1, Figure 1 · p. 1
• When trajectories gather overlapping evidence, GRPO can lose within-group advantages or confound retrieval credit with downstream reasoning and answer generation.Process rewards may refine supervision for sampled trajectories but do not undo the many-to-one mapping from free-form queries to retrieval outcomes.Source: Introduction · pp. 1–2
• Harness-G organizes a corpus as a paragraph–sentence–entity graph and offers finite Select, Lookup, and Answer actions that the environment validates and executes.The environment constructs queries, updates retrieval state, and filters duplicates; the bounded menu makes feasible same-state alternatives explicit and verifiable.Source: Introduction · p. 2, Figure 2 · p. 2
• With feasible actions and environment transitions held fixed, menu selection preserves multiple retrieval-distinct outcomes per rollout group, while free-query groups rapidly collapse to one class.This matched comparison varies whether the policy selects a menu entry directly or reaches the same target through a generated query.Source: Figure 1 · p. 1, Introduction · p. 2
• Structured Non-myopic Credit compares a selected action with same-state alternatives using a frozen answer scorer and propagates downstream gains to earlier enabling actions.Read-only previews provide frontier-relative credit, while trajectory dependencies let enablement credit assign delayed gains to early bridge actions.Source: Introduction · p. 2
• 10.74 points at 1.5B and 3.98 points at 3B: Harness-G achieves the highest average F1 across six QA benchmarks, outperforming Graph-R1.The benchmarks span multi-hop and open-domain question answering.Source: Introduction · p. 2

Related Work

• Unlike Harness-1’s open-ended policy interface and prior free-query methods, Harness-G exposes a finite, verifiable, previewable graph menu for comparing feasible actions and assigning structured non-myopic credit (Jiang et al. 2026).Harness-1 externalizes search state but leaves search open-ended at the policy interface.Source: Related Work · p. 2
• Credit-assignment methods add per-turn information gain, intermediate supervision, retrospective critics, or tree/anchor-state contrasts, but retain free-query targets and their many-to-one string-to-retrieval mapping (Wang et al. 2025a; Wang et al. 2025c; Zhang et al. 2025; Zhang et al. 2026; Ji et al. 2025; Feng et al. 2025).These approaches respond to training instabilities with denser or structured credit signals.Source: Related Work · p. 2
• Search-R1, R1-Searcher, and DeepResearcher train outcome-reward free-query agents, building on earlier RL query reformulation and tool-using search work (Jin et al. 2025; Song et al. 2025; Zheng et al. 2025; Nogueira and Cho 2017; Buck et al. 2018; Nakano et al. 2021; Yao et al. 2023).Other work adds cold starts, live-web research, or simulated retrieval (Chen et al. 2025b; Li et al. 2025c; Sun et al. 2025).Source: Related Work · p. 2
• GraphRAG systems often perform one-shot multihop aggregation, while Graph-R1 adds interactive RL but retains free-form queries; Harness-G brings discrete navigation to open documents (Edge et al. 2024; Guo et al. 2024; Gutiérrez et al. 2024, 2025; Luo et al. 2025a, 2025b).KGQA path agents select entities or relations on curated schemas, whereas Harness-G uses a relation-free paragraph–sentence–entity graph with deterministic query construction and state updates (Das et al. 2018; Sun et al. 2024; Lin et al. 2025).Source: Related Work · p. 2

Method

• Harness-G changes the policy’s action space rather than adding a retriever: it selects evidence or entities from a finite menu, or answers, while the environment constructs retrieval queries deterministically.Read-only previews of menu actions provide the basis for SNC.Source: Method · p. 3

Retrieval as Menu Navigation

• The finite-horizon process tracks committed evidence, visible sentences, visited entities, and history, replacing arbitrary query strings with discrete operators that have explicit semantics and deterministic transitions.Free-form query actions have large retrieval-equivalence classes that erode within-group contrast.Source: Retrieval as Menu Navigation · p. 3

Corpus-Induced Tripartite Graph

• Harness-G builds a relation-free paragraph–sentence–entity graph whose mention, adjacency, and normalized entity links support context expansion and cross-document hops without generative-LLM fact extraction.The graph is lightweight, enumerable, traceable, built in near-linear time, and reused across runs.Source: Corpus-Induced Tripartite Graph · p. 3

The Menu Environment

• The environment offers Select, Lookup, and Answer actions; bounded menus merge duplicates, block invalid choices, and allow read-only previews of the evidence each candidate would add.Lookup uses the selected entity to form a deterministic query, while Select commits a visible sentence and Answer terminates retrieval.Source: The Menu Environment · p. 3

SNC: Structured Non-Myopic Credit

• SNC propagates downstream gains backward over provenance edges from later steps to the earlier actions that enabled them, crediting multi-hop evidence chains that immediate gains can undervalue.The two SNC credit terms are computed by the environment without an external reward model or extra rollouts.Source: SNC: Structured Non-Myopic Credit · pp. 3–4
• SNC assigns frontier-relative credit by comparing a selected information action’s frozen-answerer score gain with gains from bounded, same-state alternatives; equivalent evidence yields near-zero credit.The scorer’s cumulative context includes surfaced sentences, so re-surfacing observed text gains exactly zero.Source: SNC: Structured Non-Myopic Credit · pp. 3–4

Main Results (RQ1)

• Harness-G leads average F1 at both scales: 55.24 at 3B (+3.98 vs. Graph-R1) and 50.83 at 1.5B (+10.74), beating Graph-R1 on every 1.5B dataset.At 3B, it improves F1 by 7.97, 9.12, and 5.95 points on 2Wiki, HotpotQA, and MuSiQue, respectively, while trailing Graph-R1 by 1.84 on NQ.Source: Table 1 · p. 5, Main Results (RQ1) · p. 6

Experimental Setup

• Experiments use six standard RAG benchmarks and evaluate Exact Match, F1, retrieval similarity, and generation evaluation.Harness-G uses Qwen2.5-1.5B/3B-Instruct with GRPO, group size 8, batch size 128, 120 training steps, and at most six interaction turns.Source: Experimental Setup · pp. 5–6

Ablation and Comparative Analysis (RQ2)

• The action menu improves F1 by more than 17 points over free-query under both credit regimes and by more than 35 on MuSiQue with outcome-only training.Menu+SNC is best on all three multi-hop datasets; free-query rollouts collapse into few retrieval-equivalence classes, while menu rollouts remain diverse.Source: Table 2 · p. 5, Ablation and Comparative Analysis (RQ2) · p. 6

Training Stability and Robustness (RQ3)

• Across six 3B runs, F1 rises without persistent late collapse and gradients remain bounded; three backbones and four RL algorithms train competitively, with GRPO slightly ahead of DAPO.On 2Wiki, Harness-G leads Search-R1, IGPO, and Graph-R1 throughout training and finishes with the highest F1.Source: Figure 4 · p. 6, Figure 6 · p. 6, Training Stability and Robustness (RQ3) · pp. 6–7

Ablation and Comparative Analysis (RQ2)

• Harness-G preserves retrieval diversity rather than collapsing early; its zero-advantage rate bottoms at 23% near step 10, then rebounds through all-correct groups.The rebound indicates successful convergence after diverse exploration, unlike Search-R1’s early collapse.Source: Figure 7 · p. 6, Ablation and Comparative Analysis (RQ2) · p. 6
• Full SNC performs best on every multi-hop dataset; removing both terms reduces F1 by 3.08, 4.55, and 2.88 on 2Wiki, HotpotQA, and MuSiQue.Removing either term also hurts performance.Source: Table 3 · p. 6, Ablation and Comparative Analysis (RQ2) · p. 6

Analysis of Interaction Efficiency (RQ4)

• Harness-G uses more late-stage interaction turns but shorter response trajectories, adds limited SNC overhead, and programmatic graph construction avoids API costs.It uses 3.6 turns versus 2.3 for Search-R1 and 2.9 for Graph-R1, while response-side trajectories are 2.5k versus 4.3k and 3.1k tokens; SNC adds 9–11% per-step wall-clock time. Construction costs $0 in API fees, with 0.12 s per 1K corpus tokens and 5.1 s per query.Source: Table 6 · p. 7, Analysis of Interaction Efficiency (RQ4) · p. 7

Cross-Dataset Generalization (RQ5)

• In train-on-one, evaluate-on-all-six transfer, Harness-G wins 21 of 30 cross-dataset pairs and raises mean O.O.D. F1 from 44.10 to 47.38 (+3.29).Gains are larger on multi-hop than single-hop targets, and performance remains stable across source–target pairs.Source: Figure 9 · p. 7, Cross-Dataset Generalization (RQ5) · p. 7

Conclusion

• 10.74 points at 1.5B and 3.98 points at 3B are the average F1 improvements over Graph-R1 across six QA benchmarks.Source: Conclusion · p. 7
• Free-form query diversity can mask nearly identical evidence retrieval, weakening group-relative learning through retrieval-equivalence collapse.Source: Conclusion · p. 7
• Harness-G replaces free-form queries with finite evidence or entity selections over an induced graph, while SNC assigns local and delayed credit without additional agent rollouts.Source: Conclusion · p. 7
• Controlled ablations isolate gains from both the menu interface and SNC, while cross-dataset evaluation indicates stronger transfer.Source: Conclusion · p. 7
• The results support action-space design as complementary to reward design; Harness-G remains text-only, with multimodal actions and SNC identified as a next step.Source: Conclusion · pp. 7–8

A Appendix Overview

• Appendices B–E specify datasets and preprocessing, baselines, evaluation measures and normalization, and training hyperparameters and optimization.Source: A Appendix Overview · p. 10
• Appendices F–H cover training dynamics, robustness, retrieval-equivalence and zero-advantage diagnostics, matched ablation protocols, and multi-dataset and domain-transfer results.Source: A Appendix Overview · p. 10
• Appendices I–O detail SNC computation, interface screening and invalid-target filtering, knowledge-construction and query costs, related work, held-out trajectories, and environment protocols.Source: A Appendix Overview · p. 10

B Dataset Details

• Harness-G is evaluated on six widely used RAG benchmarks covering multihop and single-hop question answering.The suite comprises 2WikiMultiHopQA, HotpotQA, MuSiQue, Natural Questions, PopQA, and TriviaQA.Source: B Dataset Details · p. 10
• 2WikiMultiHopQA, HotpotQA, and MuSiQue test compositional or longer-chain multihop reasoning using evidence across pages, supporting-fact annotations, or composed questions.2Wiki includes comparison questions; HotpotQA includes bridge and comparison questions; MuSiQue aims to reduce shortcut-prone reasoning.Source: B Dataset Details · p. 10
• Natural Questions, PopQA, and TriviaQA cover real search-engine queries, entity-centric questions across popularity levels, and trivia paired with distantly supervised evidence, respectively.Their evidence is paired with Wikipedia for Natural Questions and with evidence documents for TriviaQA.Source: B Dataset Details · p. 10
• Each dataset has 5,120 training and 128 held-out questions; the final checkpoint is evaluated without held-out-based selection, and each corpus is chunked into 1,200-token segments with 100-token overlap.The split matches Graph-R1 for protocol comparability; each dataset uses its own corpus and graph, reused across controlled variants.Source: B Dataset Details · p. 10

C Baseline Details

• Baselines are grouped as prompt-only or Qwen2.5-based trained methods, with separate distinctions for knowledge interface and whether generative LLMs construct or organize knowledge.Named-entity recognizers and dense encoders alone do not count as LLM-based construction.Source: C Baseline Details · p. 10
• Prompt-only baselines range from no-retrieval Naive Generation and Standard RAG (Lewis et al. 2020) to graph methods including GraphRAG (Edge et al. 2024), LightRAG (Guo et al. 2024), PathRAG (Chen et al. 2025a), HippoRAG2 (Gutiérrez et al. 2025), and HyperGraphRAG (Luo et al. 2025b).GraphRAG-family methods use GPT-4o-mini and are marked LLM-based.Source: C Baseline Details · p. 10
• Qwen2.5-based comparisons include non-updated Naive Generation and Standard RAG, SFT, direct-answer R1 (Shao et al. 2024), Search-R1 (Jin et al. 2025), IGPO (Wang et al. 2025a), R1-Searcher (Song et al. 2025), and Graph-R1 (Luo et al. 2025a).Search-R1 uses free-query search with outcome-reward GRPO; IGPO retains that interface with per-turn information-gain credit, while Graph-R1 uses an LLM-constructed graph.Source: C Baseline Details · p. 10
• Harness-G uses programmatic, LLM-free graph construction, while Table 1 baseline values are published cross-stack Graph-R1 references; only RQ2 interface and credit variants are matched under the shared protocol.Its graph uses sentence segmentation, entity recognition and normalization, and dense encoding rather than generative-LLM extraction or organization.Source: C Baseline Details · p. 10

• Evaluation uses the Graph-R1 suite and reports all metrics on a 0–100 scale.Source: D Evaluation Details · p. 11
• EM lowercases text, removes articles and punctuation, canonicalizes whitespace, and counts predictions matching any accepted alias.Source: D Evaluation Details · p. 11
• Token F1 uses normalized token multisets and takes the maximum overlap F1 across gold aliases.Source: D Evaluation Details · p. 11
• R-S measures semantic agreement between retrieved and gold evidence using the Graph-R1 metric encoder.Source: D Evaluation Details · p. 11
• GPT-4o-mini rates responses from 0 to 10 on comprehensiveness, knowledgeability, correctness, relevance, diversity, logical coherence, and factuality; scores are scaled by 10 to align with the 0–100 metrics.Source: D Evaluation Details · p. 11
• Table 1 reports per-dataset F1 and G-E; its Avg. columns are unweighted macro-averages across six datasets for EM, F1, R-S, and G-E, not pooled questions.Source: D Evaluation Details · p. 11

Training configuration

• The canonical Qwen2.5-3B setup uses one policy for menu actions and answers, trains with verl on eight A100 GPUs, and runs 120 steps for about three training passes.The batch configuration is specified in Table 7.Source: E Implementation Details · p. 11

Objective and safeguards

• GRPO and SNC update only model-generated tokens, use the fixed initial reference checkpoint as SNC’s answerer, and apply dual-clip, fp32 probabilities, gradient checks, and clamped credit scoring.The dual-clip constant is C=3.0, and the batch-global credit scale floor is smin=5×10−4.Source: E Implementation Details · p. 11

Rollout and training loop

• Each rollout renders evidence, visible sentences, and a menu, samples an action, previews it read-only, executes it, then scores the final answer and previews before updating the policy.Rollouts stop when the policy chooses Answer; per-step SNC credit combines frontier and entity rewards.Source: E Implementation Details · p. 11

Graph construction

• Graphs preserve paragraph and sentence structure, link extracted entities and adjacent sentences, add qualifying top-five entity neighbors, and are cached without LLM-based construction.Entity mentions are extracted with spaCy en_core_web_sm and canonicalized; neighbor links use bge-large-en-v1.5 with cosine similarity of at least 0.80.Source: E Implementation Details · p. 11

Stable optimization across datasets

• Training-batch F1 rose by 0.36–0.62 across six Qwen2.5-3B runs without persistent late-stage collapse, while gradient norms stayed at or below 12.7.Gradient norms were typically below 9 late in training.Source: F Training Dynamics and Robustness… · p. 12

Backbone transfer

• Qwen2.5-3B, Qwen3.5-4B, and Llama-3.2-3B achieved 63.4–67.5 F1; Qwen3.5-4B was best, and Llama-3.2-3B stayed within 2.5 points of the default.The HotpotQA data and 120-step budget were fixed while the backbone family varied under the same GRPO recipe.Source: F Training Dynamics and Robustness… · p. 12

Compatibility with multiple RL algorithms

• With Qwen2.5-3B and a 120-step budget, GRPO, PPO, REINFORCE++, and DAPO training-batch F1 rose rapidly and remained stable; GRPO performed best, slightly ahead of DAPO.REINFORCE++ and PPO were similarly stable; other experiments use GRPO.Source: F Training Dynamics and Robustness… · p. 12

Retrieval-equivalence diagnostics

• Retrieval-equivalence classes group trajectories within the same-question rollout group by evidence overlap; collapse is Dret moving toward zero and Neq toward one while Dquery remains high.The latter regime is termed illusory exploration.Source: G Additional Evaluation Protocols · p. 12

Zero-advantage diagnostics

• A same-question group is zero-advantage when it has at least two rollouts and reward standard deviation below 10^-6, then is stratified as all wrong or all correct.Figure 7 reports the batch fraction of each class for Search-R1 and Harness-G on 2Wiki with matched Qwen2.5-3B and group size 8.Source: G Additional Evaluation Protocols · p. 12

Transition-matched interface comparison

• The interface comparison fixes reachable states and retrieval substrate, varying only whether targets are finite executable menu choices or reached through an open string-to-target mapping.Both arms share transitions, retrieval budgets, data order, and random seeds.Source: G Additional Evaluation Protocols · pp. 12–13

Factorial credit control

• Outcome-only arms add no step advantage; IGPO uses policybelief information gain, while SNC is menu-only, testing structured-frontier and provenance credit beyond trajectory-local information gain.The two IGPO arms share turn-level normalization, cumulative turn advantage, and reward scaling; IGPO is cited as Wang et al. 2025a.Source: G Additional Evaluation Protocols · p. 13

Cross-dataset generalization

• For cross-dataset generalization, each source policy is tested on held-out splits of all six datasets; the O.O.D. mean covers 30 off-diagonal train–test pairs.Evaluation follows the protocol of Table 1.Source: G Additional Evaluation Protocols · p. 13

H Additional Generalization Results

• 47.38 vs 44.10 mean O.O.D. F1 (+3.29) favors Harness-G, which wins 21/30 off-diagonal pairs and gains +6.45 multi-hop versus +0.12 single-hop.The multi-hop targets are 2Wiki, HotpotQA, and MuSiQue; the single-hop targets are NQ, PopQA, and TriviaQA.Source: H Additional Generalization Results · p. 13
• Harness-G attains the highest average F1 in I.I.D., single-O.O.D., and combined training, with the same regime ordering as free-query agents but smaller gaps.The three 3B training regimes are compared in Table 8.Source: H Additional Generalization Results · p. 13, Table 8 · p. 14

I SNC Computation per Trajectory

• SNC scores the chosen information action against up to four type-stratified frontier alternatives with frozen-answerer previews, then propagates delayed credit to direct provenance producers and selected-action tokens.It reuses the realized trajectory without an additional policy rollout; observation tokens and tool responses receive zero SNC credit.Source: I SNC Computation per Trajectory · p. 13

J Action-Space and Constraint Screening

• The final design separates the four-action policy menu from environment-side constraints, with screening evaluated on 2WikiMultiHopQA under a matched 3B training budget.F1, EM, and mean interaction turns are used as complementary metrics.Source: J Action-Space and Constraint Screening · pp. 13–14

H Additional Generalization Results

• 3.16 average F1 over Graph-R1 and 3.69 over HyperGraphRAG mark Harness-G’s best average EM/F1 in zero-shot specialized-domain transfer.The combined-trained 3B policies were evaluated without training data from the five specialized domains.Source: H Additional Generalization Results · p. 13, Table 9 · p. 14

J.1 Action-menu evolution

• The legacy typed menu underperformed Search-R1; removing Rewrite_Query and Open_Context, merging entity hops into Lookup, adding Answer_With, and internalizing Lookup queries simplify navigation before SNC adds credit assignment.The screening progression is reported on 2WikiMultiHopQA; the final Menu+SNC row matches the main result.Source: §J.1 · p. 15, Table 10 · p. 16

J.2 Environment constraints

• Environment constraints filter weak Lookup anchors and prevent repeated entity or need-key lookups; these rules are not policy actions and stabilize turns and feasible frontier choices.The full configuration combines the filtered, deduplicated menu with SNC.Source: §J.2 · p. 15, Table 11 · p. 16

K Invalid-Target Filter

• Lookup targets exclude typed mentions (date, time, cardinal, ordinal, quantity, percent, money, norp), malformed surfaces, and nationality or ethnic adjectives, while mentions remain usable in sentence text.Filtering changes only the feasible Lookup menu and is applied before rendering, so the policy need not learn penalties for deterministically invalid actions.Source: K Invalid-Target Filter · pp. 15–16

L Knowledge-Construction and Query Cost

• On one A100, Harness-G processed 3,247,308 corpus tokens in 376.83 s (0.116 s TP1KT); query latency was 5.09 s TPQ across 16 held-out questions.Query latency used eight data-parallel A100 workers after one warm-up query per worker, excluding model initialization.Source: L Knowledge-Construction and Query Cost · p. 16
• SNC adds 9–11% wall-clock time per training step (51–69 s), uses no extra agent rollouts, and adds no model calls at inference; a full 3B run costs about 170 GPU·h on 8×A100.The overhead covers read-only previews and frozen-answerer scoring of the selected action and alternatives; it scales with the bounded preview frontier.Source: L Knowledge-Construction and Query Cost · p. 16

M Extended Related Work

• Search-R1 (Jin et al. 2025) and related agents retain unrestricted natural-language queries, while denser credit methods leave linguistic retrieval-equivalence classes intact.SNC is interface-dependent: menu previews enable same-state, frontier-relative step credit through environment-side bookkeeping without tree sampling.Source: M Extended Related Work · pp. 16–17
• Prior graph-structured retrieval often relies on generative-LLM extraction, while Harness-G builds a relation-free paragraph–sentence–entity graph and makes query construction and state updates deterministic environment operations.HippoRAG 2 uses a passage–entity graph for one-shot retrieval; Graph-R1 uses an LLM-extracted graph with free-form queries.Source: M Extended Related Work · p. 17
• Harness-G combines a programmatic open-corpus graph with a finite, verifiable, previewable action menu, addressing gaps left by free-query retrieval and discrete actions on curated knowledge graphs.Sentences are atomic evidence units and entities are bridge nodes without relation labels; the environment constructs and validates menu actions.Source: M Extended Related Work · p. 17

N Illustrative Trajectories

• Across the held-out trajectories, typed validated actions, rejected menu alternatives, minimal committed evidence sets, adaptive early stopping, and provenance-based credit for enabling bridge hops illustrate the menu and SNC.Comparison and composition hops often terminate with zero lookups once operands are visible; bridge hops may gain immediate answer likelihood only after a later Lookup.Source: N Illustrative Trajectories · p. 17
• Swapping free queries for the menu moves F1 by more than 17 points, and full SNC further improves multi-hop scores over leave-one-out variants in controlled ablations.The illustrative cases include bridge retrieval, film-year, birth-date and country comparisons, MuSiQue composition, and additional incorrect HotpotQA trajectories.Source: N Illustrative Trajectories · pp. 17–18

N.1 Menu Navigation: Process and Effectiveness

• Across five menu trajectories, bridge retrieval uses one Lookup while comparison and composition cases use none, with bounded choices, explicit distractors, and minimal committed evidence.The difficult decision varies by behavior: choosing an entity to look up, stopping and rejecting near-name operands, or committing the premises of a join.Source: §N.1 · p. 19

N.2 Interface Contrast: Free Query vs. Menu

• More than 17 F1 points overall and more than 35 on MuSiQue under outcome-only training separate the menu from free query, beyond gains recoverable with IGPO.The menu makes typed alternatives explicit and prevents distractors from entering committed evidence without an explicit action.Source: §N.2 · p. 20

N.3 SNC: Frontier, Enablement, and Credit Comparison

• Lookup Caroline Leaf earns a +0.61 preview gain against near-zero same-state alternatives, yielding +0.59 frontier-relative credit for beating feasible competitors.The baseline is the mean preview gain of in-menu alternatives, not an empty context; calculating it uses read-only previews and a frozen scorer, without tree search or extra rollouts.Source: §N.3 · p. 20
• The bridge Select has approximately zero immediate gain but enables a later +0.61 Lookup, so SNC propagates +0.59 credit back to the necessary first hop.The provenance edge records which earlier action produced the entity consumed later; this bookkeeping avoids learned value models or intermediate-state counterfactual rollouts.Source: §N.3 · pp. 20–21
• λ=0, which collapses SNC to outcome-only credit, drops F1 by 3.08 on 2Wiki, 4.55 on HotpotQA, and 2.88 on MuSiQue.Outcome-only credit spreads success across tokens, while myopic gains under-credit the enabling bridge hop; SNC assigns credit to that hop and retains frontier-relative credit on the later lookup.Source: §N.3 · p. 21

N.4 Additional Failure Cases

• EM = 0 and F1 = 0: on the 1984-birth-year HotpotQA item, the policy answers Arjen Robben before gold scorer Obafemi Martins is a typed target.The opening menu exposes Robben’s 1984 birth year but no League Cup final sentence or Obafemi Martins; the trajectory takes two turns.Source: §N.4 · p. 21
• A held-out rollout (index 14) recovers both DOB sentences from the typed menu and terminates on the later birth; once both year-bearing sentences are typed, the comparison is 1856 < 1991.No open-ended birth-date query is used.Source: §N.4 · p. 22
• EM = 0 and F1 = 0: on the Powerpuff Girls Z item, the policy chooses the anime-side bridge and answers Toei Animation, while Craig McCracken is not visible.The four-turn trajectory commits a Toei Animation production sentence, yielding a coherent adaptation-side answer rather than the source-series answer.Source: §N.4 · p. 22
• EM = 0 and F1 = 0: in the Back Roads item, Tommy Lee Jones is selectable, but the policy emits Tom Conti after looking up both actors.The log exposes the selectable cast sentence and answer-string mismatch; pronoun-only Oscar sentences remain visible alongside it.Source: §N.4 · p. 22

N.5 Synthesis

• Harness-G’s interface reparameterization and SNC are coupled: typed menus make decisions explicit, while non-myopic credit assigns delayed gains to actions that enable them.The synthesis contrasts this with free-query credit densification, which does not remove retrieval aliasing, and outcome-only training under the menu, which leaves delayed bridge credit unresolved.Source: §N.5 · pp. 22–23

D SNC ANSWERER (TRAINING ONLY)

• The frozen SNC answer scorer computes teacher-forced gold-answer likelihoods during training and never enters the policy context.The full Harness-G configuration retains the default Qwen2.5 system message rather than adding a custom system prompt.Source: O Environment Protocol and Prompts · p. 23

O Environment Protocol and Prompts

• The environment presents committed evidence, visible sentences, entities, and an executable menu; the policy must choose one available action id rather than write a search query.Lookup queries are constructed from the question and committed evidence, while free-text query riders are rejected and duplicate retrievals are merged.Source: O Environment Protocol and Prompts · p. 23

B SEARCH-TOOL SCHEMA fixed Model-visible function description

• The fixed search-tool schema accepts INIT or an available action id, forbids appended free-text queries, and requires final answers to use observed evidence.Its description defines SELECT, LOOKUP, ANSWER_WITH, and ANSWER, with retrieval text built by the environment for LOOKUP.Source: B SEARCH-TOOL SCHEMA fixed… · p. 28