MTA-Zero: Self-Improving Multimodal Deep Search via Online QA-Solver Co-training

1 INTRODUCTION

• Multimodal deep search requires image-based entity identification, web evidence gathering, tool selection, and multi-hop answer composition.The Solver must integrate evidence collected across modalities.Source: §1 · p. 1
• Fixed offline corpora can become unanswerable, trivial, or unrelated to an evolving Solver’s capability gaps.Their question distributions are set before the target Solver evolves.Source: §1 · p. 1
• Figure 1 contrasts the declining supply of informative questions under a frozen QA policy with training signal retained or restored by online updates.For the frozen policy, held-out groups with non-degenerate rollout outcomes fall from 55.8% at S0 to 44.3% at S50.Source: Figure 1 · p. 2
• Online co-training addresses a difficult generation balance: questions must be image-grounded, evidence-supported, independently searchable, and neither unanswerable nor trivial.Rewarding only Solver failures can encourage unanswerable questions, while rewarding only successes can collapse the curriculum to trivial ones.Source: §1 · p. 2
• MTA-Zero co-trains separate QA Generator and Solver policies, using Solver rollout outcomes to target questions that are challenging but solvable.Each policy has separate parameters, optimizers, and gradient updates.Source: §1 · p. 2
• 36.91% to 51.58% mean accuracy: MTA-Zero improves an 8B Solver across six evaluation benchmarks.It outperforms compared 7–8B specialist agents on four of six benchmarks and approaches GPT-5 agent performance with an 8B model.Source: §1 · p. 2

2.1 MULTIMODAL DEEP SEARCH AND DATA CONSTRUCTION

• Offline multimodal deep-search pipelines curate candidate questions and trajectories into SFT or RL corpora, whereas MTA-Zero trains an image-grounded QA generator and Solver online.Its group-wise Solver success rate updates question generation toward the Solver’s shifting learning frontier.Source: §2.1 · p. 3

2.2 SELF-IMPROVING AND CO-EVOLUTION

• Open-world multimodal deep search makes adaptive task generation harder because questions must preserve visual grounding, retrieve and verify evidence, and calibrate difficulty for an independently acting Solver.A plausible image-based question may still be unanswerable, ambiguous, or unsupported by retrieved evidence; MTA-Zero combines a fixed verification harness with Solver-calibrated rewards.Source: §2.2 · p. 3

3 METHOD

• The QA Generator and Solver co-train online from image-grounded seed contexts, with separate GRPO updates and no gradient sharing.The Solver’s group rollout outcomes provide signals to both policies.Source: §3 · p. 3

3.1 ONLINE QA GENERATION WITH A HARNESS

• The trainable QA policy searches for evidence and proposes single-hop QA pairs, while a fixed harness checks and merges accepted hops into a multi-hop task.The harness validates entity anchoring, answer uniqueness, temporal stability, self-contained wording, and whether search is required.Source: §3.1 · p. 4
• Accepted QA hops are merged by removing the previous entity’s explicit mention, so the final question reaches that entity through the preceding description.For each subsequent hop, the generator searches for information about the previous answer and proposes an evidence-based question-answer pair.Source: §3.1 · p. 4

3.2 MULTI-TOOL SOLVER ROLLOUTS

• Each accepted Solver question receives K independent ReAct tool-use rollouts, with terminal correctness determining the group success rate.GRPO updates the Solver only for groups with both correct and incorrect outcomes.Source: §3.2 · p. 5

3.3 SOLVER-CALIBRATED QA SCORES

• The QA Generator’s Goldilocks reward peaks at target Solver success rate τ and assigns zero reward when all or no rollouts solve a question.The main setting uses K = 8 and τ = 0.5, favoring questions that are difficult but still solvable.Source: §3.3 · p. 5

4.1.1 DATASETS

• Evaluation covers six benchmarks requiring visual evidence understanding and open-web research, including MMSearch-Plus, BrowseComp-VL, and FVQA.The set also includes HR-MMSearch-VL, MMSearch-VL, and the testmini split of VDR-Bench.Source: §4.1.1 · p. 5
• Training starts from image-grounded one-hop VQA seeds, with no pre-generated multi-hop questions, retrieved evidence, or search trajectories in the seed pool.All multi-hop optimization questions are subsequently constructed and verified online.Source: §4.1.1 · p. 5

4.1.2 AGENT SETUP

• The Solver uses a ReAct loop capped at 10 iterations, stopping early when its stop flag is true and confidence exceeds 0.7.Its tool palette includes web and image search, webpage reading, reverse image search, OCR, and image cropping.Source: §4.1.2 · p. 6

4.1.3 BASELINES

• Baselines include commercial models evaluated both without tools and in the same agent framework, alongside prior multimodal deep-search systems.The commercial comparisons include GPT-5, Claude Sonnet 4.5, and Gemini 3 Pro.Source: §4.1.3 · p. 6

4.1.4 TRAINING SETUP

• The two policies are optimized independently with GRPO for 50 iterations, using eight sampled seed QA contexts per iteration and a Goldilocks target τ = 0.5.Each seed generates eight QA trajectories targeting five hops, and retained QAs receive eight independent Solver rollouts.Source: §4.1.4 · p. 6

4.2 MAIN RESULTS

• MTA-Zero records the highest score among listed 7–8B specialist agents on four of six benchmarks and outperforms MTA-DeepSearch on four of five shared benchmarks.The MTA-DeepSearch comparison shares the raw one-hop seed pool, but the specialist comparisons are not cost-matched because methods use different tool stacks and training pipelines.Source: §4.2 · pp. 6–7
• 36.91% to 51.58% mean accuracy (+14.67 points) across six benchmarks: MTA-Zero improves over the matched Qwen3-VL-8B-Instruct Agent Workflow baseline.Table 1 reports mean accuracy over three generation seeds; the run uses no pre-generated multi-hop corpus or additional task-specific SFT stage.Source: §4.2 · p. 6, Table 1 · p. 7
• 51.58% average accuracy places the 8B MTA-Zero Solver within 0.40 percentage points of GPT-5’s 51.98% under the same Agent Workflow.Stronger commercial agents remain ahead overall.Source: §4.2 · p. 7

5.1 DOES THE QA AGENT LEARN AN ADAPTIVE CURRICULUM?

• With Q0 fixed, the informative-group rate fell from 55.8% to 44.3% as the Solver advanced from S0 to S50, indicating mismatch with the evolving Solver.The informative-group rate counts questions with 0 < pi < 1, which yield non-degenerate Solver GRPO groups.Source: §5.1 · p. 7
• With S50 fixed, the informative-group rate rose from 44.3% for Q0 to 56.8% for Q50, a 12.5-point gain attributed to the learned QA policy.The comparison used the same 512 generated questions, seed contexts, and fixed harness.Source: Table 2 · p. 8, §5.1 · p. 8

5.2 ABLATING THE QA DIFFICULTY TARGET

• 49.1% overall for target τ = 0.25 was 3.3 points below the 52.4% for τ = 0.5 after the same 50-step co-training budget.The authors argue that τ = 0.5 preserves a wider margin of mixed-success groups for Solver learning than the lower target.Source: Table 3 · p. 8, §5.2 · p. 8

5.3 DOES UPDATING THE QA POLICY IMPROVE THE SOLVER?

• 52.40% average accuracy with an adaptive QA policy exceeded the frozen-QA control by 8.12 points and the base Solver by 16.53 points.The frozen QA policy and online harness reached 44.28% (+8.41 points) over the 35.87% base Solver under the same workflow.Source: Table 4 · p. 9, §5.3 · p. 9

5.4 SCALING TO A LARGER, NEWER BACKBONE

• 57.70% overall OOD accuracy for Qwen3.6-27B was 10.82 points above its 46.88% base accuracy, with gains on all four constituent benchmarks.Reported external 27–32B agents are not a controlled ranking because benchmark subsets, tool stacks, search budgets, and judges differ.Source: Table 5 · p. 9, §5.4 · p. 9
• Reported external 27–32B agent scores do not support a controlled ranking against MTA-Zero because evaluation subsets, tools, search budgets, and judges differ.The scaling claim is therefore based on the matched Qwen3.6-27B base-versus-MTA-Zero comparison.Source: §5.4 · p. 9

6 CONCLUSION

• Separate GRPO objectives train the Generator to propose difficult-but-solvable questions and the Solver on mixed-outcome groups, without a pre-generated multi-hop corpus or task-specific SFT stage.The policies are updated from the Solver's group rollout outcomes.Source: §6 · p. 9
• 36.91% to 51.58% mean accuracy: MTA-Zero improves Qwen3-VL-8B across six benchmarks and also improves Qwen3.6-27B.The conclusion further reports that static QA policies become increasingly mismatched as the Solver changes, while updating the QA policy restores useful training groups.Source: §6 · p. 10

AI USE STATEMENT

• The authors report using generative AI for literature organization, plotting and analysis code, and experiment orchestration and debugging, while retaining responsibility for the reported work.They state that they reviewed and tested AI-assisted code and verified results against experiment outputs and raw logs.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

• Live web retrieval can surface personal, copyrighted, biased, outdated, or unreliable material, and deep-search agents carry dual-use risks.For high-stakes deployments, the authors recommend access controls, source review, and human oversight.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The authors describe protocol-level reproduction while noting that exact replay of live retrieval and proprietary model-assisted checks may vary with web content and service versions.They separate deterministic checks from model-assisted checks and report their roles and failure behavior.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 BRIDGE-ENTITY DEFINITION AND GENERATOR STATE

• A bridge entity is an unambiguous, evidence-supported proper-name answer at a non-final hop that can anchor the next retrieval step.The final hop may instead end in a short factual answer such as a count, measurement, or date when that option is enabled.Source: §A.1 · p. 13
• The policy proposes the complete answer, while the harness derives a short canonical form for the next subquestion and a disambiguated form for the next search query.The state also includes the seed image, merged question, accepted-answer history, latest harness observation, and remaining hop/turn budget.Source: §A.1 · p. 13

A.2 A RECORDED QA GENERATION TRAJECTORY

• The recorded five-hop trajectory ends with 3/8 correct Solver rollouts, illustrating a challenging but not uniformly unsolved question.Only the accepted path is shown; rejected candidates receive no QA policy credit, and construction-time source URLs are omitted from the Solver input.Source: §A.2 · p. 13

A.2.1 MONUMENT VALLEY →WINDOW ROCK NAVAJO TRIBAL PARK

• The accepted chain moves from the image's protected area through the Navajo Nation, Window Rock, and Tśegh́ahoodźańı to the park's official name.The final merged task asks for the official name of the park where the arch is located, preserving the chain back to the image.Source: §A.2.1 · p. 14

B QA HARNESS CHECKS AND FEEDBACK

• The fixed harness retrieves and filters evidence, validates proposed hops, and merges accepted QA pairs; its checks and parameters are not optimized during co-training.The harness exposes structured accept/reject feedback to the online policy and masks rejected action spans.Source: B QA HARNESS CHECKS AND FEEDBACK · p. 15

B.1 RECORDED HARNESS FEEDBACK

• Accepted feedback supplies the merged question and next entity, whereas rejection preserves the last accepted state, gives a diagnosis, and requests another search angle.Only accepted-hop search actions and proposals receive QA policy-gradient signal; rejected spans remain visible in context but are masked from the loss.Source: B QA HARNESS CHECKS AND FEEDBACK · p. 15, §B.1 · p. 16

B.2 EXTERNAL QUALITY AUDIT OF GENERATED QA TASKS

• The audit assesses 512 tasks from each of Q0 and Q50 under a fixed rubric covering validity, evidence-chain support, uniqueness, self-containedness, leakage, and image dependence.The authors caution that the audit does not rule out every shortcut or establish that every mixed-outcome group tests the intended reasoning capability.Source: §B.2 · pp. 16–17
• Evidence/chain support rises by 10.0 percentage points and self-containedness by 3.3 points from Q0 to Q50, while image dependence is 73.8% and 73.0%, respectively.The authors report increases on all five structural checks and no detectable degradation in direct signatures of ambiguity, unsupported chains, answer leakage, or image irrelevance.Source: §B.2 · p. 17

B.3 LEARNING BOUNDARY OF THE FIXED HARNESS

• The fixed harness constructs tasks but does not select search actions or author candidate QA pairs; those are outputs of the trainable Generator.Harness observations are masked from QA loss, while Generator-produced spans for accepted hops remain trainable and rejected-attempt spans are masked.Source: §B.3 · p. 17
• The Generator's Goldilocks reward peaks at the target Solver success rate, while Solver GRPO uses individual binary outcomes and retains only mixed-success groups.Both policies update from the same pre-update rollout batch, with separate parameters and optimizers; the Solver does not receive Generator intermediate answers or evidence.Source: §B.3 · pp. 17–18

C SOLVER INTERACTION AND TOOL INTERFACES

• Each Solver question receives eight complete ReAct rollouts, scored through answer normalization and matching, with a semantic judge for unresolved aliases or paraphrases.A rollout ends with a final answer, a stop decision above 0.7 confidence, or the 10-step budget; the reported multi-tool environment has six callable tools.Source: C SOLVER INTERACTION AND TOOL INTERFACES · p. 19, §C.1 · p. 19

D TRAINING AND EVALUATION DETAILS

• Seed contexts must have an image-dependent question, a visually correct answer, and one specific answer entity; only the image, question, and answer are retained.Previously generated extensions, evidence, and tool trajectories are discarded.Source: §D.1 · p. 20
• The fixed seed pool contains 1,019 contexts: 398 from InfoSeek, 300 from FVQA-train, 190 from InfoVQA, and 131 from OK-VQA.The online loader withholds source-chain and answer metadata, requiring MTA-Zero to construct each training-time multi-hop task online.Source: §D.1 · p. 20
• Although source records store a disambiguated answer field, the Generator receives only the image and one-hop question–answer pair.The comparison to MTA-Agent starts from its filtered one-hop contexts, while MTA-Zero builds the extensions online.Source: §D.1 · p. 20

• Both policies begin from separate copies of Qwen3-VL-8B-Instruct.Source: §D.2 · p. 20
• The fixed seed pool combines four public one-hop VQA sources, and each seed supplies an image and one question–answer pair.The pool contains neither multi-hop questions nor retrieved evidence.Source: §D.2 · p. 20
• Each iteration schedules 64 QA-generation trajectories and 512 Solver rollout inputs, with separate parameters, optimizers, and GRPO updates.Source: §D.2 · p. 20
• The only Solver-to-Generator signal is a scalar rollout-derived score; trajectories, hidden states, and gradients are not shared.The main run does not enable a KL/reference-policy loss.Source: §D.2 · p. 20

• The main 8B run samples 400 seed-context draws from a fixed pool of 1,019 public image-grounded VQA contexts.Source: §D.3 · p. 20
• The pool comprises 398 InfoSeek, 300 FVQA-train, 190 InfoVQA, and 131 OK-VQA examples.Source: §D.3 · p. 20
• Each source item contains an image and one-hop QA pair, with no retrieved evidence, multi-hop question, or tool trajectory.Source: §D.3 · p. 20
• Table 14 reports verified allocation, timing, and external-service workload for the same main 8B run.Source: §D.3 · p. 20

• Evaluation extracts the final boxed answer when present, otherwise the final response text, and first applies deterministic normalized matching against acceptable answers.The rule stage handles capitalization, whitespace, and standard multiple-choice formatting.Source: §D.4 · pp. 20–21
• When rule-based matching fails, gpt-5-mini judges whether the extracted answer conveys the same core information as an accepted reference.A positive verdict from either stage counts as correct.Source: §D.4 · p. 21
• 98% agreement with human correctness labels was found in a manual re-evaluation of 100 LLM-judged answers.Source: §D.4 · p. 21