PaperlayerView exact revision on arXiv

Source-linked AI summary

LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPO

Saeed Khaki, Nima Safaei, Kamal Ginotra

arXiv:2608.29357v1cs.AI

TL;DR

Small multimodal search agents need more than visual question answering: they must retrieve evidence, use tools, and know when to answer. LiteSearch-VL distills released teacher trajectories into Qwen3-VL-2B/4B with parameter-efficient training and synthetic step preferences. Full-trajectory SFT primarily transfers the agent contract, while later refinements yield smaller gains and extra search turns expose verification as the remaining bottleneck.

  • Problem

    The paper asks what released multimodal search trajectories transfer to small VLMs when stronger systems require proprietary models or large-scale agentic training.

  • Method

    LiteSearch-VL combines released OpenSearch-VL trajectories, LoRA adaptation, compact tool distillation, and GPT-5-generated step-level DPO negatives for Qwen3-VL-2B/4B.

  • Results

    Full-trajectory SFT raises the 2B model from 0.1% to 28.4% macro Pass@1, while the best 4B configuration reaches 30.8% macro Pass@1.

  • Takeaways & Limitations

    The dominant transfer is behavioral agent-contract competence; DPO and compact tools refine it, while extra VDR search mainly changes abstentions into wrong-entity errors.

  • Takeaways & Limitations

    After SFT, wrong_entity remains roughly 30% of 2B questions, and DPO is non-uniform across domains, including a 2.8-point LiveVQA loss at 4B.

Abstract

from arXiv · show

Multimodal search agents answer visual questions by interleaving image understanding, web retrieval, tool use, and evidence synthesis. Strong systems exist, but in two expensive regimes: proprietary frontier models such as GPT-5 and Gemini, or large open vision-language backbones trained with substantial agentic data and reinforcement learning. We ask a different question: when released agent trajectories are distilled into much smaller backbones under a single-node budget, what is actually transferred? We study this with LiteSearch-VL, a low-compute recipe for Qwen3-VL-2B and Qwen3-VL-4B that uses only released OpenSearch-VL trajectories, parameter-efficient LoRA adapters, and synthetic step-level preferences: DPO on GPT-5-generated hard negatives targeting five local failure modes (premature answer, wrong tool, weak query, repeated query, ignored image). Across 12,400 GPT-5-judged rollouts on SimpleVQA, FVQA, LiveVQA, and VDR-Bench-testmini, the dominant effect is behavioral rather than a uniform accuracy lift: full-trajectory supervised fine-tuning transfers the agent contract, taking the 2B model from almost never emitting a usable answer (1,237/1,240 no_answer rollouts) to 28.4% macro Pass@1, matching or slightly exceeding the off-the-shelf 4B base (25.6%). Synthetic preference learning and compact tool distillation act as refinements rather than phase transitions (best 4B configuration: 30.8% macro Pass@1). Finally, a controlled VDR step-budget ablation shows that extra search turns convert abstentions into wrong_entity errors rather than correct answers, identifying answer verification, not search depth, as the next bottleneck for small multimodal agents.

1 Introduction

LiteSearch-VL asks what released multimodal search trajectories transfer to small VLMs under a constrained training budget. The main transfer is behavioral: full-trajectory SFT teaches the 2B model to operate as an agent, while later methods provide smaller refinements.

  • Multimodal search agents combine visual understanding, retrieval, image processing, and evidence-based answer decisions for knowledge-intensive visual questions.
  • Released OpenSearch-VL trajectories enable studying whether small VLMs learn factual competence or the basic agent contract of tool use, image handles, evidence integration, and termination.
  • LiteSearch-VL trains Qwen3-VL-2B and Qwen3-VL-4B with LoRA, released trajectories, compact tool distillation, and GPT-5-generated step-level preference pairs.
  • 28.4% macro Pass@1 follows full-trajectory SFT for the 2B model, versus 0.1% before training, matching or slightly exceeding the off-the-shelf 4B base.
  • Synthetic DPO and compact tool distillation act as refinements, while extra VDR search turns mainly convert no_answer into wrong_entity errors, implicating verification rather than search depth.

2 Setup and Method

The method distills released teacher trajectories into small multimodal agents through a staged pipeline: agent-loop modeling, supervised trajectory learning, compact tool distillation, and synthetic step-level preferences.

  • Agent loop: Each agent step conditions on visual context, the question, prior actions, and observations, then emits reasoning plus either a tool call or final response.
  • Tool environment: The full environment preserves OpenSearch-VL’s seven-tool schema, observation tokenization, and indexed image registry for reusable visual outputs.
  • Stage A: supervised distillation: Stage A applies supervised distillation to the released SearchVL-SFT-36k corpus after compressing observations and creating deterministic stratified splits.
  • Stage B: tool-distilled SFT: Stage B restricts agents to image_search, text_search, and crop, filtering disallowed-tool trajectories and shortening the prompt surface for a compact action space.
  • Stage C: synthetic step-level preference optimization: Stage C uses GPT-5 to generate step-level hard negatives across five local failure modes, providing synthetic preferences for DPO.

3 Evaluation Protocol

The evaluation measures small-agent behavior across four knowledge-intensive visual-question benchmarks using standardized rollout, judging, and uncertainty procedures.

  • Benchmarks and inference: The study evaluates ten model cells across SimpleVQA, FVQA, LiveVQA, and VDR-Bench-testmini.
  • Benchmarks and inference: Each evaluation pass contains 12,400 GPT-5-judged rollouts, with 1,240 rollouts per model and a default five-step budget.
  • LLM-as-judge and models compared: GPT-5 judges correctness, partial correctness, error mode, confidence, and rationale from the image, question, reference answer, and final response.
  • LLM-as-judge and models compared: Pass@1 is the fraction judged correct, while partial-credit rate equals Pass@1 plus half the partial-credit fraction.
  • LLM-as-judge and models compared: Bootstrap confidence intervals over benchmark questions are the primary uncertainty measure, and human spot-checking is recommended before treating individual judgments as ground truth.

4 Results

Across four benchmarks, full-trajectory SFT produces the largest behavioral change, while DPO, step-level SFT, compact tools, and extra VDR steps mainly expose narrower trade-offs and verification limits.

  • DPO refinement: +1.0 average points on 2B and +1.7 average points on 4B come from synthetic step-level DPO, with a largest 4B FVQA gain of +8.5 points but a 2.8-point LiveVQA loss.The macro confidence intervals overlap, so DPO is characterized as a directional refinement rather than the main transition.
  • SFT and agent behavior: 1.5 average points are lost by step-level SFT on 2B, while 4B gains 1.0 points despite more no_answer cases, indicating a backbone-dependent terminal-behavior trade-off.Full-trajectory supervision more reliably teaches the complete contract to the smaller model.
  • Tool-surface trade-offs: –9.0 average points result from restricting 2B to three tools, compared with –1.8 average points at 4B; the 4B compact interface is strongest on FVQA at 71.2% Pass@1.At 4B, fewer wrong_entity errors accompany more no_answer failures.
  • VDR step-budget ablation: 33–50% fewer no_answer cases result from raising VDR MAX_STEPS from 5 to 8, but net Pass@1 stays within ±1 point as additional rollouts mostly become wrong_entity.The ablation identifies evidence verification, rather than search depth alone, as the remaining bottleneck.

5 Discussion and Conclusion

LiteSearch-VL finds that small multimodal agents primarily acquire the agent contract through full-trajectory SFT, while DPO and tool reduction provide narrower refinements. Remaining errors expose capacity-dependent tool trade-offs and answer verification as the main bottleneck.

  • Agent-contract transfer: Full-trajectory SFT transfers the agent contract, whereas the 2B SFT model still produces wrong_entity errors on roughly 30% of questions.The 2B base emits a final response on only 3 of 1,240 questions, while SFT substantially repairs terminal behavior.
  • Preference optimization: +1.0 points on 2B and +1.7 points on 4B: synthetic step-level DPO improves macro Pass@1 but acts as a refinement rather than a substitute for SFT.The effect is non-uniform: 4B FVQA gains +8.5 points, while 4B LiveVQA loses 2.8 points.
  • Tool space and capacity: The compact three-tool interface exposes a capacity-dependent trade-off between reducing cognitive overhead and losing specialized OCR or perspective-correction operations.Larger backbones appear more robust to the full tool set’s overhead, but compact tools can remove coverage needed by some questions.
  • Verification bottleneck: Three extra VDR steps mostly convert no_answer into wrong_entity, indicating verification rather than search depth as the remaining bottleneck.The agent can identify relevant regions and retrieve plausible passages but lacks reliable cross-checking against the question.
  • Reproducibility: All ten model cells use LoRA on one eight-GPU node and share the same agent environment and GPT-5 judge.The authors release training configurations, code, data pipelines, the judge prompt, and the search-cache schema.

B Pipeline Overview, Data Statistics, and Compute Footprint

The pipeline overview connects released trajectory data to three training families and a shared evaluation environment, while the appendix organizes corpus, preference-pair, compute, and judge-confidence diagnostics. These materials frame the study’s controlled low-compute regime and evaluation procedure.

  • Pipeline overview: Figure 2 reuses the SearchVL-SFT-36k corpus for full-trajectory SFT, three-tool distilled SFT, and SFT+DPO with mode-conditioned negatives.Every checkpoint is evaluated in the same seven-tool environment with GPT-5 using a fixed five-field rubric.
  • Synthetic preferences: Figure 3 reports validated synthetic preference pairs across five hard-negative failure modes, with each step record contributing up to one pair per applicable mode.The modes are generated and filtered as step-level training signals.
  • Compute footprint: Table 4 compares LiteSearch-VL’s maximum concurrent training allocation with the reported OpenSearch-VL allocation to clarify compute regime.The comparison is not normalized for hardware throughput.
  • Judge diagnostics: All 12,400 main-evaluation judgments report confidence=2, so the confidence field functions as a pipeline-stability diagnostic rather than discriminative evaluation signal.The confidence field is not used to filter, reweight, or affect Pass@1, partial-credit, or error-mode results.

D Additional Results Tables and Figures

The additional results clarify how partial credit, error modes, uncertainty, training-stage gains, and step budgets shape the evaluation picture. Across these views, 4B+DPO leads on macro partial credit, SFT mainly removes abstentions, and extra VDR steps mostly create wrong-entity errors.

  • Additional results: 4B+DPO is the clear winner on macro partial-credit rate, which counts Pass@1 plus half-credit for partial answers.Partial credit captures right-entity, wrong-sub-attribute answers that strict Pass@1 misses.
  • Error modes: The 2B base model produces almost no final answers, making no_answer the primary error pattern addressed by SFT.Table 6 counts 1,240 judged questions per model and distinguishes correct, partial, and mutually exclusive failure modes.
  • Uncertainty: Bootstrap intervals in Table 7 quantify uncertainty for macro Pass@1 by resampling questions independently within each benchmark.The intervals use 10,000 bootstrap replicates.
  • Training-stage gains: Figure 4 compares absolute gains from each training stage with average Pass@1 across regimes for the 2B and 4B backbones.The figure provides both per-stage and cross-backbone views of the main result.
  • Error modes: Figure 5 shows SFT collapsing the base model’s no_answer pile into a roughly even split of correct and wrong_entity outcomes.On 4B, DPO further reduces no_answer from 587 to 511 and increases correct from 248 to 254.
  • Step-budget ablation: Extra VDR step budget reduces no_answer by roughly one third to one half, but mostly produces wrong_entity errors and keeps Pass@1 movement within ±1 point.The ablation compares four models and indicates that additional search turns do not materially improve correct answers.

E Hyperparameters and Compute

The compute setup standardizes training and inference interfaces across runs while fitting SFT and DPO on single eight-GPU nodes. The appendix also documents the compact three-tool variant and the GPT-5 judging protocol.

  • Compute: SFT and DPO each run on a single eight-GPU node, using ZeRO-3 for SFT and ZeRO-2 for DPO.Backbones, LoRA configuration, and effective batch size remain identical across stages for clean comparisons.
  • SFT hyperparameters: LoRA uses rank r = 16 and scale α = 32 on language-tower linear projections, while the vision tower stays frozen and the multimodal projector remains trainable.Training also uses 0.05 dropout, cosine schedules, 3% warmup, checkpointing, and images capped at approximately 256 × 256.
  • DPO hyperparameters: DPO uses sigmoid preference loss with β = 0.3 and learning rate 10^-6, with the corresponding SFT checkpoint as reference model.The standard β = 0.1 and 5×10^-6 recipe was unstable on 2B.
  • Evaluation: The judge receives the benchmark image, question, reference answer, and model response before applying a fixed structured rubric.The per-item prompt ends with a strict-JSON directive.
  • Tool environment: The inference environment exposes seven distinct tools, with web_search aliased to text_search; the distilled variant restricts actions to image_search, text_search, and crop.The dispatcher maintains image handles across turns and visual tools return new references.

H Synthetic Preference Generator

The synthetic preference generator creates plausible, hard, step-level alternatives for defined failure modes and filters them with strict programmatic checks. Prompt tightening reduced the audited issue rate from 54% to 0%.

  • Generation: Each synthetic negative is conditioned on the image, conversation prefix, teacher action, and gold answer used only for leak detection.GPT-5 generates an in-character alternative action for each applicable failure mode.
  • Generation constraints: The generator enforces in-character behavior, valid existing image handles, and compliant tool schemas for alternative actions.These rules prevent negatives from revealing their mistake or fabricating image references and tools.
  • Validation: 54% to 0%: adding stay-in-character and existing-image-handle rules eliminated audited issues on a fixed sample.The validator checks format, mode-specific rules, answer leakage, meta-language leakage, and image-argument sanity before writing pairs.

J Reproducibility Checklist

The reproducibility materials document shared configuration, deterministic preprocessing, identical agent execution, and fixed judging, then illustrate behavior with benchmark-specific rollout cases. The examples include named visual entities and a multi-item price question.

  • Reproducibility: All ten model cells share a configuration template, differing only in base model, dataset, and training stage.This supports consistent reproduction of the reported runs.
  • Reproducibility: The preprocessing script deterministically rebuilds train-validation splits and step-level expansions from a fixed seed.
  • Inference: The shared agent loop, dispatcher, and search cache ensure identical inference behavior in the playground and evaluation harness.
  • Judging: The fixed GPT-5 judge prompt is applied identically to every model, although its uniformly maximum self-rated confidence is not treated as a reliability guarantee.The paper attributes this confidence pattern to clear-cut factoid matching rather than calibration.
  • Case studies: The rollout appendix compares five real evaluation cases using questions, gold answers, image descriptions, trajectories, final answers, and judge verdicts.
  • Case studies: The FVQA plant case identifies the pictured plant as Dodonaea viscosa, also known as hop-bush or hopseed bush.
  • Case studies: The FVQA crest case identifies the football club as FK Ekranas, a Lithuanian club.
  • Case studies: The SimpleVQA price case asks for the combined price of the Merlot and Cabernet bottles shown in the image.The displayed tags list the Merlot at $139.99 and the Cabernet at $109.99.

K.4 Case 4: DPO avoids over-specification (4B SFT-partial →4B DPO-correct)

This FVQA case presents a steel-arch bridge over a tropical waterway and asks when it officially opened; the recorded answer is 1962.

  • The image shows a steel-arch road bridge spanning a tropical waterway, with green hills in the background.
  • The question asks: “In which year did this bridge officially open?”
  • 1962 is given as the opening year for the Bridge of the Americas / Puente de las Américas in Panama.
  • The paper cautions that these agents should not be treated as verified answerers because they can retrieve real web evidence yet confidently identify the wrong primary entity.
Loading 2608.29357v1…