LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPO

• The study targets the gap between costly frontier or large-scale agent training and the question of what released search trajectories teach small VLMs.Multimodal search agents combine retrieval and image-processing tools because many knowledge-intensive visual questions cannot be answered from the image alone.Source: §1 · p. 1
• The low-compute recipe trains LoRA adapters on released trajectories, probes a compact three-tool variant, and builds GPT-5-generated step-level DPO pairs.The vision tower is frozen and the multimodal projector is trainable; training uses a single eight-GPU node.Source: §1 · p. 2
• After one epoch of full-trajectory SFT, the 2B model rises from 0.1% to 28.4% macro Pass@1 and matches the 4B base on three easier benchmarks.The 2B base emitted no usable final answer on 1,237 of 1,240 questions; its SFT gain is larger than the 4B base's because the 4B already follows the agent contract on many questions.Source: §1 · p. 2
• Synthetic step-level DPO raises macro average by +1.0 points on 2B and +1.7 on 4B, while compact tools reduce 2B performance but are nearly cost-free on 4B overall.DPO's largest localized gain is on FVQA, with non-uniform effects across domains; the compact interface is strongest on FVQA.Source: §1 · p. 2
• On VDR, increasing the step budget from 5 to 8 substantially reduces no_answer but barely changes Pass@1, as recovered rollouts mostly become wrong_entity.This points to answer verification, rather than search depth, as the remaining bottleneck.Source: §1 · p. 2

Agent loop

• The agent conditions each step on the growing visual context, question, prior actions, and observations, then emits reasoning plus a tool call or final response.The deterministic environment returns an observation that is appended to the history; a rollout ends at a response block or when the step budget is reached.Source: §2 · p. 2

Tool environment

• The reimplemented environment matches OpenSearch-VL’s seven-tool schema, prompt format, and observation tokenization, with indexed image outputs available to later calls.Tool observations are wrapped in <observation> tags to match training-time tokenization for stable inference.Source: §2 · pp. 2–3

Stage A: supervised distillation from released trajectories

• Stage A fine-tunes Qwen3-VL-2B and 4B on 36,092 released trajectories, expanding them into 133,439 prefix-to-next-step examples for controlled comparison.The stratified split includes 500 validation trajectories; LoRA adapters use rank 16 and α = 32, while Step-SFT holds optimizer steps and batch size constant to isolate the supervision unit.Source: §2 · p. 3

Stage B: tool-distilled SFT

• Stage B restricts the action space to image_search, text_search, and crop, retaining 33,243 training and 500 validation trajectories after filtering.The whitelist removes 2,849 records (7.8%) from the 36,592-record pool; a compact prompt enables a 6,144-token cutoff, with other training settings matched to Stage A.Source: §2 · p. 3

Stage C: synthetic step-level preference optimization

• Stage C uses GPT-5 to generate step-level hard negatives for five failure modes: premature answers, wrong tools, weak or repeated queries, and ignored images.A five-check programmatic validator filters candidates; prompt refinements reduce the audited issue rate from 54% to 0%, and the pipeline yields 324,121 accepted pairs from about 96K teacher decisions.Source: §2 · p. 3
• DPO [6] trains for one epoch on the filtered preference set using β = 0.3 and a learning rate of 10^-6, after the standard larger-backbone recipe collapsed the 2B model early.Length filtering retains about 234K training and 3K validation pairs, approximately 73% of the full collection; hyperparameters are selected using held-out preference reward gaps.Source: §2 · p. 3

Benchmarks and inference

• Across FVQA, LiveVQA [13], SimpleVQA [14], and VDR [15], ten model cells each generate 1,240 rollouts (12,400 total) under a five-step default budget.Models use eight parallel single-GPU replicas; images are downscaled to a maximum edge of 360 px, and persistent caching reduces per-pass search API costs by roughly an order of magnitude.Source: §3 · p. 4

LLM-as-judge and models compared

• GPT-5 judges final answers against each image, question, and reference; Pass@1 is the fraction with correct=1, and partial-credit rate is Pass@1 + 0.5 · partial.All 12,400 judgments parse as JSON; confidence saturates at 2 and is treated only as a pipeline-stability diagnostic.Source: §3 · p. 4
• Bootstrap confidence intervals over benchmark questions are the primary uncertainty measure, and the authors recommend human spot-checking before treating any single judgment as ground truth.The intervals are reported in Table 7, Appendix D.Source: §3 · p. 4
• The comparison covers 2B/4B base, full-trajectory SFT, step-level SFT, 3-tool SFT, and SFT+DPO, excluding larger OpenSearch-VL agents to assess a single-node operating point.The training-regime contrast is in Table 4, Appendix B.Source: §3 · p. 4

4 Results

• +1.0 average points on 2B and +1.7 on 4B from synthetic step-level DPO are directional refinements, with overlapping macro confidence intervals.At 4B, FVQA rises from 61.0% to 69.5% (+8.5 points), while LiveVQA falls 2.8 points relative to SFT.Source: §4 · p. 4
• 28.4% macro Pass@1 after full-trajectory SFT lifts 2B from 1,237 unusable-answer rollouts to 514 and matches or exceeds 4B base on FVQA, SimpleVQA, and LiveVQA.The 2B SFT scores are 64.4% vs. 61.0% on FVQA, 30.8% vs. 24.6% on SimpleVQA, and 16.6% vs. 16.0% on LiveVQA.Source: §4 · p. 4, Table 1 · p. 5
• On 2B, step-level SFT loses 1.5 average points and raises no_answer from 514 to 780; on 4B, it gains 1.0 point while no_answer rises from 587 to 682.The results indicate that full-trajectory supervision more reliably teaches the smaller model the complete agent contract, while local transition supervision can help the larger model choose intermediate actions.Source: §4 · p. 5
• Restricting the interface to three tools costs 2B 9.0 average points and 4B 1.8, but 4B 3-tool SFT reaches 71.2% Pass@1 on FVQA.The 4B 3-tool model has fewer wrong_entity errors than full-tool 4B SFT (242 vs. 298), but more no_answer failures (766 vs. 587).Source: §4 · p. 5
• SFT collapses the base models’ dominant no_answer pile into a roughly even split of correct and wrong_entity outcomes; DPO reduces 4B no_answer from 587 to 511 and raises correct from 248 to 254.These error-mode totals aggregate the GPT-5 judge’s labels across 1,240 questions per model cell.Source: §4 · p. 5
• Raising VDR MAX_STEPS from 5 to 8 cuts no_answer by 33–50% on every model, but net Pass@1 stays within ±1 point as extra rollouts mostly become wrong_entity errors.Both DPO checkpoints improve slightly: 2B by 0.8 points and 4B by 1.0; the results identify evidence verification, not search depth alone, as the bottleneck.Source: §4 · p. 5, Table 2 · p. 6

5 Discussion and Conclusion

• SFT repairs 2B’s near-total failure to emit a final response, but wrong_entity errors remain around 30%; on 4B, SFT mainly polishes tool selection.The 2B base emits a final <response> on only 3 of 1,240 questions, and only one is correct.Source: §5 · p. 6
• Synthetic DPO is a refinement rather than a substitute for SFT, with gains on average but regressions and redistribution across benchmarks.At 4B, FVQA Pass@1 rises 8.5 points while LiveVQA falls 2.8 points; no_answer across four benchmarks drops 13% (587 →511), and 2B partial-credit is 35.4 in both SFT and DPO rows.Source: §5 · p. 6
• The 3-tool ablation shows a capacity-dependent trade-off: the full tool set adds overhead, while the compact set loses specialized operations needed by some questions.Larger backbones appear more robust to the full set’s overhead, but can still lose coverage when OCR or perspective correction is required.Source: §5 · p. 6
• Three extra VDR steps turn no_answer into wrong_entity rather than correct answers, pointing to verification—not exploration—as the small-scale bottleneck.The agent can identify relevant regions and retrieve plausible passages but does not reliably cross-check evidence against the question; second-pass verification or process reward models [17] [18] are proposed directions.Source: §5 · p. 6
• All ten model cells use LoRA on one eight-GPU node, with a shared agent environment and GPT-5 judge; configurations, code, pipelines, judge prompt, and search-cache schema are released.The release materials are described in Appendix J.Source: §5 · p. 6
• Full-trajectory SFT takes 2B from almost never answering to 28.4% macro Pass@1; the best 4B configuration reaches 30.8% macro Pass@1 and 37.1% partial-credit.The conclusion reports results across four short-answer benchmarks and 12,400 GPT-5-judged rollouts.Source: §5 · p. 6

A Related Work

• Prior agentic-search work includes browser QA with human feedback in WebGPT [19], think-then-act prompting in ReAct [20], tool learning in Toolformer [21], and retrieval-focused RL in Search-R1 [22].OpenSearch-VL [3] released 36K supervised trajectories and an 8K RL prompt set, training 8B–32B Qwen3-VL backbones with fatal-aware GRPO [27]; LiteSearch-VL reuses its data and tool schema for a smaller, cheaper setting.Source: A Related Work · p. 9
• LiteSearch-VL applies adapter tuning and local step preferences to multimodal agent rollouts, building on LoRA [28], QLoRA, DPO [6], Step-DPO [32], and process reward models [17] [18].The evaluation uses Qwen3-VL [33] and four short-answer benchmarks—FVQA, LiveVQA [13], SimpleVQA [14], and VDR-Bench [15]—with LlamaFactory [41], vLLM [16], and DeepSpeed for training and serving.Source: A Related Work · p. 9

B Pipeline Overview, Data Statistics, and Compute Footprint

• The pipeline trains full-trajectory SFT, 3-tool distilled SFT, and SFT+DPO checkpoints from SearchVL-SFT-36k, then evaluates each in the same seven-tool environment.GPT-5 adjudicates every rollout with a fixed five-field rubric.Source: Figure 2 · p. 10
• The appendix reports SearchVL-SFT-36k corpus composition and defines median turns as the per-trajectory median across the train split, alongside validated preference-pair counts across five hard-negative modes.Each step record can contribute up to one pair per applicable mode.Source: Table 3 · p. 10, Figure 3 · p. 11
• Table 4 compares LiteSearch-VL’s maximum concurrent training allocation with OpenSearch-VL’s reported allocations, without normalizing for hardware throughput.The comparison is intended to clarify the compute regime.Source: Table 4 · p. 11

C Judge Confidence Saturation

• Confidence=2 appears in all 12,400 main-evaluation judgments and all 14,400 rows including the VDR step-budget ablation; confidence=0 or 1 never appears.The field therefore provides no discriminative signal and is a pipeline-stability diagnostic, not an independent reliability check.Source: C Judge Confidence Saturation · p. 11
• Short factoid references make reference matching plausibly clear-cut, while LLM self-rated confidence can saturate toward the top of its scale.The authors do not interpret uniform confidence as evidence that every judgment is correct.Source: C Judge Confidence Saturation · p. 11
• Because confidence does not filter or reweight reported Pass@1, partial-credit, or error-mode results, its saturation does not bias those reported numbers.The field is emitted consistently, but should be read as a JSON-parsing and value-emission diagnostic.Source: C Judge Confidence Saturation · p. 11

D Additional Results Tables and Figures

• The 4B+DPO checkpoint is the clear winner on macro partial-credit rate, while relative model rankings remain similar to strict Pass@1.Partial-credit rate is Pass@1 + 0.5 · partial; two cells have one-row discrepancies between implied partial responses and discrete error-mode labels, without affecting rates or rankings.Source: Table 5 · p. 12
• Macro Pass@1 uncertainty is estimated with benchmark-stratified question bootstrapping and percentile 95% confidence intervals over 10,000 replicates.Source: Table 7 · p. 13
• DPO reduces 4B no_answer from 587 to 511 and increases correct outcomes from 248 to 254.The figure describes SFT as collapsing the base model’s no_answer pile into a roughly even split of correct and wrong_entity outcomes.Source: Table 6 · p. 12, Figure 5 · p. 13
• Extra VDR step budget reduces no_answer by roughly one third to one half on every model, but mostly adds wrong_entity errors; Pass@1 moves within ±1 point.Source: Figure 6 · p. 14

E Hyperparameters and Compute

• SFT and DPO each run on a single eight-GPU node, using ZeRO-3 and ZeRO-2 respectively, with backbones, LoRA settings, and effective batch size held constant across stages.The hyperparameter table covers four distinct configurations; the 2B and 4B SFT runs differ only in backbone.Source: E Hyperparameters and Compute · p. 14
• SFT uses LoRA with r = 16 and α = 32 on language-tower linear projections, freezes the vision tower, and trains the multimodal projector.Training also uses 0.05 dropout, cosine learning-rate schedules with 3% warmup, gradient checkpointing, and image inputs clipped to 65,536 pixels.Source: E Hyperparameters and Compute · p. 14
• DPO uses sigmoid preference loss with β = 0.3, learning rate 10^-6, and the corresponding SFT checkpoint as its reference model.The standard β = 0.1 and learning rate 5×10^-6 recipe was unstable on the 2B backbone, a related instability to the DPO failure mode discussed in [10].Source: E Hyperparameters and Compute · p. 14

F Tool Environment

• The inference environment exposes seven distinct tools across eight callable names because web_search aliases text_search; the 3-tool variant retains image_search, text_search, and crop.The dispatcher maintains an img_N registry across turns, visual tools return a new img_N+1 reference, and observations use the <observation>Tool execution result: ... </observation> wrapper.Source: F Tool Environment · p. 14, Table 9 · p. 15

G GPT-5 Judge Prompt

• The GPT-5 judge receives each benchmark image, question, reference answer, and model response, with the system rubric reproduced in the paper.The per-item prompt appends the question, reference, and truncated model response in labeled sections and ends with “Return STRICT JSON only.”Source: G GPT-5 Judge Prompt · pp. 14–15

H Synthetic Preference Generator

• For each step, GPT-5 receives the image, conversation prefix, chosen next action, and gold answer solely for leak detection, then generates a hard alternative for each applicable failure mode.Source: H Synthetic Preference Generator · p. 15
• Eight rules require alternatives to stay in character, use valid image handles and tool schemas, match the failure mode, avoid answer leaks, preserve format, and remain plausible.Output must be strict JSON keyed by mode, with one entry per applicable mode.Source: H Synthetic Preference Generator · p. 16
• 54% to 0%: after adding in-character and existing-image-handle rules, the audited issue rate fell to zero on a fixed sample.A programmatic validator drops entries that violate formatting, mode-specific rules, answer-leak, meta-language, or image-argument checks.Source: H Synthetic Preference Generator · p. 16

I Per-Domain Validation Statistics

• The held-out validation split contains 500 records across domains; its fvqa and livevqa subsets are promoted to standalone benchmark tracks.Source: Table 10 · p. 16, I Per-Domain Validation Statistics · p. 16
• The fvqa and livevqa tracks use their corresponding subsets, while the other five domains are retained for ablations and omitted from the main table.Source: Table 10 · p. 16
• The FVQA track evaluates 59 of 60 fvqa records because one record lacks the public URL required by image_search.Source: Table 10 · p. 16

J Reproducibility Checklist

• The ten model cells use configuration files sharing a template, with only the base model, dataset, and training stage differing.Source: J Reproducibility Checklist · p. 16
• The train/validation split and step-level expansion are deterministic given a fixed seed, and the playground and evaluation harness share the same agent loop, dispatcher, and search cache.Source: J Reproducibility Checklist · p. 16
• The fixed GPT-5 judge prompt gives every judged row maximum self-rated confidence, which the authors do not treat as a reliability guarantee; search results are cached across model cells.Error-shaped observations are not cached.Source: J Reproducibility Checklist · p. 17

K Qualitative Case Studies

• Five rollouts from the evaluation harness illustrate training-stage changes; the first four cases cover stage-wise progress and DPO improvements, while the final case examines VDR verification failures.Trajectories are excerpted from evaluation outputs, with long reasoning blocks, observations, and intermediate calls elided for readability.Source: K Qualitative Case Studies · p. 17

K.1 Case 1: SFT teaches the agent contract (2B Base →2B SFT)

• FVQA item fvqa_002606 shows an ornamental shrub branch bearing clusters of small pink, winged seed pods and asks, “What is the name of this plant with pink seed pods?”Source: §K.1 · p. 17

K.3 Case 3: DPO closes the loop on a multi-tool rollout (4B SFT →4B DPO)

• In SimpleVQA item simplevqa_1057, the question asks for the total price of a Merlot at $139.99 and a Cabernet at $109.99 shown in a Costco wine display.Source: §K.3 · p. 19

K.4 Case 4: DPO avoids over-specification (4B SFT-partial →4B DPO-correct)

• The K.4 case uses FVQA item fvqa_004018, depicting a steel-arch road bridge over a tropical waterway, and asks when it officially opened.The supplied passages describe the case setup and question, not its outcome.Source: §K.4 · p. 20

K.5 Case 5: A failure that survives DPO — verification on VDR

• The K.5 VDR case asks for the name of a broad-leaved plant at the top of an overhead garden-bed image.The supplied passages describe the case setup and question, not its outcome.Source: §K.5 · p. 21

L Broader Impacts

• Making tool-using multimodal search agents workable at 2B–4B parameters lowers compute and cost barriers for on-device and low-resource settings.This accessibility motivates the work.Source: L Broader Impacts · p. 23
• Agents may retrieve real web evidence yet confidently commit to a wrong primary entity on a substantial fraction of questions.The paper points to its error analysis in Section 4 and Appendix K.Source: L Broader Impacts · p. 23
• In the VDR ablation, more search turns convert abstentions into confident errors rather than correct answers.The paper identifies verification-aware training as a priority for future work.Source: L Broader Impacts · p. 23
• Deployments should not treat these agents’ answers as verified, and the submission releases no new model weights or scraped corpora.The paper frames verification-aware training as future work.Source: L Broader Impacts · p. 23