PaperlayerView exact revision on arXiv

Source-linked AI summary

ProRetrieval: Learning to Orchestrate Hybrid Search via Executable Program Synthesis

Chengsong You, Zhen Sun, Yunhai Hu, Junwei Zhou, Xiaoyu Cao, Binyu Li, Ziyan Zhao, Weiyao Wang, Liren Lu, Zhijie Ye, Yumo Cao, Yitao Long, Yiwei Xu, Qiyi Jiang, Xuanyi Fu, Yufan Chen, Yilun Li, Rongkang Xiong, Yiran Zou, Nan Du

arXiv:2608.27017v1cs.IR

TL;DR

Hybrid retrieval must combine structured, textual, and visual constraints under arbitrary Boolean logic, but existing methods restrict either composition or backend orchestration. ProRetrieval has a language model synthesize executable SQL-centered hybrid DSL programs and trains them with hierarchical reinforcement-learning rewards. On Amazon and Enron benchmarks, its 4B model surpasses GPT-5.5 and broad baseline suites, while evaluation remains bounded by the constructed domains, logical depth, and schema assumptions.

  • Problem

    Existing hybrid retrieval methods have limited logical composition or operate over a single backend, leaving heterogeneous retrieval orchestration outside the model’s action space.

  • Method

    ProRetrieval trains a language model to synthesize executable programs interleaving SQL over structured fields with text and image vector primitives, optimized with hierarchical reinforcement learning.

  • Results

    Hit@1 0.81 vs. 0.69 on e-commerce and 0.91 vs. 0.86 on email show the 4B model surpasses GPT-5.5 on both benchmarks and broad baseline suites.

  • Takeaways & Limitations

    Retrieval program synthesis over a compositional DSL is presented as a promising direction for unified hybrid retrieval.

  • Takeaways & Limitations

    The evaluation is limited to two domains, bounded logical depth, moderate corpus scale, and a defined schema, with schema-free or evolving corpora left for future work.

Abstract

from arXiv · show

Real-world retrieval often composes structured constraints with semantic intents over text and images through arbitrary Boolean logic. Existing hybrid pipelines such as reciprocal rank fusion or self-querying retrievers admit only a fixed form of composition, while recent reinforcement-learning retrievers train the language model as a query generator for a single backend, leaving the orchestration of heterogeneous retrieval paths outside its action space. We propose ProRetrieval, which recasts the language model as a retrieval orchestrator: given a natural-language query, it synthesizes an executable program in a hybrid DSL interleaving SQL operators over structured fields with vector-retrieval primitives over text and images, with SQL itself providing the logical algebra that fuses heterogeneous candidate sets. We train Qwen3-4B with GRPO and DAPO under a hierarchical four-term reward, and evaluate on two new benchmarks built from Amazon products and Enron email. Our 4B model surpasses GPT-5.5 (Hit@1 0.81 vs. 0.69 on e-commerce; 0.91 vs. 0.86 on email) and Claude Opus 4.7 and a comprehensive suite of retrieval, LLM-augmented, structured-query, and graph-based baselines. Code: https://anonymous.4open.science/r/ProRetrieval/; data: https://huggingface.co/datasets/anonymous-7219/ProRetrieval.

1 Introduction

ProRetrieval addresses hybrid retrieval queries that combine heterogeneous modalities with arbitrary Boolean logic by synthesizing executable retrieval programs. It evaluates this approach on Amazon e-commerce and Enron email benchmarks, where its 4B model surpasses GPT-5.5 and broad baseline suites.

  • Motivation: Hybrid retrieval queries combine structured fields, text, and images under arbitrary Boolean connectives, requiring both retrieval-path selection and logical candidate-set composition.The motivating examples include nested disjunction and negation across heterogeneous backends.
  • Motivation: Existing retrievers and fusion pipelines are limited to single modalities, weighted disjunction, simple conjunctions, or a single backend, motivating a retrieval-orchestrator role.The proposed action space is an executable program rather than only a textual query.
  • Approach: ProRetrieval synthesizes executable hybrid DSL programs that interleave SQL operators with text and image vector primitives, using SQL as the logical orchestration skeleton.Vector results enter SQL through candidate-set placeholders, enabling structured and semantic predicates to be combined.
  • Approach: The model is trained with supervised fine-tuning followed by GRPO or DAPO reinforcement learning under a hierarchical four-term reward for format, execution, result, and length.The reward reflects layered correctness requirements for executable program generation.
  • Evaluation: The benchmarks cover Amazon e-commerce with structured, text, and image data and Enron email with structured and text data, spanning single-leaf to nested Boolean expressions.These benchmarks are constructed from public corpora.
  • Evaluation: Hit@1 0.81 vs. 0.69 on e-commerce and 0.91 vs. 0.86 on email show the 4B model surpasses GPT-5.5 on both benchmarks.The comparison is reported alongside wins over retrieval, reranking, structured-query, and graph-based baselines.

2 Related Work

Prior hybrid retrieval work combines modalities or applies reinforcement learning within constrained orchestration spaces. ProRetrieval instead treats the DSL as a learnable policy space optimized end-to-end for executable hybrid programs.

  • Hybrid retrieval: Sparse, dense, and multimodal retrievers operate within single representational spaces, while reciprocal rank fusion combines them only through weighted disjunction.Self-querying retrievers add simple key–value conjunctions but remain limited in logical expressiveness.
  • Hybrid retrieval: GraphRAG adds an orthogonal structured signal, but related pipelines keep orchestration logic fixed by the pipeline designer rather than adapting per query.This contrasts with ProRetrieval’s field-centric orchestration design.
  • RL for retrieval: Recent retrieval RL methods improve interaction with or reformulation for a single backend, including search, Boolean/SQL-like, and text-to-SQL settings.Their retrieval actions do not compose heterogeneous stores in one executable program.
  • DSLs for retrieval: Retrieval DSLs bridge language models and symbolic computation, with prior systems extending SQL for conversational QA or embedding vector predicates inside database-native engines.Text2VectorSQL couples retrieval and fusion within the database engine.
  • ProRetrieval: ProRetrieval differs by treating its DSL as a learnable policy space optimized end-to-end with reinforcement learning and a hierarchical reward tailored to executable hybrid programs.This extends beyond the single-signal rewards used by prior RL retrievers.

3 Method

ProRetrieval defines a hybrid DSL in which SQL over structured fields invokes text and image vector retrieval through placeholders, then trains policies to generate executable programs. Its datasets and reward design support compositional retrieval with relational fusion.

  • Framework: The framework combines a hybrid retrieval DSL, an automated query-construction pipeline, and two-stage supervised-plus-reinforcement-learning training.The pipeline produces natural-language query, gold DSL, and ground-truth document triples.
  • Hybrid Retrieval DSL: A DSL program contains an SQL field and a retrieval_list of text or image primitives specifying modality, field, and embedding query.The SQL query operates over structured fields and can reference retrieval entries by placeholders.
  • Hybrid Retrieval DSL: SQL placeholders turn vector-retrieval outputs into candidate-set membership constraints, allowing AND, OR, NOT, and nested subqueries to combine structured and semantic predicates.This gives the DSL SQL’s logical algebra for heterogeneous retrieval composition.
  • Execution: Each vector primitive is dispatched to a text or image encoder and approximate nearest-neighbor backend, returns top-K candidates with K=20, and is fused by the relational SQL engine.Execution therefore proceeds in two phases: vector retrieval followed by relational fusion.
  • Expressiveness: The full DSL strictly subsumes reciprocal rank fusion, self-querying retrieval, and pure text-to-SQL, while additionally expressing cross-modal logical compositions.Examples include structured AND semantic AND visual constraints.
  • Benchmark construction: The benchmarks are generated by sampling leaves and Boolean templates, grounding them to corpus documents, compiling gold DSL programs, executing them, and verbalizing them with an LLM.Metadata records query level, leaves, groups, operators, and modality usage.
  • Training: Training warm-starts the policy with gold NL-to-DSL pairs, samples rollouts for group-relative advantages, and optimizes them with clipped policy objectives using GRPO or DAPO.DAPO uses asymmetric clipping and token-level normalization for long DSL outputs.
  • Training: The hierarchical reward prioritizes syntactic validity, executability, result accuracy, and length efficiency through format, execution, result, and length terms.Rank-1 hit is used for result quality because each query is anchored to a single ground-truth document.

4 Experiments

The experiments evaluate ProRetrieval across hybrid e-commerce and email benchmarks, comparing retrieval paradigms, training methods, action spaces, modalities, and generalization. Results show that RL-trained orchestration delivers strong, balanced retrieval performance and preserves gains out of distribution.

  • Experimental Setup: The evaluation uses Amazon e-commerce and Enron email benchmarks with Hit@1, Hit@3, MRR, NDCG@3, and execution success rate.E-commerce combines structured, text, and image fields; email combines structured and text fields.
  • Main Results: 82.0% Hit@1 is achieved by 8B SFT on e-commerce, while DAPO achieves 90.9% on email and improves consistently across four metrics.The e-commerce result surpasses BGE-Reranker by +20.2 pp, while the email result surpasses GPT-5.5 by +5.4 pp.
  • Training Comparison: 0.809 Hit@1 is reached by RL on the full DSL, exceeding 0.680 for full-DSL SFT and 0.693 for GPT-5.5 on e-commerce.The RL gain is +12.9 pp over SFT and +11.6 pp over GPT-5.5, with statistically non-overlapping bootstrap confidence intervals.
  • Action Space Ablation: 0.752 Hit@1 for SQL+text SFT exceeds 0.680 for the full three-modality DSL, whereas full-DSL RL reaches 0.809.Under SFT, image-query Hit@1 is 0.323 for the full DSL and 0.547 for SQL+text; RL raises image performance to 0.716.
  • Modality Breakdown: 0.866 structured Hit@1 for SQL-R1 and 0.819 image Hit@1 for BGE-Reranker reveal modality specialization, while RL-trained models exceed 0.68 across all three modalities.The RL-trained models are the only approach described as lacking a catastrophic modality blind spot.
  • Generalization and Reliability: IID-to-OOD gaps remain within 3 pp, and execution success remains above 99% on OOD data.On e-commerce, DAPO’s OOD drop is −1.5 pp versus −2.6 pp for GRPO; DAPO and GRPO reach similar peak performance.

5 Conclusion

ProRetrieval frames hybrid retrieval as executable program synthesis, and its 4B model achieves strong results across e-commerce and email while revealing when reinforcement learning is most useful.

  • 80.9%/90.9% Hit@1 on e-commerce/email surpasses the best commercial LLM by +11.6/+5.4 pp.
  • ProRetrieval synthesizes executable hybrid programs that interleave SQL with text and image vector primitives under a hierarchical four-term reward.
  • OOD evaluation shows generalization within 3 pp.
  • RL overcomes the full DSL’s SFT underperformance on image retrieval, producing a +39.3 pp improvement.
  • At 8B scale, SFT alone captures the key patterns, suggesting RL is most critical when model capacity is the bottleneck.

Limitations

The study’s scope is bounded by schema dependence, shallow training expressions, two evaluation domains, moderate corpus sizes, single-turn operation, embedding quality, and deployment limitations.

  • The DSL assumes a predefined schema, leaving schema-free or evolving corpora to future work.
  • Training samples logical expressions up to two nesting levels, so deeper nesting is underrepresented despite being syntactically expressible.
  • Evaluation covers only e-commerce and email, leaving broader generalization to domains such as scientific literature and legal documents untested.
  • The benchmarks contain 3,000 products and 5,000 emails, and scaling to millions of documents may challenge execution latency and candidate-set composition.
  • The framework synthesizes one program per query and depends on multimodal embedding alignment for image retrieval quality.
  • The study omits cross-seed variance and automatic fallback when DSL generation fails, while GPT-4o query verbalization may introduce stylistic bias.

Ethics Statement

ProRetrieval retrieves existing documents rather than generating user-facing content, but filtering and embedding biases remain relevant deployment concerns.

  • ProRetrieval locates existing documents and does not generate user-facing content, posing minimal risk of harmful or misleading text.
  • Experiments use public Amazon ESCI and Enron research datasets without attempting to re-identify individuals.
  • Structured filters may systematically exclude groups or information sources, and embedding biases may propagate into retrieval rankings.
  • Deployment in sensitive domains should audit filter distributions and embedding fairness.

A Complete Baseline Results

The paper provides complete omitted baseline results and documents model sizes and backbone choices for comparison fairness and reproducibility.

  • Tables 5 and 6 report complete results for baselines omitted from the main text for space.
  • The main models use Qwen3-4B and Qwen3-8B, while DeepRetrieval and Search-R1 use Qwen3-4B for fair comparison.
  • BGE-Reranker uses 568M parameters, and LightRAG combines GPT-5.4 with all-MiniLM-L6-v2 embeddings.
  • Commercial LLM parameter counts for GPT-5.5 and Opus 4.7 are undisclosed.

B Query Complexity Breakdown

Query complexity changes which retrieval strategies are strongest: the DSL-based models excel on simple structured queries, while traditional retrievers benefit from richer keyword context in complex queries. Across query levels, the proposed method retains a substantial aggregate advantage.

  • By query complexity: 0.864 vs. 0.473 Hit@1: GRPO leads BGE-Reranker on L1 queries by 39.1 percentage points.L1 queries often reduce to single-condition structured filtering that maps directly onto SQL.
  • By query complexity: 14.4 pp vs. 15.4 pp: GRPO shows a smaller L1-to-L3 degradation than SFT.The result indicates improved handling of complex logical combinations such as OR, NOT, and nesting.
  • By query complexity: 0.802 vs. 0.720 Hit@1: RankGPT is competitive with the proposed models on L3 queries.BGE-Reranker also reaches 0.741 on L3, where richer keyword context benefits term matching and reranking.
  • By query complexity: +26.8 pp: BGE-Reranker improves from L1 to L3, contrasting with the DSL models’ strongest advantage on L1.The L3 advantage is attributed to the richer distinctive keywords in complex queries.
  • By retrieval modality: The proposed method is the only balanced solution across all three retrieval modalities, whereas SQL-R1 and BGE-Reranker are extreme specialists.Figure 6 presents this comparison alongside the query-complexity breakdown.

C Training Dynamics

Training dynamics differ substantially between GRPO and DAPO, despite similar peak performance. Larger supervised models can outperform smaller RL-trained models on e-commerce, while RL remains useful for smaller models mastering the full action space.

  • Training dynamics: 0.809 vs. 0.808 peak Hit@1: GRPO and DAPO reach similar best performance on e-commerce.GRPO peaks at step 300, while DAPO peaks at step 420.
  • Training dynamics: After step 550, GRPO performance degrades and drops sharply by step 1,200.The passage associates this instability with degenerate programs exploiting reward loopholes.
  • Training dynamics: DAPO maintains stable performance throughout training, benefiting from asymmetric clipping and dynamic sampling.Figure 7 contrasts its stability with GRPO’s sensitivity to extended training.
  • Model scale and training stage: 0.820 vs. 0.809 Hit@1: 8B SFT surpasses the best 4B RL result on e-commerce.The comparison suggests that larger-model SFT can capture orchestration patterns that smaller models acquire through RL exploration.
  • Model scale and training stage: 8B SFT remains at ≤0.820 after 400 steps of GRPO and DAPO, while RL provides modest but consistent gains over SFT for the 4B email model.The 8B result is preliminary because its stronger starting point changes the optimization landscape.

E Detailed Error Analysis

The error analysis identifies SQL syntax as the dominant failure mode among trained models, while commercial models execute nearly perfectly but retrieve less effectively. The system’s hybrid execution pipeline is also latency-efficient because vector retrieval dominates runtime and SQL-side fusion is negligible.

  • Error types: Unescaped apostrophes in brand names are the dominant SQL syntax error across trained models.Names such as “Victoria’s Secret” can break the SQL parser; the passage characterizes this as a tokenization artifact rather than a logical failure.
  • Error types: 100% vs. 99.9% execution: GPT-5.5 and Opus 4.7 achieve near-perfect execution but lag trained models on retrieval quality.This indicates that large-model performance is bottlenecked by orchestration strategy rather than syntax compliance.
  • Statistical validation: 10,000 resamples produce bootstrap 95% confidence intervals confirming the statistical significance of pairwise differences reported in the main text.For e-commerce, GRPO’s interval is [0.795, 0.823], compared with SFT [0.663, 0.697] and the base model [0.105, 0.128].
  • Execution and latency: <1 ms: placeholder injection and SQL execution each add negligible latency on K=20 candidate sets, while vector retrieval accounts for ∼80% of total latency.Total per-query latency is comparable to single-backend retrieval and faster than multi-turn Search-R1.
  • DSL execution: Structured fields such as brand, price, average_rating, year, and sender or recipient addresses are queried directly in SQL.Text fields use text placeholders, while image retrieval uses image placeholders for candidate-set injection.
  • DSL execution: SQL combines structured conditions with text and image candidate sets through product_id IN or NOT IN placeholders.The DSL supports standard SQL operators, Boolean combinations, and independently assigned text/image placeholders.
Loading 2608.27017v1…