Source-linked AI summary
The Fellowship of the Query: Learning Retrieval Actions
Mohammed Al-Maamari, Saber Zerhoudi, Michael Granitzer, Jelena Mitrović
TL;DR
Multi-hop RAG requires decisions about when to search, extract, synthesize, verify, and stop, so answer quality depends on action control as well as generation. The paper fine-tunes models on accepted teacher trajectories and evaluates action prediction and an end-to-end controller/generator pipeline. Granite 4.1 3B reaches 0.6536 test macro-F1, and the evaluated pipeline shows increased evidence-fact recording with a fine-tuned controller.
Problem
Multi-hop RAG requires sequential retrieval decisions, so final-answer quality depends on both generation and the actions that build the evidence state.
Source: §1 · p. 1Method
The study fine-tunes SLMs and xSLMs to predict structured next actions from accepted teacher trajectories, then evaluates selected models in a BM25 search pipeline.
Source: §1 · p. 2, §2.1 · p. 2Results
Granite 4.1 3B reaches 0.6536 test macro-F1 on held-out 7-way action prediction, while the fine-tuned controller increases evidence-fact recording in the evaluated pipeline.
Source: §5 · pp. 4–5Takeaways & Limitations
Trajectory supervision can improve action prediction and evidence-fact recording for an SLM in the evaluated pipeline, while robust final-answer generation remains open.
Source: §5 · p. 5Evaluation is conditional on accepted teacher trajectories and uses a per-question candidate corpus, rather than unfiltered questions or open retrieval.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Retrieval-augmented question answering requires control decisions about when to decompose a question, search, reformulate, extract evidence, synthesize facts, verify progress, and stop. We study whether trajectory fine-tuning can improve small language models (SLMs) as next-action controllers. We additionally evaluate a low-resource setting in which a single SLM serves as both the controller and the final-answer generator. From accepted teacher search traces, we build a seven-way action-prediction task, where the model predicts the next structured teacher action from the current trajectory state, and evaluate LoRA-supervised fine-tuning across SLMs and xSLMs as controllers. On 1,646 held-out action examples, Granite 4.1 3B trained on 13,194 actions reaches macro-F1 0.6536, compared with 0.1736 for zero-shot prompting of the same model and 0.5399 for a TF-IDF logistic-regression baseline. In an end-to-end controller/generator swap evaluation over 149 held-out trajectories, using the fine-tuned model for both roles improves Exact Match from 0.7530 to 0.7946 and token F1 from 0.7783 to 0.8295 compared with using the base model as both controller and generator. The cross-role conditions show that the fine-tuned controller increases evidence-fact recording when the generator is fixed, while controller-only final-answer gains are not statistically clear. Overall, trajectory supervision improves action prediction and evidence-recording behaviour in this evaluated pipeline. Code is available at https://github.com/padas-lab-de/agent-action-controller
1 Introduction
Multi-hop RAG requires a controller to choose a sequence of retrieval and reasoning actions, not just generate an answer. The paper tests whether trajectory supervision improves small models’ next-action control and evidence-fact recording.
1 Introduction
- Final-answer quality depends on both generation and the action sequence that builds the evidence state.The sequence may include decomposing, searching, reformulating, extracting, synthesizing, verifying, and finishing.Source: §1 · p. 1
- The paper asks whether trajectory supervision can improve small models as next-action controllers, given smaller models’ difficulty following action protocols.The paper uses xSLMs for models below 1B parameters and SLMs for models in the 1B–3B range.Source: §1 · p. 1
- Prior work studies interleaved retrieval and reasoning [26], retrieval self-reflection [3], query refinement [6], and search-agent training [13].The paper evaluates whether LoRA supervision enables xSLMs and SLMs to imitate a structured seven-action protocol; accepted teacher actions are not assumed optimal.Source: §1 · p. 1
- The study predicts the next structured action from the question, trajectory state, and evidence context, then evaluates selected models in an end-to-end BM25 search pipeline.It tests multiple SLMs and xSLMs, learning formulations, and training-set sizes, and evaluates final answers with Exact Match and token F1.Source: §1 · p. 2
2 Method
The method converts accepted teacher search traces into supervised next-action examples, compares learning formats, and tests a selected model in a constrained end-to-end search pipeline.
2.1 Trajectory-to-Action Supervision
- Of 5,000 teacher rollouts, 1,490 accepted trajectories supply 16,488 action steps; trajectory- and query-level splits yield 13,194 training and 1,646 test actions.Inputs include the question, prior state, tool history, and visible evidence, while teacher thoughts and gold answers are excluded; the seven target actions are decompose, search, reformulate, extract, synthesize, verify, and finish.Source: §2.1 · p. 2
2.2 Learning Formats
- The learning-format comparison includes zero-shot prompting, TF-IDF logistic regression, LoRA sequence classification, LoRA SFT, and one-vs-rest binary variants.The LoRA setups adapt attention projections while keeping the base model fixed.Source: §2.2 · p. 2
2.3 End-to-End Pipeline
- The end-to-end controller searches a deterministic BM25 retriever over a per-question corpus of 5–20 candidate documents, with gold-supporting documents included for mapped examples.Evidence facts are matched using substring or token-overlap heuristics, and the answer generator receives accumulated state and retrieved evidence through a separated prompt; if the controller does not finish within 12 steps, generation still proceeds.Source: §2.3 · pp. 2–3
3 Experiments
The experiments test trajectory fine-tuning for action prediction and whether those gains carry into a constrained end-to-end search pipeline. They compare models, training sizes, and action formulations, then evaluate controller and generator role swaps.
3 Experiments
- The experiments test action prediction across models and formulations, then evaluate the selected controller in role swaps on 149 held-out accepted trajectories.The end-to-end conditions use the same BM25 environment and a maximum of 12 controller steps.Source: §3 · p. 3, §3.3 · p. 4
3.1 RQ1: Action Prediction Across Models
- Granite 4.1 3B LoRA SFT reaches test macro-F1 0.6536 on the 1,646-example action-prediction test split.Ministral 3B sequence classification has the highest reported accuracy but lower macro-F1 than the best LoRA SFT results; verify remains difficult with 17 held-out examples.Source: §3.1 · p. 3
3.2 RQ2: Data Size and Action Formulation
- At 3,000 training samples, Granite 4.1 3B obtains macro-F1 0.6489, close to 0.6625 at 12,000 and 0.6536 on the full split.Accuracy and weighted-F1 increase slightly at larger sizes, while macro-F1 peaks at 12,000 and is similar on the full training set.Source: §3.2 · p. 3, Figure 1 · p. 3
- The multiclass controller has higher macro-F1 and macro-recall, while independent binary controllers have higher macro-precision but lose recall on some low-support or confusable classes.A naive ensemble of binary scores performs worse than the multiclass controller, which is used in the end-to-end pipeline.Source: §3.2 · p. 3, Table 2 · p. 4
3.3 RQ3: End-to-End Search
- EM rises from 0.7530 to 0.7946 and token F1 from 0.7783 to 0.8295 when the fine-tuned model fills both roles rather than base/base.The combined token-F1 gain has a paired-bootstrap interval of [0.0021, 0.1040], while the Exact Match interval crosses zero.Source: Table 3 · p. 4, §3.3 · p. 4
- With either generator fixed, replacing the base controller with the fine-tuned controller increases evidence-fact rate by 0.60 or 0.62, while controller-only answer-quality intervals cross zero.The corresponding increases in mean evidence facts are 2.25 and 2.28.Source: §3.3 · p. 4
4 Discussion
The discussion finds that trajectory supervision improves evidence-recording behavior, while controller-only answer-score gains remain statistically unclear. It also reports a general-capability trade-off and identifies constraints on interpreting the evaluation.
4 Discussion
- The trained controller increases evidence-matched intermediate facts under the automatic heuristic when the generator is fixed, while controller-only answer gains remain modest and statistically unclear.Using the fine-tuned model for both controller and generator gives a small token-F1 gain over base/base.Source: §4 · p. 4
- Fine-tuning reduces MMLU accuracy from 0.6554 to 0.5520, HellaSwag from 0.7911 to 0.6852, and ARC-Challenge from 0.6502 to 0.5589.The fine-tuned model used for both roles nevertheless gives the strongest answer scores among evaluated role combinations.Source: §4 · p. 4
- Evaluation is limited to questions with accepted teacher trajectories, a per-question candidate corpus, and a separated final-answer prompt; evidence matching is heuristic rather than human-verified factuality.Source: §4 · p. 4
5 Conclusion
The conclusion reports improved action prediction and evidence-fact recording in the evaluated pipeline. It also notes that unfiltered-query evaluation and robust final-answer generation remain open issues.
5 Conclusion
- 0.6536 macro-F1 is reached by Granite 4.1 3B LoRA SFT trained on 13,194 samples for acceptance-filtered held-out 7-way action prediction, improving over zero-shot prompting and a TF-IDF baseline.Source: §5 · p. 4
- 3,000 training samples yield macro-F1 0.6489, close to the reported best observed value of 0.6625.Source: §5 · p. 4
- A single multiclass controller is stronger in aggregate than independent binary action controllers in the evaluated Qwen 3.5 0.8B setting.Source: §5 · pp. 4–5
- Using fine-tuned Granite 4.1 3B for both controller and generator improves token F1 relative to base/base, while replacing only the controller substantially increases evidence-fact recording under a fixed generator.Source: §5 · p. 5
- Unfiltered-query evaluation and robust final-answer generation remain open issues for the reported pipeline.Source: §5 · p. 5
6 GenAI Usage Disclosure
The authors disclose using GenAI for editing, code support, and implementation-related questions, with generated text reviewed before submission.
6 GenAI Usage Disclosure
- GenAI tools supported text editing, development, and implementation-related code adaptation; generated text was carefully reviewed before submission.Source: §6 · p. 5