PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA

Nirmal Roy, Leonardo F. R. Ribeiro, Rexhina Blloshmi, Kevin Small

arXiv:2409.15515v1cs.CLcs.AI

TL;DR

Conversational QA requires systems to interpret context across turns, yet how to handle contextual search intent is understudied. SELF-multi-RAG adapts retrieval decisions and context summarization to conversations, and experiments report improved response generation over prior approaches, including an average ∼13% improvement over SELF-RAG by human annotation.

  • Problem

    Conversational QA requires understanding current intent in a longer history, while contextual search intent and multi-turn passage management remain understudied challenges.

    Source: §1 · pp. 1–2

  • Method

    SELF-multi-RAG trains LLMs to decide when retrieval is needed, summarize conversational context for retrieval, and reflect on passages and generated responses.

    Source: §1 · p. 2

  • Results

    ∼13% average improvement over SELF-RAG in conversational response quality, measured by human annotations; summary-based queries also improve retrieval effectiveness by 13.5% average R@5 over query-rewriting baselines.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SELF-multi-RAG improves conversational QA performance by better handling longer context and using summaries to increase retrieval effectiveness.

    Source: §1 · p. 2, §6 · p. 9

    Experiments use English-language QA datasets, one critic and generator model, one retrieval model, and offline retrieval.

    Source: §7 · p. 9

Abstract

from arXiv · show

Augmenting Large Language Models (LLMs) with information retrieval capabilities (i.e., Retrieval-Augmented Generation (RAG)) has proven beneficial for knowledge-intensive tasks. However, understanding users' contextual search intent when generating responses is an understudied topic for conversational question answering (QA). This conversational extension leads to additional concerns when compared to single-turn QA as it is more challenging for systems to comprehend conversational context and manage retrieved passages over multiple turns. In this work, we propose a method for enabling LLMs to decide when to retrieve in RAG settings given a conversational context. When retrieval is deemed necessary, the LLM then rewrites the conversation for passage retrieval and judges the relevance of returned passages before response generation. Operationally, we build on the single-turn SELF-RAG framework (Asai et al., 2023) and propose SELF-multi-RAG for conversational settings. SELF-multi-RAG demonstrates improved capabilities over single-turn variants with respect to retrieving relevant passages (by using summarized conversational context) and assessing the quality of generated responses. Experiments on three conversational QA datasets validate the enhanced response generation capabilities of SELF-multi-RAG, with improvements of ~13% measured by human annotation.

1 Introduction

Conversational QA must interpret current intent across a longer history while deciding whether retrieval is useful. SELF-multi-RAG adapts SELF-RAG to this setting by deciding when to retrieve, summarizing context for retrieval, and reflecting on retrieved passages; experiments report improved response quality.

1 Introduction

  • Indiscriminate retrieval can introduce unrelated or conflicting passages and compromise response quality, so deciding when grounding is useful matters.Retrieved passages can also impede LLM flexibility; factual errors and hallucinations remain persistent problems for LLM-based systems.Source: §1 · p. 1
  • Longer conversation histories make it challenging to infer current intent, avoid repetition, maintain engagement, and decide whether retrieval is needed.The retrieval decision may depend on the conversation context rather than only the latest turn.Source: §1 · p. 2
  • Conversation summaries can preserve useful retrieval signals that full-history queries may obscure and traditional rewrites may omit.The paper notes that longer histories and single-question rewrites can lose information needed for effective retrieval.Source: §1 · p. 2, Figure 2 · p. 2
  • SELF-multi-RAG trains LLMs to decide whether retrieval is needed from conversational context and summarize the conversation as a query for off-the-shelf retrieval models.The framework also reflects on retrieved passages in multi-turn settings.Source: §1 · p. 2
  • ∼13% average improvement over SELF-RAG in conversational response quality, measured by human annotations.SELF-multi-RAG summarization also improves retrieval effectiveness by 13.5% on average at R@5 versus query-rewriting baselines.Source: §1 · p. 2

2 Related Work

Prior work uses retrieval augmentation, self-reflection, conversational fine-tuning, and query rewriting, but does not explicitly train systems for conversational retrieval decisions and passage handling. The paper distinguishes its aim of improving retrieval effectiveness as well as reducing redundancy.

2 Related Work

  • SELF-RAG filters irrelevant passages through self-reflection and evaluates output quality, but is not trained to comprehend conversational context.A related RL approach can reduce retrieval costs, while SELF-multi-RAG also targets retrieval effectiveness.Source: §2 · pp. 2–3
  • Instruction fine-tuning supports multi-turn response generation, while conversational history modeling rewrites user questions using prior turns.The cited passage notes that LLM-based rewriting can improve retrieval effectiveness, but these works do not explicitly train reflection on whether retrieval is needed or how to handle irrelevant passages.Source: §2 · p. 3
  • Existing conversational query-rewriting work improves retrieval, but does not evaluate how retrieved passages affect response generation.The authors hypothesize that summarizing context rather than rewriting it as one question can improve retrieval and consequent response generation.Source: §2 · p. 3

3 SELF-multi-RAG

SELF-multi-RAG extends SELF-RAG to conversational QA, training an LLM to decide when retrieval is needed, summarize conversation history for retrieval, and critique passages and responses (Asai et al., 2023). Its framework uses conversational context and previously retrieved passages to guide retrieval and response selection.

3 SELF-multi-RAG

  • SELF-multi-RAG uses longer conversational contexts to decide whether retrieval is needed and to critique retrieved passages and generated responses.It can also reuse relevant passages retrieved in earlier turns, avoiding unnecessary retrieval.Source: §3 · pp. 3–4
  • Summarizing conversation history as a retrieval query may preserve relevant signals that a single-question rewrite can lose, especially in long conversations.The summary is generated whenever retrieval is deemed necessary.Source: §3 · p. 4, §3.1 · p. 4

3.1 Components

  • The critic produces reflection-token labels for conversational retrieval decisions and passage relevance, while the generator learns to predict reflection tokens and summarize history.The critic adapts five tasks introduced by Asai et al. (2023) from single questions to conversational history.Source: §3.1 · p. 4
  • The retriever is a separate black-box component that supplies passages when the model adaptively calls retrieval.The paper uses an off-the-shelf Contriever trained on MS-MARCO and a 54M-passage Wikipedia and Common Crawl knowledge base.Source: §3.1 · p. 4

3.2 Overall Framework

  • The model selects among [Retrieve], [No Retrieve], and [Continue to use evidence] based on the conversation and previously retrieved passages.When retrieval is called, it rewrites conversation history as a query, assesses passage relevance and response groundedness, and assigns usefulness scores.Source: §3.2 · p. 5
  • The final response is chosen using usefulness, groundedness, and passage relevance, with segment-level beam search returning the best sequence.Figure 3 highlights multi-turn-specific components in yellow and shows the critic supplying reflection tokens the generator is trained to predict.Source: Figure 3 · p. 4, §3.2 · p. 5

4 Experimental Setup

The experiments use QReCC and UltraChat data to train critic and generator models, then evaluate response generation and critic performance across conversational benchmarks. Evaluation combines automatic metrics and human ratings, with tables and a figure covering model performance and turn configurations.

4 Experimental Setup

  • Evaluation covers QReCC, UltraChat, and out-of-domain MT-Eval, whose conversations differ in knowledge grounding and instructional content.QReCC supplies gold passage annotations; UltraChat and MT-Eval contain diverse instructional conversations.Source: §4 · p. 5

4.1 Training Data

  • Critic training uses GPT-4-labeled QReCC and UltraChat multi-turn data, alongside flattened single-turn and original SELF-RAG data; generator training also includes conversation summaries.The summary dataset contains roughly 5,000 conversations.Source: §4.1 · pp. 5–6

4.3 Evaluation

  • The evaluation tables cover critic accuracy and response-generation performance, while Figure 4 relates retrieval calls and BERTScore to turn configurations.Table 2 distinguishes retrieval judgments with and without prior passages; Table 4 distinguishes rewritten or full conversation context and adaptive or no retrieval.Source: Table 2 · p. 6, Table 4 · p. 7, Figure 4 · p. 7

4.2 Models

  • Critics start from Mistral-7B-Instruct-v0.2 and compare single-turn with multi-turn training, while the final generator is trained end-to-end on both settings and conversation summarization.Table 3 identifies the configurations explored and defines ST, MT, C, G, and CSD.Source: §4.2 · p. 6, Table 3 · p. 6

4.3 Evaluation

  • Critic evaluation measures special-token prediction accuracy, while generator responses are rated with BERTScore, UniEval, GPT-4 scores, and human judgments.Human annotators rate coherence, engagingness, and understandability; the overall score averages those dimensions.Source: §4.3 · pp. 6–7

5 Results

SELF-multi-RAG improves critic performance, conversational response generation, and retrieval using summarized conversation context. Its handling of prior-turn passages supports a trade-off between response performance and retrieval efficiency.

5 Results

  • Criticsm, trained on single- and multi-turn data, has the highest accuracy on critic tasks using GPT-4 labels, including retrieval, passage relevance, and answer utility judgments.It also performs best when deciding whether to retrieve with previous-turn passages in context, although this task is harder than retrieval without those passages.Source: §5 · p. 7, §5.2 · p. 8
  • SELF-multi-RAG improves response generation across conversational benchmarks, with better context comprehension, retrieval decisions, groundedness, and MT-Eval performance than single-turn counterparts.Responses were better at all conversation turns up to six; unsatisfactory answers typically occurred when retrieved documents lacked relevant answers.Source: §5 · pp. 7–8

5.1 Summarizing Conversations

  • SELF-multi-RAG summaries outperform single-question T5QR rewrites for QReCC retrieval with both sparse and dense methods.Using its generated summary as the retrieval query also gives the best response generation on both datasets among the tested conversation-history representations.Source: §5.1 · p. 8, Table 5 · p. 8, Table 6 · p. 9

5.2 Handling Previous-turn Retrieved Passages

  • Including previously retrieved passages in context lets SELF-multi-RAG generate better responses while calling retrieval less often than its single-turn baseline.The configuration without prior passages is suited to prioritizing performance; the configuration with prior passages is suited to efficiency with a slight performance reduction.Source: §5.2 · pp. 8–9

6 Conclusion

SELF-multi-RAG trains LLMs to decide when to retrieve and generate responses for conversational QA. Evaluation across three benchmarks shows improved performance over prior approaches, alongside improved retrieval effectiveness from conversation summaries.

6 Conclusion

  • SELF-multi-RAG trains LLMs to decide when to retrieve and generate responses for conversational QA.The framework addresses retrieval decisions, document relevance, and answer usefulness in multi-turn dialogue.Source: §6 · p. 9
  • Across three conversational QA benchmarks, SELF-multi-RAG improves performance over previous approaches by better handling multi-turn context in critic and generator tasks.The conclusion attributes improved generator performance to better critic performance and comprehension of longer conversational contexts.Source: §6 · p. 9
  • SELF-multi-RAG-generated summaries improve retrieval effectiveness when used as passage-retrieval queries, and this is associated with improved response generation.Source: §6 · p. 9

7 Limitations

The experiments are limited to English-language QA datasets, one critic and generator model, one retrieval model, and offline retrieval. The appendix also describes how GPT-4 prompts were used to collect critic and summarization training data.

7 Limitations

  • Experiments use English-language QA datasets, so findings do not establish generalizability across languages, datasets, or domains.Source: §7 · p. 9
  • The evaluation uses one critic and generator model, Mistral-7B-Instruct-v0.2, and one retrieval model, Contriever.Extension of SELF-multi-RAG to other models is left for future work.Source: §7 · p. 9
  • Retrieval is performed offline to reduce computation overhead, whereas a more realistic scenario would retrieve online during response generation.Source: §7 · p. 9

A GPT-4 Prompt: Critic Training Data & Conversation Summarization Training Data

B Benchmark Statistics

QReCC differs from UltraChat and MT-Eval in grounding, retrieval needs, and question length. These differences affect how retrieval decisions should be interpreted across the benchmarks.

B Benchmark Statistics

  • QReCC turns are grounded in Wikipedia or Common Crawl passages, while UltraChat and MT-Eval include turns with passages already present, follow-up context, or creative questions.Retrieval is ideally needed at every QReCC turn, whereas the other datasets require judging retrieval need from context; their questions and gold responses are also longer than QReCC's.Source: B Benchmark Statistics · p. 11

C Implementation Details

The implementation section specifies the critic and generator training setup and identifies the prompts and dataset information supplied in the appendices.

C Implementation Details

  • Critic training used Mistral-7B-Instruct-v0.2 for 3 epochs; generator training used 5 nodes for 2 epochs, with the reported learning rates and warm-up ratios.Critic training used 4 nodes with 8 NVIDIA A10G GPUs each, a learning rate of 2e−5, and a warm-up ratio of 0.01; generator training used a learning rate of 2e−5 and a warm-up ratio of 0.03.Source: C Implementation Details · p. 11
  • Appendix tables present GPT-4 prompts for training retrieval, relevance, groundedness, utility, and conversation-summarization components.The listed prompt tables cover Retrieval critic, Multi-Retrieval critic, Relevance critic, Groundedness critic, Utility critic, and generator training for conversation summarization.Source: Table 8 · p. 12, Table 9 · p. 13, Table 12 · p. 16
  • The section includes examples of judging evidence relevance and evaluating whether a response is supported by evidence or conversation history.The response-support example uses a three-level scale: Fully supported, Partially supported, and No support / Contradictory.Source: C Implementation Details · pp. 14–15
  • A table provides statistics and examples for the three datasets considered in the study.Source: Table 14 · p. 18

D GPT-4 Evaluation Form

The evaluation form describes GPT-4 assessment of responses generated by different SELF-RAG models. It specifies generation limits and points to the prompt table.

D GPT-4 Evaluation Form

E Mechanical Turk Setup

The paper describes human ratings of generated answers and gives an example of SELF-multi-RAG handling longer conversational context better than a single-turn counterpart.

E Mechanical Turk Setup

  • Human annotators rated answer coherence, engagingness, and understandability; eligible workers were English-language-country residents with at least 100 HITs and a 95% or higher acceptance rate.Workers who passed automatic quality checks received a bonus; annotators were fairly compensated.Source: E Mechanical Turk Setup · p. 19
  • The paper provides human annotation scores for the three response-quality dimensions and a screenshot of the evaluation template.Source: E Mechanical Turk Setup · p. 19, Table 16 · p. 20, Figure 5 · p. 22

F Examples of Generated Responses

  • An example illustrates SELF-multi-RAG comprehending longer conversational contexts better than its single-turn counterpart.Table 17 is captioned as showing examples of SELF-multi-RAG improvements in conversational QA over SELF-RAGs.Source: F Examples of Generated Responses · p. 19, Table 17 · p. 21

Related papers

Loading 2409.15515v1…