Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models

1 Introduction

• Grounding responses in retrieved evidence restricts the output space, potentially reducing fluency while improving attribution.The paper evaluates this hypothesized tradeoff across settings with different degrees of attribution.Source: §1 · p. 1
• Retrieval augmentation can still produce non-factual responses because queries, external evidence, memorization conflicts, and sampling introduce failure modes.The paper highlights hallucinated retrieval queries, irrelevant evidence, conflicts between parametric memory and evidence, and higher-temperature sampling.Source: §1 · pp. 1–2
• The experiments use QReCC golden evidence to control retrieval imperfections and prompting to vary the balance between memorization and grounding.This design simplifies the first two identified sources of non-factuality while preserving controllable context conditions.Source: §1 · p. 2
• The study measures how model parameters and supplied context affect fluency and attribution across extensive controlled experiments.It varies scale, decoding temperature, dialog turns, retrieved facts, and instructions, while introducing global and local tradeoff concepts.Source: §1 · p. 2
• The work aims to identify ways to reduce fluency–attribution tradeoffs while characterizing how model and context parameters influence them.Its framing distinguishes global and local tradeoffs and seeks mitigation strategies.Source: §1 · p. 2

2 Related Work

• Existing work examines truthfulness versus informativeness and attribution versus preservation, rather than directly studying fluency–attribution tradeoffs.Zhou et al. [12] vary instructions, while Gao et al. [13] compare models and datasets using attribution and preservation.Source: §2 · p. 2
• The paper presents its tradeoff framing as distinct from the two related views because it studies both model-level and inference-level variation.The related papers use different interpretations of what reducing a tradeoff means.Source: §2 · p. 2
• The global view compares absolute positions across different models and datasets, so mitigation means selecting the best available model for a task.This view is associated with Gao et al. [13].Source: §2 · p. 2
• The local view fixes the model and dataset while changing prompt instructions to reach preferred positions in the fluency–attribution space.This view is associated with Zhou et al. [12] and treats mitigation as optimizing inference for a chosen task and metric preference.Source: §2 · p. 2

3 QReCC Dataset

• QReCC is an end-to-end open-domain question-answering dataset with 14K conversations and 81K question–answer pairs.Each conversation contains dialog history, a final unanswered query, a golden answer, and its source webpage.Source: §3 · p. 2
• The study uses a decontextualized QReCC version because some contextualized conversations were difficult for humans to parse.For some dialogs, the final user question alone contains enough information for a sensible response.Source: §3 · p. 2
• Decontextualization is incomplete: many dialogs require more than the final user turn to produce a sensible response.The dataset therefore retains cases where dialog history remains necessary.Source: §3 · p. 3
• QReCC aligns sensible responses with attribution to evidence, unlike more conversational datasets where sensibleness may instead reflect engagement or interest.This alignment materially affects how the fluency–attribution tradeoff behaves.Source: §3 · p. 3
• The authors filter conversations with assumed known articles, overly long evidence, malformed histories, or single-word answers before sampling 100 validated examples.The 100 examples are randomly selected from 324 remaining development examples and manually checked for dialog quality.Source: §3 · p. 3

4.1 Pilot

• The pilot uses SSA as a fluency proxy and AIS as an attribution framework, both originally based on human raters.The study later seeks aligned automatic metrics to reduce the cost of large-scale evaluation.Source: §4.1 · p. 3, §4.2 · p. 3
• The pilot evaluates 400 responses from 100 QReCC dialogs generated by PaLM 540B under four setups, using human sensibleness and attribution ratings.All four setups receive SSA evaluation, while AIS is collected for the evidence-containing setups; no-evidence setups are assigned zero attribution.Source: §4.1 · p. 3

4.2 “Merged” Setup for AIS

• Merged evaluation asks raters to score paired outputs for the same example consecutively, reducing repeated reading of shared evidence and dialog history.This setup is introduced as an optimization because the pilot reuses the same examples across conditions.Source: §4.2 · p. 3

4.3 Alignment of “Merged” vs “Separate”

• The AIS aggregate metrics use majority voting across five raters for each binary question, followed by averaging over 100 examples.The authors compare merged ratings against separate ratings, treating separate evaluation as ground truth for estimating uncertainty.Source: §4.3 · p. 4
• Merged evaluation systematically shifts AIS ratings: questions (2), (4), and (6) favor T=0.0 over T=0.7 in the merged condition.The direction is better for scenario B than A and worse for D than C, while overall superiority of merged or separate ratings remains unclear.Source: §4.3 · p. 4

5.1 End-to-end NLI for Auto-AIS

• The v3 NLI formulation reached 75% accuracy against human AIS, compared with 60% for a majority-label baseline.V3 checks entailment between golden evidence and the preceding question concatenated with the answer.Source: §5.1 · p. 4
• 87% accuracy against human AIS was achieved by localized NLI with two-sentence evidence windows, approaching the 93% estimated human uncertainty rate.The method splits long evidence into sentence windows, computes NLI scores, and takes the maximum localized score.Source: §5.1 · p. 4

5.2 PaLM for Auto-SSA

• PaLM-based few-shot prompts were created to predict sensibleness ratings from dialog responses.The prompts were built from human pilot ratings and used aggregated rater judgments as target scores.Source: §5.2 · p. 4
• Approximately 4% average error versus human raters was obtained by the final PaLM sensibleness prompt on validation sets.The validation used pilot slices corresponding to rows 1, 2, and 4 of Table 1.Source: §5.2 · p. 4, Table 1 · p. 4

6.1 Native Dialog Prompt

• Native dialog prompts encode turn index, parent turn, and speaker identity, supporting multiple speakers and nonlinear conversation structure.The format also lets generation stop at the next [eot] token and is intended to keep responses sensible within dialog history.Source: §6.1 · p. 5

6.2 Advanced Promptings

• Advanced prompts structure instructions, facts, and dialog history to make sensible responses attributable to supplied evidence.The setup includes golden, retrieved, absent, and non-evidence conditions.Source: §6.2 · pp. 5–6
• A representative prompt combines an instruction to use a fact, retrieved factual text, and indexed dialog turns separated by [eot] markers.The example asks a follow-up question after a prior answer about Alan Kulwicki’s racing career.Source: §6.2 · p. 5
• Retrieval conditions use BM25 with top-1, top-2, or top-3 evidence, while prompts independently vary instructions, dialog history, and evidence presence.These dimensions form the prompt-structure grid used to study component effects.Source: §6.2 · pp. 5–6
• The full experiment grid crosses 8B, 62B, and 540B models with sampling temperatures 0.0 and 0.7.Auto-AIS and Auto-SSA are applied to the full grid because comprehensive human rating would be impractical.Source: §6.2 · p. 6

7.1 Four Clusters

• Context structure, especially evidence and dialog-history availability, has a large-scale effect on the global tradeoff.The full-grid experiments identify four clusters, with evidence and dialog-history presence separating major regions.Source: §7.1 · p. 6, Table 2 · p. 6

7.2 Golden Evidence

• Reducing temperature and increasing model size produce comparable medium-scale effects on F1 under golden evidence and full conversation history.Models and temperatures fall into distinct F1 bands, allowing a smaller greedy model or a larger higher-temperature model to occupy similar bands.Source: §7.2 · p. 7

7.3 Large Model

• Comparing local tradeoffs requires experiments with similarly sized contexts because additional dialog, evidence, and instructions provide different degrees of freedom.Within fixed large-model settings, prompt differences alone usually do not yield systematic local tradeoffs; context size is proposed as the missing constraint.Source: §7.3 · pp. 7, 9

7.4 Restricted Context and Retrieval

• For a fixed context ratio, shifting capacity between dialog and evidence creates a clear local tradeoff, and top-k retrieval often raises attribution while reducing fluency relative to top-1.The synthetic setup interprets added evidence as additional retrieved items; higher recall@3 moves faster along retrieval lines but in a direction associated with lower sensibleness.Source: §7.4 · pp. 9–10

7.5 Re-Ranking or Input-Level Ensembling

• An 8B model with multiple inferences and auto-metric re-ranking achieves global-tradeoff F1 values comparable to an unre-ranked 540B model.The proposed recipe combines small-model inference, varied retrieved-item counts, and Auto-Fluency followed by Auto-Attribution re-ranking.Source: §7.5 · p. 11, §7.7 · pp. 12–13
• Re-ranking outputs under shared constraints creates a local fluency–attribution tradeoff and yields medium-scale F1 improvements comparable to reducing temperature while increasing model size.The reported improvement is also comparable to increasing model scale from 8B to 540B without re-ranking.Source: §7.5 · pp. 10–11

7.6 Discussion

• Global tradeoffs make larger moves across iso-F1 levels, whereas local tradeoffs generally move alongside iso-F1 curves and therefore preserve F1 more closely.Local tradeoffs require explicit constraints, such as context size, dialog–evidence allocation, or bounded prompt and re-ranking spaces.Source: §7.6 · pp. 11–12

7.7 Putting Things Together

• The proposed small-model recipe retrieves broadly, fits a smaller number of items within context limits, samples multiple prompt configurations, and re-ranks them with fluency before attribution.It is intended for settings where only an 8B model is affordable but performance comparable to large models with oracle evidence is desired.Source: §7.7 · pp. 12–13

• Human ratings were used to align auto-metrics before evaluating a much larger grid of model- and context-parameter experiments.The study examines fluency and attribution on the factuality-focused QReCC dialog dataset.Source: §8 · p. 13
• The analysis distinguishes global and local tradeoffs between fluency and attribution and combines both views into a retrieval-augmentation recipe.The paper presents these concepts as a way to summarize its experimental findings.Source: §8 · p. 13

9.1 QReCC Filtering

• The appendix identifies QReCC filtering as a separate preprocessing step and points to Table 3 for the full filter list and dataset-size impacts.Source: §9.1 · p. 14, Table 3 · p. 14

9.2 Note on Definition of “Attributable”

• Attributability for no-evidence pilot setups is assigned AIS=0, while alternative definitions compare generated text with golden or newly retrieved evidence.The appendix notes that the choice depends on how attributability is defined.Source: §9.2 · p. 14

9.3 Full List of Prompt Types

• The experiments vary prompt types including golden or retrieved evidence, one-shot prompting, no history or evidence, and random or next-best non-evidence.Retrieved evidence is selected with BM25 or randomly, and retrieval quality can be simulated through specified recall values.Source: §9.3 · p. 14

9.4 Final Sensibleness Prompt

• The final sensibleness prompt asks raters whether a dialog reply makes sense in context, assigning 1.0 to completely reasonable responses and 0.0 to complete nonsense.An example dialog is shown with an answer rated 0.8.Source: §9.4 · p. 15