GINGER: Grounded Information Nugget-Based Generation of Responses

1 Introduction

• RAG responses must address factual correctness, source attribution, verifiability, consistency, and coverage, challenges compounded by redundant or overly long contexts.The paper notes that relevant information in the middle of long contexts can be difficult for models to retrieve.Source: §1 · p. 1
• GINGER operates on atomic information nuggets and processes them through detection, facet clustering, ranking, summarization, and fluency refinement.The pipeline aims to include the maximum number of unique query-answering facts while grounding the final response in source passages.Source: §1 · p. 1
• GINGER performs on par with top-performing TREC RAG’24 submissions and significantly outperforms two strong baselines.The evaluation uses the TREC RAG’24 augmented generation task and AutoNuggetizer.Source: §1 · p. 1
• The paper contributes a modular nugget-based response-generation pipeline and an extensive evaluation on TREC RAG’24 using AutoNuggetizer.Source: §1 · p. 1

2 Related Work

• RAG generates a coherent response from retrieved sources rather than returning a ranked document list, but partially relevant retrieval can introduce text that hinders response quality.Source: §2 · p. 1
• Long-context generation can struggle when relevant information lies in the middle, while fluent responses and abstractive summaries can still contain unsupported claims or inaccurate citations.Source: §2 · p. 2
• Context-curation approaches include token removal, multi-agent aggregation, and trained extractors or condensers; GINGER additionally decomposes generation by operating on atomic information pieces.Source: §2 · p. 2

• An LLM detects nuggets by marking key answer information copied from passages without altering their content or adding symbols.Source: §3 · p. 2
• BERTopic [10], tuned on validation data, groups semantically close nuggets into query facets to reduce redundancy and increase information density.Source: §3 · pp. 2–3
• Because facet-cluster counts are relatively low, GINGER uses pairwise duoT5 reranking [29] to prioritize clusters for response inclusion.Source: §3 · p. 3
• The response summarizes the top n clusters independently, with n setting the desired length and a prompt word cap limiting each sentence.The prompt design aims for concise summaries containing only supplied information.Source: §3 · p. 3
• An LLM rephrases the concatenated cluster summaries to improve fluency and consistency without changing their information or adding content.Source: §3 · p. 3

Dataset

• The evaluation covers 301 TREC RAG’24 queries, using fixed ranked passages from MS MARCO V2.1 and input rankings of 5, 10, or 20 passages.The task assumes the query and ranked passages are provided as input.Source: §4 · p. 3

TREC Evaluation

• AutoNuggetizer creates vital or okay nuggets from relevant documents, then assigns nuggets by checking whether the response supports them.The system score averages query scores based on supported vital nuggets.Source: §4 · p. 3

LLM-based Evaluation

• The reimplemented AutoNuggetizer averages scores from three LLM judges; scores were higher than official TREC scores, but system ordering stayed the same.The judges were GPT-4o, Claude 3.5 Haiku, and Gemini 1.5 Flash.Source: §4 · p. 3

Baseline

• The baselines use GPT-4 with the top five passages, either standard retrieval-augmented QA prompting or Chain-of-Thought prompting with one in-context example.Responses were limited to around 100 words or three sentences.Source: §4 · p. 4

Implementation Details

• GINGER uses GPT-4 for nugget detection and query-biased summarization, BERTopic for clustering, and duoT5 for cluster ranking.The top three ranked clusters are summarized, with each summary limited to one sentence and around 35 words.Source: §4 · p. 4

Overall Performance

• GINGER’s best-performing variant outperforms both baselines and is among the top-performing TREC RAG’24 augmented-generation submissions.The comparison with TREC submissions identifies GINGER-top20 without rewriting as among the top performers.Source: §5 · p. 4

Implementation Detail

• When fewer than four nuggets are found, GINGER skips clustering and treats each nugget as an independent cluster before ranking and summarization.This is the specified exception to the clustering stage.Source: §5 · p. 4

Input Information

• Responses from 20 passages score higher than those from 10 passages at the same length limit, suggesting the system includes more vital facts despite possible redundancy.The paper attributes the pattern to increased input context and reports it for GINGER without rewriting.Source: §5 · p. 4

Component Ablation

• Replacing nugget clustering with LSA or cluster ranking with BM25 does not strongly affect response quality; the authors identify nugget-based operation as the primary factor.The ablation uses 20 relevant passages and a 400-word response limit.Source: §5 · p. 4

Fluency Enhancement

• Adding LLM-based fluency enhancement to top-five responses produces no significant difference from the version without rewriting.The authors conclude fluency can be refined without sacrificing quality or grounding.Source: §5 · p. 4

Conclusion

• GINGER uses nuggets from retrieved passages in a multistage process to produce concise, information-rich responses free of redundancy.The pipeline includes clustering, reranking, summarization, and fluency enhancement.Source: §6 · p. 4
• AutoNuggetizer evaluation shows GINGER outperforms two strong baselines and performs comparably to top TREC RAG’24 submissions.Source: §6 · p. 4
• Future directions include varying response length, controlling completeness, and managing redundancy in multi-turn conversations using previously discussed facets.Source: §6 · p. 4