Source-linked AI summary
Internet-Augmented Dialogue Generation
Mojtaba Komeili, Kurt Shuster, Jason Weston
TL;DR
The paper addresses static and hallucination-prone dialogue models by giving conversational agents dynamic internet access. It generates search queries from context, conditions responses on retrieved results, and evaluates this approach on human-human conversations with an internet-searching wizard. Internet-augmented models outperform non-augmented generation, while the authors note risks from toxic, biased, or inaccurate websites.
Problem
Dialogue models freeze knowledge at training time and can generate factually incorrect statements, motivating dynamic access to current information.
Method
The model generates a search query from dialogue context, retrieves internet results, and uses them to generate a response in a newly collected wizard-grounded conversation task.
Results
Internet-augmented models outperform conventional non-augmented models on automatic metrics and human evaluations, while search-engine models achieve validation perplexity 16.4 on WizInt and 16.1 on WoW+WizInt for CC+Wikipedia.
Takeaways & Limitations
Search-query-based internet access provides a way for dialogue systems to use current web knowledge and generate less factually incorrect responses.
Takeaways & Limitations
Internet augmentation can introduce toxic, biased, or factually incorrect information from the websites it retrieves.
Abstract
from arXiv · showhide
The largest store of continually updating knowledge on our planet can be accessed via internet search. In this work we study giving access to this information to conversational agents. Large language models, even though they store an impressive amount of knowledge within their weights, are known to hallucinate facts when generating dialogue (Shuster et al., 2021); moreover, those facts are frozen in time at the point of model training. In contrast, we propose an approach that learns to generate an internet search query based on the context, and then conditions on the search results to finally generate a response, a method that can employ up-to-the-minute relevant information. We train and evaluate such models on a newly collected dataset of human-human conversations whereby one of the speakers is given access to internet search during knowledgedriven discussions in order to ground their responses. We find that search-query based access of the internet in conversation provides superior performance compared to existing approaches that either use no augmentation or FAISS-based retrieval (Lewis et al., 2020).
1 Introduction
Static dialogue models store knowledge in fixed parameters, so they cannot reflect newly arriving information and may hallucinate factual errors. The paper instead gives conversational agents dynamic internet access through generated search queries and reports stronger performance for internet-augmented models.
- Static language models freeze knowledge at dataset collection and cannot account for information that changes by the day or minute.
- Hallucination occurs when models generate plausible statements that are factually incorrect.
- The proposed model generates a search query from dialogue context, retrieves relevant knowledge, and conditions response generation on the retrieved documents.
- The authors collect human-human knowledge-grounded conversations in which one worker searches the internet as a wizard to inform responses.
- Internet-augmented models trained to replace the wizard outperform conventional non-augmented models on automatic metrics and human evaluations.
2 Related Work
Prior dialogue systems generally store knowledge in model weights or retrieve from constrained resources, while related work increasingly augments generation with external knowledge. This paper extends knowledge-grounded dialogue toward broader internet coverage and human-directed searching.
- Standard dialogue generation trains Transformer models on conversational data without external knowledge augmentation.
- External-knowledge approaches include TFIDF, neural retrieval, RAG, and FiD methods developed for question answering and dialogue.
- Wizard of Wikipedia grounds conversations in Wikipedia using TFIDF retrieval from that database.
- The proposed task covers publicly available internet information and more diverse conversational topics than Wikipedia-only grounding.
- Related work also studies neural retrieval in the loop and search engines for downstream language-generation tasks.
3 Internet-Augmented Generation
The paper compares cached FAISS retrieval with direct search-engine access and proposes a two-stage system that generates a query before encoding retrieved documents for response generation. It also regularizes models toward using selected retrieved knowledge.
- FAISS-based methods: FAISS-based methods retrieve passages from indexed Common Crawl or Wikipedia documents using dense DPR representations.The Common Crawl index uses a November 2020 dump with approximately 109M webpages split into 250M passages.
- FAISS-based methods: RAG jointly trains neural retrieval and response generation, whereas FiD separately encodes retrieved documents and fuses them during decoding.
- FAISS-based methods: FiD-RAG first trains the retriever in a RAG setup and then applies FiD, combining both approaches.
- FAISS-based methods: Search-query retrieval generates a query from context, uses it for dense FAISS matching, and feeds returned documents into response generation.
- Search Engine-Augmented Generation: FAISS deployments may be difficult to update in real time, limited in storage, and unable to exploit search engines’ mature ranking.
- Search Engine-Augmented Generation: The proposed system combines a query generator with a FiD-style encoder-decoder that encodes documents individually before generating the response.
- Search Engine-Augmented Generation: The search engine is treated as a replaceable black box; experiments use Bing URLs mapped to page content from a Common Crawl snapshot.
- Knowledge Response Regularization: Knowledge-response regularization multitasks response generation with generating selected retrieved knowledge, encouraging models to attend to and copy from retrieved documents.
4 Wizard of the Internet Task
The Wizard of the Internet task pairs crowdworkers in conversations centered on an apprentice’s interests, with the wizard searching online to ground responses. The resulting dataset contains diverse, frequently searched topics and human-selected grounding evidence.
- The dataset pairs a wizard who can search with an apprentice who lacks search access and has an assigned persona.
- Conversations target in-depth discussion of the apprentice’s assigned interests and contain 5–6 turns, or 10–12 utterances.
- Apprentices choose personas such as tennis interests, with choices mined from Persona-Chat interests.
- Wizards may open the conversation or respond to the apprentice, using searches to address the apprentice’s interests.
- Wizards enter free-text searches, inspect expandable results, select relevant sentences, and use them to compose responses.
- The collection contains 9633 dialogues and 82952 training utterances, with 5781 validation and 4932 test utterances.
- 84.81% of wizard turns involve search, while selected search-result sentences are used 80.3% of the time.
- Training spans 26192 unique selected URLs across 10895 domains, and Wikipedia accounts for 8.56% of validation queries.
5 Experiments
The experiments compare pre-trained models and retrieval strategies on WizInt, finding that search-engine augmentation performs best overall while knowledge use involves a measurable trade-off with conversational overlap.
- Experimental setup: The study evaluates pre-trained Transformers, retrieval methods, search-query systems, and knowledge-response regularization on WizInt using automatic and human metrics.Models include BlenderBot, BART-Large, and T5-Large; retrieval methods use five returned documents.
- Pre-training models: Larger BlenderBot models improve perplexity, F1, and KF1 in the no-knowledge setting, while BART-Large and T5-Large improve F1 and KF1 at comparable model sizes.The authors select BART-Large as the base for subsequent experiments.
- Baseline comparisons: Gold knowledge substantially improves validation performance over no knowledge across the reported Transformer baselines.For BART-Large, validation F1 increases from 17.6 to 25.4 and KF1 from 6.8 to 23.1.
- Retrieval and search augmentation: Query generation followed by FAISS retrieval performs slightly worse than DPR+FAISS, with perplexity no better than the no-knowledge model except for the Wikipedia database case.Reported perplexities are 17.6 for CC and 17.2 for Wikipedia, compared with 17.6 for no knowledge.
- Retrieval and search augmentation: Search-engine models achieve the best validation perplexity, reaching 16.4 on WizInt and 16.1 when trained jointly on WoW and WizInt for the CC+Wikipedia case.F1 and KF1 are hardly impacted, while test-set perplexity shows a similar reduction.
- Knowledge-response regularization: Knowledge-response regularization increases KF1 at the expense of some F1, providing smooth control over knowledge use.The authors do not use this regularization in the remaining results but identify it as potentially useful for retrieval-augmented systems.
- Human evaluation: In human evaluation, search-engine augmentation is judged more knowledgeable, less factually incorrect, more consistent, and more engaging than the no-knowledge baseline.Knowledgeable judgments are 46.5% versus 38.6%, factual incorrectness 5.3% versus 7.1%, and consistency 76.1% versus 66.5%.
- Example conversations: The error analysis identifies generation mistakes, poor query formulation, wrong knowledge selection, and failure to use correct available knowledge.These errors occur even when the relevant knowledge is present in retrieved sources.
6 Conclusions
The work addresses siloed knowledge in language models by augmenting generation with internet access, enabling more knowledgeable and less factually incorrect dialogue while motivating broader future applications.
- Internet augmentation addresses language models’ inability to access world knowledge beyond their fixed training sets.
- The authors report that internet-augmented models can display more knowledge and generate less factually incorrect information during dialogue with humans.
- Future work should improve architectures and extend evaluation to additional situations, including goal-directed tasks.
7 Societal Impact
The paper discusses environmental, safety, toxicity, bias, and factuality concerns associated with language models and internet augmentation, while noting possible mitigations and interpretability benefits.
- Large language models require substantial resources for training and deployment and raise concerns about toxic language, bias, and related generation issues.
- The fine-tuning task used crowdworker instructions against toxic language, while the initial pre-training data contained potentially offensive internet-procured text.
- Internet augmentation can improve interpretability and debuggability because models generate readable queries and expose source documents.
- Retrieved websites may themselves contain toxic, biased, or factually incorrect information, making fact checking a separate concern.
- Using search engines instead of FAISS retrieval can leverage existing search-engine safety mitigations rather than rebuilding them.
A Wizard of Internet Task
The Wizard of Internet task uses crowdworkers to create diverse knowledge-grounded conversations through persona or topic choices, refinement, and screenshots of the collection and evaluation tasks.
- Crowdworker collection and evaluation tasks are documented with screenshots in Figures 8 and 10.
- Persona choices combine Persona-Chat entries used 10% of the time with topic-based choices used 90% of the time.
- Persona-Chat entries were selected when they contained searchable entities or location-dependent topics, while generic or unrelated topics were excluded.
- Topic-based collection covers fashion, books, music, movies or television, sports, hobbies or games, and purchasable items.
- Workers choose between two topics, specify a favorite item, and add an imaginative refinement sentence to promote diverse conversations.
B.1 Model Training Details
Most retrieval-augmented BART-Large models were trained on four 32-GB GPUs with Adam, learning-rate sweeps, warmup, and validation-based early stopping, using five retrieved documents per example.
- Most retrieval-augmented BART-Large models were trained on 4 32-GB GPUs with Adam and learning rates between 1e-6 and 5e-5.
- Training used batchsize 16, a linear learning-rate scheduler with 100 warmup updates, and early stopping based on validation perplexity.
- Each example used N = 5 retrieved documents; FAISS documents were supplied in 100-word chunks, whereas search-engine documents used their first 256 tokens.
B.2.1 Training Details
Search query generators are BART-Large models trained to produce human search queries from dialogue context, using specified optimization settings and validation-based early stopping.
- BART-Large models generate human search queries from dialogue context.Training used 4 32-GB GPUs.
- Training used Adam with a 1e-5 learning rate, batch size 64, and a linear scheduler with 100 warmup updates.
- Early stopping was based on validation-set perplexity.
B.2.2 Query Generation Performance
Query-generation performance was assessed through retrieval metrics comparing generated search results with gold and wizard-selected documents. The generators retrieved the gold document nearly 25% of the time, while beam-size and minimum-length changes produced little automatic-metric variation.
- Evaluation setup: The evaluation compares gold retrieved documents, wizard-selected documents, and search results produced by generated queries.These sets are used to assess downstream retrieval quality rather than query-generation metrics alone.
- Retrieval metrics: % in Top 5 measures the percentage of gold documents present among generated search results.
- Retrieval metrics: Average F1 averages each generated result’s maximum word-overlap F1 against the gold retrieved documents.
- Retrieval metrics: Gold Recall at 5 measures how often any wizard-selected document appears among the generated search results.
- Results: Nearly 25% of the time, the query generator retrieved the gold document.The result is reported for two decoding schemes in Table 8.
- Decoding effects: Changing beam size and minimum beam length produced little change in automatic metrics.This result comes from experiments with search-engine FiD using CC+Wikipedia.