PaperlayerView exact revision on arXiv

Source-linked AI summary

WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences

Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, Jie Tang

arXiv:2306.07906v1cs.CLcs.AI

TL;DR

WebGLM addresses the challenge of building efficient web-enhanced QA systems that handle knowledge beyond an LLM’s capacity and avoid WebGPT’s deployment costs. It combines LLM-augmented retrieval, bootstrapped generation, and human preference-aware scoring, and its 10B model outperforms WebGPT-13B while performing comparably to WebGPT-175B.

  • Problem

    LLMs can fail on rare-knowledge questions, while WebGPT requires costly expert annotations and slow, expensive behavior cloning for real-world deployment.

  • Method

    WebGLM combines two-stage LLM-augmented retrieval, an LLM-bootstrapped quoted long-form QA generator, and a scorer trained from online user feedback.

  • Results

    WebGLM (10B) outperforms WebGPT (13B), performs comparably to WebGPT (175B), and achieves the highest performance in fluency, truthfulness, and redundancy among the compared systems.

  • Takeaways & Limitations

    WebGLM demonstrates that a practical, lower-cost web-enhanced QA system can achieve performance close to a much larger WebGPT model.

Abstract

from arXiv · show

We present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web search and retrieval capabilities while being efficient for real-world deployments. To achieve this, we develop WebGLM with strategies for the LLM-augmented retriever, bootstrapped generator, and human preference-aware scorer. Specifically, we identify and address the limitations of WebGPT (OpenAI), through which WebGLM is enabled with accuracy, efficiency, and cost-effectiveness advantages. In addition, we propose systematic criteria for evaluating web-enhanced QA systems. We conduct multi-dimensional human evaluation and quantitative ablation studies, which suggest the outperformance of the proposed WebGLM designs over existing systems. WebGLM with the 10-billion-parameter GLM (10B) is shown to perform better than the similar-sized WebGPT (13B) and even comparably to WebGPT (175B) in human evaluation. The code, demo, and data are at \url{https://github.com/THUDM/WebGLM}.

1 Introduction

WebGLM addresses the limitations of WebGPT for practical web-enhanced question answering by combining efficient retrieval, bootstrapped generation, and human preference-aware scoring. Its 10B system achieves comparable quality to WebGPT-175B while outperforming WebGPT-13B and Perplexity.ai.

  • 1 Introduction: The system augments GLM-10B with web search and retrieval to address rare-knowledge questions beyond the capacity of closed-book language models.WebGLM is presented as a practical web-enhanced QA system based on the 10-billion-parameter GLM.
  • 1 Introduction: WebGPT’s real-world deployment is constrained by costly expert annotations, behavior cloning, and multi-turn browser interaction.These requirements increase the time, training, and expense needed to build and deploy the system.
  • 1 Introduction: WebGLM combines an LLM-augmented retriever, a bootstrapped generator, and a human preference-aware scorer.The retriever uses coarse web search and fine-grained LLM-distilled retrieval; the generator uses quoted long-form QA data; the scorer learns from online user feedback.
  • 1 Introduction: WebGLM (10B) surpasses WebGPT (13B), performs comparably to WebGPT (175B), and improves over Perplexity.ai in human evaluation.The system is reported as efficient, cost-effective, and human preference-aware.
  • 1 Introduction: The paper introduces human evaluation metrics and uses extensive human evaluation plus quantitative ablations to assess web-enhanced QA systems.These evaluations support claims about WebGLM’s accuracy, efficiency, and effectiveness.

2 Related Work

Related work spans large language models, open-domain question answering, retrieval augmentation, and learning from human feedback. WebGLM targets the gap between short-answer QA and costly long-form, referenced web QA.

  • 2 Related Work: Web-enhanced QA combines large language models, open-domain question answering, retrieval augmentation, and reinforcement learning from human feedback.The paper frames these as cross-domain components of a systematic construction process.
  • 2 Related Work: Open-domain QA addresses the open world but is more challenging than datasets assuming an available reference.Examples include Natural Questions, Web Questions, and MS Marco.
  • 2 Related Work: Most open-domain QA datasets and models focus on short answers, whereas users often prefer informative long-form answers with references.Constructing and evaluating such datasets is difficult because open-world references require expert-level annotation.
  • 2 Related Work: Retrieval augmentation includes sparse methods such as BM25 and TF-IDF and dense methods such as DPR and Contriever.Related retrieval-augmented language-model systems include REALM, RAG, Fusion-in-Decoder, and Atlas.
  • 2 Related Work: Human-feedback scoring extends evaluation beyond overlap metrics such as BLEU and ROUGE to learned scorers based on human preferences.The related literature also includes METEOR and BERTScore.

3 The WebGLM System

WebGLM combines efficient web retrieval, LLM-bootstrapped answer generation, and human-preference scoring to address the cost and deployment limitations of WebGPT. Its 10B system uses filtered quoted QA data and achieves fast retrieval while maintaining strong answer quality.

  • 3 The WebGLM System: WebGLM combines a two-stage retriever, bootstrapped generator, and human preference-aware scorer for efficient web-enhanced QA.The retriever searches and filters web content, the generator produces cited answers, and the scorer selects the highest-scored candidate.
  • 3.1.2 Fine-grained LLM-augmented Retrieval: The retriever’s median total latency is about 4.07s, with 90% of searches loaded within 10s; fetching pages is the main bottleneck.Parallel asynchronous loading enables most pages to load within 5s, whereas synchronous loading can take 2–3 minutes.
  • 3.2 Bootstrapped Generator: The bootstrapped generator creates 83k quoted long-form QA samples and retains 45k high-quality samples for GLM training.LLM in-context learning, citation correction, and filtering replace costly expert-written training data.
  • 3.3 Human Preference-aware Scorer: The preference scorer uses cleaned forum thumb-up feedback, with length-bias mitigation and rank-separated answer pairs to improve comparison training.The preprocessing keeps answers with more than 3 thumb-ups, selects questions with at least 8 valid answers, and pairs answers separated by more than 5 rank positions.

4 Human Evaluation Criteria

The paper introduces human evaluation criteria for both retrieved references and generated answers, combining binary and four-level scoring to balance objectivity and scale. The criteria cover informativeness, factuality, safety, linguistic quality, answer quality, citation use, and redundancy.

  • Evaluation framework: Human evaluation uses binary scores for objective metrics and four-level scores for subjective metrics, balancing objectivity with evaluation scale.Higher scores are better for upward metrics, while lower scores are better for toxicity, social bias, and redundancy.
  • Reference criteria: Reference evaluation is performed per question-reference pair using relevancy, density, truthfulness, toxicity, and social-bias criteria.Relevancy measures informativeness recall, while density measures informativeness precision.
  • Answer criteria: Answer evaluation is performed triple-wise over the question, answer, and references, covering fluency, correctness, citation accuracy, truthfulness, objectivity, and redundancy.Correctness includes factual consistency, while citation accuracy penalizes incorrect or missing citations.
  • Answer criteria: Objectivity measures whether an answer can be supported by the provided references rather than the model’s latent pre-training knowledge.Redundancy penalizes duplicate content because it reduces informativeness within limited text length.

5 Experiment

Experiments evaluate WebGLM through human assessment, a Turing test, open-domain QA benchmarks, and component ablations. Results show strong quality relative to WebGPT and Perplexity.ai, while ablations support the proposed filtering, scorer, retriever, generator, and reward-model designs.

  • 5.1 Main Results: WebGLM achieves the highest fluency, truthfulness, and redundancy performance, and reaches 2.81 correctness, close to WebGPT-175B and above Perplexity.ai and WebGPT-13B.The main human evaluation compares systems on questions from the WebGPT demo website.
  • 5.2 Turing Test: WebGLM wins against humans 43% of the time, versus 21% for Perplexity.ai and 18% for WebGPT-13B, nearly matching WebGPT-175B at 45%.The Turing test ranks shuffled system and human answers on correctness, informativeness, and truthfulness.
  • 5.3 Test on QA Benchmarks: WebGLM outperforms Perplexity.ai on 400 sampled Natural Questions and Web Questions.The benchmark evaluation uses randomly sampled questions from both datasets.
  • 5.4.1 WebGLM-QA Filtering: Rouge-1 filtering improves reference relevancy and density, answer correctness, factual consistency, and objectivity, making citation accuracy closely related to reference and answer quality.The experiments compare unfiltered, Rouge-L-filtered, and Rouge-1-filtered datasets.
  • 5.4.1 WebGLM-QA Filtering: Models trained on the Rouge-1 dataset outperform alternatives in fluency, citation accuracy, and objectivity, motivating its use for training WebGLM-10B.The three datasets show little difference in correctness, but Rouge-1 improves the other reported metrics.
  • Scorer Ablation: The human preference-aware scorer performs best on accuracy and Spearman coefficient, while larger training amounts substantially improve the reward model.Its reward curve is also more discriminative on the ELI5 test set than those of other models.
  • Component Ablations: WebGLM retrieval performs similarly to WebGPT-175B and significantly better than non-retrieval, while WebGLM generation is slightly better than GPT-3 across several quality metrics.Reward-model scoring improves fluency, correctness, citation accuracy, truthfulness, and redundancy over original sampled answers.

6 Conclusion

WebGLM combines web retrieval, LLM-bootstrapped quoted long-form QA data, and a human preference-aware scorer to produce preferred answers. Human evaluation and Turing tests show competitive performance against Perplexity.ai and WebGPT.

  • 6 Conclusion: WebGLM combines web retrieval, an LLM-bootstrapped quoted long-form QA dataset, and a human preference-aware scorer that selects the highest-scored candidate answer.The scorer is trained to assign marks to generated responses and choose the answer humans prefer most.
  • 6 Conclusion: Human evaluation and Turing tests demonstrate competitive WebGLM performance against Perplexity.ai and WebGPT.The conclusion frames these experiments as evidence for the system’s competitive performance among web-enhanced QA systems.

A.1 WebGLM vs Others in WebGPT Reference

Using WebGPT-175B references, WebGLM-Rouge1 achieves competitive human-evaluated generation quality despite being more than ten times smaller. It matches larger systems on several metrics and exceeds them on fluency and redundancy.

  • A.1 WebGLM vs Others in WebGPT Reference: WebGLM-Rouge1 matches WebGPT-175B and GPT-3 on correctness, citation accuracy, objectivity, and truthfulness, while outperforming them on fluency and redundancy.The comparison uses WebGPT-175B references and human evaluation of generated answers.
  • A.1 WebGLM vs Others in WebGPT Reference: WebGLM-Rouge1 achieves this competitive performance with a model more than ten times smaller than GPT-3 and WebGPT-175B.The result is presented as evidence that the retrieval paradigm compensates for model-size differences.

B Detailed Efficiency Analysis

The efficiency analysis compares WebGLM’s retrieval pipeline with WebGPT’s browser-based generation, showing substantially lower query latency for WebGLM.

  • WebGLM retrieves references through one search, parallel page fetching, paragraph extraction, Contriever ranking, and selection of the top five paragraphs.The retrieval-stage time is decomposed into search, fetching, extraction, and ranking.
  • WebGPT simulates browser operations, and its browsing cost depends on command-generation time plus repeated action costs.The analysis counts action types and generated tokens for 272 questions to estimate WebGPT latency.
  • With a 500-token prompt, GPT-3 generates about 20 tokens per second at 175B parameters and 100 tokens per second at 13B parameters.
  • WebGLM consumes 5.36 seconds per query, compared with 52.48 seconds for WebGPT-175B and 31.12 seconds for WebGPT-13B.The comparison assumes the same network conditions.
  • WebGLM’s efficiency is about 10 times that of WebGPT-175B and 6 times that of WebGPT-13B.

C Choice of Prompts and Instructions

The study evaluates prompt and instruction strategies for bootstrapping WebGLM-QA data, finding that reference-question ordering is especially influential.

  • Zero-shot bootstrapping methods were limited in effectiveness when generating answers with appropriate citation marks.
  • Bootstrapped data can contain inconsistent citation formats, including [2] and.
  • Bootstrapped data may cite useless references even when the instruction says not to cite them, so this method does not work reliably.
  • The bootstrap context uses one in-context example and five references because additional examples or references can exceed the token limit.The example includes useless references that should not be cited in the answer.
  • Placing the question after the references is the most effective prompt strategy in the experiments.The finding comes from experiments comparing prompt arrangements for bootstrapped data.
  • The selected concise instruction asks the model to read provided references and answer the corresponding question.

D Dataset Examples

The dataset examples illustrate WebGLM-QA construction, retrieval outputs, answer quality, and the effect of reward-model training.

  • Table 14 illustrates WebGLM-QA with an unrelated fifth reference that is intentionally left uncited as dirty training data.Unrelated references are included to train the model to select relevant information.
  • The examples compare retrieved references from WebGLM and Perplexity AI with those retrieved by WebGPT systems.
  • WebGLM’s answer is consistent with the question in one example, while both WebGPT answers are beside the point in the conclusion.
  • WebGLM performs better after implementing the reward model, as illustrated by the comparison between GLM-10B-RM and GLM-10B-Rouge1.

H Criteria Details

The paper presents evaluation criteria for references and answers alongside a web demo and real examples spanning technical, entertainment, current-events, and general-interest questions.

  • Human-evaluation criteria are provided separately for references and answers in Tables 20 and 21.
  • The paper provides a WebGLM web demo and real examples showing responses to diverse questions.Examples include career, software, machine learning, games, COVID-19, politics, films, and AI systems.
  • The real examples include questions about transformers in computer vision, Music Transformer, Toolformer, and CyGames.
  • Additional examples cover COVID-19, the current United States president, Black Panther 2, Hogwarts Legacy, Google Bard, and popular AI technology in 2023.
  • The examples also address NBA All-Star teams, Copilot, ChatGPT’s core technique, Copilot training data, and the model behind Perplexity AI.
Loading 2306.07906v1…