Source-linked AI summary
Recent Advances in Neural Question Generation
Liangming Pan, Wenqiang Lei, Tat-Seng Chua, Min-Yen Kan
TL;DR
Neural QG must combine understanding input with generating appropriate questions, and the survey addresses the lack of comprehensive coverage of recent work. It synthesizes corpora, methods, and evaluation, identifying trends toward joint learning, broader inputs, and deeper questions.
Problem
Neural QG combines input understanding with question generation, but the field lacked a comprehensive survey of its recent developments.
Source: §1 · p. 1Method
The paper systematically surveys NQG corpora, methodologies, and evaluation methods, organizing findings around learning paradigms, input modalities, and cognitive levels.
Source: §1 · p. 1Results
The survey identifies emerging trends in multi-task learning, wider input modalities, and deep-question generation.
Source: §1 · p. 1Takeaways & Limitations
Personalized QG through explicit user-state modeling remains an unexplored direction described by the survey.
Source: §7 · p. 9Most NQG models use SQuAD, leaving other question types, including deep multiple-choice questions, less investigated.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Emerging research in Neural Question Generation (NQG) has started to integrate a larger variety of inputs, and generating questions requiring higher levels of cognition. These trends point to NQG as a bellwether for NLP, about how human intelligence embodies the skills of curiosity and integration. We present a comprehensive survey of neural question generation, examining the corpora, methodologies, and evaluation methods. From this, we elaborate on what we see as emerging on NQG's trend: in terms of the learning paradigms, input modalities, and cognitive levels considered by NQG. We end by pointing out the potential directions ahead.
1 Introduction
Question generation combines understanding and reasoning over input with the challenge of producing grammatical, semantically correct questions. This survey addresses the lack of a comprehensive account of neural QG by organizing recent work around its emerging trends.
1 Introduction
- QG requires understanding input and relevant contexts, then generating grammatically and semantically correct questions.It is presented as a challenging task complementary to question answering.Source: §1 · p. 1
- QG can support educational assessment and self-learning, dialogue systems, and the creation of large-scale QA and reading-comprehension datasets.The paper notes applications including generating assessments from course materials and initiating conversations or obtaining information from users.Source: §1 · p. 1
- The survey fills a gap in comprehensive coverage by examining neural QG’s learning paradigms, input modalities, and generation of deeper questions.It situates these trends alongside recent advances in end-to-end neural methods and broader applications.Source: §1 · p. 1
2 Fundamental Aspects of NQG
The survey frames NQG through its learning paradigm, input modalities, and the cognitive levels of questions. It contrasts older, separated selection-and-construction pipelines with neural approaches and traces expanding inputs and ambitions for deeper questions.
2.1 Learning Paradigm
- Neural Seq2Seq models jointly learn content selection and question construction, replacing parsing-based selection with attention or copying and making question construction data-driven.Earlier pipelines separated the stages and relied on symbolic representations, transformation rules, or templates; neural approaches allow more language flexibility with less labor.Source: §2.1 · p. 2
2.2 Input Modality
- Traditional QG largely used textual inputs, while neural work has expanded to knowledge bases and images alongside growth in KBQA and VQA applications.The survey relates this expansion to neural models’ feature representations for images and knowledge.Source: §2.2 · p. 2
2.3 Cognitive Levels
- Traditional QG mainly produces shallow factoid questions, whereas higher Bloom levels include questions such as Why, Whatif, and How that require reasoning over input.The survey examines recent work exploring comprehension and reasoning, while noting limitations of current Seq2Seq models for deep-question generation.Source: §2.3 · p. 3
3 Corpora
The survey describes NQG corpora by question-answering difficulty and answer type, and notes that benchmark concentration leaves several forms of generation underexplored.
3 Corpora
- Corpus difficulty depends partly on the cognitive level needed to answer: NQG performs promisingly on shallow factoid datasets but drops on deep-question datasets.The section names SQuAD and MS MARCO as shallow-question examples and LearningQ as a deep-question dataset.Source: §3 · p. 3
- Corpora also differ by answer type: passage spans, abstractive human answers, multiple-choice options with distractors, or no supplied answer.The system design differs accordingly.Source: §3 · p. 3
- SQuAD is the most-used NQG benchmark in the survey, leaving deep multiple-choice questions and other QG types less investigated.The authors call for future research to benchmark against a wider variety of corpora.Source: §3 · p. 3
4 Evaluation Metrics
The survey describes human and automatic ways to evaluate generated questions, while emphasizing that question quality is difficult to quantify and existing approaches have limitations.
4 Evaluation Metrics
- Human evaluation commonly has annotators rate sampled generated questions on a five-point Likert scale, reporting average ranks or the share ranked best.Meaningfulness, grammaticality, semantic soundness, and naturalness are useful but difficult to quantify.Source: §4 · p. 3
- BLEU, METEOR, and ROUGE are widely used because human evaluation is time-consuming, but they do not correlate well with fluency, adequacy, or coherence.The cited concern is that these metrics essentially compute n-gram similarity between the source sentence and generated question.Source: §4 · p. 3
- Table 1 groups NQG datasets by cognitive level and answer type and lists document counts, question counts, and average questions per document.These dimensions allow corpus statistics to be compared across dataset groups.Source: Table 1 · p. 4
- An answerability metric based on question type, content words, function words, and named entities had not yet been applied to evaluate an NQG system.The survey calls for improved schemes that investigate the mechanism of question asking, especially for deep questions.Source: §4 · p. 4
5 Methodology
Neural question generation commonly uses Seq2Seq models, with variations for answer focus, question-word choice, broader context, and content selection. The survey also reviews general NLG techniques and reports comparable SQuAD results, with Zhao et al. (2018) state of the art at the time of writing.
5 Methodology
- Seq2Seq models condition question generation on a passage and possibly an answer; the survey also reviews attention and QG-specific and general NLG techniques.Du et al. (2017) introduced an attention-based model, while later models vary in answer encoding, question-word generation, context use, copying, linguistic features, and reinforcement learning.Source: §5 · p. 4
5.1 Encoding Answers
- Answer-aware models mark answer positions or encode answers separately, while answer indicators can also encourage answer-revealing questions.Sun et al. (2018) added relative-position embeddings to attention, gaining 0.89 BLEU-4 on SQuAD; separate passage and answer encoding is presented as more flexible but more complex to train.Source: §5.1 · pp. 4–5
5.2 Question Word Generation
- Duan et al. (2017) generate question templates before filling them, while Sun et al. (2018) add a decoding mode for question words.The survey notes that question-word and answer-type mismatches are common and points to question pragmatics, including intent, as a possible direction.Source: §5.2 · p. 5
5.3 Paragraph-level Contexts
- Context methods either refine representations across a full passage with gated self-attention or pre-filter context using coreference clusters.Zhao et al. (2018) report state-of-the-art SQuAD results for their gated self-attention encoder.Source: §5.3 · p. 5
5.4 Answer-unaware QG
- Without a target answer, systems select question-worthy sentences or keyphrases before using Seq2Seq question construction; multi-piece reasoning remains challenging.The survey identifies reasoning patterns such as inductive, deductive, causal, and analogical reasoning as future study areas.Source: §5.4 · pp. 5–6
5.5 Technical Considerations
- NQG models use copying and linguistic features, while policy-gradient rewards such as BLEU or ROUGE can distribute probability across equivalent question expressions.The survey notes that optimizing only ground-truth likelihood ignores equivalent ways of asking a question.Source: §5.5 · p. 6
5.6 The State of the Art
- Existing NQG models achieve comparable SQuAD results, with Zhao et al. (2018) state of the art at the time of writing.The survey summarizes model techniques and best-reported SQuAD performance in Table 2.Source: §5.6 · p. 6
- Copying can produce invalid answer-revealing questions, while paragraph-context methods show promise for performance and deeper question generation.The survey says the future role of copying requires further investigation.Source: §5.6 · p. 6
6 Emerging Trends
The survey highlights three emerging NQG trends: multi-task learning, wider input modalities, and generation of deeper questions. It reviews how these trends are being explored and identifies directions for deep question generation.
6.1 Multi-task Learning
- QG-derived pseudo-labeled pairs raised semantic-parsing performance 3%, while iterative QA–QG self-training expands data and joint objectives can leave QG below state of the art.In the described self-training process, QG creates questions from unlabeled text, QA answers them, and the resulting pairs are used to retrain both models.Source: §6.1 · pp. 6–7
6.2 Wider Input Modalities
- NQG extends beyond text to knowledge bases and images; for KBs, copying and textual context address sparse entities, while related-triple keywords counter overgeneralization.A single KB triple may provide too little information, producing questions that are too broad.Source: §6.2 · p. 7
- Grounded VQG targets image-supported answers, whereas open-ended VQG seeks natural, engaging questions involving higher cognition.Grounded systems use image captions to varying degrees; open-ended approaches have used VAEs and GANs to encourage creativity.Source: §6.2 · pp. 7–8
6.3 Generation of Deep Questions
- Deep questions concern events, evaluations, opinions, syntheses, or reasons, and college students asked only about 6 deep-reasoning questions per hour in one tutoring study.The cited study illustrates the difficulty of producing high-quality deep questions even for humans.Source: §6.3 · p. 8
- LearningQ's Seq2Seq results were < 4 BLEU-4 and < 9 METEOR, versus > 12 BLEU-4 and > 16 METEOR on SQuAD.The authors suggest that handling long inputs and reasoning across multiple pieces of information may explain the difficulty.Source: §6.3 · p. 8
- Deep-question approaches include semantic representations, crowdsourced templates, and executable question programs; the roadmap emphasizes cross-sentence relations, reasoning patterns, and human question-asking.The crowdsourcing workflow is semi-automatic, combining an intermediate ontology, question templates, and template retrieval and ranking.Source: §6.3 · p. 8
7 Conclusion – What’s the Outlook?
The survey organizes NQG models by QG-specific and general technical variations and identifies multi-task learning, wider input modalities, and deep question generation as emerging trends. It points to future work on when machines should ask and on personalizing questions to users.
7 Conclusion – What’s the Outlook?
- The survey categorizes NQG models by technical variations and summarizes trends in multi-task learning, wider input modalities, and deep question generation.It also identifies potential future directions for the field.Source: §7 · p. 8
- When to ask remains unexplored for engaging machine questions in dialog, despite its relevance to tutoring and conversational systems.The survey proposes interactive and dynamic question asking as a possible research topic.Source: §7 · pp. 8–9
- Personalized QG remains unexplored in dialog management and recommendation systems, although people ask different questions based on character and knowledge background.The proposed direction explicitly models user state and awareness.Source: §7 · p. 9