Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base

1 Introduction

• Stepwise KB-QA approaches can propagate upstream errors downstream, while independently learned subtasks cannot share supervision.Entity detection and linking errors are identified as examples of upstream errors.Source: §1 · p. 1
• The pointer-equipped parser uses entity-mention positions and contextual features to support joint learning with entity detection and conversational coreference resolution.The authors contrast its contextual features with prior low-level entity representations such as mean-pooled word embeddings.Source: §1 · pp. 1–2
• Type-aware entity detection jointly predicts entity mentions and types, then uses predicted types to filter entity-linking results at inference.The authors state that jointly learning entity detection and parsing shares supervision, and that type prediction further reduces entity-linking errors.Source: §1 · p. 2
• 12.56% overall F1 improvement over strong baselines, with gains consistent across all question types.The reported result is on CSQA.Source: §1 · p. 2

2 Task Definition

• Questions concatenate dialog history to handle ellipsis or coreference, and are labeled with their answers.Entity mentions in the resulting utterance also receive IOB tags and linked KB entities for entity-detection training.Source: §2 · p. 2
• The approach detects entities for linking and generates a logical form that is executed on the knowledge base to retrieve an answer.Entity detection uses IOB tags and linked entities as supervision; semantic parsing uses a gold logical form, which may require intensive BFS when only final answers are available.Source: §2 · p. 2

3.1 Grammar and Logical Form

• Logical forms are ordered trees serialized by depth-first traversal; entry categories are question constants, while intermediate categories are operator outputs.The grammar constrains child nodes by the parent operator’s nonterminal category.Source: §3.1 · p. 3

3.2 Proposed Model

• MaSP comprises word embedding, contextual encoding, entity detection, and a pointer-equipped logical-form decoder, with semantic parsing and entity detection learned in one model.The combined training lets each task leverage supervision signals from the other.Source: §3.2 · p. 3, §3.3 · p. 5

3.2.1 Embedding and Contextual Encoder

• The input concatenates the current utterance and dialog history, then token embeddings and a Transformer encoder produce contextual representations.A [CTX] token represents the entire question.Source: §3.2.1 · p. 3

3.2.2 Pointer-Equipped Decoder

• The decoder generates sequence-formatted logical forms, using a small token vocabulary and separate scoring for predicate and type candidates.Grammar guidance constrains decoding to operators legitimate for the current nonterminal, while beam search and early execution filter decoding candidates.Source: §3.2.2 · p. 4, §3.3 · p. 5
• Pointer networks locate target entities and numbers in question mentions, addressing coreference while reducing the decoding vocabulary from millions of KB entities to question positions.The decoder uses separate pointer networks for entities and numbers; for multiword entities, it points to the first word.Source: §3.2.2 · p. 4

3.2.3 Entity Detection and Linking

• A joint IOB-tag and entity-type prediction space distinguishes same-text entities, and predicted types filter KB linking candidates before the logical form is executed.Linking locates mentions, retrieves candidates with an inverted index, and selects the highest-scoring candidate among those matching the predicted type.Source: §3.2.3 · pp. 4–5

3.3 Learning and Inference

• Training combines semantic-parsing and entity-detection losses, weighting the former by α; gold logical forms are searched when training data provides only weak supervision.The paper describes the shared model as enabling each task to use the other’s supervision.Source: §3.3 · p. 5

4.1 Experimental Settings

• CSQA contains about 1.6M question-answer pairs across ∼200K dialogs and a Wikidata KB with 21.2M triplets and 12.8M entities.Entity-answer questions are evaluated with F1; boolean- or number-answer questions use accuracy. HRED+KVmem and D2A are the comparison baselines.Source: §4.1 · pp. 5–6

4.2 Model Comparisons

• ∼12% overall F1 improvement over D2A, with gains of 2% to 45% across question types, establishes MaSP as a new state of the art on CSQA.The entity accuracy in final logical forms rises from 55% to 72% versus D2A; the authors also attribute improvement to better coreference handling.Source: §4.2 · p. 6, Table 2 · p. 6
• The shallow model improves logical-reasoning F1 over D2A by only 1.73%; deeper modeling with BERT raises that question-type result by 10% over D2A.The authors suggest the shallow model struggles to parse the multiple entities often needed for logical-reasoning forms.Source: §4.2 · p. 7

Effect of Entity Type Prediction (w/o ET)

• Removing entity-type prediction causes a 9% drop in overall F1, and both precision and recall for entity linking fall significantly without type filtering.Figure 3 presents entity-linking performance; “w/o ET” removes entity type filtering.Source: Effect of Entity Type Prediction (w/o… · p. 7, Figure 3 · p. 7, Table 3 · p. 7

Effect of Multi-Task Learning (w/o Multi)

• Separate training lowers F1 by 3% to 14% across question types; joint learning improves logical-form component accuracy while entity-detection F1 changes only from 99.5% to 99.4%.The authors conclude that multi-task learning improves final question answering while maintaining satisfactory entity detection.Source: Effect of Multi-Task Learning (w/o… · p. 7, Table 4 · p. 7, Table 3 · p. 7
• The combined ablation removes both entity-type filtering and joint learning, producing a model similar to Coarse-to-Fine semantic parsing for small-scale KBs.The reported comparison describes the model configuration, not an additional numerical result.Source: Effect of Multi-Task Learning (w/o… · p. 7, Table 3 · p. 7

4.4 Model Setting Analysis

• A pre-trained BERT-base encoder improves F1 by 1.3% over vanilla MaSP, while increasing beam size from 4 to 8 improves F1 by 2.3%.BERT is fine-tuned during training; the larger beam is used during inference.Source: §4.4 · p. 8, Table 5 · p. 8

4.5 Error Analysis

• Error analysis identifies same-type entity-name ambiguity, inaccurate logical-form components, and spurious BFS-derived training forms as sources of errors.The authors report sampling 100 examples; spurious forms can introduce training noise and negatively affect KB-QA performance.Source: §4.5 · p. 8

5 Related Work

• Traditional semantic parsing systems learn lexicon-based parsers and scoring models to construct logical forms from natural-language questions.Examples include CCG and shift-reduce parsers.Source: §5 · p. 8
• Neural semantic parsers have used syntax-aware logical-form methods and attention-based encoder-decoders to translate questions into tree-structured forms.The passage cites Rabinovich et al. (2017), Krishnamurthy et al. (2017), Cheng et al. (2017), Yin and Neubig (2017), Dong and Lapata (2016), and Alvarez-Melis and Jaakkola (2017).Source: §5 · p. 8
• Earlier large-KB approaches first identify entity candidates and then map questions to logical forms, or predict a logical-form sketch before generating the full form.The cited examples include Liang et al. (2016), Dong and Lapata (2016), Guo et al. (2018), and Dong and Lapata (2018).Source: §5 · pp. 8–9
• MaSP replaces logical-form entities with the starting positions of their question mentions, enabling conversational coreference handling and joint learning with entity detection.It also incorporates entity type prediction into entity-mention IOB labeling to filter linking candidates by type.Source: §5 · p. 9

6 Conclusion

• MaSP jointly learns type-aware entity detection and pointer-equipped logical-form generation, and experiments on a large-scale dataset verify the framework’s effectiveness.The authors identify testing on more datasets and handling spurious logical forms in weakly supervised KB-QA as future work.Source: §6 · p. 9

A.1 Word Embedding

• Input tokens, including an appended [CTX] token, are mapped to low-dimensional embeddings using a trainable word-embedding matrix.The embeddings may be randomly initialized or pretrained.Source: §A.1 · p. 11

A.2.1 Encoder of Seq2seq Model

• The encoder uses stacked two-layer multi-head attention with additive positional encoding to produce context-aware token representations.Each layer also includes a feed-forward network, residual connection, and layer normalization.Source: §A.2.1 · p. 11

A.2.2 Decoder of Seq2seq Model

• The decoder embeds gold logical-form sketch tokens and uses two Transformer-style layers with masked self-attention, encoder cross-attention, and feed-forward networks.Its hidden states form the decoder output sequence.Source: §A.2.2 · p. 11

A.3 Multi-task Learning

• Entity-detection IOB supervision supplies question entity spans and improves performance over separately learning the detection task.The authors prefer this joint setup to direct span prediction because direct prediction loses supervision for question entities absent from the gold logical form, requires a deeper network, and may aggregate pointer errors.Source: §A.3 · pp. 11–12

A.4 Inverted Index

• The inverted index maps entity-name substrings to full Wikidata entity texts using Levenshtein-distance scores, retaining the highest-scoring map when substrings collide.Substrings are considered when their length is no less than the full text length minus a threshold.Source: §A.4 · p. 12

B Supplemental Experiment Results

• Table 12 presents recall and precision results for the proposed model and baselines.The passage identifies the table’s scope but does not provide its values.Source: §B.1 · p. 12
• Table 13 presents the recall and precision ablation study.The passage identifies the table’s scope but does not provide its values.Source: §B.1 · p. 12
• Table 14 compares performance improvement after sophisticated strategies are applied.The passage identifies the table’s scope but does not provide its values.Source: §B.1 · p. 12

• 2× less average entity candidates are produced by the proposed model than by D2A.The passage says this provides the downstream subtask with more accurate entity-linking results.Source: §B.2 · p. 12, Table 6 · p. 12
• The proposed model obtains a lower ratio of empty logical forms than D2A.Source: §B.2 · p. 12, Table 7 · p. 12
• Entity accuracy in predicted logical forms improves with the proposed approach during entity detection and entity prediction.The comparison includes D2A, the standard approach, and a BERT-based model.Source: §B.2 · p. 12, Table 8 · p. 12

B.3 Multi-task Learning

• Evaluation covers logical-form generation and entity detection, and uses BFS on the test set to search for gold logical forms.The supplied tables identify sketch accuracy and IOB tagging/entity-type prediction as evaluation measures.Source: §B.3 · pp. 12–13, Table 9 · p. 13, Table 10 · p. 13
• Logical-form generation improves with multi-task learning, while entity detection falls by approximately 0.002, with only a very minor effect on KB-QA.Source: §B.3 · p. 13

B.4 BFS Success Ratio

• BFS search for gold logical forms can fail because of time and buffer limits, and its success ratio is listed by question type.Source: §B.4 · p. 13, Table 11 · p. 13

• Logical reasoning gains over D2A are relatively small; final-form entity accuracy is 8%, and the shallow network struggles with cases needing two entities.In the example, adding BERT produces the correct logical form where the standard approach and MaSP each err in an operator or entity.Source: C Supplemental Analysis · p. 13
• Table 12 presents KB-QA comparisons against baselines.Source: Table 12 · p. 14
• Table 13 ablates entity-type prediction and multi-task learning, separately and together.Source: Table 13 · p. 14
• Table 14 compares vanilla settings, BERT incorporation, and increasing beam size from 4 to 8.Source: Table 14 · p. 14