Source-linked AI summary
A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling
Yu Wang, Yilin Shen, Hongxia Jin
TL;DR
Intent detection and slot filling are often modeled separately or with a single joint network that may not fully exploit their cross-impact. The paper introduces correlated Bi-model RNN structures with two BLSTM-based task-networks, with or without a decoder, and reports state-of-the-art performance on ATIS while surpassing a previous best model on multi-domain data.
Problem
Prior approaches either treat intent detection and slot filling as separate tasks or use one joint neural model, potentially limiting their use of cross-impact.
Method
The paper jointly models the tasks with two correlated BLSTM-based task-networks, using an optional LSTM decoder and asynchronous training based on separate cost functions.
Results
The Bi-model structures achieve state-of-the-art results on ATIS for both tasks and surpass the previous best SLU model on multi-domain data; the decoder variant performs best.
Takeaways & Limitations
Modeling intent detection and slot filling through separate but correlated task-networks supports strong joint semantic frame parsing across benchmark and multi-domain evaluations.
Abstract
from arXiv · showhide
Intent detection and slot filling are two main tasks for building a spoken language understanding(SLU) system. Multiple deep learning based models have demonstrated good results on these tasks . The most effective algorithms are based on the structures of sequence to sequence models (or "encoder-decoder" models), and generate the intents and semantic tags either using separate models or a joint model. Most of the previous studies, however, either treat the intent detection and slot filling as two separate parallel tasks, or use a sequence to sequence model to generate both semantic tags and intent. Most of these approaches use one (joint) NN based model (including encoder-decoder structure) to model two tasks, hence may not fully take advantage of the cross-impact between them. In this paper, new Bi-model based RNN semantic frame parsing network structures are designed to perform the intent detection and slot filling tasks jointly, by considering their cross-impact to each other using two correlated bidirectional LSTMs (BLSTM). Our Bi-model structure with a decoder achieves state-of-the-art result on the benchmark ATIS data, with about 0.5$\%$ intent accuracy improvement and 0.9 $\%$ slot filling improvement.
1 Introduction
SLU systems center on intent detection and slot filling, tasks often treated separately despite possible cross-impact. The paper proposes correlated Bi-model RNN structures to model them jointly and evaluates them on ATIS and multi-domain data.
- Intent detection is framed as utterance classification, while slot filling is formulated as sequence labeling.Conventional classifiers, CRFs, and RNNs are established approaches for these tasks.
- Prior work includes joint RNN or encoder-decoder models that generate intent and semantic tags together.These models aim to exploit shared modeling of both tasks.
- The proposed Bi-model structures use two task-networks, each containing a BLSTM with or without an LSTM decoder.The structures generate intent and semantic tags concurrently while accounting for cross-impact between the tasks.
- The paper evaluates the approach on the public ATIS benchmark and a self-collected dataset spanning Food, Home, and Movie domains.The ATIS experiments target state-of-the-art performance for both semantic parsing tasks, while the multi-domain experiment compares against a prior joint RNN model.
2 Background
Background work treats intent detection as domain-specific classification and slot filling as sequence labeling with multiple outputs. Joint architectures combine these tasks through multiple decoders or attention-based intent generation.
- Deep neural network for intent detection: Intent detection is a domain-specific classification problem that can use feed-forward models or RNNs.In the ATIS example, the flight-reservation domain has 18 intent labels.
- Recurrent Neural network for slot filling: Slot filling produces multiple outputs and is commonly modeled with RNNs that generate semantic tags sequentially.A single RNN requires the number of generated slot tags to match the number of input words.
- Recurrent Neural network for slot filling: Encoder-decoder slot-filling models allow input utterances and output slot tags to have different lengths without alignment.These models use separate RNN encoder and decoder components.
- Joint models: Joint models include one encoder with two decoders or intent generation from slot-filling hidden states through attention.Both approaches are reported to achieve very good results on the ATIS dataset.
3 Bi-model RNN structures for joint semantic frame parsing
The Bi-model approach jointly models intent detection and slot filling with two interconnected BLSTM-based task networks, optionally paired with decoders. Separate task costs and shared hidden states support asynchronous learning of cross-task information.
- Motivation: The proposed Bi-model structures replace predominantly single-model approaches with two task networks that model intent detection and slot filling jointly.The structures are designed to account for cross-impact between the tasks.
- Shared architecture: Two interconnected BLSTMs process the utterance for intent detection and slot filling, producing task-specific hidden-state sequences.Each BLSTM reads the input sequence in both directions and concatenates forward and backward states.
- Bi-model structure with a decoder: In the decoder-based structure, each task network includes a BLSTM and an LSTM decoder, with intent and semantic-tag outputs generated concurrently.The slot-filling network produces semantic tags at each time step, while the intent network uses the other task's hidden state.
- Bi-model structure with a decoder: The intent network combines its hidden state with the slot-filling network's hidden state to generate intent states and final intent-label probabilities.The intent output is associated with the final time step in the decoder-based formulation.
- Bi-model structure without a decoder: The decoder-free structure generates one intent label at the utterance's final time step and one slot tag at each time step.This model omits the LSTM decoder used in the alternative structure.
- Asynchronous training: Asynchronous training optimizes separate cross-entropy costs L1 and L2 while exchanging hidden states between the two task networks.The authors state that separate costs filter negative task interference and allow cross-impact to be learned through shared hidden states.
4 Experiments
Experiments evaluate the Bi-model structures on ATIS and a three-domain internal dataset, using intent accuracy and slot-filling F1. The decoder variant achieves the strongest reported performance across both evaluations.
- Datasets: The experiments use the public ATIS dataset and an internal dataset spanning food, home, and movie domains.ATIS contains 4,978 training utterances, 893 test utterances, 18 intent classes, and 127 slot labels.
- Evaluation: Performance is evaluated with classification accuracy for intent detection and F1-score for slot filling.
- ATIS results: 0.9% improvement in F1 score and 0.5% improvement in intent accuracy make the Bi-model with a decoder state of the art on ATIS.
- ATIS results: The Bi-model with a decoder outperforms the version without a decoder on the ATIS dataset.
- Model comparison: On ATIS, the joint model performs better on intent detection but shows slight degradation on slot filling, whereas the Bi-model generates the two results separately.
- Multi-domain results: The Bi-model with a decoder gives the best performance in all internal-dataset cases, with at least 0.5% intent-accuracy improvement and approximately 1% to 3% F1-score improvement.
5 Conclusion
The paper proposes and evaluates Bi-model RNN semantic frame parsing structures for joint intent detection and slot filling. These structures achieve state-of-the-art performance on ATIS and surpass the previous best model on multi-domain data.
- The proposed Bi-model RNN structures jointly address intent detection and slot filling, with variants using a decoder or not.
- The Bi-model structures achieve state-of-the-art performance on both tasks on ATIS benchmark data.
- The Bi-model structures surpass the previous best spoken-language-understanding model on multi-domain data.
- The decoder-based Bi-model outperforms the Bi-model without a decoder on both ATIS and multi-domain data.