Source-linked AI summary
Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbots
Jia-Chen Gu, Zhen-Hua Ling, Xiaodan Zhu, Quan Liu
TL;DR
Personalized response selection must account for persona information as well as conversational context, while prior fusion approaches do not model persona-response interactions. DIM performs parallel matching between responses and contexts and between responses and personas. On PERSONA-CHAT, it reports a 14.5% hits@1 improvement over its IMN persona-fusion baseline and a 27.7% margin over the current state-of-the-art model.
Problem
Prior persona-fusion methods enhance a context representation using persona similarities but ignore utterance-level distinctions and persona-response interactions.
Source: §1 · p. 1, §2.2 · p. 3Method
DIM formulates personalized response selection as dual matching, interacting in parallel between each response candidate and the context and between the candidate and persona.
Source: §1 · pp. 1–2Results
On PERSONA-CHAT, DIM exceeds the current state-of-the-art model by 27.7% in top-1 accuracy hits@1 and its IMN context-level persona-fusion baseline by 14.5% in hits@1.
Source: §1 · p. 2Takeaways & Limitations
The paper reports that DIM achieves new state-of-the-art performance on PERSONA-CHAT response selection.
Source: §8 · p. 9
Abstract
from arXiv · showhide
This paper proposes a dually interactive matching network (DIM) for presenting the personalities of dialogue agents in retrieval-based chatbots. This model develops from the interactive matching network (IMN) which models the matching degree between a context composed of multiple utterances and a response candidate. Compared with previous persona fusion approaches which enhance the representation of a context by calculating its similarity with a given persona, the DIM model adopts a dual matching architecture, which performs interactive matching between responses and contexts and between responses and personas respectively for ranking response candidates. Experimental results on PERSONA-CHAT dataset show that the DIM model outperforms its baseline model, i.e., IMN with persona fusion, by a margin of 14.5% and outperforms the current state-of-the-art model by a margin of 27.7% in terms of top-1 accuracy hits@1.
1 Introduction
Personalized response selection extends context-based retrieval by considering dialogue-agent personas. DIM matches each response candidate with both the conversation context and persona, and achieves higher hits@1 on PERSONA-CHAT than the reported baseline and state-of-the-art model.
1 Introduction
- The earlier persona-fusion approach enhances a context representation using persona similarities, but ignores utterance-specific roles and persona-response interactions.It attends from the whole context to profile sentences before combining the persona and context representations.Source: §1 · p. 1
- DIM treats personalized response selection as finding a response that matches the context and persona simultaneously, with parallel context-response and persona-response interactions.The matching interactions produce the final matching feature for response selection.Source: §1 · pp. 1–2
- 27.7% higher top-1 accuracy hits@1 than the current state-of-the-art model; DIM also improves hits@1 by 14.5% over the IMN context-level persona-fusion baseline.Both comparisons are reported on PERSONA-CHAT.Source: §1 · p. 2
- The proposed IMN-based utterance-level persona-fusion model improves hits@1 by 2.4% over the context-level fusion baseline on PERSONA-CHAT.The paper also introduces this fine-grained model alongside DIM.Source: §1 · p. 2
2 Related Work
Prior response-selection work matches candidates to dialogue context, while persona-based systems add profile information to personalize responses. The paper identifies coarse context-level fusion and ignored persona-response interactions as limitations of those systems.
2.1 Response Selection
- Multi-turn response-selection models match responses with context at different granularities and use bidirectional or global interactions to derive matching features.Examples include the sequential matching network, deep attention matching network, and interactive matching network.Source: §2.1 · pp. 2–3
2.2 Persona for Chatbots
- Persona-chatbot methods commonly fuse persona information at the whole-context level, leaving utterance-level representations and persona-response interactions unused.Prior systems obtain context embeddings, compare the whole context with profile sentences, then combine persona and context representations.Source: §2.2 · p. 3
3 Task Definition
The task represents each example using a dialogue context, a persona, a response candidate, and a binary label. It learns to rank response candidates by their match to the context-persona pair.
3 Task Definition
- Each dialogue example is represented as (c, p, r, y), with context utterances, persona profile sentences, a response candidate, and a label.The label is 1 when the response is proper for the context-persona pair and 0 otherwise.Source: §3 · p. 3
- The model learns g(c, p, r) to measure how well a response matches a context-persona pair, using a softmax over candidates and multi-class cross-entropy training.The supplied text identifies a context as a set of utterances and a persona as a set of profile sentences.Source: §3 · p. 3
4 IMN-Based Persona Fusion
The baseline adds persona information to an IMN context representation using context-level fusion. The improved IMN model instead matches each context utterance with profile sentences before aggregating enhanced utterances into a context representation.
4 IMN-Based Persona Fusion
- The IMNctx baseline applies context-level persona fusion to IMN embeddings and estimates all model parameters end-to-end.The persona representation is combined with the context representation through concatenation or addition.Source: §4 · pp. 3–4
- IMNutr computes similarities between each context utterance and each profile sentence to form enhanced utterance representations.This fine-grained fusion is proposed because utterances may play different roles when matching profile sentences.Source: §4 · p. 4
- Enhanced utterance representations are aggregated into an enhanced context representation, using either an RNN or attention-based aggregation.The aggregation follows the utterance-level persona fusion step.Source: §4 · p. 4
5 Dually Interactive Matching Network
DIM treats context and persona as equally important counterparts to a response, matching each with the response in parallel to rank candidates. Its five-layer architecture encodes the inputs, builds and aggregates matching features, then predicts whether each context–persona–response triple matches.
5.1 Model Overview
- DIM formulates personalized response selection as dual matching: the chosen response should match both the context and persona, which interact with it in parallel.The model treats personas as equally important counterparts to contexts when ranking response candidates.Source: §5.1 · p. 4
5.2 Word Representation Layer
- Context utterances, persona profile sentences, and response candidates are represented as sequences of d-dimensional word embeddings.The model combines general pre-trained, task-specific, and character-level embeddings to address out-of-vocabulary words.Source: §5.2 · p. 4
5.3 Sentence Encoding Layer
- BiLSTMs encode the context utterances, profile sentences, and response candidate.The three BiLSTMs share parameters in the implementation.Source: §5.3 · p. 4
5.4 Matching Layer
- The matching layer applies global, bidirectional interactions and soft alignment to context–response and persona–response sequences, forming relevant representations and enhanced features.The persona–response matching follows the same strategy as context–response matching; differences and element-wise products are combined with the original vectors.Source: §5.4 · pp. 4–5
5.5 Aggregation Layer
- The aggregation layer turns context-utterance, profile-sentence, and response matching matrices into a final matching feature vector.It uses BiLSTMs and max and last-hidden-state pooling; context utterances are aggregated chronologically, while independent profile sentences use attention-based aggregation.Source: §5.5 · pp. 5–6
5.6 Prediction Layer
- A softmax MLP predicts whether each context–persona–response triple matches and returns its matching probability.Source: §5.6 · p. 6
6 Experiments
The experiments evaluate personalized response selection on PERSONA-CHAT, comparing DIM and persona-fusion baselines across persona configurations. DIM outperforms its IMN baseline and previous models, with results varying by persona type.
6.1 Dataset
- PERSONA-CHAT contains 8,939 training, 1,000 validation, and 968 test dialogues; turn-level selection expands these to 65,719, 7,801, and 7,512 examples.The task has a 1:19 positive-to-negative response ratio, and each persona has 3 to 5 profile sentences.Source: §6.1 · p. 6
6.2 Evaluation Metrics
- Evaluation uses hits@1 for retrieving the true response and MRR to account for its rank among candidates.Source: §6.2 · p. 6
6.3 Training Details
- Models use Adam with batch size 16, dropout 0.2, and fixed word embeddings combining GloVe, Word2Vec, and character-level representations.The development set selects the best model for testing.Source: §6.3 · pp. 6–7
6.4 Experimental Results
- On original self personas, context-level persona fusion significantly improves previous models, but DIM’s gain is larger because it uses dual matching.The authors suggest context representations that already incorporate response information may be inappropriate for context-level persona fusion.Source: §6.4 · p. 7
- With revised self personas, DIM improves hits@1 by 6.9% and MRR by 5.4%, and outperforms pretrained FT-PC by 10.0% hits@1.The paper attributes DIM’s gains with revised personas to its direct, interactive persona-response matching.Source: §6.4 · pp. 7–8
- DIM exceeds IMNctx by 14.5% hits@1 and 10.5% MRR, and exceeds previous models by more than 27.7% hits@1 on original self personas.The comparison with IMNctx is statistically significant at p-value < 0.01.Source: §6.4 · p. 7, Table 3 · p. 8
- Partner-persona conditioning yields no significant gains, while proposed methods avoid the 2.4%–7.3% hits@1 losses seen in previous models.Source: §6.4 · p. 8
7 Analysis
The analysis tests DIM’s matching components, visualizes its two attention pathways, and evaluates transfer between original and revised personas. Both matching components contribute, and transfer works better when training and test persona types match.
7.1 Ablations
- Ablations show that both context-response and persona-response matching contribute to DIM’s performance.The single persona-response matching variant reaches hits@1 of 48.8% and MRR of 60.9%.Source: Table 4 · p. 8, §7.1 · p. 8
7.3 Transfer Test
- DIM performs better when tested on the same persona type used for training; revised-to-original transfer loses less than original-to-revised transfer.The authors interpret this asymmetry as evidence that revised personas provide better generalization ability than original ones.Source: §7.3 · pp. 8–9, Table 5 · p. 8
7.2 Interactive Matching in DIM
- For one test sample, Figure 3 visualizes response-to-context and response-to-persona attention weights, with darker units indicating larger values.Source: Figure 3 · p. 9
8 Conclusions
The paper formulates personalized response selection as matching a response to both the conversation context and persona. DIM performs these interactions in parallel and achieves state-of-the-art performance on PERSONA-CHAT.
8 Conclusions
- DIM improves over IMN models with context-level or utterance-level persona fusion and achieves state-of-the-art performance on PERSONA-CHAT.Source: §8 · p. 9
- DIM formulates personalized response selection as matching a response to the context and persona simultaneously, deriving final matching features from parallel interactions.Source: §8 · p. 9
- Future work will explore models that make better use of dialogue partners’ personas for response selection.Source: §8 · p. 9