Source-linked AI summary
Sentence Similarity Learning by Lexical Decomposition and Composition
Zhiguo Wang, Haitao Mi, Abraham Ittycheriah
TL;DR
Existing sentence similarity methods often focus on similar parts while overlooking dissimilar information, despite challenges involving lexical gaps and multiple semantic granularities. The paper decomposes word semantics into similar and dissimilar components, composes them with a two-channel CNN, and predicts similarity. It reports state-of-the-art performance on answer sentence selection and comparable performance on paraphrase identification.
Problem
Conventional methods overlook dissimilar sentence parts, while sentence similarity also requires handling lexical gaps and word-, phrase-, and syntax-level matching.
Method
The model computes semantic matching vectors across sentence pairs, decomposes word vectors into similar and dissimilar components, and composes them with a two-channel CNN.
Results
The model achieves state-of-the-art performance on answer sentence selection and a comparable result on paraphrase identification.
Takeaways & Limitations
Decomposing and composing lexical semantics provides a model that addresses similarity and dissimilarity across the evaluated sentence-similarity tasks.
Abstract
from arXiv · showhide
Most conventional sentence similarity methods only focus on similar parts of two input sentences, and simply ignore the dissimilar parts, which usually give us some clues and semantic meanings about the sentences. In this work, we propose a model to take into account both the similarities and dissimilarities by decomposing and composing lexical semantics over sentences. The model represents each word as a vector, and calculates a semantic matching vector for each word based on all words in the other sentence. Then, each word vector is decomposed into a similar component and a dissimilar component based on the semantic matching vector. After this, a two-channel CNN model is employed to capture features by composing the similar and dissimilar components. Finally, a similarity score is estimated over the composed feature vectors. Experimental results show that our model gets the state-of-the-art performance on the answer sentence selection task, and achieves a comparable result on the paraphrase identification task.
1 Introduction
Sentence similarity learning must address lexical gaps, multiple semantic granularities, and informative dissimilarities between sentences. The proposed model jointly handles these challenges and reports state-of-the-art answer sentence selection performance alongside comparable paraphrase-identification results.
- Sentence similarity supports paraphrase identification, question answering, and information retrieval by measuring relationships between sentence pairs.
- Lexical gaps can make semantically equivalent sentences appear different because they use different words.E1 and E2 have similar meanings but different lexicons.
- Similarity must be measured across word, phrase, and syntax levels, including cases where a phrase matches a single word.The phrase “not related” is treated as an indivisible unit when matching “irrelevant.”
- Dissimilar parts also provide clues: contrasting salmon-related terms distinguishes shared meaning from more specific or unrelated meanings.Sockeye and coho share the salmon family, whereas flounder does not; sockeye additionally means red salmon.
- The proposed model decomposes and composes lexical semantics to address all three challenges jointly.It represents words as vectors, computes cross-sentence matching vectors, separates similar and dissimilar components, and predicts similarity using composed features.
- The model achieves state-of-the-art performance on answer sentence selection and a comparable result on paraphrase identification.
2 Model Overview
The model represents sentence words as vectors, matches each word against the other sentence, and decomposes each word into similar and dissimilar components. A two-channel CNN composes these components into features used to predict sentence similarity.
- Word Representation: The model represents each sentence as a matrix of d-dimensional word vectors derived from pretrained word embeddings.Sentence lengths are m and n for sentences S and T.
- Semantic Matching: Semantic matching computes a vector for each word by composing part or all of the word vectors in the other sentence.Matching is performed in both directions, producing ˆs_i for words in S and ˆt_j for words in T.
- Decomposition: The matching vector represents semantic coverage, allowing each word to be decomposed into similar and dissimilar components.For example, salmon partially matches sockeye, while red remains a dissimilar component of red salmon.
- Composition: The model composes similar and dissimilar component matrices into feature vectors using a composition function.It combines both types of information because considering either alone can make sentence comparisons difficult.
- Similarity Assessing: The final stage concatenates the two sentence feature vectors and predicts the similarity score.
3 An End-to-End Implementation
The model matches words across sentences, decomposes each word into similar and dissimilar components, composes both through a two-channel CNN, and predicts similarity.
- 3.1 Semantic Matching Functions: Semantic matching generates each word’s matching vector by composing vectors from the other sentence using global, max, or local-w functions.The global function uses all words, max selects the most similar word, and local-w uses a window around that word.
- 3.2 Decomposition Functions: The model decomposes each word vector into similar and dissimilar components based on its semantic matching vector.Rigid, linear, and orthogonal decomposition operations provide alternative ways to separate covered from uncovered semantics.
- 3.2 Decomposition Functions: Orthogonal decomposition treats the parallel component as similar and the perpendicular component as dissimilar in the geometric space.This decomposition uses the semantic matching vector as the reference direction.
- 3.3 Composition: A two-channel CNN extracts multi-granularity features from the similar and dissimilar component matrices using filters for different n-gram orders.Convolution is followed by max-pooling to produce fixed-size feature values from variable-length feature sequences.
- 3.4 Similarity Assessment and Training: A linear layer sums the composed features and a sigmoid function produces a similarity score in the range [0, 1].Training maximizes likelihood over sentence pairs labeled as similar or dissimilar for the two evaluated tasks.
4 Experiment
Experiments evaluate configuration choices and compare the model on answer sentence selection and paraphrase identification, with stronger performance reported on WikiQA than prior models.
- 4 Experiment: The model is evaluated on answer sentence selection and paraphrase identification using QASent, WikiQA, and MSRP datasets.Answer sentence selection is measured with mean average precision (MAP) and mean reciprocal rank (MRR).
- 4.2 Configuration Analysis: Max semantic matching outperformed global on both MAP and MRR, while local-l improved progressively as the window grew below 4.These configuration experiments used QASent development-set results with other options fixed.
- 4.2 Configuration Analysis: Rigid decomposition performed worst; orthogonal achieved similar MAP to linear but better MRR and was selected for subsequent experiments.The reported explanation is that rigid decomposition relies on exact word matching.
- 4.2 Configuration Analysis: Adding higher-order n-gram filters initially improved performance, with performance reaching a peak as filter windows expanded.The experiment compared five filter groups ranging from unigram-only to filters including 5-grams.
- 4.3 Comparing with State-of-the-art Models: Earlier QASent comparisons indicate that word overlap, word vectors, lower-level granularity, and attention each contributed useful performance factors.The authors describe their model as combining these factors with sentence-pair dissimilarities.
- 4.3 Comparing with State-of-the-art Models: On WikiQA, the model was reported as more effective than the other compared models while using both similarity and dissimilarity.The comparison included CNN, LSTM, attention-based, and word-overlap approaches.
5 Related Work
The model extends attention-based CNN sentence matching by decomposing the attention-derived similarity information into separate similar and dissimilar channels.
- 5 Related Work: Unlike prior attention-based CNN work that directly feeds an attention matrix as a channel, this model decomposes it into similar and dissimilar component matrices.The two matrices are then supplied to a two-channel CNN.
- 5 Related Work: The approach introduces an attention mechanism into CNN modeling and focuses on interactions between similar and dissimilar sentence parts.The semantic matching functions are inspired by attention-based neural machine translation.
6 Conclusion
The proposed model assesses sentence similarity by decomposing and composing lexical semantics, extracting features from both similar and dissimilar components across multiple granularities. It is effective on answer sentence selection and paraphrase identification.
- The model represents words with context vectors and decomposes each word vector into similar and dissimilar components.
- A two-channel CNN with multiple ngram filter types extracts features at multiple levels of granularity.
- The model is effective on both the answer sentence selection task and the paraphrase identification task.