Source-linked AI summary
Evolution of Semantic Similarity -- A Survey
Dhivya Chandrasekaran, Vijay Mago
TL;DR
Semantic similarity remains difficult because natural language complicates rule-based measurement and existing surveys often focus on individual techniques or applications. This survey synthesizes knowledge-based, corpus-based, deep neural network-based, and hybrid methods, finding promising recent hybrid and transformer approaches while no single model is best.
Problem
Semantic similarity is difficult to measure because natural language is versatile, while prior surveys largely cover individual techniques or applications rather than the field comprehensively.
Method
The survey traces semantic similarity methods across knowledge-based, corpus-based, deep neural network-based, and hybrid approaches, discussing their strengths and weaknesses.
Results
The survey finds that recent hybrid and transformer methods show promising results, although choosing one best model remains difficult.
Takeaways & Limitations
Semantic similarity research still requires balancing computational efficiency and performance while addressing domain-specific embeddings and the lack of an ideal corpus.
Takeaways & Limitations
Deep neural network-based methods require substantial computational resources and are often difficult to interpret as black-box models.
Abstract
from arXiv · showhide
Estimating the semantic similarity between text data is one of the challenging and open research problems in the field of Natural Language Processing (NLP). The versatility of natural language makes it difficult to define rule-based methods for determining semantic similarity measures. In order to address this issue, various semantic similarity methods have been proposed over the years. This survey article traces the evolution of such methods, categorizing them based on their underlying principles as knowledge-based, corpus-based, deep neural network-based methods, and hybrid methods. Discussing the strengths and weaknesses of each method, this survey provides a comprehensive view of existing systems in place, for new researchers to experiment and develop innovative ideas to address the issue of semantic similarity.
1 INTRODUCTION
The introduction motivates semantic similarity as a central NLP problem across diverse tasks and presents the survey as an evolution-focused classification of existing techniques. It also defines Semantic Textual Similarity as measuring semantic equivalence between two text blocks rather than making a binary similarity decision.
- Motivation: Semantic similarity between words, sentences, and documents supports NLP tasks including information retrieval, summarization, classification, essay evaluation, translation, and question answering.The motivation is linked to the exponential growth of generated text data.
- Background: Semantic Textual Similarity measures semantic equivalence between two text blocks and typically produces a similarity ranking or percentage instead of a binary decision.These techniques were proposed to address limitations of lexical measures that ignored semantic and syntactic properties.
- Survey Scope: Recent surveys commonly examine one semantic similarity technique or application, motivating a broader synthesis of the field.The introduction contrasts this narrower survey coverage with the present survey’s wider scope.
- Survey Scope: This survey traces semantic similarity techniques over past decades and distinguishes them according to their underlying methods.Its structure includes a dataset review followed by sections describing broad method categories.
2 DATASETS
Semantic similarity algorithms are evaluated using datasets of word or sentence pairs with standard similarity values, typically by correlating algorithm outputs with those standards. The section reviews widely used datasets chronologically, including foundational human-rated word-pair benchmarks.
- Evaluation datasets contain word or sentence pairs with associated standard similarity values, and algorithm performance is measured by correlation with those standards.These datasets provide reference measures for assessing semantic similarity algorithms.
- Rubenstein and Goodenough is described as the first and most widely used semantic similarity dataset.It was created from ratings by 51 native English-speaking undergraduate students on 65 noun pairs using a 0-to-4 similarity scale.
- Miller and Charles repeated the Rubenstein and Goodenough experiment with 30 word pairs, rated by 38 human subjects on a 0-to-4 scale.The subset came from the original 65 word pairs.
- WS353 is listed among the widely used semantic similarity datasets and contains 353 word pairs.
3 KNOWLEDGE-BASED SEMANTIC-SIMILARITY METHODS
Knowledge-based methods estimate semantic similarity from structured knowledge sources that represent concepts and their semantic relations. They include edge-counting, feature-based, and information-content approaches, offering efficient ambiguity handling but depending heavily on available and maintained knowledge bases.
- Foundations: Knowledge-based methods derive similarity from ontologies, lexical databases, thesauri, or dictionaries that structurally connect concepts through semantic relations.Using the terms’ actual meanings provides an ambiguity-free semantic measure.
- Knowledge Sources: WordNet is a widely used lexical database containing more than 100,000 English concepts, represented as graph nodes linked by semantic relations.Its structure is primarily based on synonyms and synsets representing different word meanings.
- Method Categories: Knowledge-based methods comprise edge-counting, feature-based, and information-content-based approaches.Edge-counting uses taxonomic path distance, feature-based methods compare properties such as glosses, and information-content methods use concept specificity derived from context.
- Information Content: Corpus-based information-content wp a t h measures achieved greater correlation in most gold-standard datasets than intrinsic information-content variants.The method applies across knowledge graphs such as WordNet, DBPedia, and YAGO, while k must be tuned for each graph and domain.
- Strengths and Limitations: Knowledge-based methods are computationally simple, handle synonyms, idioms, and phrases efficiently, and can extend to sentence similarity through aggregation rules.Their limitations include dependence on frequently updated knowledge sources, maintenance costs, and weaker resource availability across languages and domains.
4 CORPUS-BASED SEMANTIC-SIMILARITY METHODS
Corpus-based methods estimate semantic similarity from distributional information in large corpora, commonly representing words as vectors and comparing them with cosine similarity. They include diverse approaches such as word embeddings, co-occurrence and alignment models, while offering language and domain independence but requiring substantial corpora and processing resources.
- Distributional foundations: Corpus-based methods apply the distributional hypothesis, using information from large corpora to estimate similarity without directly considering word meaning.The approach assumes that similar words occur together frequently and uses vector representations with semantic distance measures.
- Distributional foundations: Cosine similarity became the most widely used measure among distributional semantic methods.Distributional techniques construct text vectors and compare them using semantic distance measures.
- Word embeddings: Word embeddings represent words as vectors that retain linguistic relationships, using neural networks, word co-occurrence matrices, or contextual representations.They can also measure cross-language similarity by mapping one language’s embeddings into another vector space using translation pairs.
- Corpus-based approaches: LSA, HAL, ESA, dependency-based models, kernel methods, and word-alignment models construct corpus-based representations through distinct statistical, contextual, syntactic, or structural mechanisms.LSA uses co-occurrence matrices and SVD; HAL uses distance-weighted window associations; ESA uses Wikipedia concepts; alignment models compare aligned context words.
- Kernel-based approaches: Tree-kernel-based methods combine parsed subject, verb, and object components with knowledge-base similarity and averaging techniques to estimate sentence similarity.A tested algorithm using related kernel formulations performed better on 12 out of 19 selected STS datasets.
- Strengths and limitations: Corpus-based systems are language and domain independent and can be adapted across languages, but their performance depends on corpus quality and processing large corpora is time-consuming and resource-dependent.Web crawling can facilitate construction of large corpora, yet building an efficient corpus remains paramount.
5 DEEP NEURAL NETWORK-BASED METHODS
Deep neural network-based methods use architectures including CNNs, LSTMs, Bi-LSTMs, Recursive Tree LSTMs, and transformers to improve semantic similarity estimation. They generally outperform traditional methods, but require substantial computational resources and may trade performance for efficiency.
- Deep neural network-based methods: Deep neural network-based semantic similarity methods commonly use CNN, LSTM, Bi-LSTM, and Recursive Tree LSTM architectures.These models are built around convolution and pooling operations.
- Deep neural network-based methods: One CNN model combines GloVe embeddings, handcrafted lexical and POS features, 300 filters, and max pooling to construct sentence vectors.Inputs are normalized to 30 words through removal or padding before processing.
- Deep neural network-based methods: An LSTM-based model computes cosine, Euclidean, and Manhattan distances between hidden states and uses these distances as interaction weights instead of attention weights.Its network alternates spatial convolution and max-pooling layers before fully connected and LogSoftmax layers.
- Deep neural network-based methods: Transformer-based models achieved state-of-the-art semantic similarity results, with performance increasing as training data, model size, and computational power increase.The described T5 variants range from 60 million to 11 billion parameters, and replicating the best model requires five GPUs among other resources.
- Deep neural network-based methods: Deep neural network methods outperform most traditional approaches, but reducing computational requirements can lower performance, as illustrated by TinyBERT.The section identifies large computational resource requirements as a central implementation limitation.
6 HYBRID METHODS
Hybrid methods combine knowledge-based structure with corpus-based versatility, often using semantic resources to build sense-aware representations. The survey highlights NASARI, MSSA, Sensembed, and ensemble systems as examples of this direction.
- 6 HYBRID METHODS: NASARI uses BabelNet and linked Wikipedia pages to construct concept-specific corpora and vector representations.Its NASARIembedded extension maps lexical vectors into word-embedding space and measures similarity with cosine similarity.
- 6 HYBRID METHODS: MSSA, MSSA-NR, and MSSA-D form word-sense embeddings by associating corpus words with WordNet synsets during word-sense disambiguation.The passage introduces three proposed methodologies for constructing word-sense embeddings.
- 6 HYBRID METHODS: Hybrid methods combine the structural efficiency of knowledge-based approaches with the versatility of corpus-based methods.They aim to exploit the advantages of both method families while addressing their respective limitations.
- 6 HYBRID METHODS: Sensembed combines BabelNet sense annotation with word2vec to produce different vectors for different word senses.This approach incorporates the actual meanings of words into their vector representations.
- 6 HYBRID METHODS: The first 5 places in SemEval 2017 semantic similarity tasks were awarded to ensemble models, indicating a shift toward hybrid systems.The survey presents this outcome as evidence of growing research interest in hybrid models.
7 ANALYSIS OF SURVEY
The survey selected research articles through a keyword-based Google Scholar search refined by publication quality and impact parameters. Analysis of 118 abstracts used preprocessing and word-cloud generation to show that the selected literature broadly focuses on semantic similarity.
- Article Selection: Articles were retrieved from Google Scholar using keywords spanning semantic similarity, embeddings, knowledge-based methods, corpus-based methods, deep neural networks, LSTMs, text processing, and datasets.The search results were fine-tuned using journal ranking, Google Scholar Index, citation count, and publication year.
- Abstract Analysis: The survey analyzed 118 article abstracts to construct a dataset for word-cloud generation.A simple Python program was used to generate the word cloud from the abstracts.
- Abstract Analysis: Abstracts were lowercased, stripped of punctuation, and cleared of common English stop words before word-cloud construction.Stop words were removed using the nltk library, and the word cloud was built with the wordcloud Python library.
- Findings: The word cloud indicated that, despite varied search keywords, the selected articles generally focused on semantic similarity.Word size in the cloud represents frequency of use.
8 CONCLUSION
The survey reviews the evolution, advantages, and disadvantages of semantic-similarity methods. It concludes that no single best model is evident, while hybrid methods show promising results over independent models.
- Knowledge-based methods consider the actual meaning of text but are not adaptable across different domains and languages.
- Hybrid methods combine different approaches to exploit their benefits and compensate for their shortcomings.
- Recent research focuses on more semantically aware word embeddings, while transformer models have shown promise.
Appendix A SEMANTIC DISTANCE MEASURES AND THEIR FORMULAE
Appendix A presents a table of semantic measures and their formulae, adapted from Mohammad and Hurst [81]. The table continues from a previous page.
- Semantic distance measures and their formulae: Table 4 is marked as continuing from the previous page.This continuation label accompanies the semantic-measures table.
- Semantic distance measures and their formulae: Appendix A compiles semantic measures together with their formulae in a table adapted from Mohammad and Hurst [81].The supplied passage identifies this as Table 4.
Appendix B TABLE OF REFERENCES · Title Year Authors Venue SJR Quartile
Appendix B lists reference metadata by title, year, authors, venue, SJR, and quartile, including entries from 2009 and 2015.
- Title Year Authors Venue SJR Quartile: 2009 entry lists Agirre, Eneko and Alfonseca, Enrique and Hall, Keith and Kravalova, Jana and Pasca, Marius and Soroa, Aitor.
- Title Year Authors Venue SJR Quartile: The 2009 reference appeared in Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
- Title Year Authors Venue SJR Quartile: 2015 entry lists Agirre, Eneko and Banea, Carmen and Cardie, Claire and Cer, Daniel and Diab, Mona and Gonzalez-Agirre, Aitor and Guo, Weiwei and Lopez-Gazpio, Inigo and Maritxalar, Montse and Mihalcea, Rada and others.
Table 5 continued from previous page Citation … Title Year Authors Venue SJR Quartile
The section continues a bibliographic table of cited works, recording publication years, authors, venues, SJR quartiles, and associated citation or ranking values. The entries span works published from 1996 to 2019 across artificial intelligence, information processing, expert systems, behavior research, and IEEE venues.
- Title Year Authors Venue SJR Quartile: 2016 Camacho-Collados, Pilehvar, and Navigli are listed with Artificial Intelligence, Q1, 135, and 117.
- Title Year Authors Venue SJR Quartile: 2017 Jiang, Bai, Zhang, and Hu are listed with Information Processing & Management.
- Title Year Authors Venue SJR Quartile: 2015 Jiang, Zhang, Tang, and Nie are included as a cited work.
- Title Year Authors Venue SJR Quartile: 2019 Lopez-Gazpio, Maritxalar, Lapata, and Agirre are listed with Expert Systems with Applications, Q1, 162, and 2.
- Title Year Authors Venue SJR Quartile: 1996 Lund and Burgess are listed in Behavior research methods with Q1, 114, and 1869.
- Title Year Authors Venue SJR Quartile: 2019 Pawar and Mago are listed with IEEE Access, Q1, 56, and 11.
- Title Year Authors Venue SJR Quartile: 2007 Pedersen, Pakhomov, Patwardhan, and Chute are included as a cited work.
Table 5 continued from previous page
This section continues Table 5, which lists references used in the survey’s analysis.
- Table 5 provides references used in the analysis of the survey.
- The table continues from the previous page.