PaperlayerView exact revision on arXiv

Source-linked AI summary

Text Relatedness Based on a Word Thesaurus

George Tsatsaronis, Iraklis Varlamis, Michalis Vazirgiannis

arXiv:1401.5699v1cs.CL

TL;DR

The paper addresses how to measure text relatedness when lexical overlap alone misses semantic connections. It introduces thesaurus-based SR and Omiotis, which use semantic paths and extend word relatedness to texts. Experiments report stronger performance than lexicon-based measures across the selected tasks and datasets, with competitive results against corpus-based and hybrid approaches.

  • Problem

    Text relatedness requires capturing both lexical overlap and semantic relations between words, because lexically different fragments may convey related meanings.

  • Method

    SR derives word relatedness from WordNet semantic paths using path length, node depth, and edge types, while Omiotis extends SR to text relatedness.

  • Results

    Omiotis performed very well on sentence similarity and paraphrase recognition, while SR outperformed state-of-the-art measures overall on the used word-to-word datasets.

  • Takeaways & Limitations

    A thesaurus-only approach can support semantic relatedness measurement across word-to-word, sentence-similarity, and paraphrase-recognition tasks.

  • Takeaways & Limitations

    The paper identifies drawbacks in NWPL, including inability to distinguish relatedness among concepts in one hierarchy and increased computational cost.

Abstract

from arXiv · show

The computation of relatedness between two fragments of text in an automated manner requires taking into account a wide range of factors pertaining to the meaning the two fragments convey, and the pairwise relations between their words. Without doubt, a measure of relatedness between text segments must take into account both the lexical and the semantic relatedness between words. Such a measure that captures well both aspects of text relatedness may help in many tasks, such as text retrieval, classification and clustering. In this paper we present a new approach for measuring the semantic relatedness between words based on their implicit semantic links. The approach exploits only a word thesaurus in order to devise implicit semantic links between words. Based on this approach, we introduce Omiotis, a new measure of semantic relatedness between texts which capitalizes on the word-to-word semantic relatedness measure (SR) and extends it to measure the relatedness between texts. We gradually validate our method: we first evaluate the performance of the semantic relatedness measure between individual words, covering word-to-word similarity and relatedness, synonym identification and word analogy; then, we proceed with evaluating the performance of our method in measuring text-to-text semantic relatedness in two tasks, namely sentence-to-sentence similarity and paraphrase recognition. Experimental evaluation shows that the proposed method outperforms every lexicon-based method of semantic relatedness in the selected tasks and the used data sets, and competes well against corpus-based and hybrid approaches.

1. Introduction

The paper motivates text relatedness measures that capture both lexical overlap and semantic relations between words. It introduces Omiotis, a thesaurus-based text measure extending the word-level SR measure, and evaluates it across word and text tasks.

  • 1. Introduction: Traditional vectorial similarity measures capture lexical relatedness but can fail when text segments use different words with similar meanings.The introduction contrasts standard measures such as Cosine, Dice, and Jaccard with semantically related but lexically different sentences.
  • 1. Introduction: The proposed measure combines lexical and semantic relatedness to handle cases that traditional lexical matching models miss.This is the stated motivation for developing the approach.
  • 1. Introduction: Omiotis extends the word-level semantic relatedness measure SR to compute semantic relatedness between texts using semantic links derived from WordNet.SR itself uses thesaurus-based paths between word senses, while Omiotis applies the resulting word relations to texts.
  • 1. Introduction: The paper evaluates SR on word similarity, relatedness, synonym identification, and analogy, then evaluates Omiotis on sentence similarity and paraphrase recognition.The listed experiments cover both individual-word and text-to-text relatedness tasks.
  • 1. Introduction: The paper proceeds from preliminary concepts and related work to formal definitions, experiments, analysis, conclusions, and future work.The introduction provides this section-level organization of the paper.

2. Preliminaries and Related Work

The preliminaries review lexical, corpus-based, dictionary-based, and hybrid approaches to semantic relatedness. They position WordNet as the thesaurus underlying Omiotis and describe prior applications, measures, and semantic relations.

  • 2. Preliminaries and Related Work: Word thesauri such as WordNet support semantic-network models for tasks including document similarity, classification, retrieval, summarization, and word-sense disambiguation.The paper surveys these uses across prior research.
  • 2. Preliminaries and Related Work: WordNet organizes English nouns, verbs, adjectives, and adverbs into synsets connected by numerous semantic relations, including cross-part-of-speech links.The listed relations include hypernymy, meronymy, synonymy, antonymy, entailment, domain terms, and derivationally related forms.
  • 2. Preliminaries and Related Work: Existing semantic relatedness measures are classified as dictionary-based, corpus-based, or hybrid.The review includes thesaurus, latent-semantic-analysis, information-content, and supervised combination approaches.
  • 2. Preliminaries and Related Work: Earlier dictionary-based measures commonly use path length, taxonomy depth, or concept density, while some assume all semantic edges have equal importance.The paper identifies equal edge weighting as a limitation of early approaches.
  • 2. Preliminaries and Related Work: Information-content measures combine thesaurus hierarchy with corpus statistics, using the least common subsumer and its probability of occurrence.Resnik’s formulation is presented as a hybrid measure based on the information content of the shared subsumer.
  • 2. Preliminaries and Related Work: The paper then introduces its formal Omiotis definitions, complexity analysis, design justifications, and potential text-related applications.This marks the transition from reviewed approaches to the paper’s method.

3. Measuring Word-to-Word and Text-to-Text Semantic Relatedness

Omiotis measures semantic relatedness at multiple granularities, moving from WordNet senses to words and then to texts. This design supports relatedness calculations across different linguistic units.

  • 3. Measuring Word-to-Word and Text-to-Text Semantic Relatedness: Omiotis extends semantic relatedness from WordNet senses to words and finally to texts.The measure is defined to operate across these levels of granularity.
  • 3. Measuring Word-to-Word and Text-to-Text Semantic Relatedness: The approach is intended for a wide range of linguistic and text-related applications because it captures relatedness at every stated granularity.The paper explicitly links the multi-level definition to broad applicability.

3.1 Construct Semantic Links between Words

The method constructs semantic paths between word senses through WordNet and scores them using path compactness, node depth, and edge weighting. SR selects the path maximizing the combined relatedness score.

  • 3.1 Construct Semantic Links between Words: The first step is finding implicit semantic links between words in the two text fragments through paths connecting thesaurus concepts.The word-level path definition is later extended to text relatedness.
  • 3.1.1 SEMANTIC NETWORK CONSTRUCTION FROM WORD THESAURI: WordNet senses are recursively expanded into a semantic network until a path between the selected senses is found; no path means no semantic relatedness.Semantic links become edges and linked senses become network nodes during expansion.
  • 3.1.2 SEMANTIC RELATEDNESS BETWEEN A PAIR OF CONCEPTS: The paper justifies combining path length, node depth, and edge type because these factors jointly determine semantic relatedness in its formulation.Edge types receive weights based on their relative frequency in WordNet.
  • 3.1.2 SEMANTIC RELATEDNESS BETWEEN A PAIR OF CONCEPTS: Semantic compactness captures path length through the product of edge weights, with no path yielding zero and identical senses yielding one.The compactness definition makes shorter, stronger paths preferable.
  • 3.1.2 SEMANTIC RELATEDNESS BETWEEN A PAIR OF CONCEPTS: Semantic path elaboration incorporates the depths of path nodes relative to the maximum thesaurus depth.Deeper senses represent more specific concepts, while shallow senses are more general.
  • 3.1.2 SEMANTIC RELATEDNESS BETWEEN A PAIR OF CONCEPTS: SR defines sense relatedness as the maximum product of semantic compactness and semantic path elaboration over candidate paths.The combined score is written as maxP {SCM(S, O, P) · SPE(S, O, P)}.
  • 3.1.2 SEMANTIC RELATEDNESS BETWEEN A PAIR OF CONCEPTS: When multiple paths connect two senses, a modified Dijkstra algorithm returns the path with maximum edge-weight product.The algorithm tracks maximum relatedness values and predecessors to reconstruct the selected path.

3.2 Analysis of the SR Measure

The SR measure is designed to capture semantic relatedness through richer WordNet paths, using all parts of speech, relation types, edge weights, and depth scaling. Comparisons with alternative path formulations motivate product-based semantic compactness.

  • Design decisions: SR uses senses across all parts of speech and all WordNet semantic edge types, rather than only noun senses and IS-A relations.This broadens the semantic graph and allows evidence from verb and cross-POS relations, such as links between espionage and spying or convicted and penalty.
  • Design decisions: Non-hierarchical WordNet links can reveal relatedness missed by measures restricted to noun taxonomies, including the connection between child care and school.The paper presents this connection as a path in WordNet that its richer semantic representation can detect.
  • Design decisions: 7 percentage points lower Spearman correlation with human judgements occurred on Miller and Charles when the depth scaling factor was omitted.The result supports retaining depth scaling in the edge-weighting mechanism.
  • Path formulation: PR selects paths maximizing the product of semantic compactness, while NWPL selects paths using normalized sums of edge weights.The comparison isolates product-based and normalized-sum formulations of path relatedness.
  • Path formulation: For car–accelerator, NWPL returns 0.61 from a long hypernym/hyponym path, whereas PR returns 0.0367 from the immediate part-meronym path.NWPL therefore assigns the same 0.61 value to every concept pair shown in the hierarchy, while PR distinguishes most pairs.
  • Path formulation: NWPL produces the same 0.61 semantic-relatedness value in almost 40% of Miller and Charles word pairs, creating many ties.The paper concludes that PR is preferable as the semantic compactness factor because it distinguishes relatedness more effectively.

3.3 Omiotis

Omiotis extends SR to compare texts using both semantic relatedness and lexical similarity. It matches terms across texts in both directions and aggregates the strongest pairwise relevance estimates.

  • Text relatedness: Omiotis extends SR to account for both semantic relatedness and lexical similarity between terms in two texts.Lexical similarity helps handle specialized terms absent from WordNet, while semantics complements matching when different terms express similar concepts.
  • Text relatedness: Lexical relevance is estimated from the harmonic mean of the terms’ TF-IDF values.The harmonic mean is chosen because it provides a tighter upper bound than an average for the two term-strength quantities.
  • Matching: For each term in text A, Omiotis selects the term in text B maximizing the product of semantic relatedness and lexical similarity.The selected match is denoted b*, with an analogous a* defined for terms in text B.
  • Matching: Omiotis aggregates relevance scores from A to B and repeats the process from B to A to handle texts with unequal numbers of terms.The bidirectional estimates are combined into the final relevance between texts.

3.4 Applications of Semantic Relatedness

This section applies semantic relatedness to word analogies, sentence similarity, and paraphrase recognition. The approach combines thesaurus-based word relations with lexical weighting to compare text segments and analogy candidates.

  • SAT analogy tests: SAT analogy solving selects, from five candidate pairs, the pair judged most analogous to the target pair.The task evaluates whether a candidate relation matches the relation expressed by the target word pair.
  • SAT analogy tests: The reported SAT baselines score 56.1% for the top corpus-based approach, 42% for the top lexicon-based approach, and 33.2% for the top hybrid approach.The average US college applicant score reported in the same comparison is 57%.
  • SAT analogy tests: SR evaluates word analogies by comparing target and candidate pairs through horizontal and vertical analogy scores.The candidate with the maximum combined score is selected, capturing both alignment between pair members and differences in SR scores.
  • Paraphrase recognition: Paraphrase detection applies Omiotis directly to text-segment pairs, interpreting higher Omiotis scores as greater relatedness.The evaluation uses the Microsoft Research Paraphrase Corpus.
  • Sentence similarity: Omiotis computes sentence relatedness with the same Equation 10 and Algorithm 2 procedure used for text-segment comparison.Sentence evaluation uses 30 sentence pairs from Li et al. (2006), with human scores available for comparison.

3.5 Complexity and Implementation Issues

Omiotis can be computationally expensive because its complexity depends on constructing thesaurus-based semantic networks. Precomputing SR values in a database reduces processing time and makes the system feasible for scalable use.

  • Complexity: Constructing the semantic network for SR takes O(2 · k^(l+1)), where k is the thesaurus branching factor and l is the maximum semantic path length.Omiotis inherits this cost through its dependence on the SR base measure.
  • Implementation: Precomputing SR values for every possible synset pair transfers the major computation to a one-time database-building cost.The implementation stores WordNet nodes, weighted edges, and paths in a relational database.
  • Implementation: Processing 100 term pairs takes approximately 1 second after SR similarities are precomputed.The authors present this result as evidence that Omiotis is feasible and scalable.

4. Experimental Evaluation

The evaluation tests SR on word-level similarity, relatedness, synonym identification, and analogy, then tests Omiotis on sentence similarity and paraphrase recognition. Across these tasks, the results show strong performance against lexicon-based and other approaches, with context and efficient execution affecting comparisons.

  • Evaluation design: The evaluation covers three SR word-level tasks—similarity and relatedness, synonym identification, and SAT analogy—and two Omiotis text-level tasks.The word-level evaluation uses benchmark rankings, TOEFL and ESL synonym questions, and 374 SAT questions; the text-level evaluation covers sentence similarity and paraphrase recognition.
  • Word similarity and relatedness: SR correlations are reported with Spearman’s ρ and Pearson’s r on the R&G, M&C, and 353-C word-pair data sets, with statistical significance tested using Fisher’s z-transformation.The 353-C collection emphasizes semantic relatedness, whereas R&G and M&C focus more on semantic similarity; inter-judge agreement is also lower for some 353-C cases.
  • Word similarity and relatedness: SR values follow a pattern similar to human ratings in Figure 5, while SR performs well on 353-C but trails Wikipedia-based approaches there.The paper attributes the 353-C disadvantage partly to missing context and limited WordNet coverage, while reporting statistically significant SR advantages over those approaches on R&G and M&C.
  • Synonym identification: SR ranks first on the ESL synonym-identification data set, matching JS and outperforming all compared corpus-based methods.The result is reported in Table 6 and is used to argue that lexicon-based methods are promising for synonym identification.
  • SAT analogy questions: S ranks second among the compared lexicon-based SAT measures, while combining s1 and s2 reaches 52.4% and Naive Bayes adds a 4.1% boost.The paper reports that S processes 374 SAT questions in less than 3 minutes, compared with about 6 hours for B and almost 8 days for LRA.
  • Text-level evaluation: Omiotis surpasses all lexicon-based methods and matches Mihalcea et al.’s combined method on paraphrase recognition, with tuned F-Measure reaching 81.7.The reported evaluation uses a simpler tuning, while the authors present the tuned result as evidence that Omiotis performs competitively with state-of-the-art methods.

5. Conclusions and Future Work

The paper concludes that SR and Omiotis perform strongly across word- and text-relatedness tasks, while identifying room to improve word analogy results.

  • SR outperformed all state-of-the-art measures overall on the evaluated word-to-word tasks.
  • Omiotis performed very well on sentence similarity and paraphrase recognition.
  • Combining path length, node depth, and edge type in one formula supported applying the measures to different text-based tasks.
  • Word analogy results were satisfactory but may improve because the method received no special tuning.
Loading 1401.5699v1…