Source-linked AI summary
Semantic Similarity from Natural Language and Ontology Analysis
Sébastien Harispe, Sylvie Ranwez, Stefan Janaqi, Jacky Montmain
TL;DR
Machines tasked with complex cognitive work need ways to compare semantic entities by meaning. This book organizes semantic-measure terminology and classifications, including reviews of Wikipedia-based approaches. It presents a structured account of the field’s diverse measures and interdisciplinary foundations.
Problem
Machines tasked with complex cognitive work need ways to compare semantic entities by meaning.
Source: Preface · p. 7, Introduction to semantic measures · p. 9Method
The book organizes semantic-measure terminology and classifications, and reviews approaches including Wikipedia-based vector methods.
Source: Text analysis of Wikipedia content and… · p. 98, Conclusion and research directions · p. 123Results
The book introduces similarity models and organizes the terminology and classifications used to describe semantic measures.
Source: Conclusion and research directions · p. 123Takeaways & Limitations
This organized account helps readers understand the field’s breadth and interdisciplinary depth.
Source: Conclusion and research directions · p. 123Distributional measures are commonly considered to capture relatedness rather than word similarity, since co-occurrence can reflect many kinds of relationships.
Source: §2.5.2 · p. 45
Abstract
from arXiv · showhide
Artificial Intelligence federates numerous scientific fields in the aim of developing machines able to assist human operators performing complex treatments -- most of which demand high cognitive skills (e.g. learning or decision processes). Central to this quest is to give machines the ability to estimate the likeness or similarity between things in the way human beings estimate the similarity between stimuli. In this context, this book focuses on semantic measures: approaches designed for comparing semantic entities such as units of language, e.g. words, sentences, or concepts and instances defined into knowledge bases. The aim of these measures is to assess the similarity or relatedness of such semantic entities by taking into account their semantics, i.e. their meaning -- intuitively, the words tea and coffee, which both refer to stimulating beverage, will be estimated to be more semantically similar than the words toffee (confection) and coffee, despite that the last pair has a higher syntactic similarity. The two state-of-the-art approaches for estimating and quantifying semantic similarities/relatedness of semantic entities are presented in detail: the first one relies on corpora analysis and is based on Natural Language Processing techniques and semantic models while the second is based on more or less formal, computer-readable and workable forms of knowledge such as semantic networks, thesaurus or ontologies. (...) Beyond a simple inventory and categorization of existing measures, the aim of this monograph is to convey novices as well as researchers of these domains towards a better understanding of semantic similarity estimation and more generally semantic measures.
Introduction to semantic measures
This section introduces semantic measures for estimating likeness or relatedness among language units, concepts, and knowledge-base instances using semantic evidence. It outlines text- and ontology-based approaches and clarifies how terminology, comparison evidence, and representation shape their scope, including the distinction between relatedness and similarity.
Introduction to semantic measures
- Semantic measures compare meaning-bearing entities using evidence extracted from text corpora or structured knowledge models such as ontologies.Unlike syntactic comparison, they use semantic evidence to characterize meaning; corpus approaches can use word co-occurrence, while ontology approaches use explicit relationships.Source: Introduction to semantic measures · pp. 9–10
1.1.1 Natural Language Processing
- Semantic measures support language tasks including paraphrase and plagiarism detection, thesaurus or text generation, summarization, discourse analysis, question answering, and ambiguity resolution.Their use in natural language processing reflects the need to compare words, sentences, paragraphs, and documents by meaning.Source: §1.1.1 · p. 10
1.1.2 Knowledge engineering, Semantic Web and Linked Data
- Ontologies and structured vocabularies provide formal, machine-processable knowledge for comparing domain concepts and instances, with concept taxonomies especially useful for estimating similarity.Applications include ontology alignment and instance matching, inexact ontology search, learning, and Linked Data recommendation systems.Source: §1.1.2 · p. 11
1.1.3 Biomedical Informatics & Bioinformatics
- In bioinformatics, ontology-based semantic measures compare genes through annotations of their functions, locations, and biological processes, supporting analysis and knowledge inference.Gene Ontology annotations enable comparison based on formalized biological aspects rather than only sequence, structure, or expression properties.Source: §1.1.3 · p. 11
1.1.4 Other applications
- Semantic measures extend information retrieval beyond exact term matching and also support similarity comparisons between locations using geographic-feature descriptions.Ontology-based retrieval can query non-textual resources, such as genes annotated with concepts.Source: §1.1.4 · p. 12
1.2.1 Human cognition, similarity and existing models
- Similarity underlies human learning, memory retrieval, categorisation, pattern recognition, problem solving, and reasoning, motivating its study in intelligent systems.Cognitive models compare mental representations, and human judgments can be asymmetric, challenging models that assume distance axioms.Source: §1.2.1 · pp. 12–13
Spatial models
- Spatial models represent objects as points in multidimensional psychological space and treat similarity as a function of distance, but human judgments can violate distance axioms.Empirical work has questioned the framework for inconsistencies involving symmetry, triangle inequality, and identity of indiscernibles.Source: Spatial models · p. 14
Feature models
- Feature models compare objects through common and distinct features, using a matching function that is expected to increase similarity as common features are added or distinct features removed.The model represents objects as feature sets and assesses their overlap and differences.Source: Feature models · p. 14
- Feature models weight shared and distinct features by salience, while parameters α and β let them produce symmetric or asymmetric similarity measures.Feature salience captures specificity and can allow less than maximal self-similarity.Source: Feature models · p. 15, Unification of cognitive models of… · p. 15
Structural alignment models
- Structural alignment models compare structured representations by matching correspondences; more correspondences imply greater similarity, potentially across both features and relationships.This approach responds to the criticism that feature models treat compared objects as unstructured.Source: Structural alignment models · p. 15
Tranformational models
- Transformational models treat similarity as decreasing with the complexity of the modifications needed to transform one mental representation into another.The account frames similarity in representational distortion and connects it to Kolmogorov complexity theory.Source: Tranformational models · p. 15
Unification of cognitive models of similarity
- Cognitive models differ in their assumptions about mental representation and comparison, yet researchers have proposed frameworks that unify spatial, feature-based, structural, and transformational approaches.Tenenbaum and Griffiths propose a Bayesian unification of spatial, feature-based, and structure-based models; Hahn proposes transformational models as a generalisation of spatial, feature, and structure-based models.Source: Unification of cognitive models of… · pp. 15–16
Generalities
- Semantic measures estimate relationships between language units, concepts, or instances from information supporting meaning, but terminology and the relation to human judgments remain contested.Measures may follow evidence in a semantic proxy rather than human judgments, and the book aims to clarify definitions amid continuing debate.Source: Generalities · pp. 16–18
Semantic relatedness and semantic similarity
- Semantic relatedness allows any type of semantic link, whereas semantic similarity is treated here as a subset based only on taxonomic relationships.The book distinguishes the concepts because related elements need not be similar: Tea and Cup are related, but share few constitutive properties.Source: Semantic relatedness and semantic… · p. 18
The diversity of types of semantic measures
- The terminology includes relatedness, similarity, distance, and dissimilarity; the book introduces semantic unrelatedness for inverse-relatedness measures that need not satisfy distance properties.Semantic distance is often used for inverse relatedness whether or not it meets mathematical distance requirements.Source: The diversity of types of semantic… · p. 19
Distance and similarity in Mathematics
- A mathematical distance must be non-negative, symmetric, and reflexive; metric-space distance additionally requires identity of indiscernibles and the triangle inequality.The section introduces mathematical definitions to rigorously characterize semantic measures and clarify the intuitive terminology used in the field.Source: §1.2.3 · p. 20, Distance and similarity in Mathematics · p. 20
- Formal mathematical properties characterize measures, but empirical performance can challenge the value of requiring those properties.The text notes that measures can perform well despite failing properties such as the triangle inequality or identity of indiscernibles.Source: Distance and similarity in Mathematics · p. 21
Flexibility of semantic measures
- Semantic measure terminology describes function meaning rather than mathematical properties: distance captures unrelatedness, while relatedness increases with stronger semantic interactions.The book uses “measure” broadly for functions that assess semantic distance, relatedness, similarity, and related notions, without requiring measure-theory definitions.Source: Flexibility of semantic measures · pp. 21–22
1.3.1 How to classify semantic measures
- Measures can be classified by the elements compared, their semantic proxies, the evidence and assumptions used, and the elements’ canonical representations.These four aspects provide a framework for distinguishing the large variety of semantic-measure approaches.Source: §1.3.1 · p. 22
1.3. CLASSIFICATION OF SEMANTIC MEASURES
- Semantic measures compare language units, concepts or groups of concepts, and semantically characterized instances.Examples of language units include words, sentences, paragraphs, and documents.Source: §1.3 · p. 23
Semantic proxies from which semantics is distilled
- Semantic proxies are information sources used to characterize elements and extract semantics, including texts and structured resources such as ontologies and taxonomies.Text-based proxies include corpora, controlled vocabularies, and dictionaries.Source: Semantic proxies from which semantics… · p. 23
Semantic evidence and considered assumptions
- The proxy determines the semantic evidence and assumptions behind a measure; for text analysis, word co-occurrence can indicate relatedness.This example assumes that the degree of co-occurrence reflects the strength of term relatedness.Source: Semantic evidence and considered… · p. 23
Canonical forms used to represent compared elements
- A measure’s canonical form affects its representational granularity and the interpretation of its scores, so each measure processes a specific element type in a specific form.That interpretation matters when drawing inferences from the resulting scores.Source: Canonical forms used to represent… · p. 23
1.3.2 A general classification of semantic measures
- The chapter classifies semantic measures into corpus-based and knowledge-based approaches, organizing a diverse field and highlighting areas that remain unexplored.It also distinguishes terminology such as similarity, relatedness, and distance, while noting the difficulty of defining these notions.Source: §1.3.2 · p. 24
Corpus-based semantic measures
Corpus-based semantic measures use evidence from word occurrence contexts to build representations and estimate similarity or relatedness between language units. This section surveys distributional models and alternatives—including geometric, set-based, and probabilistic approaches—and the choices and challenges involved in applying them across relationships and larger language units.
2.1. FROM TEXT ANALYSIS TO SEMANTIC MEASURES
- The chapter distinguishes distributional measures from approaches such as information-retrieval-based measures, while focusing mainly on distributional methods and reserving graph-based models for knowledge-based measures.Most corpus-based work studies word pairs; measures for larger language units are often extensions of word measures or use similar approaches.Source: §2.1 · pp. 27, 30
2.1 From text analysis to semantic measures
- Corpus-based measures build a semantic model from a text corpus, derive processable word representations, and compare them; defining the model is central, while comparison is often a technical step.The chapter mainly focuses on word comparisons, rather than measures for sentences or texts.Source: §2.1 · pp. 27, 28, 30
2.2.1 The meaning of words
- What word meaning and semantic relatedness denote is generally left intuitive, and the question of how finely to define meaning for similarity measurement remains open.Corpus-based measures generally compare linguistic meaning through word usage in text, often treating words in shared contexts as related.Source: §2.2.1 · p. 31
2.2.2 Structural relationships: Paradigmatic and Syntagmatic
- Paradigmatic relatedness concerns substitutability and is captured through indirect co-occurrences, whereas syntagmatic relatedness concerns association in a sequence and is captured through direct co-occurrence.Paradigmatic association can reflect relations such as synonymy, hyponymy, hypernymy, or antonymy.Source: §2.2.2 · p. 32
2.2.3 The notion of context
- Context is central to analyzing word meaning; in the Vector Space Model, words can be represented by vectors marking the documents in which they occur, whose similarity estimates word relatedness.This approach treats each document as a topical unit and derives a document-word matrix from the corpus.Source: §2.2.3 · pp. 32–33
- A semantic model represents each word as a vector whose dimensions correspond to contexts, making context central to word comparison.Contexts may be documents or smaller units such as paragraphs, sentences, or word windows.Source: §2.2.3 · p. 33
Syntagmatic contexts
- Syntagmatic contexts capture sequential co-occurrences, using larger text regions or sliding word windows to build word-context or co-occurrence matrices.Narrower windows tend to produce sparser co-occurrence matrices; oriented windows and distance weighting are among the possible refinements.Source: Syntagmatic contexts · p. 34
Paradigmatic contexts
- Paradigmatic contexts capture indirect co-occurrences, often through grammatical dependencies, to characterize words by shared contexts and lexical substitutability.For example, the pattern ⟨w, SUBJ, GROW⟩ can characterize plant and tree, which occur as subjects of “grow.”Source: Paradigmatic contexts · p. 35
2.2.4 Distributional semantics
- Distributional semantics assumes that words occurring in similar contexts tend to be semantically related, while distributional similarity may be weaker than semantic similarity under some definitions.Distributional analysis is presented as a practical basis for corpus-based measures, despite debate over how fully it characterizes linguistic meaning.Source: §2.2.4 · pp. 36–37
2.3 Distributional measures
- Distributional measures build semantic spaces from corpus contexts and represent words with profiles that support varied relatedness measures.Different models and profiles produce a wide diversity of proposed and evaluated measures.Source: §2.3 · p. 37
2.3.1 Implementation of the distributional hypothesis
- Distributional models depend on choices about corpus preprocessing, context type, frequency weighting, and optional dimension reduction.TF-IDF can downweight frequent contexts, while matrix factorization such as SVD can reduce dimensions, address sparsity, and reveal latent dimensions.Source: §2.3.1 · p. 38
2.3.2 From distributional model to word similarity
- Distributional models characterize words through usage contexts, and their statistics can support geometric, set-based, or probabilistic similarity measures.The three approaches compare vector positions, context-set overlap, or probabilistic evidence, respectively.Source: §2.3.2 · p. 39, §2.3.1 · p. 39
The geometric or spatial approach
- Geometric measures compare word vectors in a context-defined semantic space using distances, cosine similarity, or correlation.Examples include LSA's reduced word-context matrix, ESA's word-concept matrix, and HAL's weighted co-occurrence matrix.Source: The geometric or spatial approach · pp. 39–40
The set-based approach
- Set-based measures compare words by the overlap and difference between their context sets, using measures such as Dice or Jaccard.Fuzzy-set weighting has also been proposed.Source: The set-based approach · p. 41
The probabilistic approach
- PMI estimates word association from co-occurrence and individual frequencies, but it favors rare words; NPMI and PPMI are among proposed adaptations.PMImax is another variation, designed to account for multiple word senses.Source: The probabilistic approach · p. 42
- Probabilistic methods compare normalized word-distribution profiles with measures such as KL divergence, while PLSA and LDA provide statistical topic-model approaches.KL divergence is positive and asymmetric, has zero self-distance, and does not satisfy the triangle inequality.Source: The probabilistic approach · pp. 42–43
2.3.3 Capturing deeper co-occurrences
- First-order co-occurrence methods can miss similar words that do not share contexts, so deeper co-occurrences can be modeled through matrix factorization, SOC-PMI, or dependency-based contexts.The text gives road and street as an example of words that can be synonymous despite rarely co-occurring in the same word window.Source: §2.3.3 · p. 44
2.4 Other corpus-based measures
- Other corpus-based approaches compare information-retrieval results or use dictionary, thesaurus, and semantic-network structure to estimate word similarity.One dictionary-based method builds a graph from definitions and evaluates similarity with a random walk.Source: §2.4 · p. 44, §2.5 · p. 45
2.5.1 Advantages of corpus-based measures
- Corpus-based measures are unsupervised and can compare relatedness in corpora without prior knowledge of word meaning or usage.Source: §2.5.1 · p. 45
- Corpus-based measures can compare language units from words to texts and be tuned to capture particular syntagmatic or paradigmatic relationships, including antonymy.Source: §2.5.1 · p. 45
2.5.2 Limits of corpus-based measures
- Corpus-based measures require words to occur in the corpus and depend on its contents; representative corpora and sense-tagged data may be unavailable, while disambiguation complicates comparisons beyond words.Disambiguation can be computationally costly on large corpora, and most approaches cannot compare multi-word expressions.Source: §2.5.2 · p. 45
- Distributional measures generally capture relatedness rather than similarity, though specific preprocessing or paradigmatic contexts can target relations such as synonymy and antonymy.The passage describes work on synonymy extraction and modified LSA for assessing synonymy and antonymy between words.Source: §2.5.2 · p. 45
- Scores can be difficult to interpret because general distributional approaches do not make the semantics of co-occurrences readily traceable.Source: §2.5.2 · p. 45
2.6 Conclusion
- Corpus-based measures span diverse approaches and language-unit sizes, but extending word-level models to sentences and documents raises challenges in compositionality and representing expressions such as anaphora and negation.The passage characterizes current distributional models as providing only a rough approximation of natural-language meaning and notes adaptation research.Source: §2.6 · p. 46
- A further research aim is to unify corpus-based measures so one approach can compare lexical units at different granularities, such as words and paragraphs.Source: §2.6 · p. 46
Knowledge-based semantic measures
This section surveys knowledge-based semantic measures for comparing ontology-defined concepts and entities using graph and taxonomy structure, semantic and information-content evidence, and pairwise or group-comparison strategies. It also addresses their assumptions and limitations, adaptations to multiple inheritance and multiple ontologies, and hybrid approaches that combine corpus and knowledge-base evidence.
Knowledge-based semantic measures
- Knowledge-based measures use formal knowledge to compare concepts or instances, and this chapter focuses on semantic-graph and taxonomy measures rather than logic-based or multi-ontology measures.Unlike corpus-based measures, they can compare any formally described entities, not only language units.Source: Knowledge-based semantic measures · pp. 47–48
- Ontology availability limits knowledge-based measures, but wider adoption of ontology standards, Linked Data, and shared schemas has expanded access across domains.The passage notes that search-engine companies jointly designed Schema.org to help publishers describe web-page content unambiguously.Source: Knowledge-based semantic measures · p. 47
Taxonomies and partially ordered sets
- A taxonomy organizes similar elements into ordered classes and models concepts as a reflexive, antisymmetric, transitive partial order, generally represented by a rooted directed acyclic graph.Predicate axioms formally state the taxonomic relationship's properties and can support inference and algorithm coherence.Source: Taxonomies and partially ordered sets · pp. 48–49
General discussion on ontologies as graphs
- Even expressive ontologies can often be partially reduced to graphs, with concepts and instances represented as nodes and instance-to-class links as relationships.The chapter's graph formalism extends a taxonomy structure to include instances, data values, and multiple predicates.Source: General discussion on ontologies as… · pp. 49–50
Types of ontologies considered in this chapter
- The adopted semantic-graph formalism represents concepts, predicates, instances, data values, directed predicate-typed relationships, and axioms, with concept and predicate taxonomies treated as rooted connected directed acyclic graphs.The manuscript mainly manipulates ontologies without predicate taxonomies.Source: Types of ontologies considered in this… · pp. 50–51
3.1.4 Notations for taxonomies
- The taxonomic reduction GT contains only concepts and their taxonomic relationships, and the chapter uses it to compute concept similarity and define subsumption, ancestor, and descendant relations.A concept v subsumes u when u is ordered below v in the taxonomy.Source: Types of ontologies considered in this… · p. 51, §3.1.4 · pp. 52–53
3.1.2 Relationships
- Graph relationships are directed subject-predicate-object triplets, and each relationship implicitly entails an inverse relationship even when it is not explicitly defined.Relationships are central to defining graph algorithms and characterizing paths.Source: §3.1.2 · p. 52
3.1.3 Graph traversals
- A graph path is a sequence of relationships, while a path pattern lists predicates and can specify paths using a predicate or its inverse repeatedly.These notations concisely characterize path families between graph elements.Source: §3.1.3 · p. 52
3.1.4 Notations for taxonomies
- Taxonomy notation identifies a concept’s ancestors, descendants, and leaves, providing structural vocabulary for describing concept hierarchies.A taxonomic tree is a special case in which each concept has fewer than two parents.Source: §3.1.4 · p. 53
3.2 Types of semantic measures and graph properties
- Semantic measures divide into those for potentially cyclic graphs with multiple predicates and those for acyclic taxonomies with one transitive predicate.Measures using all predicates in cyclic graphs generally estimate relatedness rather than similarity and can compare concepts or instances.Source: §3.2 · p. 54, §3.2.1 · p. 54
Semantic measures based on graph traversals
- Graph-traversal measures include shortest paths, random walks, and other interconnection methods; they are unsupervised but offer limited control over semantics and score interpretation.Tuning predicate contributions can provide finer-grained control.Source: Semantic measures based on graph… · p. 54
Shortest path approaches
- Shortest-path measures estimate relatedness from the weighted distance between nodes, but unconstrained paths treat taxonomic and complex multi-predicate paths alike.Penalties or predicate-specific weights can address this limitation.Source: Shortest path approaches · p. 55
Random walk approaches
- Random-walk measures use Markov-chain transition probabilities to compare nodes through quantities such as hitting time and commute time, but rely on hard-to-interpret second-order information.Commute time combines the expected travel time in both directions.Source: Random walk approaches · pp. 55–56
Other measures based on interaction analysis
- Other interaction measures use indirect paths between nodes; simRank is a normalized example, with an adaptation proposed for Linked Data graphs.The matrix-forest theorem is another approach to exploiting graph structure.Source: Other measures based on interaction… · p. 56
Semantic measures for the graph property model
- Graph-property measures compare entities through their direct and indirect properties, applying property-specific comparisons and aggregating their scores into overall relatedness.Indirect properties can be context-selected or recursively incorporated, and may enable comparisons through derived properties such as body mass index.Source: Elements represented as a list of… · p. 56, §3.2 · p. 57, Consideration of indirect properties… · p. 57
3.2.2 Semantic measures on acyclic graphs
- For acyclic graph reductions, specialized techniques apply, and many measures focus on taxonomic relations while some generalize similarity to other acyclic relation subsets.A semantic graph’s explicit structure, such as taxonomy, also carries implicit evidence interpretable through the meanings of its ontology and elements.Source: Consideration of indirect properties… · pp. 57–58, §3.2.2 · p. 58, §3.3 · p. 58
3.3 Semantic evidence in semantic graphs and their interpretations
- Semantic evidence derives graph properties for normalization, specificity, and relationship weighting; taxonomy evidence is intentional (topology) or extensional (topology plus usage), with global or local scope.Its interpretation depends on assumptions or theoretical justification grounded in the ontology’s semantics.Source: §3.3 · p. 58, §3.3.1 · p. 59
Extensional evidence
- Instance usage supports local correction factors and specificity estimates, but counts must respect taxonomic inheritance to avoid incoherent distributions.Fewer instances are taken to indicate greater concept specificity.Source: §3.3.1 · p. 59, Extensional evidence · p. 60
Intentional evidence
- Taxonomy depth indicates granularity, diameter is treated as evidence of coverage, and local density or a concept’s depth can inform estimates of concept generality or expressiveness.These interpretations rely on assumptions about how structural properties relate to the ontology and its concepts.Source: Global properties · p. 59, Local properties · pp. 59–60
3.3.2 Concept specificity
- Specificity can be inferred from topology or instance counts for ordered concepts, but incomparable concepts cannot be exactly ranked; θ functions must respect taxonomy order, and Table 3.1 surveys proposals.Evidence from non-comparable common ancestors gives only a lower bound on shared properties.Source: §3.3.2 · pp. 61–62, List of functions defined to estimate… · p. 64, Table 3.1 · p. 65
Basic intrinsic estimators of concept specificity
- Simple intrinsic estimators use depth or ancestor and descendant counts, but can assign similar specificity to concepts that differ in detail.The text identifies this as a strong assumption and motivates more refined estimators.Source: Basic intrinsic estimators of concept… · p. 62
Extrinsic information content
- Extrinsic information content is inversely proportional to concept probability, but it reflects usage bias and text-based estimation can require costly disambiguation.Usage bias may be desirable in some cases, but can distort estimates when it is not appropriate.Source: Extrinsic information content · pp. 62–63
Intrinsic information content; Non-taxonomic information content
- Intrinsic IC uses ontology topology to avoid usage bias but remains sensitive to structural bias; extended IC incorporates non-taxonomic predicates, whose contributions can be averaged or weighted.A proposed modification avoids assigning identical intrinsic IC to every leaf.Source: Intrinsic information content · p. 63, Non-taxonomic information content · pp. 63–64
3.3.3 Strength of connotations between concepts
- Connotation strength models the non-uniform semantics of taxonomic links, generally treating links between deeper concepts as stronger and their taxonomic distance as smaller.Taxonomies specify partial ordering but do not explicitly encode relationship strength.Source: §3.3.3 · p. 66
- Connotation strength depends on the specificity of linked concepts and how their specificities vary.For a taxonomic link u ⪯ v, one example defines strength as w(u, v) = IC(u) − IC(v).Source: §3.3.3 · p. 66
3.4 Semantic similarity between a pair of concepts
- Concept-pair measures include structural, feature-based, information-theoretic, and hybrid approaches, but these categories overlap and are not mutually exclusive.Measures can also be classified as intentional, extensional, or hybrid according to whether they use concept definitions, instances, or both.Source: §3.4 · pp. 67–68
3.4.1 Structural approach
- Structural measures use graph connections, especially shortest paths, while variants incorporate concept specificity and non-uniform connotation strength.Uniform edge counting fails to reflect concept salience, and shortest-path computation can hamper use on large semantic graphs; redundant links can also affect interpretation.Source: §3.4.1 · pp. 68–70
Towards other estimators of semantic similarity
- Shortest paths can be decomposed into differences from a shared least common ancestor, which serves as a proxy for the concepts’ commonality.This decomposition underpins feature-based and information-theoretic strategies that characterize concepts through their commonalities and differences.Source: Towards other estimators of semantic… · p. 71
- An LCA can provide only an upper bound on commonality, since highly similar concepts may share a general ancestor that captures only part of what they have in common.In some cases, the set of non-comparable common ancestors also contains concepts other than the LCA.Source: Towards other estimators of semantic… · p. 71
3.4.2 Feature-based approach
- Feature-based measures represent concepts through features and compare their shared and distinct characteristics, using taxonomic ancestors, instances, or other representations.Feature collections can also be compared as vectors; instance-based comparison is unsuitable for concepts that are not ordered because their instance intersection tends to be empty.Source: §3.4.2 · pp. 72–73
3.4.3 Information theoretical approach
- Resnik measures similarity by the information content of the most informative common ancestor, but this gives equal scores to pairs sharing that ancestor regardless of their own specificities.Refinements incorporate the compared concepts’ specificities; some also lower similarity when the shared ancestor is general, while an alternative can use intrinsic information content without extensional data.Source: §3.4.3 · pp. 73–74
3.4.4 Hybrid approach
- Hybrid measures combine taxonomy structure and information content: Jiang and Conrath’s approach weights a shortest path containing the concepts’ LCA, with parameters controlling density and depth.Setting α = 0, β = 1, and T(u, v) = 1 yields their information-theoretic distJC measure.Source: §3.4.4 · p. 74
3.4.5 Considerations when comparing concepts in semantic graphs
- Measures developed for trees require reconsideration in semantic graphs, where multiple inheritance can give non-ordered concepts common descendants and paths that bypass their common ancestors.For example, the shortest path between two concepts can avoid a common ancestor when multiple-inheritance edges are included.Source: Shortest path · p. 75
Shortest path
- A common graph-based convention constrains the shortest path to pass through one common ancestor, but the best ancestor for similarity may differ from the one selected by path length.The constraint can also be relaxed to allow paths with no common ancestor or multiple common ancestors.Source: Shortest path · pp. 75–76
Notion of depth
- In a graph or redundant taxonomy, depth must remain monotonic with concept ordering, so it is defined using the longest path from a concept to the root.This supports a graph generalization of Pekar and Staab’s measure.Source: Notion of depth · p. 76
Notion of least common ancestors
- Because LCA- or MICA-based measures overlook other non-comparable common ancestors in graphs, GraSM and DiShIn adapt existing measures to account for the full set.One proposed adaptation replaces the MICA’s information content with the average information content of the NCCAs; another averages similarity across multiple NCCAs.Source: Notion of least common ancestors · p. 76
3.4.6 List of pairwise semantic similarity measures
- Wu and Palmer (1994) use concept depth and predicate weights, while Pekar and Staab (2002) account for the depth of the concepts’ lowest common ancestor.Source: §3.4.6 · pp. 78–79
- Blanchard et al. (2006) compare concept and lowest-common-ancestor depth, while Alvarez and Yan (2011) combine weighted path, LCA, and literal-definition components.Source: §3.4.6 · p. 80
- Resnik (1995) defines similarity using the information content of the MICA, whereas Jiang and Conrath (1997) define taxonomic distance as a function of the compared classes’ information content.Source: §3.4.6 · p. 82
- Lin (1998a) computes similarity from the MICA’s information content relative to that of the compared classes; Schlicker et al. (2006) modify Lin’s measure to account for MICA specificity.Source: §3.4.6 · p. 82
- A separate table identifies feature-based measures for comparing concepts in a taxonomy or elements in a partially ordered set.Source: Table 3.4 · p. 84
3.5 Semantic similarity between groups of concepts
- Groupwise measures compare concept sets either directly using information about each set in a taxonomy or indirectly by aggregating pairwise similarities across the Cartesian product; sets are generally assumed to contain no semantically redundant concepts.The indirect approach reuses pairwise measures, while the direct approach generalizes pairwise methods to sets.Source: §3.5 · p. 86, §3.5.2 · p. 86
Structural approach; Feature-based approach; Information theoretical measures
- Groupwise approaches include structural measures based on paths in ancestor-induced graphs, feature-based measures such as simUI and simNTO, and information-theoretic measures that use concept information content.The cited feature-based proposals include simUI and simNTO; information-content-based measures include a proposal by Pesquita et al. (2007).Source: Structural approach · p. 86, Feature-based approach · p. 86, Information theoretical measures · p. 86
Improvements of direct measures using concept similarity
- Basic vector comparisons ignore relationships between dimensions, so direct measures can weight dimensions by specificity or use pairwise similarity in vector products to account for concept relationships.The text also notes that ordinary set-based and vector comparisons may fail to reflect similarity between set elements, yielding zero for semantically related sets.Source: §3.5.1 · p. 86, Improvements of direct measures using… · p. 87
Aggregation strategies
- Indirect aggregation first computes pairwise scores between concepts in the two sets, then combines them using strategies such as max, min, average, Max-Avg, BMM, or BMA.BMM means Best Match Max, and BMA means Best Match Average.Source: Aggregation strategies · p. 87
3.6.1 Semantic measures based on logic-based semantics
- Logic-based semantic measures address complex class or instance descriptions that relational measures cannot compare directly, including OWL concepts, with proposals spanning simple to more expressive description logics.Some approaches extend graph-analysis measures for simple DLs, while others exploit logics such as ALC, ALN, SHI, and ELH.Source: §3.6.1 · p. 91
3.6.2 Semantic measures for multiple ontologies
- Multiple-ontology measures enable comparison across ontologies and can incorporate more information; although less studied than single-ontology measures, they have been classified as structural, feature-based, or information-theoretic.The text notes applications in data integration, clustering, and information retrieval.Source: §3.6.2 · p. 91, Main approaches for the definition of… · p. 92
Comparison with ontology alignment/mapping and instance matching
- Unlike ontology alignment and instance matching, which generally seek exact matches, multiple-ontology semantic measures can estimate relatedness and compare non-equivalent elements while using existing mappings.The section describes alignment and instance matching as processes that use specific similarity measures and can also support the design of multiple-ontology measures.Source: Comparison with ontology… · p. 92
Main approaches for the definition of semantic measures using multiple ontologies
- A hybrid approach using multiple ontologies is attributed to Rodríguez and Egenhofer (2003).Source: Main approaches for the definition of… · p. 92
Advantages
- Knowledge-based measures compare ontology-defined terms, concepts, and instances, including entities that text analysis cannot compare, and allow fine control over the semantic relationships used.They are generally easier and less complex to compute than corpus-based measures because they avoid complex preprocessing, although measures using complex logic constructors can pose problems.Source: Advantages · p. 93
Limits
- Knowledge-based measures require an ontology, logic-based measures can be computationally complex on large ontologies, and graph-based measures may not support non-binary relationships.Knowledge bases generated or enriched through text analysis may help address the ontology requirement; graph-based limitations depend on how an ontology is mapped to a semantic graph.Source: Limits · p. 93
3.8.1 Generalities
- Studies report performance gains from mixing knowledge-based and corpus-based approaches.The section distinguishes pure-hybrid measures, which directly combine corpus and ontology analysis, from aggregated measures, which combine scores using functions such as averages, minima, maxima, or medians.Source: §3.8.1 · p. 94
3.8.2 Wikipedia-based measure: how to benefit from structured encyclopaedia knowledge
- Wikipedia-based measures generally combine textual evidence with topic organization or structured representations, often using hyperlinks that are informative but are not necessarily ontologies or semantic models.They assess relatedness between words or Wikipedia topics and cannot be placed uniformly within the chapter’s measure classification.Source: §3.8.2 · p. 94
2. The text content of these articles
This section surveys semantic measures based on knowledge resources, including Wikipedia article text, hyperlinks, and categories, as well as ontology taxonomies. It covers methods that use these resources separately or in combination, graph-based concept comparisons, and computational and noise limitations.
Measuring semantic relatedness by exploiting Wikipedia’s hyperlink relationships
- Wikipedia’s hyperlink measures compare linked-article sets, but word-level methods require disambiguation and can incur high computation time; Wikipedia links also require pruning because they may be noisy.WLM compares links among articles associated with words, while Turdakov and Velikhov use weighted incoming and outgoing links and heuristics; the section also describes the FLP shortest-path approach, which is outperformed by vector-based approaches.Source: Measuring semantic relatedness by… · pp. 95–97
- Random-walk methods use Wikipedia’s broader graph to compare relatedness; Yazdani and Popescu-Belis report applications across several tasks and improvements intended to reduce computational cost.Their graph combines hyperlinks with content-similarity links, and their method is reported to outperform TF-IDF cosine similarity for document clustering; the approach also requires substantial memory and computation.Source: Measuring semantic relatedness by… · pp. 96–97
- Human navigation paths can capture intuitive associations beyond hyperlink structure, but West et al.’s measure covers only encountered paths; Singer et al.’s co-occurrence vectors allow comparisons beyond shared paths.Singer et al. also identify useful navigational-path characteristics; the section notes that path-based measures can compare entities such as images and textual pages.Source: Measuring semantic relatedness by… · p. 97
Text analysis of Wikipedia content and vector-based approaches
- ESA represents words and texts as weighted Wikipedia-concept vectors, while TSA adds temporal information and is reported to improve on ESA across multiple benchmarks.Vector-based measures outperform path-length measures on collaboratively constructed resources; Wiktionary ranks word pairs better than Wikipedia, though it performs worse on a German dataset. CPRel combines term frequency and link probability, but ESA performs best overall.Source: Text analysis of Wikipedia content and… · pp. 98–99
- Cross-lingual ESA uses interlanguage links to compare concept vectors, and its benchmark correlations slightly outperform monolingual measures despite variation in Wikipedia language coverage.The method modifies the similarity metric and concept weights, and applies preprocessing to detect missing interlanguage links and enforce reflexivity.Source: Text analysis of Wikipedia content and… · pp. 98–99
Wikipedia relies on a category organisation that may be used to measure semantic relatedness
- Wikipedia’s categories form a light ontology that supports semantic comparison alongside corpus-based methods; category-based and hybrid approaches report good results, sometimes outperforming ESA.WikiRelate! combines article-content overlap with category relatedness; BabelRelate! uses BabelNet’s multilingual graph and reports better results with more languages, while Taieb et al. combine article, hyperlink, and category information.Source: Wikipedia relies on a category… · p. 100
3.9. CONCLUSION
- Wikipedia’s multilingual articles, hyperlinks, categories, and infoboxes have supported accurate semantic measures, and its knowledge-base derivatives may enable further hybrid approaches.The section also directs readers to an appendix for software implementations and to Chapter 4 for evaluation protocols and datasets.Source: Wikipedia relies on a category… · p. 100, §3.9 · p. 101
3.9 Conclusion
- The chapter analyzes graph-based measures for taxonomy concepts and distinguishes structure-based, feature-based, and information-theoretical approaches.It uses this analysis to discuss the foundations and core elements of knowledge-based semantic measures.Source: §3.9 · p. 101
- It briefly introduces measures based on complex logic constructs and on analyzing several ontologies.Source: §3.9 · p. 101
- It discusses the advantages and limits of knowledge-based measures and introduces hybrid measures combining corpus-based and knowledge-based approaches.Source: §3.9 · p. 101
Methods and datasets for the evaluation of semantic measures
The chapter surveys methodological and practical aspects of evaluating semantic measures and selecting them for particular needs. It emphasizes that comparisons depend on criteria and use cases, while noting that it will not aggregate performance results across evaluation settings.
Methods and datasets for the evaluation of semantic measures
- The chapter surveys evaluation methods and resources, including measure properties, mathematical frameworks, protocols, and datasets, for researchers and users.It addresses both objective evaluation and selection for specific needs.Source: Methods and datasets for the… · p. 103
- The chapter will not analyze and aggregate existing results to summarize measure performance across different evaluation settings.Its broad overview of evaluation approaches is not an in-depth analysis of every relevant aspect.Source: Methods and datasets for the… · p. 103
4.1 A general introduction to semantic measure evaluation
- There is no best semantic measure in absolute; measures outperform others only under specific conditions, making performance conclusions difficult to generalize.Evaluation criteria and their interpretation depend on the purpose of the comparison.Source: §4.1 · pp. 103–104
- Users must specify constraints, available information, and whether they seek similarity or relatedness before choosing a measure for comparing concepts or terms.The chapter notes that these needs shape which measure suits the use case.Source: §4.1 · p. 104
3. Which criteria must be considered to evaluate measures in a specific context?
This section frames evaluation as context-dependent, surveying criteria for characterizing semantic measures and protocols and datasets for comparing them. It emphasizes that evaluation results do not automatically generalize across usage contexts.
4.2 Criteria for semantic measure evaluation
- Evaluation considers accuracy, precision, robustness, computational complexity, mathematical properties, and technical details.These criteria can be studied theoretically or empirically and address different facets of semantic measures.Source: §4.2 · p. 104
4.2.1 Accuracy, Precision and Robustness
- Accuracy is commonly assessed by correlation with averaged human expectations, while robustness concerns sensitivity to uncertainty in ratings or semantic proxies.Accuracy depends on the dataset, parameters, and resources; results for one setting are not guaranteed to generalize to another.Source: §4.2 · pp. 105–106
4.2.2 Computational complexity
- Computational complexity matters for practical use, but its analysis is limited and often depends on implementation-specific data structures and algorithms.Consequently, comparisons are difficult and commonly rely on empirical evaluation of particular implementations.Source: §4.2.2 · p. 106
4.2.3 Mathematical properties
- Mathematical properties such as symmetry and normalization characterize measures and can guide application choice and valid computational optimizations.Violating properties such as symmetry or identity of the indiscernibles may produce undesirable results in some applications.Source: §4.2.3 · p. 106, §4.2 · p. 107
4.2.4 Semantics
- A measure’s semantics arise from its design assumptions, cognitive model, semantic proxy, and evidence, and can change conclusions supported by its outputs.For example, similarity-based recommendations around Coffee may favor substitutes, whereas semantic proximity may also return related concepts such as Cup or Coffee bean.Source: §4.2.4 · p. 107
4.2.5 Technical details
- Practical selection also considers implementation availability and licensing, resource dependencies, and evaluations across multiple settings.Resource availability and the sensitivity of accuracy to those resources may matter in the target domain.Source: §4.2.5 · p. 107
4.3.1 Protocols used to compare measures
- Direct evaluation correlates measure scores with expected human ratings, whereas indirect evaluation assesses dependent applications or domain-specific metrics.Indirect strategies include evaluating a dependent system’s performance or correlating scores with measures such as gene similarity derived from DNA sequences.Source: §4.3.1 · pp. 108–109, Figure 4.1 · p. 108
4.3.2 Datasets
- Human-rated pair datasets support direct evaluation, but participant instructions, agreement, language, domain, and concept mappings vary across datasets.For knowledge-based evaluation, mappings to concepts are often undocumented, and some studies select the best score when words map to multiple concepts.Source: §4.3.2 · p. 109
(Rubenstein and Goodenough, 1965) – noun similarity
This section surveys human-rated benchmarks and datasets for evaluating semantic similarity and relatedness measures, including how they were constructed and their reported agreement or reliability. It also describes direct and indirect evaluation approaches and identifies open challenges involving benchmarks, reproducibility, and dataset biases and scope.
(Rubenstein and Goodenough, 1965) – noun similarity
- 65 noun pairs were rated for similarity of meaning by 51 undergraduates on a 0–4 scale in Rubenstein and Goodenough’s benchmark.Repeat ratings produced an intra-subject Pearson correlation of 0.85, and mean judgments from the two groups correlated at 0.99; the results were merged.Source: (Rubenstein and Goodenough, 1965) –… · p. 112
(Miller and Charles, 1991) – noun similarity
- Miller and Charles’s benchmark selects 30 pairs from Rubenstein and Goodenough’s set, balanced across high, intermediate, and low similarity.Thirty-eight participants rated the pairs; inter-subject correlation was 0.8848, and its ratings correlated 0.97 with Rubenstein and Goodenough’s. A typographical error changed cord/smile to chord/smile.Source: (Miller and Charles, 1991) – noun… · pp. 112–113
TOEFL (Landauer and Dumais, 1997) – word similarity; ESL (Turney, 2001) – word similarity
- The TOEFL benchmark uses 80 multiple-choice synonym questions, while ESL uses 50, to assess word similarity across nouns, verbs, and adjectives.TOEFL applicant averages were 51.6 correct answers (64.5%), or 52.7% after correction for guessing; ESL inter-agreement was not reported.Source: TOEFL (Landauer and Dumais, 1997) –… · p. 113, ESL (Turney, 2001) – word similarity · p. 113
SAT Analogy questions (Turney and Littman, 2003) – similarity of word relationships; (Boyd-Graber et al., 2006) – WordNet synset relatedness; (Yang and Powers, 2006) – verb relatedness; (Pedersen et al., 2007) – concept relatedness
- The SAT benchmark tests similarity between word relationships through 374 analogy questions, while Boyd-Graber et al. assess relatedness for about 100K WordNet synset pairs.SAT items match a problem pair to one of five candidate pairs, and human performance is not reported. The synset benchmark has two datasets, built using trained undergraduates and Amazon Mechanical Turk participants, and is reported to contain noise. Yang and Powers provide 144 verb pairs rated on a 0–4 scale (participant correlation r = 0.866); Pedersen et al. use medical concept pairs, with a 29-pair subset showing physician/coder correlation of 0.85.Source: SAT Analogy questions (Turney and… · pp. 113–114, (Boyd-Graber et al., 2006) – WordNet… · p. 114, (Yang and Powers, 2006) – verb… · p. 114
WS Sim (Agirre et al., 2009) – cross-lingual word similarity; (Pakhomov et al., 2010) – word similarity/relatedness; ConceptSim (Schwartz and Gomez, 2011) – concept similarity
- WS Sim adapts Rubenstein and Goodenough and WordSim353 pairs for Spanish–English evaluation; Pakhomov et al. rate medical terms, and ConceptSim maps words to WordNet synsets.The WS Sim translators agreed on 72% and 84% of the respective pairs. Pakhomov et al. provide separate sets for similarity and relatedness, while ConceptSim retains agreed disambiguations by two annotators.Source: WS Sim (Agirre et al., 2009) –… · p. 115, (Pakhomov et al., 2010) – word… · p. 115, ConceptSim (Schwartz and Gomez, 2011)… · p. 115
(Radinsky et al., 2011) – word relatedness; (Ziegler et al., 2012) – concept relatedness; SCWS (Huang et al., 2012) – word similarity in context; RW (Luong et al., 2013) – Rare word relatedness
- Radinsky et al. provide 280 word pairs; related benchmarks cover 771 pairs, concept or instance pairs, contextual word pairs, and rare-word pairs.Mturk-771 uses at least 20 ratings per pair and quality-control items. Ziegler et al. use two sets of 25 and 30 concept/instance pairs; SCWS rates 2,003 word pairs in sentence contexts, and RW rates 2,034 rare-word pairs.Source: (Radinsky et al., 2011) – word… · p. 115, Mturk-771 (Halawi et al., 2012) – word… · p. 116, (Ziegler et al., 2012) – concept… · p. 116
MEN Test (Bruni et al., 2014) – word relatedness; Sim-Lex-999 (Hill et al., 2014) – word similarity; (Baker et al., 2014) – verb similarity
- MEN uses comparative judgments for 3,000 word pairs, Sim-Lex-999 rates similarity for 999 pairs, and Baker et al. provide relatedness scores for 143 verb pairs.MEN converts 50 binary comparisons per pair into a 0–50 score. Sim-Lex-999 averages 50 ratings per pair on a 0–7 scale; Baker et al. use ten raters per pair on a 0–10 scale.Source: MEN Test (Bruni et al., 2014) – word… · p. 117, Sim-Lex-999 (Hill et al., 2014) – word… · p. 118, (Baker et al., 2014) – verb similarity · p. 118
The Semantic Textual Similarity campaign and related datasets
- The STS campaign evaluates sentence similarity as semantic equivalence, using participant ratings on a 0–5 scale; related resources cover definitions, documents, and paraphrases.STS datasets draw on paraphrase and machine-translation pairs, and the campaign includes cross-level evaluation. Li et al. assess sentence definitions, Lee et al. rate document pairs, and the MSR Paraphrase benchmark can be enriched with graded relatedness judgments.Source: The Semantic Textual Similarity… · p. 119
- The Microsoft research Video Paraphrase Corpus uses 120k descriptions collected via Amazon Mechanical Turk for 2000 videos to generate 1500 sentence pairs from sentences describing the same or different videos.Source: The Semantic Textual Similarity… · p. 119
Other evaluations not based on human ratings
- Indirect evaluations assess how a measure affects system performance, including disambiguation, biomedical grouping or interaction tasks, and correlations between ontology annotations and DNA-sequence similarity.Direct evaluation compares measure scores with expected human ratings; system evaluations commonly use accuracy and precision.Source: Other evaluations not based on human… · pp. 119–120
4.4 Discussions
- Measure selection must account for the use case, available resources, and constraints; no generic turnkey solution fits every context.Existing benchmarks and implementations can support fine-grained evaluation and tuning, while unrepresentative use cases may require new datasets.Source: §4.4 · p. 120
- Evaluation remains an open challenge: current datasets could better capture individual uncertainty, imprecise scores, pairwise comparisons, and unequal difficulty across concept pairs.The text notes that datasets often summarize assessments as averages and use finite linear scales such as 0–4.Source: §4.4 · p. 120
- The field needs reproducible shared evaluation platforms and systematic comparisons, including the STS campaign and initiatives.These efforts could help compare proposals and build general knowledge about their benefits and drawbacks.Source: §4.4 · pp. 120–121
4.4. DISCUSSIONS
- Benchmarks need scrutiny of contextualized versus ambiguous-word similarity and differences among nouns, verbs, and adjectives, since word-pair datasets leave sense selection to participants.That participant-led sense selection can affect evaluation quality and potentially its meaning.Source: §4.4 · p. 121
- Results based on datasets with only dozens of pairs, such as Miller and Charles, may yield biased or questionable conclusions, so larger datasets need further evaluation.Source: §4.4 · p. 121
- Improved evaluation will support better comparisons and more precise characterization of the dimensions encompassed by semantic similarity, proximity, and distance.Source: §4.4 · p. 121
Conclusion and research directions
The book surveys semantic measures for comparing language units and ontology entities by meaning, including corpus-based and knowledge-based approaches. It closes by identifying research directions for characterisation, tools, and context-specific use.
Conclusion and research directions
- The book surveys semantic measures for comparing language units and ontology entities by meaning, covering corpus-based and knowledge-based approaches.Source: Conclusion and research directions · p. 123
- The book is an introduction rather than an exhaustive treatment, leaving evaluation and context-specific measure selection outside detailed discussion.Source: Conclusion and research directions · p. 123
- Research priorities include better characterising semantic measures, developing theoretical and software tools, standardising ontology handling, and improving compositionality models.Source: Conclusion and research directions · p. 123
- Further directions include studying language-specific aspects of semantic measures and promoting interdisciplinary work.Source: Conclusion and research directions · p. 123
- The agenda also calls for studying measure complexity and supporting context-specific method selection.Source: Conclusion and research directions · p. 123
Research directions
The book identifies research directions for better understanding, evaluating, and applying semantic measures. These include characterising measure semantics, expanding shared resources and tools, and addressing challenges in ontology handling, language, complexity, and context-specific use.
Better characterise semantic measures and their semantics
- Measure semantics depend on the semantic proxy, mathematical properties, and evidence and assumptions; ad hoc benchmarks often cannot predict behavior in other contexts.A deeper characterisation can support selection for particular uses and inform cognitive models of similarity.Source: Better characterise semantic measures… · p. 124
- Designers should characterise measures more fully, while communities should define quality relative to usage context and investigate properties such as algorithmic complexity.The literature has discussed complexity poorly, despite its importance in concrete applications.Source: Better characterise semantic measures… · pp. 124–125
Develop datasets
- Benchmarks should grow across domains, include conceptualised word-to-word pairs for knowledge-based measures, and use indirect evaluations beyond human judgments.Existing benchmarks are often small and based on limited pools of subjects.Source: Develop datasets · p. 125
Develop generic open-source software
- Generic open-source software, standard tests, and shared evaluation frameworks are needed to support broad comparisons and reproducible results across ontology-based measures.Ontology-specific implementations can be difficult to reuse, and different implementations may produce different results.Source: Develop generic open-source software · pp. 125–126
Develop theoretical tools
- Theoretical unification can group independently proposed knowledge-based measures into general forms and help analyse which core elements affect accuracy in a specific context.The book also calls for more theoretical tools to deepen analysis of measure families and their limitations.Source: Develop theoretical tools · pp. 126–127
Standardise ontology handling
- Ontology-to-graph transformations need clearer documentation and standardisation because reductions can affect results and hinder reproducibility.Some expressive logic-based ontologies cannot be mapped to semantic graphs without reductions.Source: Standardise ontology handling · p. 127
Compositionality for comparing large units of language; Corpus-based semantic measures and language specificities
- Further work should address semantic composition for sentences and documents, as well as evaluation and development of semantic measures for multiple languages.Language-specific processes may also be studied to improve accuracy.Source: Compositionality for comparing large… · p. 127, Corpus-based semantic measures and… · p. 127
Promote interdisciplinarity; Study the algorithmic complexity of semantic measures; Support context-specific selection of semantic measures
- Interdisciplinary collaboration, complexity analysis, and large-scale comparisons can help guide measure selection toward application-specific needs.Users may prefer lower accuracy when it substantially reduces computational time and resources.Source: Promote interdisciplinarity · p. 128, Study the algorithmic complexity of… · p. 129, Support context-specific selection of… · p. 129
Examples of syntagmatic contexts
The examples show how different definitions of context produce different word representations: contexts may be sentences or local sliding windows, with word similarities illustrated through two-dimensional projections.
Examples of syntagmatic contexts
- With sentence contexts, each word is represented by a vector whose dimensions are sentences, with a cell set to 1 when the word occurs in that sentence.The example uses a word-sentence matrix and projects similarities between word vectors with MDS.Source: Examples of syntagmatic contexts · p. 131, Figure A.1 · p. 132
- A word-word matrix can count every pair of words co-occurring within each sentence, yielding a representation based on sentence-level context.The corresponding MDS figure shows a two-dimensional projection of similarities between word vectors.Source: Examples of syntagmatic contexts · p. 131, Figure A.2 · p. 133
- A five-word sliding window uses two words on each side of the focal word and updates co-occurrence counts as it advances one word at a time.Its word-word matrix supplies vectors whose similarities are projected in Figure A.3.Source: Examples of syntagmatic contexts · pp. 131–132, Figure A.3 · p. 134
A brief introduction to Singular Value Decomposition
The section introduces SVD as a way to reduce large, sparse distributional matrices while retaining dimensions with high variability. It also distinguishes distributional models that represent word meaning from language models designed for prediction tasks.
A brief introduction to Singular Value Decomposition
- SVD factorizes high-dimensional, sparse matrices into a lower-dimensional subspace that preserves maximal variability, reducing model size, noise, and sparsity.The resulting dimensions are linear combinations of original dimensions; around 300 dimensions are generally used for word representations.Source: A brief introduction to Singular Value… · p. 135, Figure B.1 · p. 136
- SVD reduction can lose information by projecting distinct words to the same vector, and its high algorithmic complexity can constrain large-model processing and require recomputation after updates.The section points readers to technical treatments of SVD and its use in distributional models.Source: A brief introduction to Singular Value… · p. 135
A brief overview of other models for representing units of language; Language models
- Distributional models derive word representations from usage statistics, whereas language models are designed to answer prediction tasks such as likely next words or likely contexts.Such language-model predictions can support tasks including question answering, spellchecking, and disambiguation.Source: Language models · p. 137, A brief overview of other models for… · p. 137
N-gram models
The section surveys corpus-based language models that derive word representations from usage, from n-gram probabilities to learned neural representations and compositional models. It also notes practical limits in data requirements and software support.
N-gram models
- N-gram models estimate next-word probabilities from observed context sequences and can turn conditional probabilities into word vectors, but their parameter space grows exponentially with n.Unobserved n-grams receive zero likelihood without smoothing, and the approach depends on very large corpora.Source: N-gram models · p. 138
Neural network models
- Neural network language models generalize across similar contexts by projecting observations into a low-dimensional space and sharing parameters, while requiring greater computation than n-grams.Their learned word representations can be extracted for comparison; they can achieve efficient results with reduced input datasets.Source: Neural network models · pp. 139–140
Compositionality: representing complex units of language
- Compositional models combine constituent word representations to represent phrases and sentences, but commutative linear operations miss ordering, syntax, and negation.Nonlinear and parse-tree approaches address some of these limits, while more complex models can produce high-dimensional representations with many parameters.Source: Compositionality: representing complex… · pp. 140–141
Software tools and source code libraries
- The book describes software libraries as useful for measure use and evaluation, while noting that no extensive tool federates the field’s communities and many tools serve particular contexts.The appendix orders tools using subjective criteria and reports information gathered in January 2015.Source: Software tools and source code libraries · p. 143
Tools for corpus-based semantic measures
The section surveys software and libraries for corpus-based semantic measures, including tools for comparing words or sentences, accessing models, and conducting evaluations.
- The listed tools compute similarity or relatedness for word and sentence pairs, and most implemented measures analyze syntagmatic relationships.The table identifies tool names, references, object types, tool types, and programming languages; documentation may use “semantic similarity” differently from the book’s definition.Source: Tools for corpus-based semantic measures · pp. 143–144, Table D.1 · p. 144
- DKPro Similarity, Semilar, and NLTK provide varied access to corpus-based measures, models, or interfaces for comparing words and sentences.DKPro includes measures for WordNet synsets; Semilar offers a Java library and GUI, while NLTK includes WordNet measures and a large-scale corpus-based module.Source: Tools for corpus-based semantic measures · p. 144
- Gensim and SemSim provide distributional-model functionality, while Disco supports word similarity across multiple languages.Gensim is a Python platform, SemSim can compute distributional models from text, and Disco is a Java library.Source: Tools for corpus-based semantic measures · pp. 144–145
- WikiBrain supports Wikipedia-based relatedness for sentences and pages, while TakeLab Semantic Text Similarity compares sentences and Mechaglot remains an alpha developer version.The section also directs readers to an updated list of web services because links and service support may change.Source: Tools for corpus-based semantic measures · p. 145
Tools for knowledge-based semantic measures
The section surveys libraries and tools for computing knowledge-based semantic measures, noting that many are tailored to specific ontologies while some support generic formats.
- Many knowledge-based tools target particular ontologies or structured terminologies, whereas support for OWL, RDF, or OBO can enable use with any ontology in those formats.The table characterizes tools by supported ontologies and formats, interface or library type, language, and pairwise or groupwise comparison capability.Source: Tools for knowledge-based semantic… · p. 145, Table D.2 · p. 146
- SML supports numerous measures for ontology concepts and concept groups, multiple formats and domain ontologies, parameter tuning, and command-line use.Its library and toolkit are described as optimized for large datasets and fast computation.Source: Tools for knowledge-based semantic… · p. 146
- FastSemSim handles concept pairs and groups and can load OBO ontologies, while SimPack compares ontology concepts and loads RDF or OWL.Other listed tools include SemMF for RDF graph objects and OntoSim for ontology comparison and RDF graph objects.Source: Tools for knowledge-based semantic… · pp. 146–147
- The survey also lists ontology-specific options such as YTEX Semantic Similarity for UMLS, SNOMED-CT, and MeSH, and Disease Ontology packages DOSim and DOSE.WordNet-focused tools include WordNet-Similarity and WS4J; UMLS-Similarity targets UMLS concepts.Source: Tools for knowledge-based semantic… · p. 147
Tools for the analysis of semantic measures
The section describes tools and resources for evaluating semantic measures, including datasets, reproducible experiments, and platforms for comparing measure configurations. It also identifies SML’s project leader and developer.
Tools for the analysis of semantic measures
- DKPro Similarity provides commonly used datasets and experiments for comparing corpus-based measures, and can reproduce the SemEval-2012 best system.CESSM supports evaluation strategies for measures over the Gene Ontology and makes evaluation datasets downloadable.Source: Tools for the analysis of semantic… · pp. 147–148
- SML supports accuracy comparisons of knowledge-based measure configurations by comparing estimated similarity with human expected values.In Figure D.1, surfaces represent accuracy and points correspond to specific configurations; Tversky ratio and contrast models are expressed through alpha and beta parameters.Source: Figure D.1 · p. 148, Tools for the analysis of semantic… · p. 148
S´ebastien Harispe
- Sébastien Harispe led and primarily developed the Semantic Measures Library project, following theoretical and practical work on semantic measures.His thesis analyzed knowledge-based semantic similarity measures and proposed a unifying theoretical framework.Source: S´ebastien Harispe · p. 175