Source-linked AI summary
Cross-lingual Entity Alignment via Joint Attribute-Preserving Embedding
Zequn Sun, Wei Hu, Chengkai Li
TL;DR
Cross-lingual entity alignment must handle imperfect machine translation and the limited use of attribute information in embedding methods. This paper jointly embeds relationship structures and attribute correlations in a unified space, and experiments report superior results to two state-of-the-art embedding approaches while allowing complementarity with machine translation.
Problem
Cross-lingual alignment methods face imperfect machine translation, while existing embedding models largely leave the substantial attribute information in knowledge bases unexplored.
Method
The model combines structure embedding of relationship triples with attribute embedding of attribute correlations to jointly represent two knowledge bases in a unified vector space.
Results
Experiments on real-world datasets show that the approach achieves superior results to two state-of-the-art embedding approaches and can be complemented by machine-translation methods.
Takeaways & Limitations
Attribute-preserving joint embedding provides an embedding-based route to cross-lingual entity alignment without depending on machine translation.
Takeaways & Limitations
The approach suffers from multi-mapping relations, discards attribute values because of their diversity and cross-linguality, and has not yet been evaluated on more heterogeneous knowledge bases.
Abstract
from arXiv · showhide
Entity alignment is the task of finding entities in two knowledge bases (KBs) that represent the same real-world object. When facing KBs in different natural languages, conventional cross-lingual entity alignment methods rely on machine translation to eliminate the language barriers. These approaches often suffer from the uneven quality of translations between languages. While recent embedding-based techniques encode entities and relationships in KBs and do not need machine translation for cross-lingual entity alignment, a significant number of attributes remain largely unexplored. In this paper, we propose a joint attribute-preserving embedding model for cross-lingual entity alignment. It jointly embeds the structures of two KBs into a unified vector space and further refines it by leveraging attribute correlations in the KBs. Our experimental results on real-world datasets show that this approach significantly outperforms the state-of-the-art embedding approaches for cross-lingual entity alignment and could be complemented with methods based on machine translation.
1 Introduction
Cross-lingual entity alignment integrates knowledge bases but is challenged by imperfect machine translation and underused attribute information. The paper proposes jointly embedding relationships and attribute correlations in a unified space, reporting stronger performance than embedding baselines.
- Motivation: Cross-lingual entity alignment finds entities referring to the same real-world object and supports integrating knowledge bases across languages.The task can help construct coherent knowledge bases and handle different expressions of knowledge across natural languages.
- Motivation: Machine-translation-based alignment remains vulnerable to uneven translation quality, making alignment quality dependent on translation quality.
- Research gap: 58,181,947 attribute triples in English DBpedia are three times as numerous as its 18,598,409 relationship triples.Attributes can provide additional entity information, but their incorporation into cross-lingual embedding models remains largely unexplored.
- Approach: The proposed approach jointly embeds relationship triples through structure embedding and refines entities using attribute-triple correlations.It combines both modules to embed entities from two knowledge bases into a unified vector space.
- Contributions: The model avoids dependence on machine translation and addresses cross-lingual alignment using embedding-based representations.
- Results: Experiments on real-world DBpedia datasets largely outperformed two state-of-the-art embedding-based methods and could complement machine-translation methods.
2 Related Work
Related work spans knowledge-base embedding and cross-lingual knowledge-base alignment. Prior approaches model relationships, types, or selected attributes, while cross-lingual alignment methods commonly rely on translation or seed-alignment-based embedding.
- KB embedding: Knowledge-base embedding research began with translation-based models such as TransE, which represent relationships as translations between head and tail entities.TransE is effective for modeling 1-to-1 relations and knowledge-base completion.
- KB embedding: Some embedding models add type constraints or attribute information, but KR-EAR learns attribute embeddings only within a single knowledge base.
- KB embedding: KR-EAR targets attributes with small value sets and may fail on sparse, heterogeneous attributes such as names, labels, and coordinates.
- Cross-lingual KB alignment: Cross-lingual knowledge-base alignment includes ontology matching and entity alignment, with machine translation integral to the ontology-matching approaches described.
- Cross-lingual KB alignment: MTransE learns language-specific spaces and translations between them using seed alignment, whereas JE embeds knowledge bases into one unified space with similar seed-aligned entities.
3 Cross-lingual Entity Alignment via KB Embedding
The model uses seed alignments to jointly embed two cross-lingual knowledge bases in one vector space, combining relationship structure with attribute correlations. Structure embedding scores triples geometrically, while negative triples and shared seed representations guide the training process.
- Framework: Given two cross-lingual knowledge bases and seed-aligned entities or properties, the framework learns representations while positioning latent aligned entities closely.
- Structure Embedding: Structure embedding models relationship triples with TransE-style translations and shares seed-aligned representations to bridge the two knowledge bases.This creates an overlay relationship graph and jointly learns entity representations in a unified space.
- Attribute Embedding: Attribute embedding abstracts complex attribute values to range types, captures attribute correlations, and refines entity representations alongside structure embedding.The joint model therefore preserves both relationship and attribute information.
- Design motivation: Structure embedding can leave sparse-relationship entities disorganized, motivating attribute-based refinement of the learned representations.
- Structure Embedding: For a relationship triple tr = (h, r, t), structure embedding expects h + r = t and assigns lower scores to more plausible triples.The score function is f(tr) = ∥h + r − t∥2.
- Structure Embedding: Negative triples are generated by replacing either the head or tail with a random entity, encouraging higher scores for corrupted triples.The positive-to-negative weighting ratio α lies in [0, 1].
3.3 Attribute Embedding and Entity Similarity Calculation
The attribute embedding module models correlations among attributes across the two KBs and uses them to learn attribute vectors and entity representations.
- Attribute correlations: Correlated attributes are those commonly used together to describe an entity, with stronger correlation assigned to longitude and latitude because they share a range type.Seed entity pairs establish cross-lingual attribute correlations in both directions.
- Attribute embedding: AE predicts correlated attributes from a given attribute, adapting the Skipgram idea to capture attribute relationships.The model uses range-type information in its objective.
- Optimization: Negative sampling replaces log p(c|a) with an efficient parameterization and prevents all attribute vectors from having the same value.Negative attribute pairs are generated from a log-uniform base distribution and treated as incorrect.
- Range types: The weighting term w_a,c equals 1 for different range types and 2 for matching range types, encouraging same-type attributes to become more similar.The model distinguishes Integer, Double, Datetime, and String range types.
Entity Similarity Calculation
Entity similarity is computed from cosine similarities among entity embeddings, producing cross-KB and within-KB similarity matrices that are thresholded to retain likely matches.
- Similarity matrices: The similarity matrices contain cosine similarities between entity pairs within the same KB or across the two KBs.Each matrix entry S_i,j measures similarity between the corresponding entities.
- Matrix structure: The resulting matrices separately represent similarities within each KB and between KB1 and KB2.The cross-KB matrix has dimensions based on the numbers of entities in the two KBs.
- Thresholding: Entries below the threshold τ are set to zero because low similarity indicates that two entities are likely different.The thresholds are τ = 0.95 for inner similarity matrices and τ = 0.9 for the cross-KB matrix.
3.4 Joint Attribute-Preserving Embedding
The joint model refines structure-based entity embeddings with attribute-based similarity information so that related entities across the two KBs are embedded closely.
- Similarity refinement: The model uses pairwise entity-similarity matrices as supervised information to cluster similar entities and refine their representations.The two inner similarity matrices apply the same mechanism within each KB.
- Hyper-parameters: The hyper-parameter β balances cross-KB similarities against inner-KB similarities in the structure-related objective.This weighting controls the relative contribution of cross-KB and within-KB similarity constraints.
- Structure embedding: Structure embedding accumulates vectors from the other KB according to cross-KB similarities, while inner similarities guide refinement within each KB.This encourages similar cross-KB entities to have close embeddings.
- Joint objective: The joint objective minimizes structure and similarity losses together to preserve both relationship structure and attribute information.The hyper-parameter δ weights the similarity objective O_S.
3.5 Discussions
The paper discusses and analyzes the joint attribute-preserving embedding model from several aspects.
- Discussions: The paper presents subsequent discussion and analysis of its joint attribute-preserving embedding model.The passage introduces the discussion rather than stating a specific finding.
Objective Function for Structure Embedding
The structure-embedding objective distinguishes positive and negative relationship triples while encouraging lower positive and higher negative scores. Training alternates separate optimizers, constrains vector lengths, and uses sparse or divide-and-conquer computation for scalability.
- SE uses a translation-based embedding model, but its objective differs from the margin-based ranking loss used by many previous KB embedding models.
- The selected objective separates positive and negative triples while also assigning lower scores to positive triples and higher scores to negative triples.
- Negative triples help distinguish relations between entities, and experiments demonstrate their effectiveness.
- Training initializes entity, relation, and attribute vectors randomly, then alternates optimizers minimizing the structure and attribute objectives.
- Unit-length embedding constraints prevent trivial norm growth, limit distributional randomness, and align inner products with cosine similarity.
- Sparse matrices and divide-and-conquer computation reduce memory costs, supporting scalable training alongside translation-based structure embedding.
Parameter Complexity
The joint model has parameter count proportional to embedding dimension times the numbers of entities, relationships, and attributes. Because relationships and attributes are much fewer than entities in practice, complexity is roughly linear in total entities, while alignment uses cosine-distance ranking.
- The joint model contains d(ne + nr + na) parameters, where ne, nr, and na denote entities, relationships, and attributes.
- Because nr and na are much smaller than ne in practice, model complexity is roughly linear in the total number of entities.
- With unit-length vectors, cosine distance between entities from the two KBs can be calculated directly.
- Nearest entities are obtained by sorting each distance-matrix row in descending order, with truly aligned targets expected among the highest-ranked candidates.
4 Evaluation
Experiments on real-world DBpedia datasets evaluate JAPE against embedding-based baselines, ablations, machine translation, and larger-scale settings. JAPE benefits from combining structure and attribute information, remains effective with limited seeds, and shows scalability, although performance decreases on DBP100K.
- Results on DBP15K: JAPE largely outperformed JE and MTransE on DBP15K because it captures both structure and attribute information.JE and MTransE have limitations related to vector constraints and separate vector spaces, respectively.
- Results on DBP15K: Negative triples reduce random entity distributions, while attribute embedding further refines them in the ablation study.Together, these improvements demonstrate that systematic entity distributions help cross-lingual entity alignment.
- Results on DBP15K: Attribute embedding narrows alignment-direction differences caused by sparse non-English relationship triples by providing additional information for entity representations.This effect is especially relevant when aligning entities involving sparse knowledge bases.
- Results on DBP15K: PCA visualizations show closely aligned Chinese-English entity pairs and attribute clusters for cellphone, city, university, and country-related attributes.The visualization separates entity categories while grouping aligned entities closely.
- Sensitivity to Proportion of Seed Alignment: 53.27% Hits@1 and 82.91% Hits@10 were achieved on DBP15KZH-EN with half of the gold standards as seed alignment.With 10% seeds, Hits@10 reached 55.04% on DBP15KZH-EN and 44.69% on DBP15KJA-EN.
- Combination with Machine Translation: Combining JAPE with machine translation produced significantly better results than either result alone, indicating mutual complementarity.Machine translation can perform strongly at Hits@1, while JAPE can place correct entities slightly lower when labels are mistranslated.
- Results at Larger Scale: Similar results and conclusions on DBP100K indicate JAPE’s scalability and stability, although all methods’ performance decreases at the larger scale.DBP100K contains sparse entities and more multi-mapping relations, while JAPE still outperformed JE and MTransE.
5 Conclusion and Future Work
The paper concludes that its joint attribute-preserving embedding model improves cross-lingual entity alignment and complements machine-translation methods, while identifying three directions for future improvement.
- The joint attribute-preserving embedding model achieved superior results to two state-of-the-art embedding approaches on real-world datasets.
- Future work will address multi-mapping relations in structure embedding through cross-lingual hyperplane projection.
- Future work will incorporate diverse, cross-lingual attribute values using cross-lingual word embeddings.
- The model should also be evaluated on more heterogeneous KBs, such as DBpedia and Wikidata.