Source-linked AI summary
Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference
Tim Wientzek
TL;DR
Large-scale phylogenetic inference is limited by the computational cost of analyzing thousands of languages and the labor-intensive creation of cognacy-based character matrices. This paper learns lexical representations from raw multilingual lexical data using a contrastive objective, achieving phylogenetic inference competitive with existing automated methods while running in minutes on a standard notebook GPU. The generalization of these representations to further tasks without adaptation remains open.
Problem
Phylogenetic inference becomes computationally substantial for thousands of languages, while creating cognacy-based character matrices is time-consuming and labor-intensive.
Method
The paper presents a neural network that learns lexical representations by organizing phonetically and phonotactically similar word forms into a coherent space.
Results
Phylogenetic inference from the learned representations is competitive with existing automated methods and runs in minutes on a standard notebook GPU, versus over 100 hours for an MSA-based workflow on a 144-core server.
Takeaways & Limitations
The approach provides a neural representation of lexical data for computationally efficient phylogenetic inference.
Takeaways & Limitations
Whether the representations generalize well to further tasks without additional adaptation remains an open question.
Abstract
from arXiv · showhide
Computational phylogenetics has become an essential tool in historical linguistics, yet its application at a global scale remains constrained by two factors: the labor-intensive manual annotation of cognacy judgments required for character-based methods and the substantial computational cost of inference on large datasets. This paper introduces a fully self-supervised contrastive learning framework that learns lexical representations directly from raw IPA-transcribed wordlists, without requiring cognacy annotations, alignments, or additional expert input. The model employs a dual contrastive objective: a word-level loss that organizes phonetically similar forms into a coherent space, and an auxiliary language-level loss that encourages the lexical space to reflect broader phonological properties of languages. From the resulting word representations, pairwise language distances are derived and used to infer a global phylogenetic tree of 3,399 language varieties. The inferred tree achieves a generalized quartet distance (GQD) to the Glottolog reference tree competitive with multiple baselines, while requiring only minutes of computation on a standard notebook GPU. Furthermore, the same representations capture diachronic concept stability: variance in pairwise distances across languages yields stability rankings that correlate significantly with established rankings. Ablation studies confirm that both the language-level objective and the use of phonetic feature vectors improved the inferred trees topology with regards to GQD. The framework thus provides a computationally efficient and fully automatic alternative for large-scale phylogenetic inference and offers a unified representation supporting downstream analyses at both the language and concept level.
1 Introduction
Large-scale phylogenetic inference is limited by labor-intensive cognacy annotation and high computational costs. The paper addresses these constraints with self-supervised lexical representations supporting both tree inference and concept-stability analysis.
- Motivation: Manual cognacy judgments and character encodings are unavailable for many languages, while inference becomes costly across thousands of languages.These constraints also hinder repeated reconstruction when datasets change or new languages are added.
- Motivation: Automated cognacy and sound-correspondence methods can produce reasonable trees but still require substantial computational resources on large datasets.One automated character-based analysis of more than 3000 languages required several days on a high-performance cluster.
- Prior approaches: Distance-based inference averages lexical-form distances across shared concepts to construct a language-distance matrix and then groups languages by overall lexical similarity.Unlike character-based methods, it does not explicitly model evolutionary history or the timing of innovations.
- Research aim: The paper investigates whether self-supervised lexical representations learned from multilingual lexical data can recover historically meaningful structure across language- and concept-level tasks.The tasks are phylogenetic inference and ranking concepts by diachronic stability.
- Research aim: The same representations retain information for pairwise language distances and concept-level statistics, unlike approaches that discard individual-form information after tree inference.This supports downstream analyses beyond reconstructing a tree.
2 Data & Methods
The study uses filtered IPA-transcribed lexical data and a hierarchical dual-encoder architecture. Word-level and language-level contrastive objectives organize lexical representations for downstream language-distance analyses.
- Data: The dataset contains 3,399 language varieties across 311 families, covering all macroareas, with IPA transcriptions for 210 core concepts.The sample uses selected languages with at least 40 covered concepts per dataset.
- Data: The study uses the full 210-concept set, whereas a comparison workflow restricted inference to its 105 highest-stability concepts.The broader set is assumed advantageous when included concepts are sufficiently stable.
- Word encoder: Each IPA character is encoded as a 39-dimensional phonetic feature vector before projection into an embedding space.The representation captures articulatory and related phonetic features, with augmentations including duplication, Gaussian noise, and Transformer dropout.
- Word encoder: The word encoder adds learned concept embeddings, processes augmented sequences with a Transformer, and obtains word representations through masked mean pooling.Concept conditioning supplies information about the meaning associated with each word form.
- Language encoder: Word representations are grouped by language, combined with learned language embeddings, processed by a second Transformer, and mean-pooled into language-level representations.The language encoder is used to shape lexical representations rather than directly in evaluation.
- Contrastive training: Two NT-Xent losses attract augmented views of the same word or language and repel other words or languages in the batch.The total loss is the word-level loss plus a weighted language-level loss, whose auxiliary objective encourages broader language-level phonological information.
3 Results
The learned lexical representations produced phylogenetic trees with competitive GQD, coherent major-family clustering, and significant correlations between lexical and geographic distances. The same representations also yielded concept-stability rankings significantly correlated with established rankings, while ablations supported the contributions of the language encoder, full sample, and phonetic feature vectors.
- Phylogenetic inference: 0.0346 mean GQD was reported across methods, with LDN performing slightly better than the other approaches.
- Phylogenetic inference: 0.013 MAD for the learned-representation tree compared with 0.060 for Jäger’s tree indicates greater confidence in rooting and the molecular-clock assumption.
- Phylogenetic inference: Most major language families generally cluster coherently, while Nuclear Trans New Guinea is the most visibly fragmented large family.
- Phylogenetic inference: The tree separates Sino-Tibetan and Austroasiatic languages more cleanly in mainland Southeast Asia, addressing a primary error identified by Jäger.
- Phylogenetic inference: Family-level F1-scores were consistently high, suggesting that GQD differences mainly reflect finer-grained within-family topology rather than broad family misclassification.
- Phylogenetic inference: 0.2021 correlation between pairwise cosine and great-circle distances was significant (p < 0.001), compared with 0.193 for PMI-based distances.
- Concept stability: Significant correlations (p < 0.05) were found between model-derived concept-stability rankings and established rankings.
4 Discussion
The framework learns lexical representations without cognacy judgments and yields globally competitive, fast phylogenetic inference, while also exposing limitations in genealogical interpretation, evaluation, and generalisation.
- Framework and representations: The self-supervised framework learns word-form representations from raw multilingual lexical data, with genealogical signal emerging as a by-product rather than from language-specific parameters.The model organizes phonetically and phonotactically similar forms, while its language embeddings primarily distinguish language views during training.
- Phylogenetic inference: Global phylogenetic inference is competitive with existing automated methods and runs in minutes on a standard notebook GPU instead of over 100 hours for an MSA-based workflow.The approach requires only raw IPA transcriptions, removing cognacy annotation as a prerequisite.
- Limitations: The learned distances retain geographic signal and cannot distinguish inherited phonological similarity from contact-induced convergence without geographic or genealogical information.The authors propose contrastive objectives or contact detection components to separate these sources in future work.
- Evaluation limits: A GQD of 0.035 indicates strong overall quartet agreement, but family-internal structure is less well resolved, with average family-wise GQD about 0.12 for families exceeding 50 languages.The authors retain GQD as a useful overall quality and comparability metric while noting its limited informativeness for family-internal resolution.
- Representation design: 39-dimensional phonetic feature vectors improve tree topology because they place phonetically similar sounds in a structured, linguistically meaningful space.The ablation results support providing this phonetic organization before training, especially in data-sparse settings.
- Concept stability: Variance in learned lexical distances correlates with established concept-stability rankings, although shorter or universally shaped forms can inflate apparent stability through false cognates.The method also benefits from larger concept sets, which provide more overlap and evidence per distance estimate.
5 Conclusion
The paper presents a neural lexical representation framework that captures information about both language relationships and diachronic concept stability. For phylogenetic inference, it combines reduced runtime with improved topological quality, while remaining primarily a proof of concept.
- The neural network learns lexical representations carrying information about language relationships and diachronic concept stability.
- Its main phylogenetic advantages are computational efficiency and substantially reduced runtime compared with previous methods.
- The approach also achieves improved topological quality on the reported evaluations.
- The model is intended primarily as a proof of concept for neural word and language representation and leaves room for further improvement.
- The learned representations may support applications across historical linguistics, natural language processing, and their interface.
Data availability
The project makes its underlying data, source code, model checkpoints, intermediate outputs, final results, and experiment script available. The data are distributed under a Creative Commons Attribution 4.0 International license.
- The project repository contains underlying linguistic data, including word lists, metadata, and stability rankings.
- It provides source code, saved model checkpoints, intermediate processing outputs, and final project results.
- The run_experiment.py script supports reproducing the reported experiments.
- The data are available under the Creative Commons Attribution 4.0 International license.