Source-linked AI summary
A Manifold-Aware Topic Modeling Approach via Rank-Based Prototypes
Thiago César Castilho Almeida, Daniel Carlos Guimarães Pedronette
TL;DR
Existing topic models either lack an architectural document anchor or assign representatives after clustering. MARETopic selects K real-document exemplars using ranked neighborhoods on a projected manifold; its variants rank first in seven of 15 dataset–metric cells, and MARETopicCorr beats neural models in Purity and NMI across all three corpora without training.
Problem
Neural topic models lack an architectural document-level anchor, while clustering-based representatives are selected post hoc and HDBSCAN does not directly control topic count.
Source: §1 · p. 1Method
MARETopic projects document embeddings onto a low-dimensional manifold, builds ranked lists, and greedily selects K real corpus documents as topic exemplars.
Source: §1 · p. 2Results
Takeaways & Limitations
Inter-topic MMR narrows the diversity gap to optimal-transport models, and the step can be applied to any competitor.
Source: §5 · p. 9Evaluation covers three moderate-size English corpora; effectiveness on multilingual or domain-specific corpora remains unverified.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Recent topic models leverage pretrained embeddings, but neural architectures produce latent representations without grounding in specific texts, and clustering-based pipelines assign representative documents only post hoc, relying on absolute distances distorted by hubness and anisotropy in high-dimensional spaces. We introduce MARETopic, a training-free framework that casts topic discovery as rank-based prototype selection. After projecting embeddings onto a low-dimensional manifold, MARETopic builds ranked lists encoding ordinal neighborhood structure. A greedy algorithm selects exactly K exemplar documents, real corpus texts, whose neighborhoods cover the corpus. Two variants share this criterion. MARETopic$_\text{Corr}$ scores candidates with a query performance predictor and a rank correlation measure, leading Purity and NMI on the two benchmarks with the most categories, ahead of both neural and clustering-based topic models. MARETopic$_\text{Diff}$ scores them with a rank-based diffusion matrix, needs neither measure, and runs 1.7 to 1.9 times faster. Without a single gradient update, MARETopic leads topic coherence on two of three datasets. A novel inter-topic Maximal Marginal Relevance step raises vocabulary diversity at little cost in coherence. Our code is available at https://github.com/thcastilho/maretopic.
1 Introduction
Existing topic models leave topics without an architectural document anchor or select representatives only after clustering, while high-dimensional distance geometry motivates a rank-based alternative. MARETopic selects exactly K real-document exemplars from ranked neighborhoods; its variants achieve strong benchmark results, and inter-topic MMR raises vocabulary diversity at a minor coherence cost.
1 Introduction
- Neural topics lack an architectural document-level anchor, while clustering-based representatives are post hoc and HDBSCAN does not directly control the number of topics.Human inspection matters because automated coherence scores can disagree with human judgments.Source: §1 · p. 1
- Ranked lists retain ordinal neighborhood structure rather than absolute distances, which are vulnerable to hubness and anisotropy in high-dimensional embedding spaces.The ordering remains unchanged under strictly monotone transformations of the underlying distance function.Source: §1 · p. 2, Figure 1 · p. 2
- MARETopic projects pretrained document embeddings onto a low-dimensional manifold, then greedily selects exactly K real corpus documents whose ranked neighborhoods cover the corpus with reduced redundancy.MARETopicCorr uses a query performance predictor and rank correlation; MARETopicDiff uses a rank-based diffusion matrix.Source: §1 · p. 2
- Inter-topic MMR penalizes vocabulary overlap, increasing topic diversity at a minor cost in coherence; each topic is also anchored to an inspectable corpus document.The exemplar provides a document-level representation rather than a latent-space-only topic.Source: §1 · p. 2
- The two MARETopic variants rank first in seven of 15 dataset–metric cells, while MARETopicCorr beats every neural topic model in Purity and NMI on all three corpora without training.The framework is evaluated on three benchmarks against seven baselines under five metrics.Source: §1 · p. 2
2 Related Work
Related work spans generative and clustering-based topic models, prototype-inspired approaches, and rank-based similarity and manifold learning. MARETopic addresses the remaining gap by using ranked-list neighborhoods for prototype selection in topic modeling.
2.1 Topic Modeling
- Generative models represent topics as vocabulary distributions, while clustering models derive topics from embedding-space groups; neither makes an actual corpus document the topic’s architectural definition.Prototype-inspired ProtoXTM and E-LDA also do not use an actual corpus document as the architectural representation of a topic.Source: §2.1 · p. 3
2.2 Ranking and Manifold Learning
- Hubness and anisotropy challenge absolute-distance neighborhoods, while ordinal comparisons and rank-based measures have shown benefits in geometry recovery, semantic similarity, and outlier detection.Mutual k-NN graphs also outperform standard k-NN graphs in the evaluated document-classification task.Source: §2.2 · p. 3
- Ranked lists encode a query’s relationships to the collection; RaDE uses diffusion and GRaCE uses query-performance prediction with rank correlation to select representative samples.Both methods produce exemplar-anchored embeddings whose dimensions correspond to real data points.Source: §2.2 · pp. 3–4
- Earlier graph-based topic models use graph structure to inform topic distributions rather than select exemplar documents, whereas MARETopic applies greedy rank-based leader selection to topic modeling.Its ranked lists encode ordinal neighborhood relations, and projection sharpens those neighborhoods.Source: §2.2 · p. 4
3 The MARETopic Framework
MARETopic formulates topic discovery as selecting K real-document leaders whose ranked-list neighborhoods cover the corpus while limiting redundancy. It encodes documents on a low-dimensional manifold and applies a shared greedy criterion across two scoring variants.
3 The MARETopic Framework
- The framework has two greedy exemplar-selection variants, Corr and Diff, which produce document–topic affinities and representative word lists using c-TF-IDF with inter-topic MMR.Corr uses query-performance prediction and rank correlation; Diff uses a diffusion matrix.Source: §3 · p. 4
3.1 Problem Formulation
- Each of K selected leaders is a real, inspectable corpus document whose neighborhood defines a topic boundary; selection seeks corpus coverage while minimizing redundancy.The framework uses no probabilistic modeling or neural training.Source: §3.1 · p. 4
3.2 Encoding and Indexing
- Sentence-BERT document embeddings are projected by UMAP, normalized, and indexed to produce ranked lists of corpus documents ordered by increasing distance.The top-k neighbors exclude the query document itself.Source: §3.2 · p. 4
3.3 Rank-Based Topic Discovery
- Both variants greedily select the next leader by balancing candidate centrality against a penalty for neighborhood overlap with already selected leaders.The variants differ in how they define the centrality and redundancy terms.Source: §3.3 · p. 4
MARETopicCorr
MARETopicCorr scores candidate leaders with reciprocal-neighborhood influence and suppresses candidates redundant with previously selected leaders. Its query-performance and rank-correlation components can be replaced independently.
MARETopicCorr
- Reciprocal Density estimates candidate influence by measuring how many of a document’s neighbors are also reciprocal neighbors of one another.The measure is used as the query performance predictor in MARETopicCorr.Source: MARETopicCorr · pp. 4–5
- JaccardMax penalizes redundancy by measuring the maximum Jaccard overlap between two ranked neighborhoods across prefix depths below k.Greater overlap suppresses the candidate’s score.Source: MARETopicCorr · p. 5
- Any query performance predictor or rank-correlation measure can replace Reciprocal Density or JaccardMax without changing the other steps.The framework’s two scoring components are therefore independently replaceable.Source: MARETopicCorr · p. 5
MARETopicDiff
MARETopicDiff builds a sparse affinity graph from ranked lists and diffuses those affinities for exemplar scoring. The selected leaders support document-topic representation, out-of-sample inference, and topic-word extraction with inter-topic diversification.
MARETopicDiff
- With p=2, diffusion propagates local rank affinities to smooth noise; diagonal scores attract candidates, while shared diffusion paths penalize redundancy with selected leaders.The affinity matrix W stores rank positions from the document lists, and the diffusion matrix is A = W^p.Source: MARETopicDiff · p. 5
3.4 Document-Topic Representation and Inference
- After selecting K leaders, each document’s topic scores use JaccardMax for MARETopicCorr or the corresponding A column for MARETopicDiff, with argmax assignments used for extraction and clustering evaluation.The document-topic matrix Θ is computed after selecting L = {ℓ1, …, ℓK}.Source: §3.4 · p. 5
- For a new document, the fitted pipeline projects its embedding into the training manifold and retrieves a ranked list without retraining; diffusion uses one-hop leader affinities instead of re-diffusing.MARETopicCorr directly compares the new ranked list with each leader’s list.Source: §3.4 · p. 5
3.5 Topic Word Extraction
- c-TF-IDF scores each word for a topic using its frequency in that topic’s documents, its total frequency across topics, and the mean token count per topic.The passage defines tf(c, w), tfD(w), and the mean number of tokens per topic.Source: §3.5 · pp. 5–6
- Inter-topic MMR processes topics from largest to smallest and penalizes words by their maximum c-TF-IDF score in already processed, larger topics.λ controls the diversity weight, and the penalty suppresses high-frequency words appearing in larger topics.Source: §3.5 · p. 6
- Because MMR follows fixed assignments and does not depend on the model producing Θ, it cannot change Purity or NMI and can also be applied to clustering-based topic models.λ affects only word-level metrics.Source: §3.5 · p. 6
4 Experimental Evaluation
Across three benchmarks, MARETopic’s results vary by metric and variant; the evaluation also examines computational cost and sensitivity to key design choices.
4.1 Experimental Setup
- MARETopic is compared with seven baselines across 20NG, NYT, and Web of Science using Cv, NPMI, TD, Purity, and NMI, with K = 50 topics and three independent runs.Source: §4 · p. 6, §4.1 · p. 6
4.2 Main Results
- MARETopicCorr achieves the highest Purity and NMI on 20NG and NYT without gradient updates, while Top2Vec leads both metrics on WoS.For Cv, Corr leads on 20NG at 0.6818 and Diff leads on WoS at 0.7198; Top2Vec leads on NYT at 0.6944. Clustering-based methods stay above zero on NPMI, and MMR raises both variants’ TD to 0.85–0.88 versus BERTopic’s 0.77–0.80.Source: Table 1 · p. 7, §4.2 · pp. 7–8
4.3 Computational Cost
- 1.7–1.9×: MARETopicCorr costs more than Diff, though both undercut every neural model on every corpus; indexing takes under 1% of runtime.Because selection uses rank positions rather than distances, the exact index can be replaced by an approximate one without changing the method.Source: §4.3 · p. 8
4.4 Ablation Studies
- The ablations test sensitivity to UMAP projection, neighborhood size k, and the MMR diversity parameter λ.They report Cv, TD, Purity, and NMI except where a design choice cannot affect a metric.Source: §4.4 · p. 8
4.4.1 Effect of UMAP Pre-processing
- UMAP improves Corr clustering on 20NG and NYT but lowers Cv; for Diff, it helps Cv and NMI on 20NG yet harms Purity on NYT and WoS.The authors retain dUMAP = 5 for Corr’s clustering gains on 20NG and NYT, while suggesting disabling UMAP for Diff or coarser-grained corpora.Source: §4.4.1 · p. 8
4.4.2 Neighborhood Size (k)
- Purity and NMI rise steeply through k = 100 for both variants; Diff keeps gaining beyond 100, while word-level metrics vary by at most 0.06 for k ≥ 50.The authors choose k = 100 as Corr’s clustering-curve elbow; Corr’s cost grows with k because JaccardMax scans every prefix l < k.Source: §4.4.2 · pp. 8–9, Figure 3 · p. 9
4.4.3 MMR Diversity Parameter (λ)
- At λ = 0.3, TD rises roughly +0.2 on every dataset while Cv drops at most 0.021; at λ = 0.7, TD nears FASTopic’s ≈0.99 with a larger Cv loss.Purity and NMI are unchanged because MMR acts on word lists after cluster assignments are fixed.Source: Table 3 · p. 9, §4.4.3 · p. 9
5 Conclusion
MARETopic uses rank-based prototype selection to anchor topics in real documents, and its performance depends on corpus structure. The conclusion identifies evaluation and scaling directions for future work.
5 Conclusion
- MARETopic ranks first in seven of fifteen dataset–metric cells, and MARETopicCorr beats every neural topic model in Purity and NMI on all three corpora without training.The framework uses ranked-list neighborhoods over a low-dimensional manifold and anchors each topic to an inspectable corpus document.Source: §5 · p. 9
- On WoS, Top2Vec achieves higher Purity and NMI; the authors associate MARETopic’s advantage with more irregular collections.They describe future work on hierarchical and dynamic topics, downstream and intrusion-based evaluation, and larger and multilingual corpora.Source: §5 · p. 9
Limitations
The evaluation and practical claims are bounded by the datasets, scale, hardware, and downstream tasks examined. Broader applicability and downstream impact remain unverified.
Limitations
- Evaluation covers three English-language corpora of 9K–19K documents and 7–20 categories; multilingual and domain-specific effectiveness remains unverified.The authors also note that very large collections (100K+ documents) were not evaluated, and would require an approximate index.Source: Limitations · p. 10
- Runtime comparisons are hardware-limited because neural baselines ran CPU-bound; the authors claim no-training and no-accelerator architecture, not faster wall-clock time.The added nearest-neighbor index has negligible measured cost, but scales with corpus size.Source: Limitations · p. 10
- Topic Diversity remains below generative models such as FASTopic, while inter-topic MMR narrows the gap; downstream evaluations are still needed to measure practical impact.The authors suggest tasks such as document classification and word/topic intrusion.Source: Limitations · p. 10
A Transferability of the Inter-Topic MMR
Inter-topic MMR transfers to BERTopic by re-ranking its c-TF-IDF word lists, substantially increasing Topic Diversity with limited coherence costs. With the same word pipeline, MARETopicCorr retains higher Cv on two corpora, while NYT results are tied within standard deviations.
A Transferability of the Inter-Topic MMR
- Topic Diversity rises by 0.132–0.172 for BERTopic, reaching 0.92–0.94 from 0.77–0.81, at a cost of at most 0.037 in Cv.On NYT, Cv improves by 0.010; diversified BERTopic reaches above the 0.85–0.88 that MARETopic itself reaches on the same corpora.Source: A Transferability of the Inter-Topic MMR · p. 12
- With c-TF-IDF followed by inter-topic MMR at λ = 0.3, MARETopicCorr reaches Cv 0.6818 against BERTopic’s 0.5991 on 20NG and 0.7150 against 0.6531 on WoS.This comparison gives both methods the same word pipeline.Source: A Transferability of the Inter-Topic MMR · p. 12
- On NYT, Cv is 0.6496 for MARETopicCorr against 0.6445 for BERTopic, a margin within either measurement’s standard deviation and therefore read as a tie.The comparison uses the same c-TF-IDF-plus-MMR word pipeline.Source: A Transferability of the Inter-Topic MMR · p. 12
B Number of Topics
At K = 100, MARETopicCorr retains its Purity and NMI lead across the three corpora among methods meeting that topic budget. The comparison also notes Top2Vec’s lower returned topic counts and metric changes as K increases.
B Number of Topics
- At λ = 0, the diversification step reproduces BERTopic’s native word lists, and Purity and NMI stay unchanged because it acts after Θ is fixed.Table 4 describes diversification applied to BERTopic’s c-TF-IDF, with deltas against its native lists.Source: Table 4 · p. 13
- At K = 100, MARETopicCorr still leads Purity and NMI on all three corpora among methods reaching that budget.MARETopicDiff is runner-up on NYT and WoS; Table 5 reports K = 100 values alongside changes from K = 50.Source: Table 5 · p. 13, B Number of Topics · p. 13
- Top2Vec returns 84, 80, and 71 topics when 100 are requested, so its results are not measured at the same budget as the other methods.Its count follows embedding-space density, and its reduction step merges topics but does not split them.Source: B Number of Topics · p. 13
- As K doubles, Cv falls for 23 of 27 method–corpus pairs and TD for 24, while Purity rises for all but four.The passage attributes the coherence and diversity pattern to a fixed vocabulary being divided among twice as many topics.Source: B Number of Topics · p. 13
- The cited discussion frames the issue as resolution rather than diffusion scoring, and evaluates greedy-selection quality at K = 100 after Section 4.2’s standard K = 50 setting.Because MARETopic selects topics greedily, raising K extends the sequence already begun rather than refitting from scratch.Source: B Number of Topics · pp. 12–13
C Encoder Robustness
The encoder tests show stable performance across the evaluated SBERT backbones, while the qualitative examples and held-out affinity visualization illustrate topic content and assignment structure.
C Encoder Robustness
- Across SBERT encoders, MARETopicCorr varies by at most 1.4 pp in Purity and 1.1 pp in NMI; MARETopicDiff’s spreads reach 4.3 pp and 2.6 pp, respectively.Topic diversity varies by up to 2.3 pp, while all encoders keep TD in the 0.85–0.88 range; no encoder dominates every setting.Source: C Encoder Robustness · pp. 13–14, Table 6 · p. 14
D Qualitative Topic Examples
- The ten representative 20NG topics span computing, sports, law enforcement, vehicles, and medicine, with concise keywords and little vocabulary overlap.Figure 4 presents each topic through a leader document and its top-five c-TF-IDF terms.Source: D Qualitative Topic Examples · p. 14, Figure 4 · p. 14
E Document–Topic Matrix Visualization
- On 7,532 held-out 20NG test documents, the affinity matrix has block-diagonal structure, with leaders attracting documents from corresponding classes and near-zero affinity elsewhere.The matrix is produced by a single transform() call without retraining; Figure 5 colors JaccardMax affinity and marks ground-truth class boundaries.Source: E Document–Topic Matrix Visualization · p. 14, Figure 5 · p. 15