PaperlayerView exact revision on arXiv

Source-linked AI summary

BERTilda: Explainable Topic Lifecycle Tracking with Split/Merge Detection via Similarity-and-Flow Temporal Graphs

Cláudia Oliveira, Álvaro Figueira

arXiv:2608.18101v1cs.CLcs.LG

TL;DR

Tracking topic change over time requires identifying not only gradual drift but also births, deaths, splits, and merges. BERTilda builds an explainable temporal topic graph from independently discovered topics using semantic similarity and bidirectional document flow, producing lifecycle labels that are often interpretable, especially for continuation and disappearance, while split and merge cases remain more ambiguous.

  • Problem

    Longitudinal topic analysis lacks well-specified methods for identifying structural events such as topic splits and merges alongside gradual drift.

  • Method

    BERTilda independently discovers topics in each time window, links adjacent topics using semantic similarity and bidirectional coverage, and applies graph rules to label lifecycle events.

  • Results

    Lifecycle outputs are often interpretable under human validation, with continuation and disappearance receiving the strongest validation rates while split and merge remain more ambiguous.

  • Takeaways & Limitations

    Temporal topic graphs provide an analyst-facing representation that exposes why split or merge labels were assigned, supporting inspection and error analysis.

  • Takeaways & Limitations

    Threshold calibration remains an open issue, especially for ambiguous split and merge cases, and overlapping windows may inflate continuity by construction.

Abstract

from arXiv · show

Longitudinal text streams exhibit topic birth and death, but also discrete structural reorganizations in which themes split into subtopics or merge into broader narratives. Many dynamic topic models emphasize smooth drift, while snapshot topic models (fit independently per time window) leave temporal correspondence underspecified. We present BERTilda, an explainable framework that discovers topics independently in each window (using an embedding-based topic model) and then constructs a temporal topic graph linking topics across adjacent windows. Links are supported by two complementary signals: (i) semantic similarity between topic representations and (ii) a bidirectional coverage signal that estimates document outflow (where a topic goes) and inflow (where a topic comes from) via cross-window tweet-to-topic attribution. Graph-based rules label continuations, splits, merges, disappearances, and unclear transitions. We evaluate BERTilda on political corpora, including U.S. congressional tweets and historical speech datasets, report topic-quality and temporal-stability diagnostics, and validate lifecycle labels on a gold-standard subset annotated by three independent annotators. On the annotated subset, BERTilda reaches majority agreement rates up to 87% and attains the highest macro-average agreement across the compared methods, with particularly strong disappearance detection relative to similarity-only and forward-only baselines.

1 Introduction · 2 Related Work

BERTilda frames longitudinal topic analysis as graph-based temporal alignment and event labeling, combining semantic similarity with bidirectional document coverage to detect topic lifecycles, splits, and merges. The paper situates this approach among snapshot, dynamic, streaming, cluster-evolution, and evaluation methods, and validates it on political corpora using diagnostics and human annotation.

  • 1 Introduction: Longitudinal political and institutional corpora exhibit topic birth, death, fragmentation into sub-narratives, and convergence into broader narratives beyond gradual drift.Explicitly identifying these structural events supports summaries of shifting and reorganized attention.
  • 1 Introduction: Classical and neural dynamic topic models encourage smooth temporal evolution, whereas snapshot models leave temporal correspondence underspecified and do not naturally produce discrete, auditable split/merge events.The introduction contrasts word-distribution models such as LDA and DTM with latent-representation models such as DETM and independent snapshot approaches.
  • 1 Introduction: BERTilda independently discovers window-specific topics, then builds an explicit temporal graph linking adjacent windows through semantic similarity and bidirectional coverage.The framework instantiates snapshot discovery with BERTopic but is model-agnostic.
  • 2.1 Topic modeling for short and noisy texts: Embedding-based topic models address short-text sparsity by clustering contextual representations or combining embeddings with sparse lexical descriptors, including Top2Vec and BERTopic.BERTopic has also been applied to congressional Twitter discourse.
  • 2.2 Dynamic and online topic models: Dynamic and online topic models encode gradual evolution, birth/death processes, streaming updates, branching or merging dependencies, and relaxed temporal identities.Representative approaches include DTM, Topics over Time, Timeline, online variants, DETM, and chain-free formulations.
  • 2.3 Cluster evolution, split/merge events, and explainability: BERTilda extends similarity-graph and post-processed cluster-evolution methods with an interpretable flow constraint that distinguishes substantive transitions from weak semantic proximity.Document-level coverage helps resolve ambiguous many-to-many matchings in dense embedding spaces.
  • 2.4 Evaluation of topic quality and temporal behavior: Because coherence measures imperfectly reflect human judgments and vary across models and domains, BERTilda combines within-window quality metrics, temporal drift/volatility diagnostics, and human validation of lifecycle labels.The evaluation also draws on prior work concerning contextualized coherence, purpose-oriented evaluation, temporal consistency, stability, and robustness.

3 Method

BERTilda independently discovers topics in timestamped windows, aligns adjacent-window topics with semantic similarity and bidirectional document-flow coverage, and applies auditable graph rules to identify lifecycle events. The resulting temporal topic graph distinguishes continuations, splits, merges, disappearances, and unclear transitions.

  • Pipeline: BERTilda segments timestamped documents into windows, fits a snapshot topic model in each window, and aligns adjacent topics in a bipartite graph.Each document receives a window-specific topic assignment, while topics are represented from their assigned documents.
  • Cross-window alignment: Topic links combine cosine similarity between cross-window topic centroids with bidirectional coverage estimated through document-to-topic attribution.Forward attribution measures outflow from a source topic, while backward attribution measures inflow to a target topic; bidirectionality helps disambiguate merges.
  • Lifecycle labeling: Validated graph edges define successor and predecessor sets used by transparent rules for continuation, split, disappearance, merge target, or unclear transitions.Splits require at least two validated successors with substantial mass flow, merges require at least two validated predecessors with substantial inflow, and disappearance requires no validated successor or negligible total outflow.
  • Temporal graph: The final directed temporal topic graph contains time-indexed topics as nodes and validated cross-window relations as edges.The pipeline is modular: its snapshot topic model may be replaced if it supplies per-window assignments and interpretable topic descriptors.

4 Experiments

Experiments evaluate BERTilda’s snapshot topic quality, temporal diagnostics, and split/merge event detection across three political corpora. Comparisons use topic-modeling, dynamic-topic, and minimal event-detection baselines under shared preprocessing and evaluation conditions.

  • Evaluation questions: Experiments assess topic coherence and diversity, temporal stability, and whether bidirectional document coverage improves split/merge interpretability over similarity-only and forward-only alternatives.The evaluation is organized around four research questions, including snapshot topic quality, event-label interpretability, and stability relative to dynamic topic models.
  • Datasets: The evaluation uses Congress tweets, UN General Debates speeches, and State of the Union addresses, with temporal segmentation defined for each corpus.The congressional corpus contains 357,896 tweets from 544 official accounts of the 119th U.S. Congress, collected from September 26, 2024 to September 22, 2025.
  • Topic discovery: Snapshot topics use BERTopic with MiniLM embeddings and HDBSCAN, discarding topics below corpus-specific document thresholds and representing topics with top-N c-TF-IDF words.The minimum topic sizes are 50 documents generally, 20 for UN Debates, and 1 for State of the Union.
  • Baselines: Baselines include Top2Vec, LDA and NMF for topic quality, DTM and DETM for temporal diagnostics, and similarity-only, lexical-only, and forward-only methods for event detection.Event-detection baselines reuse the same snapshot topics and windows as BERTilda, while all methods share default alignment and event-labeling thresholds and post-processing.
  • Diagnostics: Topic evaluation reports CV, NPMI, UMass, diversity, and an overall topic-quality score, while temporal behavior is summarized with drift and rolling short-term volatility.Automated coherence measures are treated as comparative diagnostics rather than definitive quality measures because of known failure modes.
  • Human evaluation: Three independent annotators evaluated a balanced, curated congressional subset by judging candidate topic correspondences using keywords and representative tweets.For continuation, split, and disappearance items, annotators saw the focal topic and its three most similar candidate successors in the next window.

5 Results

BERTilda produced less-fragmented congressional topic snapshots and achieved the strongest overall event-label validation among the compared methods. Its advantage persisted across threshold perturbations, while a qualitative case study illustrated unclear continuation, splitting, independent continuation, and disappearance.

  • Within-window topic quality: 2,569 topics versus Top2Vec’s 5,717 across temporal windows, with Top2Vec often exceeding 70 topics per week, indicating greater fragmentation.Top2Vec used the same MiniLM embedding model, 10 top words per topic, and a 50-document minimum for fair comparison.
  • Event-label validation: 0.775 macro-average validation rate: BERTilda ranked highest across all four event types, ahead of similarity-only at 0.571 and forward-only at 0.725.Validation used majority agreement across three annotators on 120 annotated events.
  • Event-label validation: Forward-only was slightly higher on split and merge precision, whereas BERTilda performed better on disappearance, consistent with backward attribution normalizing inflow and reducing spurious continuations.The comparison used the same snapshot topics and windows, isolating alignment and labeling mechanisms.
  • Sensitivity analysis: 0.663–0.718 macro-average validated precision for BERTilda versus 0.623–0.669 for forward-only across five threshold perturbation analyses.Disappearance detection showed the most stable advantage; split and merge were more sensitive, without degenerate label distributions under small threshold changes.
  • Qualitative case study: A Holocaust/Israel–Antisemitism case study marked an unclear continuation, followed by a split into geopolitical-conflict and campus-focused antisemitism branches that continued independently until disappearance.The transition occurred from mid-April to mid-May.

6 Discussion and Limitations

BERTilda yields interpretable topic-lifecycle labels, with continuation and disappearance validated more strongly than split and merge. Key limitations concern ambiguity in split/merge cases, threshold calibration, and continuity inflation from overlapping windows.

  • Interpretability and validation: Continuation and disappearance receive the strongest human validation, whereas split and merge labels are more ambiguous.BERTilda represents topic lifecycles explicitly through a temporal topic graph.
  • Interpretability and validation: Merge labels can reflect partial thematic overlap rather than a single unambiguous fusion.This contributes to the greater ambiguity observed for merge cases.
  • Limitations: Threshold calibration remains an open issue, especially for ambiguous split and merge cases, despite sensitivity analyses suggesting conclusions are not tied to one setting.The limitation applies even though local sensitivity analyses indicate robustness across threshold settings.
  • Limitations: Overlapping windows may inflate continuity by construction, motivating ablations with non-overlapping windows or flow computed on window deltas.These ablations are needed to quantify the effect.

7 Conclusion

BERTilda is an explainable temporal graph framework that aligns independently discovered topics using semantic similarity and bidirectional document flow to expose lifecycle events. Its results support interpretable lifecycle analysis while motivating robustness, attribution, and evaluation extensions.

  • Conclusion: BERTilda explicitly tracks topic lifecycles and makes split and merge events visible across independently discovered time-window topics.Its graph-based design addresses temporal correspondence between separately fitted topic snapshots.
  • Conclusion: Combining semantic similarity with bidirectional document flow enables explainable successor/predecessor links and rule-based labels for continuation, split, merge, disappearance, and unclear transitions.The resulting temporal topic graph provides an analyst-facing representation of topic evolution.
  • Conclusion: Document flow, particularly bidirectional coverage, is most helpful for disappearance detection, whereas split and merge detection remain harder and forward-only alignment remains competitive.Across corpora, BERTilda also achieves competitive topic-quality diagnostics and often interpretable lifecycle outputs under human validation.
  • Future work: Future work will test robustness to windowing and thresholds, calibrate against annotated subsets, improve document-to-topic attribution for multi-membership and ambiguity, and broaden domains and event types.The authors also plan to explore alternative transport-based or distributional alignment approaches.

Use of Generative AI

Generative AI supported manuscript preparation through language polishing, bibliography analysis, and formatting assistance. The authors reviewed and edited all AI-assisted content and retained responsibility for the manuscript’s originality and technical correctness.

  • AI-assisted preparation: Generative AI assisted with proofreading and language polishing, bibliography analysis, and formatting and layout, including LaTeX structuring.These uses supported readability, reference organization, and manuscript formatting.
  • Author responsibility: The authors reviewed and edited all AI-assisted content and remained fully responsible for originality and the correctness of technical content, claims, and conclusions.Author oversight covered both the generated material and the manuscript’s substantive claims.

Supplementary Material for BERTilda: Explainable Topic Lifecycle Tracking with Split/Merge Detection via Similarity-and-Flow · 1 Additional historical-corpus diagnostics

The supplement adds SOTU diagnostics and robustness evidence, complementing the main paper’s UN General Debates results. BERTilda remains competitive on historical corpora, exceeding Tomotopy DTM on coherence-oriented metrics while preserving very high diversity and supporting lifecycle analysis over time.

  • 1 Additional historical-corpus diagnostics: The supplement reports historical-corpus diagnostics on the State of the Union collection alongside additional robustness and validation evidence.These results complement, but do not replace, the paper’s core claims.
  • 1 Additional historical-corpus diagnostics: BERTilda remains competitive on SOTU and clearly exceeds Tomotopy DTM on coherence-oriented metrics while preserving very high diversity.This extends the qualitative pattern reported for the UN General Debates corpus.
  • 1 Additional historical-corpus diagnostics: DETM remains highly diverse, but its overall topic-quality profile is close to BERTilda rather than clearly superior.The comparison concerns overall topic quality on the SOTU corpus.
  • 1 Additional historical-corpus diagnostics: The SOTU diagnostics use overall topic-quality metrics reported as mean ± std across windows.This table provides the aggregate historical-corpus quality view accompanying the narrative comparison.
  • 1 Additional historical-corpus diagnostics: Topic quality on SOTU is not static across time for any model, as indicated by temporal drift values defined as drift = β1.The supplement reports these temporal-drift diagnostics separately from overall topic quality.
  • 1 Additional historical-corpus diagnostics: BERTilda does not trade off interpretability for temporal tracking, and its behavior remains stable enough to support lifecycle analysis on long historical corpora.This qualitative conclusion aligns with the main paper’s results despite the different temporal regime of historical corpora.
  • 1 Additional historical-corpus diagnostics: The supplementary comparison also includes static topic-quality metrics on the congressional tweets dataset, linking the historical diagnostics to the main case-study setting.The table is presented as a static-quality reference rather than a new quantitative claim.

2 Additional event-validation results

Under unanimous-vote validation, annotator agreement remained substantial but absolute validation rates fell, particularly for ambiguous splits and merges. BERTilda still led disappearance validation and remained competitive on split and merge cases, supporting a balanced, interpretable event layer rather than eliminating ambiguity.

  • Unanimous-vote validation: Fleiss’ κ = 0.61 and Krippendorff’s α = 0.61 under unanimous voting indicate substantial inter-annotator agreement.Absolute validation rates decreased relative to majority-vote validation, especially for ambiguous split and merge events.
  • Unanimous-vote validation: Unanimous voting provides a stricter lower bound on event interpretability because it rewards only the least ambiguous instances.The criterion explains why validation rates declined most for split and merge cases.
  • Method comparison: BERTilda remains strongest on disappearance validation and competitive on the harder split and merge cases, even under unanimous voting.This stricter analysis reinforces the majority-vote interpretation of BERTilda’s event-labeling performance.
  • Method comparison: BERTilda’s main advantage is a more balanced and interpretable event layer than simpler alignment variants, not the elimination of ambiguity.The result characterizes the method’s comparative strength under the stricter validation criterion.

3 Additional robustness results · 4 Additional examples of temporal topic graphs

Additional robustness analyses show that BERTilda remains superior to the forward-only baseline in macro-average validated precision across plausible local threshold perturbations, with especially stable disappearance detection. Additional political case studies illustrate interpretable temporal graphs across differing granularities and event mixtures.

  • 3 Additional robustness results: Across plausible threshold perturbations, BERTilda remained above forward-only in macro-average validated precision, especially for disappearance detection.Split and merge performance varied more, consistent with greater ambiguity under human evaluation.
  • 3 Additional robustness results: Small local threshold changes did not collapse BERTilda into a degenerate label distribution, although split and merge outcomes varied more.The analysis held preprocessing, windowing, snapshot topics, representations, annotated items, and majority-vote protocol fixed.
  • 3 Additional robustness results: Forward-only produced more disappearances and fewer merges, whereas BERTilda preserved a broader event distribution across tested thresholds.The broader distribution is consistent with bidirectional attribution stabilizing inflow estimates.
  • 3 Additional robustness results: Validation precision is defined on predicted events using unanimous votes from three annotators.This caption specifies the validation-rate convention used in the robustness results.
  • 3 Additional robustness results: The robustness section summarizes macro-average validated precision as min/max/mean/std across tested threshold values and reports event-count ranges.Additional local curves cover the remaining threshold perturbations, including τtopic and αout
  • 4 Additional examples of temporal topic graphs: Two additional political case studies show that graph-based event views remain interpretable across narratives with different temporal granularities and mixtures of continuation, branching, and disappearance.Examples include Town Halls and Constituent Services Assistance, Shooting Event to Political Polarization, and Israel-related topic evolution.
Loading 2608.18101v1…