Source-linked AI summary
PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes
Yang Ba, Michelle V. Mancenido, Rong Pan
TL;DR
Semantic redundancy and homogenization can narrow the coverage of generated and curated datasets, while existing metrics provide limited characterization of distribution shape. PLDiv summarizes persistence landscapes as a scalar measure of structural diversity. Across text, image, instruction, and reasoning settings, it tracks diversity changes and detects collapse, with its use bounded by what the embedding representation measures.
Problem
Generated and curated datasets can become semantically redundant, and existing diversity metrics provide limited characterization of the shape of the underlying data distribution.
Source: §1 · pp. 1–2Method
PLDiv is a reference-free measure that integrates persistence-landscape area to summarize structural features in a dataset as a scalar.
Source: §1 · p. 2Results
PLDiv tracks diversity changes in text generation and image-class coverage and detects collapse from near-duplicate instructions and homogenized reasoning trajectories.
Source: §1 · pp. 2–3, §6 · p. 11Takeaways & Limitations
The results support PLDiv as an exploratory data-analysis tool for structural complexity, semantic coverage, and collapse.
Source: §6 · p. 11PLDiv does not measure data quality, factuality, fairness, demographic representation, or the social value of examples, so the authors recommend using it alongside other analyses.
Source: Broader Impact Statement · p. 12
Abstract
from arXiv · showhide
Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets grow, they often add semantically redundant samples rather than expanding semantic coverage, which deteriorates the quality of the whole dataset. Existing diversity metrics primarily summarize lexical variation, entropy, or pairwise similarity, but provide limited characterization of the shape of the underlying data distribution. In this paper, we propose PLDiv, a diversity metric derived from persistence landscapes that quantifies the structural organization of a dataset from its geometric representation. PLDiv summarizes how data neighborhoods evolve as nearby samples progressively connect, producing an interpretable scalar measure of structural diversity without requiring task-specific supervision. Whether measuring intrinsic diversity in text generations or image embeddings, or detecting semantic collapse in instruction datasets and reasoning trajectories, PLDiv performs competitively against evaluated baselines and aligns strongly with diversity labels during semantic collapse. Consequently, it effectively supports applications in synthetic data monitoring, dataset construction, data augmentation, and generative model evaluation.
1 Introduction
The introduction frames semantic homogenization in generated and curated datasets as a challenge that existing diversity measures do not fully capture. It introduces PLDiv, a geometry-aware measure, and outlines evaluations across structural, semantic, and collapse settings.
1 Introduction
- Synthetic and curated datasets can contain near-duplicates that limit semantic coverage, while generated responses may converge in phrasing, topics, and reasoning patterns.The introduction also describes homogenization as potentially self-reinforcing when uniform generations enter later training corpora.Source: §1 · pp. 1–2
- Lexical and embedding-based diversity metrics can miss semantic collapse or fail to characterize distribution shape, including local clusters versus broader coverage.Lexical measures focus on token overlap; embedding-based measures summarize entropy, scale, or separability.Source: §1 · p. 2
- PLDiv is a reference-free, geometry-aware scalar that integrates persistence-landscape area, with each feature contributing one quarter of its squared lifetime.Its 0-dimensional form summarizes merging components, while its 1-dimensional form summarizes persistent loops.Source: §1 · p. 2
- The evaluation spans geometric changes, text and image diversity, instruction redundancy, and homogenized reasoning trajectories, with comparisons to several classes of diversity measures.The stated contributions also include axiomatic characterization and a scalable sparse estimator for 0-dimensional PLDiv.Source: §1 · pp. 2–3
2 Related Work
Prior diversity measures capture lexical variation, embedding-based summaries, or topological structure, but the related work identifies limited characterization of local and global data geometry. PLDiv is positioned as a reference-free diversity measure based on persistent homology.
2.1 Diversity Measurement
- Distinct-n, Self-BLEU, ROUGE-L, and related lexical methods capture form-level variation but do not directly characterize meaning across embedding spaces.Human evaluations motivate semantic metrics such as NLI-based diversity and dataset quality indicators.Source: §2.1 · p. 3
- Embedding metrics include reference-based FID and MAUVE, dispersion measures, spectral entropy, separability, and effective-size estimates, but global summaries can obscure local versus global structure.Reference-based methods can mix fidelity with diversity; identical statistics can also mask distinct geometric structures.Source: §2.1 · p. 3
2.2 Persistent Homology
- Persistent homology records births and deaths of connected components, loops, and voids across scales as barcodes or persistence diagrams.It provides a stable multiscale description of point-cloud structure.Source: §2.2 · p. 3
- Prior topological generative-model evaluations compare generated samples with reference data, whereas PLDiv is formulated as a reference-free dataset diversity measure.The related work also connects Vietoris–Rips H0 total persistence to a power-weighted minimum-spanning-tree functional (Steele, 1988; Hero & Michel, 1999; Cohen-Steiner et al., 2010).Source: §2.2 · pp. 3–4
3 Preliminaries
The preliminaries describe persistent homology as tracking topological features across a filtration of a point cloud. Persistence landscapes transform persistence-diagram information into functions that can support statistical summaries.
3.1 Persistence Diagrams
- A filtration grows a nested sequence of complexes as the distance threshold increases, tracking the appearance and disappearance of components, loops, and voids.In the Vietoris–Rips convention used here, the threshold ε corresponds to balls of radius ε/2; feature lifetimes are death time minus birth time.Source: §3.1 · p. 4
3.2 Persistence Landscapes
- Persistence diagrams represent features by birth–death pairs, while persistence landscapes embed diagram information as a sequence of functions in a Banach space.At each t, the first landscape function records the largest tent value, the second the next largest, and so on.Source: §3.1 · p. 4, §3.2 · pp. 4–5
4 Methodology
The methodology defines PLDiv as an area-based scalar summary of persistence landscapes and establishes its diversity axioms. For H0, it uses a sparse minimum-spanning-tree computation to avoid constructing a dense complete graph.
4.1 Diversity Measure via Persistence Landscapes
- PLDiv sums the areas under persistence-landscape layers, equivalently one quarter of the sum of squared lifetimes, and applies separately to each homology dimension.The quadratic weighting follows from the area of each landscape tent rather than a tuned hyperparameter; larger values indicate features persisting across wider filtration ranges.Source: §4.1 · p. 5
4.2 Axiomatic Properties of Diversity
- PLDiv satisfies effective size and monotonicity: separation increases the score, and uniform metric dilation scales persistence intervals.For fixed point count, the score reflects effective spread; when α > 1 and PLDiv(X) > 0, it strictly increases.Source: §4.2 · p. 6
- Adding an exact duplicate leaves PLDiv unchanged, and reordering data points does not affect its value.The duplicate adds a zero-lifetime pair, while permutation invariance follows from dependence on the metric structure and interval multiset.Source: §4.2 · p. 6
4.3 Scalable Sparse Implementation
- The empirical evaluations use Vietoris–Rips H0 for connected components and H1 for loop-sensitive analysis of reasoning-trace homogenization.Under H0, finite death times coincide with edge weights of the complete metric graph's minimum spanning tree.Source: §4.3 · p. 6
- A symmetrized k-nearest-neighbor graph and its sparse minimum spanning tree avoid materializing the dense complete graph.The passage points to the appendix and Tables 5 and 6 for the estimator, its exactness condition, and empirical validation.Source: §4.3 · p. 6
5 Experiment
Across text, instruction, reasoning, and image settings, the experiments assess whether PLDiv tracks diversity and collapse. PLDiv performs consistently well on text tasks, best tracks the non-redundant fraction in instruction data, and higher-dimensional persistence adds signal for reasoning collapse; image results are consistent across two embedding models.
5.1 Semantic Diversity in Text Embeddings
- PLDiv performs consistently well for temperature-proxy and human ABS-HDS diversity on prompt completion and dialogue, with story completion under ABS-HDS as an exception.Exact duplication affects 58.1% of story response sets and story ABS-HDS is strongly associated with the number of unique responses; PLDiv is insensitive to repeated copies of an already represented response.Source: §5.1 · pp. 7–8, Figure 2 · p. 7
5.2 Data Near-Duplicate Collapse Detection
- PLDiv achieves the strongest Pearson correlation with the non-redundant fraction among evaluated metrics as instruction-data near-duplicate replacement increases.The benchmark holds dataset size and task-category distribution fixed while progressively increasing semantic redundancy.Source: §5.2 · p. 8, Figure 3 · p. 8
5.3 Homogenization in Reasoning Traces
- PLDiv(H1) and PLDivrel(H1) show superior sensitivity and out-of-fold performance for tracking reasoning collapse, adding geometric signal beyond global dispersion.The benchmark modifies intermediate-state text while preserving trajectory order and shared question–answer endpoints.Source: §5.3 · pp. 9–10
5.4 Diversity in Image Embeddings
- On image embeddings, PLDiv has the highest Pearson correlation with class diversity and consistently outperforms evaluated baselines across Inception V3 and ResNet-18.The experiment varies class composition while fixing total sample size, providing evidence across two image embedding geometries.Source: §5.4 · pp. 10–11, Figure 5 · p. 11
6 Conclusion
PLDiv summarizes structural diversity from persistence landscapes and captures both meaningful-coverage expansion and contraction toward redundancy. Its diagnostic use is bounded by the chosen representation and distance function.
B.1 PLDiv Closed Form
- PLDiv integrates persistence-landscape area into an interpretable scalar reflecting the number and persistence of structural features.The closed form gives each feature a contribution equal to one quarter of its squared lifetime.Source: §6 · p. 11, §B.1 · pp. 16–17
6 Conclusion
- PLDiv tracks diversity changes in text generation and image-class coverage, while detecting collapse from near-duplicate instructions and homogenized reasoning trajectories.The conclusion also reports a strong association with intrinsic dimensionality in controlled PCA experiments.Source: §6 · p. 11
Broader Impact Statement
- PLDiv is intended as a diagnostic for structural diversity and semantic collapse, not as a measure of data quality, factuality, fairness, or social value.Scores depend on the chosen representation and distance function; the paper recommends combining PLDiv with other analyses and human review for consequential curation.Source: Broader Impact Statement · pp. 11–12
- The experiments reuse released datasets and collect no new personal data; automated paraphrase validation may miss errors or biases, so generated material is restricted to controlled evaluation.The paper does not use that generated material for deployment.Source: Broader Impact Statement · p. 12
A Description of Diversity Scores in Comparisons
- MagArea summarizes the magnitude function’s area, while MagDiff compares magnitude functions; DCScore measures sample separability without accounting for dataset geometry or topology.MagArea and MagDiff are described as reference-free and reference-based measures, respectively.Source: A Description of Diversity Scores in… · pp. 15–16
B.2 Axiomatic Properties of Diversity
PLDiv is characterized by effective-size, monotonicity, twin, and symmetry properties, including an upper bound attained by strictly equidistant metric spaces. The paper compares these axioms with competing diversity measures.
B.2 Axiomatic Properties of Diversity
- PLDiv satisfies effective size, monotonicity, the twin property, and symmetry as axiomatic properties of diversity measurement.Effective size reflects greater spread for more separated points; uniform dilation increases PLDiv when α > 1 and the score is positive, while adding an exact duplicate leaves it unchanged.Source: §B.2 · pp. 17, 19, 20
- For a fixed number of points and dataset diameter, PLDiv is bounded above, with equality if and only if the metric space is strictly equidistant.The result uses the condition that all points are born at zero, no merging occurs before distance c, and all components merge at c.Source: §B.2 · pp. 18–19
- PLDiv is invariant to data-point ordering because permutation preserves the metric structure used to construct its persistence diagrams.The persistence diagrams and lifetimes remain identical after reordering.Source: §B.2 · p. 20
B.3 Axiomatic Properties Comparison
- Table 2 reports that all compared methods satisfy effective size and symmetry; PLDiv, MagArea, and DCScore also satisfy monotonicity and the twin property.Vendi Score only partially satisfies the twin property and does not generally satisfy monotonicity.Source: Table 2 · p. 21, §B.3 · p. 21
C.1 Implementation of Gaussian Separation
The Gaussian-separation simulation varies the separation between two fixed base samples while keeping within-cluster noise unchanged.
C.1 Implementation of Gaussian Separation
- The simulation draws two independent base samples A and B in R^50×2 from N(0, 0.08I_2).The samples are reused across separation settings.Source: §C.1 · p. 21
- Separation is tested at s = 0, 0.4, 0.8, 1.2, 1.6, and 2.0.Reusing the base samples keeps within-cluster noise fixed as separation changes.Source: §C.1 · p. 21
- Inter-cluster separation is the only changing factor across the simulation settings.The design holds the base samples constant across values of s.Source: §C.1 · p. 21
C.2 Implementation of PCA Intrinsic Dimension
The PCA experiment varies retained intrinsic rank while holding sample size and ambient dimension fixed, then tests raw and trace-normalized conditions. PLDiv tracks rank strongly in both conditions, and a separate long-tail experiment reports that it predicts diversity as imbalance increases.
C.2 Implementation of PCA Intrinsic Dimension
- The experiment reconstructs data at seven target ranks from 2 to 32, keeping sample size and ambient dimension fixed while retained complexity increases.The 3,000-point, 50-dimensional data have coordinate variances decreasing from 5.0 to 0.1, so added components retain lower-variance structure rather than pure noise.Source: §C.2 · p. 21
- Raw reconstructions let total covariance trace grow with rank, while the normalized condition rescales each reconstruction to unit covariance trace.The two conditions distinguish rank changes from overall scale changes.Source: §C.2 · pp. 21–22, Figure 7 · p. 22
- PLDiv has the highest raw-setting correlation with intrinsic rank (r = 0.993) and remains strong after trace normalization (r = 0.984).Trace normalization raises Vendi Score to r = 0.998, MagArea to r = 0.844, and DCScore to r = 0.800; their weaker raw-setting correlations may reflect scale confounding.Source: §C.2 · p. 22
C.3 Imbalanced Long-tail data
- In D4 synthetic toy long-tail datasets, PLDiv effectively handles imbalanced data and reliably predicts diversity, which increases monotonically, as 20, 40, 60, 80, or 100 outliers are added.Each variant kept the Gaussian core at 200 - noutliers points.Source: §C.3 · p. 22, Figure 8 · p. 23
C.4 Implementation of Text Embeddings
The text-generation evaluation tests whether metrics track temperature and human diversity judgments across three tasks and five embedding models. PLDiv performs well on prompt completion and dialogue response generation, while story completion under human evaluation favors other metrics.
C.4 Implementation of Text Embeddings
- The evaluation uses 1,000 contexts across story completion, dialogue response generation, and three-word completion, with ten outputs per context at varying temperatures.A subset of 200 contexts has mean human diversity scores (ABS-HDS); five embedding models are evaluated.Source: §C.4 · p. 23
- PLDiv performs well for both temperature and human ABS-HDS targets on prompt completion and dialogue response generation, but not equally on human-evaluated story completion.The story-task difference motivates examining exact response repetition.Source: §C.4 · p. 23
- Across embedding models and cosine versus Euclidean/RBF inputs, PLDiv consistently outperforms other metrics on temperature-based diversity tasks.The comparison varies both embedding model and distance or similarity matrix.Source: §C.4 · p. 23
C.4.1 Duplicate Diagnostics for Story Completion under ABS-HDS Evaluation
The story-task analysis links exact repetition with human diversity judgments and finds that PLDiv correlates best with those judgments after duplicate-containing sets are excluded. The accompanying instruction benchmark construction keeps dataset size and category mixture approximately fixed while increasing local semantic redundancy.
C.4.1 Duplicate Diagnostics for Story Completion under ABS-HDS Evaluation
- Exact duplicates occur in 118 of 203 story sets, versus 35 of 203 prompt sets and 53 of 203 response-generation sets; unique-response count is most associated with ABS-HDS for story completion (r = 0.817).The corresponding associations are r = 0.682 for prompt completion and r = 0.622 for response generation.Source: §C.4.1 · p. 23
- A repeated copy adds no positive persistence to PLDiv, whereas replacing a distinct response with a duplicate can lower the score by removing distinct semantic information.Vendi Score and DCScore retain sample multiplicity through their kernel spectrum and softmax separability, respectively.Source: §C.4.1 · p. 24
- After restricting analysis to 85 story sets with ten unique outputs, PLDiv has the highest ABS-HDS correlation across all five encoders.This is a post-hoc sensitivity analysis under the Euclidean/RBF setting.Source: §C.4.1 · p. 24, Table 3 · p. 24
C.5 Implementation of Instruction Near-Duplicate Collapse
- The instruction benchmark retains 400 examples while replacing a fraction of balanced Dolly-15K-category anchors with same-category near neighbors, keeping category mixture approximately fixed.When a category lacks eligible external neighbors, remaining replacements are sampled with replacement from retained anchors in that category.Source: §C.5 · pp. 24, 26
- For temperature tasks, PLDiv outperforms other metrics across embedding models and cosine or Euclidean/RBF distance settings.The comparison spans different distance/similarity matrices.Source: Figure 10 · p. 25
- Nearest-neighbor replacements average cosine similarity around 0.5 to anchors, versus around 0.1 for random same-category pairs, validating the intended local-redundancy manipulation.The experiment varies collapse level ρ over 0, 0.1, 0.25, 0.5, and 0.75 with ten repeats; its validation target is 1 − ρ.Source: §C.5 · p. 26, Table 4 · p. 26
C.6 Implementation of Reasoning Homogenization
The reasoning-homogenization protocol changes semantic links between alternative and prototype reasoning steps while holding graph topology fixed, and evaluates metric responses across problems. The image and scalability implementations vary class coverage and use sparse kNN graphs to extend PLDiv to larger embeddings.
C.6 Implementation of Reasoning Homogenization
- The reasoning evaluation uses 90 problems, all with correct final answers, and compares metrics between no intervention and full intervention.For prediction, grouped 10-fold cross-validation keeps all intervention levels from each held-out problem together; scores are averaged across ten replacement-order trials.Source: §C.6 · pp. 26–28
- Reasoning homogenization replaces texts in a randomized fraction of alternative-branch steps while preserving graph structure and temporal order.Replacement texts are audited for distinctness, preservation of the prototype’s canonical number set, and consistency with the reasoning state; intervention levels are p = 0, 0.25, 0.5, 0.75, and 1.0.Source: §C.6 · p. 27
C.7 Implementation of Image Embeddings
- The image experiment holds sample size at 750 while varying the number of represented digit classes from 1 to 10.It uses ten randomized class orders and compares metrics on Inception V3 and ResNet-18 embeddings.Source: §C.7 · p. 28
C.8 Scalable Sparse PLDiv
- Sparse PLDiv values remain nearly unchanged for k = 10, 20, 50, and 100 across image-embedding sample sizes from 5k to 40k.The reported experiments suggest k = 20 provides a sufficient approximation.Source: §C.8 · p. 29, Table 5 · p. 29
- Dense exact PLDiv forms all pairwise distances, requiring O(n^2) memory, whereas sparse PLDiv stores O(kn) edges and computes the MST in O(kn log n) time after graph construction.Approximate nearest-neighbor construction can further reduce front-end search cost; exact brute-force kNN may still require pairwise search.Source: §C.8 · p. 29
D Computational Environment
- The experiments ran on an AMD EPYC 7413 24-Core server with an NVIDIA A100-80GB GPU, using Python 3.11 and Sentence Transformers for text embeddings.Source: D Computational Environment · p. 30