Source-linked AI summary
Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning
Dongyue Wu, Tao Ma
TL;DR
Existing pruning methods compress samples into embeddings that can misalign geometric similarity with fine-grained semantic coverage and provide limited dataset-level interpretability. MCL instead builds sample- and dataset-level concept graphs and greedily selects samples for coverage; experiments report strong performance under aggressive pruning with substantial efficiency gains.
Problem
Embedding-based pruning can misrepresent compositional semantic redundancy and offers limited human interpretability of preserved or discarded concepts.
Method
MCL represents image–text pairs with Entity, Event, and Attribute concept graphs, aggregates them into a dataset-level graph, and greedily maximizes semantic coverage.
Results
Across multimodal instruction tuning and object detection, MCL achieves strong performance under aggressive pruning with substantial data-efficiency improvements.
Takeaways & Limitations
MCL provides a coverage-aware pruning framework that reduces redundancy while preserving semantic diversity and exposing the dataset’s semantic landscape for inspection.
Takeaways & Limitations
MCL’s individual-sample concept extraction may be imperfect, and its coverage objective is typically NP-hard to optimize directly.
Abstract
from arXiv · showhide
Existing data pruning methods predominantly rely on high-dimensional feature embeddings to measure sample importance. However, these compressed vectors often obscure fine-grained semantic interactions, leading to suboptimal coverage of rare semantic concepts in the pruned subsets. In this paper, we propose Mapping the Concept Landscape (MCL), a novel structural perception framework for transparent data pruning. Instead of abstract embeddings, we represent each image-caption pair as an explicit sample-level graph comprising entities, events, and attributes. By integrating these individual graphs into a comprehensive dataset-level graph, we characterize the global distribution of semantic concepts and quantify their rarity across the entire corpus. Based on this structured perception, we develop a greedy concept-coverage maximization algorithm that iteratively selects samples to maximize the marginal gain of high-value, under-represented concepts. Experimental results on various benchmarks demonstrate that our method not only achieves superior pruning efficiency compared to state-of-the-art methods but also provides a transparent and interpretable audit trail for the selection process.
1 Introduction
MCL reframes data pruning as transparent, concept-level coverage optimization rather than opaque embedding-space selection. It represents samples with semantic graphs, aggregates them into a dataset-level landscape, and greedily prioritizes underrepresented concepts while achieving strong efficiency.
- Motivation: Web-scale vision–language datasets create storage, computation, redundancy, low-quality data, and noise challenges that motivate compact representative subsets.Pruning seeks to preserve model performance under strict data budgets.
- Limitations of Existing Methods: Embedding-based pruning assumes geometric similarity reflects semantic redundancy, but compositional concepts can remain distinct despite embedding proximity.Geometrically diverse samples may also overlap in conceptual content.
- Limitations of Existing Methods: Embedding-space decisions obscure which concepts are preserved or discarded, limiting human auditing, inspection, refinement, and reasoning about dataset-wide semantic distributions.This is a dataset-level interpretability limitation of dense numerical representations.
- MCL Framework: MCL maps each image–text pair to an interpretable graph of Entities, Events, and Attributes, then aggregates these graphs into a transparent dataset-level concept graph.The aggregate emphasizes relative frequencies and coverage patterns despite imperfect individual concept extraction.
- MCL Framework: MCL formulates pruning as concept-coverage maximization and greedily selects samples by marginal contribution, prioritizing underrepresented concepts while reducing redundant dominant semantics.The framework moves beyond opaque embedding-based criteria toward structured concept graphs and human-interpretable analysis.
- Reported Outcome: Using less than 10% of training time and selected data, MCL attains performance comparable to full-data baselines.This introduction-level result summarizes the reported pruning efficiency.
2 Related Work
Dataset pruning and coreset selection seek compact subsets that preserve full-dataset performance. Prior work uses training dynamics, losses, gradients, clustering, and distances to centers for sample selection.
- Dataset Pruning and Coreset Selection: Dataset pruning and coreset selection identify compact training subsets intended to preserve full-dataset performance.Prior applications include active learning, noisy learning, and continual learning.
- Dataset Pruning and Coreset Selection: Existing approaches estimate sample importance from training dynamics, loss values, gradients, clustering, or distances to centers.These signals support selecting representative instances.
3 Method
MCL prunes multimodal datasets by modeling explicit concept graphs, aggregating them into a dataset-level semantic landscape, and maximizing fine-grained concept coverage under a budget. Its importance scores combine concept rarity with structural participation, while category normalization and greedy selection support diverse, interpretable subsets.
- 3.1 Sample-level Concept Graph Construction: MCL maps each image–text pair to an interpretable concept graph containing Entities, Events, and Attributes.These categories represent objects or subjects, actions or interactions, and properties or modifiers.
- 3.4 Semantic Coverage Greedy Selection: MCL uses greedy selection to iteratively choose samples with the largest marginal contribution to concept coverage.The selected samples therefore complement the retained set by targeting rare, diverse, and relationally informative semantics.
- 3.2 Dataset-level Concept Graph Construction: Individual sample graphs are unified into a dataset-level graph whose node weights count how many samples contain each concept.High-weight concepts indicate frequent semantics, whereas low-weight concepts indicate rare or scarce semantics.
- 3.3 Concept Importance and Coverage Objective: MCL assigns concept importance using both semantic rarity and structural participation in the dataset graph.The inverse-fraction term prioritizes under-represented concepts, while the degree-based correction favors concepts involved in richer interactions.
- 3.3 Concept Importance and Coverage Objective: Importance scores are normalized independently across Entity, Event, and Attribute categories so no category dominates the objective.This makes importance scores comparable across concept types.
- 3.3 Concept Importance and Coverage Objective: The pruning objective maximizes the weighted union of concepts covered by selected samples subject to a retention budget.Once a concept is covered, additional samples containing it do not increase the objective, favoring samples that add uncovered concepts.
4 Experiment
MCL is evaluated across multimodal instruction tuning and object detection, multiple selection ratios, and concept-coverage analyses. The experiments report strong performance, efficiency, generalization, and more balanced semantic distributions after pruning.
- Main results: At 7.5% retention, most pruning methods preserve over 90% of full-data performance, while MCL achieves stronger overall relative performance than DataTailor.The authors attribute MCL’s effectiveness to fine-grained, disentangled semantic representations from concept graphs.
- Efficiency: MCL’s pruning pipeline completes in 1.7 hours on a 4 * RTX 3090 GPU server and retains 7.5% of the data with up to 99% of full-data performance.The reported pipeline includes concept graph construction, concept importance estimation, and greedy selection.
- Selection-ratio analysis: MCL consistently outperforms competing methods across selection ratios on LLaVA-1.5-mix-665k and improves over PFB and DivBS on COCO.The comparisons include COINCIDE, D2-Pruning, SemDeDup, Self-Sup, PFB, and DivBS.
- Ablation study: Greedy marginal-gain selection performs better than static ranking on COCO because static scores can overemphasize rare concepts while underrepresenting dominant concepts.The performance gap becomes larger as the selection ratio decreases.
- Concept coverage: After pruning, frequent entity concepts such as man and people occupy a smaller proportion, while less frequent event and attribute concepts gain relative prominence.The reported redistribution produces a more balanced semantic concept distribution.
- Ablation study: With 7.5% of the data, MCL reaches or slightly exceeds the full-data baseline on POPE and ScienceQA, whereas SEED-I and GQA benefit from higher data ratios.The results indicate that benchmark sensitivity to selection ratio differs across tasks.
5 Conclusion
MCL presents coverage-aware pruning through interpretable, fine-grained concept graphs, using greedy marginal gains to reduce redundancy while preserving semantic diversity.
- MCL models large-scale multimodal data with interpretable, fine-grained concept graphs.
- Greedy selection based on set-dependent marginal gains reduces redundancy while preserving semantic diversity.
- Experiments on multimodal instruction tuning and object detection show strong performance under aggressive pruning.
- MCL offers substantial data-efficiency improvements with minimal computational overhead.