Source-linked AI summary
From Detrimental to Beneficial: Dynamic Influence-based Valuation and Editing
Adrian Nyakairu, Hongfu Liu
TL;DR
Existing data valuation methods often discard or downweight detrimental samples, leaving subsequent intervention underexplored. DIVE dynamically values samples at the batch level and reverses harmful gradient directions rather than altering raw data. Across noisy image benchmarks and LLM fine-tuning regimes, it reports strong performance and computational efficiency, including an average accuracy of 0.7951 across six noisy datasets.
Problem
Existing work mainly estimates data values, while using those estimates to intervene on training data remains limited; detrimental samples are typically removed or downweighted.
Method
DIVE dynamically estimates sample values at the batch level and edits detrimental samples’ gradients by reversing their directions without modifying raw data.
Results
DIVE consistently demonstrates strong performance and computational efficiency across standard image benchmarks and LLM fine-tuning, achieving average accuracy 0.7951 across six noisy datasets.
Takeaways & Limitations
DIVE transforms detrimental samples into beneficial contributions, leveraging available data while supporting effective optimization without clean validation data via self-influence.
Takeaways & Limitations
Dynamic valuation introduces significant computational overhead because data values are evaluated frequently during training.
Abstract
from arXiv · showhide
Data valuation is a cornerstone of data-centric learning, where prior efforts primarily focus on designing algorithms to classify training samples as either beneficial or detrimental for the learning task. However, leveraging these valuation estimates for subsequent data intervention remains underexplored; conventional approaches typically discard or downweight harmful samples, thereby underutilizing available data resources. In this paper, we present Dynamic Influence-based Valuation and Editing (DIVE), a novel and efficient framework that dynamically estimates sample values at the batch level and transforms detrimental data into beneficial contributions. Rather than altering the raw data, DIVE operates at the optimization level by strategically reversing the gradient directions of harmful samples during training, ensuring seamless integration with standard learning procedures with minimal overhead. Extensive empirical evaluations demonstrate that DIVE consistently improves classification performance, maximizes data efficiency, stabilizes optimization, and effectively generalizes to large language model fine-tuning.
1 Introduction
Data-centric learning evaluates training data according to its contribution to a target model, but existing valuation work pays limited attention to using those estimates for intervention. DIVE instead dynamically values samples and transforms detrimental contributions through gradient editing.
- Data-centric learning curates data for a target model rather than designing new algorithms around fixed training data.
- Existing data valuation work mainly estimates sample value, while subsequent use of those estimates for intervention remains limited.
- Prior approaches commonly remove or downweight detrimental samples, discarding potentially useful information.
- DIVE dynamically estimates data value at the batch level and modifies detrimental samples’ gradients to make their updates beneficial.
- DIVE operates on gradients rather than raw data, integrating with standard optimization procedures with minimal overhead.
- Experiments compare DIVE with data valuation and dynamic batch curation methods and extend evaluation to large language model fine-tuning.
2 Related Work
Related work spans influence-function valuation and dynamic batch curation, while DIVE combines dynamic valuation with gradient editing to convert harmful samples without prior labels or heuristics.
- Influence Functions: Influence functions estimate sample influence on validation performance without retraining, but require computational approximations to inverse Hessians.
- Dynamic Data Valuation: Dynamic valuation computes sample value during training because aggregating checkpoint estimates can miss evolving or conflicting influence signals.
- Dynamic Batch Curation: Dynamic batch curation includes curriculum learning, adaptive selection, and reweighting based on sample difficulty, uncertainty, or multimodal criteria.
- DIVE’s Distinction: Gradient Reversal and Negative Learning adjust gradient directions but depend on external knowledge or heuristics to pre-identify detrimental samples.
- DIVE’s Distinction: DIVE unifies sample valuation and gradient editing, dynamically discovering detrimental samples and turning their harmful gradients into beneficial updates.
3 Preliminaries on Influence Functions
Influence functions quantify how perturbing training samples affects model behavior, but their conventional formulation relies on strong assumptions and a converged solution.
- Influence Function Formulation: Influence functions quantify a training sample’s effect on model parameters or validation performance through infinitesimal perturbation or removal.
- Influence Function Formulation: The formulation uses sample gradients and the Hessian matrix evaluated at the empirical-risk-minimizing parameters.
- Assumptions and Approximations: Conventional influence functions rely on strong assumptions, including model convexity and access to a fully converged solution.
- Assumptions and Approximations: Subsequent work primarily improves computational efficiency through stochastic, projection-based, factorized, low-rank, or identity-Hessian approximations.
4 Method
DIVE values samples dynamically during mini-batch training and edits detrimental gradients instead of changing raw data. Its influence estimate uses gradient alignment, while implementation supports direct sample-level computation.
- Dynamic Data Valuation: DIVE estimates data valuation dynamically within each mini-batch and uses the result to guide model updating.
- Dynamic Data Valuation: Static valuation can become outdated after retraining and often requires two training rounds, making it costly for large datasets and models.
- Dynamic Data Valuation: Dynamic valuation avoids outdated estimates and retraining but incurs frequent evaluation overhead, making efficient approximations important at scale.
- Dynamic Data Valuation: DIVE interprets influence through gradient inner products that indicate whether a sample helps or harms the target objective.
- Dynamic Gradient Editing: The method partitions each batch by influence sign and flips negative-influence gradients to produce beneficial updates.
- Dynamic Gradient Editing: DIVE edits gradients directly rather than raw data, enabling integration with standard optimization procedures and constructive contributions from every sample.
5 Experimental Results
DIVE is evaluated across noisy-label image benchmarks, ablations, computational analyses, validation-free settings, and LLM fine-tuning. It consistently improves performance by dynamically correcting detrimental samples rather than removing them, while retaining efficiency and robustness.
- Algorithmic Performance: DIVE achieves the highest average accuracy, 0.7951, across six noisy-label image benchmarks and performs best on three datasets.The result is averaged over five runs, with a paired t-test showing significant improvement over the second-best method at p < 0.1.
- Algorithmic Performance: DIVE consistently outperforms static influence methods, curriculum learning, and strong dynamic baselines on noisy-label classification.Dynamic methods generally outperform static methods, while DIVE achieves further consistent gains over InfoBatch, Ghost, and LAI.
- In-depth Exploration: Influence signs change during optimization, and DIVE’s correction rate rises after warm-up before stabilizing at approximately 20%.Individual samples may remain beneficial, switch once, or oscillate between beneficial and detrimental states, motivating adaptive correction.
- In-depth Exploration: DIVE’s accuracy improves with longer warm-up, while editing every 2 or 3 mini-batches offers the best frequency trade-off and dominates GHOST and LAI across runtime budgets.The default 30% warm-up captures most of the gain; larger editing intervals reduce runtime but eventually reduce accuracy.
- Extension on Fine-Tuning LLMs: DIVE remains effective without validation data and extends to LLM fine-tuning, where it lowers test loss on every evaluated dataset and improves accuracy when headroom exists.Without validation data, DIVE significantly outperforms the second-best method; in LLM fine-tuning, accuracy gains are +1.82 points on ANLI_r3 and +2.13 points on Civil_Comments_b.
6 Conclusion
DIVE is a lightweight dynamic data-curation framework that transforms detrimental samples into beneficial contributions during training. Experiments across image benchmarks and LLM fine-tuning show strong performance and computational efficiency.
- DIVE handles noisy and detrimental data dynamically during training through lightweight data curation.
- DIVE transforms detrimental samples into beneficial ones, recovering informative signals instead of discarding them.
- Across standard image benchmarks and LLM fine-tuning regimes, DIVE consistently demonstrates superior performance and computational efficiency.
- DIVE remains effective without clean validation data by using self-influence.
- The findings support repurposing noisy training data rather than discarding it during large-scale model optimization.
A Descriptions of Datasets and Models
The experiments use six noisy-label image classification benchmarks, with different sources and noise settings. Validation construction differs according to whether clean training labels are available, and image models use a ResNet-9 backbone.
- Datasets: The evaluation covers six noisy-label image classification benchmarks, including CIFAR-10N variants and CIFAR-100N.
- Datasets: CIFAR-10N uses aggregate, random, and worst label sets with approximately 9%, 17%, and 40% noise, respectively.
- Datasets: CIFAR-10N and CIFAR-100N contain 50,000 training images over 10 and 100 classes, respectively.
- Validation: For datasets without clean training labels, a class-stratified 1/5 of the clean test split is used for validation, with accuracy reported on the remaining 4/5.
- Models: All image experiments use a ResNet-9 backbone with convolutional, normalization, activation, downsampling, residual, and global-pooling components.
B Additional Experimental Results
Additional analyses on Animal-10N and Food-101N reproduce the main behavioral pattern: DIVE performs best after warm-up while dynamically correcting samples. With less noise, influence scores shrink toward zero as training converges.
- On Animal-10N and Food-101N, DIVE attains the highest test accuracy among curation methods after warm-up.
- DIVE keeps correcting a sizeable, time-varying fraction of the training set rather than a fixed subset.
- Tracking individual samples shows that their influence signs are not stable during training.