PaperlayerView exact revision on arXiv

Source-linked AI summary

For-Value: Efficient Forward-Only Data Valuation for finetuning LLMs and VLMs

Wenlong Deng, Qi Zeng, Jiaming Zhang, Minghui Chen, Zixin Ding, Christos Thrampoulidis, Boying Gong, Xiaoxiao Li

arXiv:2508.10180v3cs.CL

TL;DR

Existing data-valuation methods are difficult to scale because they rely on gradient computations and per-sample processing. For-Value instead estimates value from last-layer representations and prediction errors using forward-only computation, matching or outperforming prior methods while improving efficiency. Its scope is fine-tuning, not pretraining data selection, and data value may change during training.

  • Problem

    Gradient-based data-valuation methods are costly for large LLMs and VLMs because they require per-sample backpropagation, storage, or other expensive computations.

  • Method

    For-Value derives a closed-form influence score from last-layer hidden representations and token-level prediction errors and computes it with a single forward pass.

  • Results

    For-Value matches or outperforms existing methods in identifying influential and mislabeled data and selecting high-value fine-tuning subsets, while being substantially more efficient.

  • Takeaways & Limitations

    Forward-only valuation provides an effective and batch-scalable approach for data selection during LLM and VLM fine-tuning.

  • Takeaways & Limitations

    For-Value is tailored to fine-tuning and is not directly applicable to pretraining data selection when the unconstrained feature assumption may not hold.

Abstract

from arXiv · show

Data valuation is essential for enhancing the transparency and accountability of large language models (LLMs) and vision-language models (VLMs). However, existing methods typically rely on gradient computations, making them computationally prohibitive for billion-parameter models and precluding batch parallelization. In this work, we introduce For-Value, a forward-only data valuation framework that enables efficient batch-scalable value estimation while maintaining effectiveness. Leveraging the expressive power of pretrained LLMs/VLMs, we theoretically demonstrate that data valuation can be captured by the alignment between the final hidden representations and prediction errors at the last layer. In light of this insight, For-Value computes data value using a simple closed-form expression with a single forward pass, eliminating the need for costly backpropagation and enabling efficient batch calculating at scale. Extensive experiments show that For-Value matches or outperforms gradient-based baselines in detecting influential data and mislabeled data, while achieving significant efficiency improvements.

1 Introduction

Data valuation is needed to identify training samples that improve downstream LLM and VLM fine-tuning, but gradient-based methods limit scalability. For-Value addresses this with a forward-only score based on hidden representations and prediction errors.

  • LLMs and VLMs can produce factually incorrect or biased outputs when fine-tuning data is irrelevant, mislabeled, or unrepresentative.
  • Data valuation assigns each training sample a score based on its effect on model performance on a valuation set.Common assessment signals include loss, margin, and likelihood.
  • Gradient-based valuation methods require per-sample backpropagation, hindering large-batch parallel processing and creating substantial storage or computational costs.The cited methods include TracIn, DataInf, HyperInf, and online Shapley estimation.
  • For-Value derives a closed-form data-value approximation from last-layer hidden representations and token-level prediction-error similarities.The derivation is presented for LLM and VLM fine-tuning under an unconstrained feature assumption.
  • For-Value uses forward-only influence scores to support efficient large-batch parallelism and scalability to modern LLMs and VLMs.
  • For-Value matches or outperforms prior methods across influential-data, mislabeled-data, and high-value-subset fine-tuning tasks while improving efficiency.

2 Related Work

Prior data-valuation approaches range from influence and Hessian approximations to TracIn and Shapley-based methods. Despite efficiency improvements, these methods still rely on gradients, repeated training, or costly stored quantities.

  • Influence estimation methods include Hessian-based approaches, TracIn, DataInf, and HyperInf.
  • Shapley value methods provide principled alternatives but remain impractical because of repeated training or per-token gradients and activations.
  • Existing approaches generally require per-sample gradient computation during or after fine-tuning.

3 Method

For-Value estimates training-sample influence from a closed-form alignment of hidden representations and token-level prediction errors. It computes these scores with forward passes, sparse similarity operations, and ranking.

  • Data value measures how much a training sample increases the likelihood of a valuation sample during fine-tuning.The definition focuses on pretrained initialization and relates higher likelihood to lower valuation loss.
  • Under the unconstrained feature assumption, pretrained models produce embeddings that are transformed into logits and token probabilities.
  • Theorem 1 relates greater training-sample value to an expression combining hidden-representation alignment with token-level prediction-error similarity.The assumption excludes training inputs identical to the valuation input.
  • A larger For-Value score indicates a greater increase in the valuation data’s likelihood, and the score depends only on forward-pass variables.
  • For-Value obtains valuation and training embeddings and prediction-error vectors using inference, computes scores with sparse matrix similarity, and ranks samples by score.
  • The algorithm performs valuation-sample inference and batch training-sample inference before producing the data-valuation output.
  • The pipeline sorts training samples by their computed scores in descending order.

4 Experiment Setup

The experiments compare For-Value with efficiency-oriented valuation baselines on LLM and VLM tasks, including influential-data identification, noisy-data robustness, and efficiency evaluation.

  • The baselines use LoRA across transformer layers for parameter-efficient comparisons and first-order gradient similarity for Hessian-free influence estimation.
  • Table 1 reports LLM influential-data identification results as mean ± standard deviation and states that For-Value consistently achieves comparable or superior performance.
  • Experiments evaluate influential-data identification across LLM sentence-transformation and math-word-problem datasets and VLM image-to-text generation tasks.
  • The evaluation includes GSM8K for mathematics, Noise-Huatuo-Complex-CoT for noisy medical training, and PMC-Reasoning for vision-language models.

5 Results

For-Value performs competitively across influential-data identification, mislabeled-data detection, and data-selection tasks for LLMs and VLMs, while substantially reducing valuation cost. Its effectiveness remains stable across model sizes, and its prediction-error weighting improves over embedding similarity.

  • Influential and mislabeled data: For-Value consistently matches or outperforms baselines on LLM and VLM influential-data identification, including recall gains over HyperINF.It surpasses HyperINF by 6.5% recall on Qwen2.5-1.5B sentence transformation, 6% on both Qwen math datasets, and more than 7% on 11B subject generation.
  • Influential and mislabeled data: For-Value improves mislabeled-data detection on Qwen-VL-3B and matches near-perfect gradient-based detection on Llama-3.2-11B.The larger model reaches AUC > 0.99 and Recall = 1.0.
  • Data selection for fine-tuning: 48.3% accuracy results from selecting the top 5% GSM8K samples with For-Value, exceeding HyperINF by 5.5%.At a 1% selection rate, For-Value still exceeds all baselines by up to 3.3% and is over 5× faster.
  • Data selection for fine-tuning: 60.31% average accuracy on noisy medical data exceeds DataInf by 3%, while 62.35% at 10% selection exceeds HyperINF by 2.1%.For-Value requires only 0.8h, up to 6× faster than baselines.
  • Data selection for fine-tuning: 52.23% average accuracy with 10% PMC-Reasoning data exceeds the base model by over 3% and HyperINF by 0.6%.For-Value also achieves the lowest computational cost, 0.4h versus 1.6–1.7h for baseline methods.
  • Ablation and efficiency: For-Value maintains high performance across model sizes, while its prediction-error weighting outperforms embedding similarity across LLM and VLM tasks.AUC and Recall stay close to 1.0, and the α term down-weights confidently predicted tokens.
  • Ablation and efficiency: For-Value runs within a few hundred seconds as model size increases from 1.5B to 72B, whereas baselines can require hours.HyperINF takes 6 hours for the 32B model on sentence transformation.

6 Conclusion

For-Value estimates data influence from the alignment of last-layer hidden representations with token-level prediction errors, eliminating backpropagation. With one forward pass, it matches existing methods while improving efficiency in influential and mislabeled data identification and high-value subset selection.

  • 6 Conclusion: For-Value approximates data influence using alignment between last-layer hidden representations and token-level prediction errors.The formulation eliminates the need for backpropagation.
  • 6 Conclusion: A single forward pass enables For-Value to identify influential and mislabeled data and select high-value finetuning subsets.The framework is designed for pretrained LLMs and VLMs.
  • 6 Conclusion: For-Value matches existing methods while being substantially more efficient.The reported evaluation covers influential data, mislabeled data, and high-value subset selection.

7 Limitations

For-Value is scoped to data valuation during finetuning and is not directly applicable to pretraining data selection under the stated feature assumption. The authors also note that data value may change during training and leave stage-aware selection and active learning for future work.

  • 7 Limitations: For-Value is tailored to finetuning-stage data valuation and is not directly applicable to pretraining data selection.The authors attribute this boundary to the unconstrained feature assumption potentially not holding during pretraining.
  • 7 Limitations: Data value may evolve over the course of training, motivating future extensions to stage-aware data selection or active learning.The paper leaves these extensions for future work.

A.2 Additional Results

For-Value combines strong performance with low computational and memory overhead. Unlike gradient- and Hessian-based baselines, it is training-free and supports large-batch parallel computation using forward calculations alone.

  • A.2 Additional Results: For-Value maintains strong performance while remaining highly efficient compared with prior valuation methods.The complexity comparison covers computational and memory costs across influence-function, Hessian-based, embedding, and For-Value methods.
  • A.2 Additional Results: HyperINF requires about 6 hours for a Qwen-32B model because of its cubic complexity.The paper describes this cost as impractical for large LLMs.
  • A.2 Additional Results: For-Value and Emb are training-free and algorithm-agnostic, reducing overhead relative to methods based on gradient traces or Hessian computations.For-Value also benefits from typically small valuation sets, often under 2k samples.
  • A.2 Additional Results: For-Value can improve efficiency through large-batch parallel computing because it requires only forward calculations.Baseline methods require computing gradients for individual samples, limiting comparable batch parallelism.

A.3 Additional Details of Select Data for Finetuning

The additional materials describe finetuning setups, complexity comparisons, qualitative data-influence examples, and held-out evaluations for language and vision-language tasks. These details cover noisy-data selection, model-specific training procedures, and the efficiency context for For-Value.

  • A.3 Additional Details of Select Data for Finetuning: The finetuning experiments select the top 5% of training data by value scores before subsequent finetuning.The GSM8K setup uses a one-epoch warmup before scoring and selection.
  • A.3 Additional Details of Select Data for Finetuning: The complexity comparison evaluates influence-function, Hessian-free, DataInf, Emb, and For-Value methods under multilayer-perceptron cost assumptions.The table accounts for layers, dimensions, epochs, training samples, parameters, and in-batch vocabulary size.
  • A.3 Additional Details of Select Data for Finetuning: For-Value identifies influential samples that share task characteristics such as reasoning type, transformation rule, subject, or style.Qualitative examples contrast the most and least influential training samples for each target valuation sample.
  • A.3 Additional Details of Select Data for Finetuning: Additional tables report mislabeled-data performance across noise ratios and high-quality data detection accuracy.The mislabeled-data results use Qwen2.5-VL-3B-Instruct.
  • A.3 Additional Details of Select Data for Finetuning: The medical instruction setup selects the top 5% high-value training data from noisy training data before finetuning the pretrained model.The procedure uses LoRA during the initial setup and then full-parameter finetuning.
  • A.3 Additional Details of Select Data for Finetuning: The vision-language setup finetunes Qwen2.5-3B-VL for 3 epochs and evaluates it on six held-out medical and visual-question-answering datasets.Evaluation uses greedy decoding after full-parameter finetuning.

A.4 Additional Analysis on Select Data for Finetuning

The analysis evaluates influential-data selection across three LLM text-generation tasks and two VLM image-captioning tasks using AUC and recall.

  • Noisy-Data Analysis: In the Noise-Huatuo-Complex-CoT evaluation, baseline methods struggle to select high-quality data from noisy training datasets.The section attributes this difficulty to assumptions of uniqueness or convergence that are difficult to satisfy with noisy data.
  • LLM Tasks: The LLM evaluation covers sentence transformations, math word problems without reasoning, and math word problems with reasoning.Each LLM task uses 10 classes with 100 examples per class, split into 90 training and 10 test examples.
  • VLM Tasks: The VLM evaluation covers style generation and subject generation through image-to-text captioning tasks.Style generation includes cartoons, pixel art, and line sketches; subject generation uses images of 30 distinct subjects.
  • Metrics: AUC measures influential-point identification against class-matching pseudo-labels, while recall measures the class relevance of top-ranked training points.Higher AUC indicates better identification of influential points.

A.5.4 Mislabeled Data Detection Data & Metrics

The mislabeled-data detection setup converts cat-versus-dog classification into a visual-language question-answering task and evaluates whether valuation scores identify clean examples.

  • Task Construction: The task asks a visual-language model, “What is the animal in the image? It is a [label]”.The setup uses demonstrations and converts the classification data into a question-answering format.
  • Noise Injection: The dataset uses cat-versus-dog images, with 50% of labels flipped to introduce noise.The evaluation uses 200 images, with 100 images per class, for valuation.
  • Evaluation: Mislabeled detection is evaluated with AUC and recall, using pseudo-labels that require both class agreement and clean training data.Baseline comparisons use the checkpoint with the highest test AUC because influence-based methods vary substantially across checkpoints.
  • Text Noise: Noise-Huatuo-Complex-CoT contains 5,000 randomly sampled examples, with 40% corrupted by random insertion or deletion of irrelevant words.The corruption process is illustrated with examples of noisy reasoning.

B.2 Proof of Theorem 1

The proof isolates how a training sample influences a valuation sample, showing that under distinct-input conditions the effect is captured by shared hidden representations rather than token unembeddings.

  • Representation Alignment: Theorem 2 states that training-data value increases with the alignment between valuation and training hidden representations.The relevant representations are indexed by token prefixes and training time.
  • Distinct Inputs: When training and valuation inputs differ, influence arises solely through the term representing token embeddings and other parameters excluding token unembedding.This identifies the representation pathway carrying influence across distinct inputs.
  • Assumption: The distinct-input assumption requires that no training input equals the valuation input.The paper describes this as mild because inputs, especially vision-language images, are typically distinct.
  • Proof Consequence: Under the distinct-input assumption, the token-unembedding contribution vanishes and influence comes entirely through shared representation features.The proof contrasts this with cases involving identical inputs and overlapping output predictions.
Loading 2508.10180v3…