PaperlayerView exact revision on arXiv

Source-linked AI summary

Influence-Preserving Proxies for Gradient-Based Data Selection in LLM Fine-tuning

Sirui Chen, Yunzhe Qi, Mengting Ai, Yifan Sun, Ruizhong Qiu, Jiaru Zou, Jingrui He

arXiv:2602.17835v1cs.LG

TL;DR

Gradient-based data selection is expensive for multi-billion-parameter LLMs, and off-the-shelf proxies offer limited sizing and no principled influence alignment. IPROX derives flexible proxies from the target through influence-preserving compression and gradient/logit alignment, consistently outperforming off-the-shelf proxies and baselines while reducing computation.

  • Problem

    Gradient-based data selection is computationally costly for large LLMs, while off-the-shelf proxies have unclear learning dynamics, fixed sizes, and no systematic target-alignment procedure.

  • Method

    IPROX constructs proxies directly from the target using influence-preserving low-rank compression followed by gradient alignment in low-rank space and output-logit anchoring.

  • Results

    IPROX consistently outperforms off-the-shelf proxies and baselines across model families and tasks, including a 1.5B proxy outperforming a 1.7B proxy on Qwen3-4B.

  • Takeaways & Limitations

    Influence-preserving proxies provide a scalable approach to gradient-based data selection with efficiency benefits for LLM fine-tuning.

  • Takeaways & Limitations

    Proxy size cannot be reduced arbitrarily: compression below about 10% of the original rank can sharply reduce model quality and influence preservation, while embedding and output layers limit reduction.

Abstract

from arXiv · show

Supervised fine-tuning (SFT) relies critically on selecting training data that most benefits a model's downstream performance. Gradient-based data selection methods such as TracIn and Influence Functions leverage influence to identify useful samples, but their computational cost scales poorly, making them impractical for multi-billion-parameter large language models (LLMs). A common alternative is to use off-the-shelf smaller models as proxies, but they remain suboptimal since their learning dynamics are unclear, their sizes cannot be flexibly adjusted, and they cannot be further aligned with the target model in terms of gradient-based influence estimation. To address these challenges, we introduce Iprox, a two-stage framework that derives influence-preserving proxies directly from the target model. It first applies a low-rank compression stage to preserve influence information of the target model, and then an aligning stage to align both model gradients and logits, thereby constructing proxies that flexibly control computational cost while retaining the target model's influence. Experimental results across diverse LLM families and evaluation tasks show that Iprox consistently outperforms off-the-shelf proxies and baseline methods. On Qwen3-4B, a 1.5B proxy constructed with Iprox achieves stronger performance than the larger 1.7B off-the-shelf proxy. Notably, on Llama3.2, Iprox achieves better performance than baselines while reducing computational cost by more than half relative to the full 3B model. These results show that Iprox provides effective influence-preserving proxies, making gradient-based data selection more scalable for LLMs.

1 INTRODUCTION

Gradient-based data selection can identify useful SFT examples but is costly for large LLMs, while off-the-shelf proxies lack flexible sizing and principled influence alignment. IPROX addresses these gaps by compressing and aligning proxies derived directly from the target model, with stronger reported results than larger off-the-shelf proxies.

  • SFT performance depends on selecting a curated subset of training data rather than simply increasing the amount of data.Naively combining datasets can degrade downstream performance.
  • Gradient-based selection estimates sample importance from model influence but incurs substantial checkpoint, backpropagation, or inverse-Hessian costs.TracIn accumulates gradient inner products across checkpoints, whereas Influence Functions use gradients scaled by the inverse Hessian.
  • Off-the-shelf proxies have unclear learning dynamics, fixed available sizes, and no systematic way to align their influence estimates with the target model.These limitations make proxy selection depend on prior assumptions about similarity between larger models and their smaller counterparts.
  • IPROX constructs influence-preserving proxies directly from the target through low-rank compression followed by gradient and logit alignment.IPSVD retains influence-relevant components, then the aligning stage matches proxy gradients in low-rank space while anchoring output logits.
  • A 1.5B IPROX proxy outperforms the Qwen3-1.7B off-the-shelf proxy for Qwen3-4B data selection.Figure 1 identifies this as a representative comparison showing that a smaller influence-preserving proxy can achieve better data-selection performance.
  • IPROX consistently outperforms off-the-shelf proxies across diverse tasks and model families while supporting different gradient-based influence estimators.The reported gains are described across diverse evaluations rather than a single task or model family.

2 RELATED WORKS

Related work develops model-aware data selection, efficient influence estimation, and decomposition-based LLM compression. These lines of research motivate more scalable influence computation and low-rank proxy construction.

  • Gradient-based selection estimates each example’s impact through first- and second-order training dynamics rather than only dataset structure.Influence functions and trajectory-based gradient similarity are central model-aware approaches.
  • Efficient LLM data-selection methods simplify influence estimation or compute influence on subsets before extrapolating to the full dataset.These approaches respond to the growing impracticality of gradient-based selection for large models.
  • Decomposition-based compression exploits low intrinsic rank in weight matrices, with later methods incorporating activations, quantization, or broader transformer approximations.The cited progression includes SVD, activation-aware factorization, quantized low-rank methods, and Nyström approximation.

3 PRELIMINARIES AND PROBLEM DEFINITION

Model-aware data selection seeks a fixed-budget training subset that maximizes downstream validation utility, commonly using pairwise gradient-based influence scores. Because direct influence computation is expensive at LLM scale, the section frames a proxy as a smaller model that balances efficiency with alignment to the target.

  • The objective is to choose a k-example subset of Dtrain whose fine-tuned model maximizes utility on Dval.The training and validation datasets may follow either the same or different distributions.
  • Pairwise influence scores rank training samples by their estimated utility for target validation samples, enabling top-k selection instead of direct combinatorial optimization.The score I(z, z′) quantifies the utility of training on z for performance on z′.
  • TracIn approximates influence by accumulating gradient similarities between training and target samples across multiple checkpoints.Its score uses checkpoint parameters and averaged learning rates along the optimization trajectory.
  • Influence Functions estimate the effect of infinitesimally upweighting a training sample using its gradient multiplied by the inverse Hessian.This approximation models how final model parameters would change under a small reweighting.
  • The proxy-design problem is to approximate target-model influence scores while being small enough to reduce computation and aligned enough to preserve selection quality.This efficiency–alignment trade-off is the central problem addressed by the paper.
  • IPROX uses influence-preserving SVD for compression and then aligns proxy gradients in low-rank space while anchoring logits.The two stages are summarized as compression followed by gradient alignment with logit anchoring for stability.

4 PROXY CONSTRUCTION VIA INFLUENCE-PRESERVING COMPRESSION

IPROX constructs influence-preserving proxies through influence-aware low-rank compression followed by gradient alignment within the proxy parameter space. The method preserves influence-relevant components, controls proxy size through rank, and avoids high-dimensional gradient reconstruction during influence calculation.

  • Proxy construction: IPROX combines influence-preserving SVD compression with gradient alignment that matches proxy gradients and anchors the logits distribution.The two stages respectively preserve influence-relevant components and refine the proxy's gradient behavior after layerwise approximation errors accumulate.
  • Stage 1: Influence-Preserving SVD: IPSVD attains markedly higher influence retention than standard SVD as compression sparsity increases, addressing standard SVD's failure to preserve gradient-based influence.Standard SVD can retain loss while influence retention deteriorates more rapidly; IPSVD explicitly targets influence-relevant components.
  • Stage 1: Influence-Preserving SVD: The compression objective reweights approximation errors using input and upstream-gradient second moments, prioritizing weights most critical to influence estimates.The reweighting emphasizes directions with large inputs and directions where the loss is most sensitive.
  • Stage 1: Influence-Preserving SVD: The proxy rank directly controls model size, reducing each decomposed layer to r_l(m_l+n_l) parameters and balancing efficiency against approximation quality.Lower ranks produce higher sparsity, while the rank can be selected under a computational budget.
  • Efficient implementation: Skinny probe-matrix SVDs reduce the reweighted decomposition to an at-most N × N core, lowering complexity from full O(n^3) decompositions to O(N^3 + n_lN^2 + m_lN^2).The approach avoids explicitly forming large second-moment matrices and uses a small probe set of N samples.
  • Stage 2: Gradient Alignment: Gradient alignment is performed entirely in the proxy parameter space, eliminating high-dimensional reconstruction during influence calculation and preserving the proxy's efficiency.This alignment refines the initialized proxy after approximation errors compound across layers.

5 EXPERIMENTS

IPROX is evaluated across model families, tasks, influence estimators, efficiency settings, and ablations. It generally outperforms off-the-shelf proxies and baselines while offering controllable computational savings, with performance shaped by proxy size, task type, probe diversity, and alignment components.

  • 5.2 MAIN RESULTS: IPROX consistently outperforms off-the-shelf proxies across BBH and TyDiQA, remains competitive on MMLU, and surpasses a larger 1.7B proxy with only 1.5B parameters on Qwen3.The comparison spans four target model families and multiple sparsity levels.
  • 5.2 MAIN RESULTS: IPROX gains are more pronounced on TyDiQA than MMLU, consistent with the closer relationship between TyDiQA and Dolly than between MMLU and Dolly.The paper links greater training–validation distributional shift to a looser error bound.
  • 5.2 MAIN RESULTS: IPROX improves over Layer Extraction and Influence Scorer by 2.81% on Llama3.2-3B and 1.99% on Gemma3-4B on average.The baselines are cheaper but do not preserve gradient information, and perform notably worse on BBH and TyDiQA.
  • 5.3 ANALYSIS: Over 140 TFLOPs are saved at sparsity ρ=0.7 versus the full Llama3.2-3B model, while proxy-construction stages contribute only a small fraction of total FLOPs.On one GH200, influence calculation takes about 38–44 minutes and total runtime about 43–51 minutes, compared with about 90 minutes for the full model.
  • 5.3 ANALYSIS: Lower-sparsity proxies achieve higher subspace affinity, whereas higher-sparsity proxies retain selection diversity through larger 1-NND values.The analysis interprets this as task-relevant direction capture alongside variation in less dominant components.
  • 5.3 ANALYSIS: Probe-size gains saturate around 3× the default, while larger probe sets increase Stage 1 cost roughly linearly and lower probe diversity consistently degrades performance.These findings support a moderate, diverse probe set; removing KL anchoring or alignment also reduces performance, especially at higher sparsity.

6 CONCLUSION

The conclusion presents IPROX as an influence-preserving proxy framework that combines compression with gradient and output alignment for efficient LLM data selection.

  • 6 CONCLUSION: Removing KL anchoring or the entire aligning stage consistently reduces performance across tasks.
  • 6 CONCLUSION: IPROX combines influence-preserving low-rank compression with model-gradient and output alignment to reduce proxy computational cost.The framework is designed to preserve the target model’s influence information while constructing a smaller proxy.
  • 6 CONCLUSION: Experiments across multiple model families and tasks show gains over off-the-shelf proxies and baselines, alongside efficiency benefits.The authors position these results as evidence that influence-preserving proxies can scale gradient-based data selection.

A.1 BASELINE AND EVALUATION DETAILS

The appendix specifies baseline implementations, evaluation tasks, training settings, and practical construction details for IPROX and influence estimators.

  • A.1 BASELINE AND EVALUATION DETAILS: Layer extraction scores blocks by hidden-state representation change, selecting the top 70% at sparsity ρ = 0.3 from a 1% random training-data sample.
  • A.1 BASELINE AND EVALUATION DETAILS: Evaluation covers MMLU 5-shot accuracy, TyDiQA 1-shot macro exact match, and BBH 3-shot exact match under four-epoch training.The reported task metrics average performance across the specified subtasks or languages.
  • A.1 BASELINE AND EVALUATION DETAILS: TracIn uses validation-gradient cosine similarity, while Influence Functions use K-FAC inverse Hessian–vector products with 1024-sample Hessian statistics.
  • A.1 BASELINE AND EVALUATION DETAILS: IPROX is initialized from 1% of Dtrain, allocating 10% of that sample to the first stage and 90% to the second.
  • A.1 BASELINE AND EVALUATION DETAILS: The second stage searches learning rates and λKL values while aligning decomposed layers and fixing the remaining parameters.

B FURTHER DISCUSSION

The discussion situates IPROX among gradient-based selection methods and identifies approximation-error trade-offs that constrain proxy fidelity.

  • B FURTHER DISCUSSION: Gradient-based selection methods use gradient geometry as a surrogate for example utility but differ in which parts of the selection pipeline they accelerate.
  • B FURTHER DISCUSSION: Aggressive compression and layer-wise mismatches can accumulate approximation errors, especially when stochastic fine-tuning noise interacts with them.The resulting deviations may affect per-example scores or selection rankings.

C ADDITIONAL EXPERIMENT RESULTS

Additional experiments test IPROX across target models, data budgets, compression levels, candidate datasets, and alignment settings, generally favoring influence-preserving construction.

  • C ADDITIONAL EXPERIMENT RESULTS: IPROX consistently outperforms baselines across Qwen3-4B and Qwen2-7B, with TyDiQA gains of +3.85 and +7.84 over the strongest baseline.
  • C ADDITIONAL EXPERIMENT RESULTS: IPROX outperforms the off-the-shelf 1B proxy at both 1% and 20% data budgets, while the larger budget degrades performance particularly on TyDiQA.
  • C ADDITIONAL EXPERIMENT RESULTS: Replacing IPSVD with standard SVD drops average performance by approximately 2 to 3 points and by up to 6 points on TyDiQA.
  • C ADDITIONAL EXPERIMENT RESULTS: On CoT and BioInstruct data, IPROX outperforms the off-the-shelf 1B proxy and remains competitive with the full 3B model.CoT improves all benchmarks, whereas BioInstruct causes larger degradation on general benchmarks.
  • C ADDITIONAL EXPERIMENT RESULTS: At 90% sparsity, IPROX consistently outperforms Layer Extraction despite natural performance degradation under extreme compression.
  • C ADDITIONAL EXPERIMENT RESULTS: A moderate γKL = 0.1 performs best, while excessive or insufficient KL regularization degrades influence alignment or target-capability retention.

D PROOF OF PROPOSITION 4.1

Proposition 4.1 bounds how perturbing a layer’s weights changes its influence contribution under four technical assumptions. The proof controls this change through local smoothness, moment, directional-coherence, and covariate-shift conditions.

  • Technical assumptions: Assumption (A1) requires differentiable upstream gradients with a uniformly bounded Jacobian along the perturbation path.
  • Technical assumptions: Assumption (A2) requires finite second moments for layer inputs and upstream gradients across independent samples.
  • Technical assumptions: Assumptions (A3) and (A4) impose directional coherence and bounded covariate shift, respectively, linking perturbation effects to scalar errors and expectation bounds.
  • Proposition and assumptions: Under Assumptions (A1)–(A4), any layer perturbation has a finite data-dependent bound on its influence change.The proposition introduces a finite constant Cκ governing the bound.
  • Proof: The proof derives the result by differentiating influence along the perturbation path, bounding the derivative, and integrating the resulting expectation.It uses the chain rule, triangle inequality, Cauchy–Schwarz, Fubini’s theorem, and Jensen’s inequality.

E INFLUENCE-PRESERVING LOW-RANK APPROXIMATION FOR INFLUENCE FUNCTIONS

The paper extends influence-preserving low-rank approximation from TracIn to Influence Functions by accounting for reweighted gradients and Hessian structure. Under simplifying assumptions, the expected influence change is bounded by a finite constant independent of the specific perturbation.

  • Influence Functions: Influence Functions incorporate inverse-Hessian reweighting, and the analysis extends the low-rank influence-preservation argument to this curvature-aware measure.
  • Hessian approximation: A block-diagonal Hessian and layerwise independence between inputs and upstream gradients yield a tractable surrogate Hessian.The surrogate is defined from expected outer products of vectorized layer gradients.
  • Influence-preserving objective: The compression objective penalizes weight error after reweighting by activation and upstream-gradient second-moment matrices.The matrices are assumed symmetric positive definite.
  • Guarantee: Under assumptions (B1)–(B3), perturbing a layer’s weights produces a bounded expected change in the Influence Function score.
  • Proof: The final bound uses first-order Taylor terms plus a quadratic remainder, with constant Cκ depending on data properties but not on Eℓ.

F EFFICIENT IMPLEMENTATION VIA PROBE-BASED APPROXIMATION AND CORE SVD

The implementation avoids forming large second-moment matrices by estimating them from a probe dataset and reducing the computation to skinny SVDs plus a small core SVD. This makes influence-preserving compression practical when the probe set is much smaller than the layer dimensions.

  • Probe-based approximation: The approximation replaces prohibitively large second-moment matrices with statistics collected from N probe samples using one forward and backward pass.
  • Skinny SVD construction: Skinny SVDs of the probe matrices represent regularized second-moment square roots without explicitly forming the full matrices.
  • Core SVD: The large-matrix SVD is reduced to an SVD of a core matrix with dimensions at most N × N, then truncated to the desired rank rℓ.
  • Complexity: When N ≪ min(nℓ, mℓ) and rℓ ≤ N, the computation depends primarily on probe-set size rather than target-model dimensions.
  • Complexity: The dominant costs are core-matrix formation and skinny SVDs, reducing complexity from O(min(mℓ, nℓ)^3) for full second-moment SVDs.
Loading 2602.17835v1…