Do Influence Functions Work on Large Language Models?

• High-quality training data is essential because poor-quality data can severely compromise LLM performance.Identifying helpful, harmful, or adversarial samples could improve performance and make predictions more transparent.Source: §1 · p. 1
• Tracing individual training samples with leave-one-out validation or Shapley values is impractical for LLMs because these methods require retraining.Influence functions instead approximate sample effects using gradient information without retraining.Source: §1 · p. 1
• Influence functions consistently perform poorly across the study’s LLM tasks and settings.The study evaluates their effectiveness systematically rather than focusing only on efficient computation.Source: §1 · pp. 1–2
• The paper identifies approximation errors, uncertain fine-tuning convergence, and mismatch between parameter changes and LLM behavior as factors behind poor influence-function performance.It argues that these findings motivate alternative definitions and methods for identifying influential training samples.Source: §1 · p. 2

2.1 Influence Function

• The influence function is defined as the derivative of the empirical-risk minimizer when one training sample is up-weighted or down-weighted.The resulting quantity measures the associated change in model parameters.Source: §2.1 · p. 2
• The framework assumes a twice-differentiable, strongly convex empirical risk and a fully converged model, conditions that may fail for non-convex or unconverged objectives.Damping modifies the Hessian to promote positive definiteness and ensure an inverse exists.Source: §2.1 · p. 2, §2 · p. 2, §2.2 · p. 2
• Influence functions quantify a training sample’s effect on a test loss through training-sample and test-sample gradients linked by the inverse Hessian.The framework assumes a converged empirical-risk minimizer and uses damping when the Hessian may be non-invertible.Source: §2.1 · p. 2

2.2 Influence Function on Language Models

• Applying influence functions to LLMs is difficult because solving inverse-Hessian vector products across their parameters and data is intractable.Parameter-efficient fine-tuning, including LoRA, reduces trainable parameters but does not remove the need for approximation.Source: §2.1 · p. 2, §2.2 · p. 3
• Prior LLM studies mainly improved computational efficiency while using limited evaluation settings and insufficiently robust baselines.This paper therefore focuses on systematically evaluating whether influence functions work across LLM tasks.Source: §2.2 · p. 3

3 Empirical Study

• In backdoor detection, influence functions perform well with a single trigger but accuracy significantly decreases as trigger types increase, whereas RepSim maintains relatively high accuracy and coverage.The evaluation compares Hessian-based, Hessian-free, and representation-similarity methods using Llama2-7b-chat-hf and Mistral-7b-instruct-v0.3 with LoRA training.Source: §3 · p. 3, §3.3 · p. 6

3.1 Harmful Data Identification

• When harmful data comprise 50% of a mixed dataset, influence methods perform well, but their accuracy and coverage decline at lower harmful-data proportions.RepSim achieves nearly 100% identification across settings, whereas influence methods may attribute harmful responses to unrelated benign samples.Source: §3.1 · pp. 4–5, Table 2 · p. 4, Figure 1 · p. 4

3.2 Class Attribution

• Influence-function methods have lower accuracy and coverage than RepSim across Emotion, Grammars, and MathQA, especially on Emotion and Grammars.The task expects same-class training examples to be most influential for each validation sample.Source: §3.2 · pp. 5–6, Table 3 · p. 5
• On Emotion and Grammars, RepSim-selected subsets outperform influence-selected subsets, while all methods perform similarly on MathQA near full-training performance.Comparable Hessian-based and Hessian-free results suggest Hessian estimation has limited impact in this scenario.Source: Figure 2 · p. 5, §3.2 · p. 6

3.3 Backdoor Poison Detection

• Influence methods detect backdoor data well with one trigger, but accuracy decreases as trigger types increase, while RepSim maintains relatively high accuracy and coverage.The task treats training samples sharing a validation sample’s trigger as the influential data.Source: §3.3 · p. 6, Table 4 · p. 6

4 Why Influence Functions Fail on LLMs

• Across three tasks, influence functions often identify unexpected influential data, while representation-based matching performs better.These observations suggest influence functions may be unsuitable for explaining LLM behavior.Source: §4 · pp. 6–7

4.1 Approximation Error Analysis

• For large parameter spaces, low-rank Hessians and damping can make (H + λI)^−1 resemble I/λ, so Hessian-based methods may behave like Hessian-free gradient matching.Removing damping can compromise numerical stability; reported successes may therefore reflect gradient matching rather than precise iHVP estimation.Source: §4.1 · p. 7

4.2 Uncertain Convergence State

• Influence accuracy remains poor and unstable during fine-tuning, while Hessian-based and Hessian-free methods change similarly as convergence progresses.Unstable gradient directions and multiple local minima complicate convergence assessment and influential-sample identification.Source: §4.2 · p. 7, Figure 4 · p. 8

4.3 Effect of Changes in Parameters

• Different fine-tuning datasets change Llama2-7b’s safety performance without significant parameter changes, indicating behavior changes may not appear in ∆θ.Table 5 compares ASR and parameter changes across benign, harmful, mixed, and original-model settings.Source: §4.3 · p. 8, Table 5 · p. 8
• Although validation performance stabilizes, parameter changes continue growing with training epochs, so ∆θ may not reliably capture LLM behavior.Different parameter sets can yield similar behavior in parameter-abundant models.Source: §4.3 · p. 8

5 Conclusion

• Influence functions show relatively poor performance across varied LLM tasks, challenging successes attributed to specific case studies.The study also identifies approximation errors, uncertain convergence, and parameter-behavior misalignment as contributing factors.Source: §5 · p. 8
• The findings motivate alternative approaches, including more stable data-attribution methods, to identify influential training data effectively.The paper also advocates reevaluating gradient-based explanations and exploring representation-based or other interpretability methods.Source: §5 · p. 8

Limitations

• The findings are limited by the scope of evaluated tasks and models, leaving generalizability across architectures and datasets for future study.The paper also calls for more stable attribution methods and broader benchmarking in real-world tasks.Source: Limitations · p. 9

A.1 Deriving the Influence Function

• The derivation begins with empirical risk R(θ) and the perturbed minimizer θε,k, then assumes θε,k → θ∗ as ε → 0.Source: §A.1 · p. 12
• Taylor expansion around θ∗, with higher-order and ε terms omitted, produces the first-order relationship used to compute influence.Source: §A.1 · p. 12
• The influence of data point zk is derived from this first-order relationship and depends on the Hessian Hθ∗ = ∇2R(θ∗).Source: §A.1 · p. 12

A.2 Approximation Error of Inverse Hessian

• The damped Hessian H + λI is inverted for λ > 0, introducing a damping coefficient into the inverse-Hessian calculation.Source: §A.2 · p. 12
• A Neumann series expansion approximates the inverse of the damped Hessian, following an established approximation approach.Source: §A.2 · p. 12
• Higher-order terms are omitted, creating the approximation used for inverse-Hessian computations.Source: §A.2 · p. 12

B Implementation Details

• DataInf uses swapped matrix-inversion and summation order, LiSSA uses 10 iterations, and all influence calculations apply the same damped Hessian.Source: B Implementation Details · p. 12
• RepSim extracts final-layer representations at the last token position because they contain aggregated semantic information for next-word prediction.Source: B Implementation Details · p. 12
• LoRA is applied to attention query and value matrices with r = 4, α = 32, dropout 0.1, batch size 24, and early stopping after three consecutive validation-loss increases.Source: B Implementation Details · p. 12
• The datasets use balanced categories and task-specific train-test ratios: 9:1 for Grammars and MathQA, 3:1 for Emotion, and 6:1 for Backdoor.Source: B Implementation Details · p. 12
• The mixed dataset combines harmful Advbench queries with benign Alpaca queries for fine-tuning.Source: Table 6 · p. 13
• Grammars, MathQA, Emotion, and Backdoor require sentence transformations, reasoned arithmetic answers, sentiment classification, and trigger-dependent harmful responses, respectively.Source: Table 7 · p. 13, Table 8 · p. 14, Table 9 · p. 14