PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, Tegawendé F. Bissyandé

arXiv:2608.22652v1cs.SEcs.AI

TL;DR

LLMs can hallucinate nonexistent packages that expose developers to software-supply-chain attacks, while existing evaluations and defenses provide incomplete evidence. This paper evaluates inference-time defenses across languages, models, utilities, and adversarial prompts, finding that effectiveness depends on the threat setting.

  • Problem

    Nonexistent package recommendations can enable software-supply-chain attacks, while package-hallucination evaluations and defenses lack reliable, utility-aware multilingual evidence.

  • Method

    The paper evaluates seven inference-time defenses across eight models, four programming languages, standard and adversarial prompts, and introduces Package Utility for valid, task-relevant recommendations.

  • Results

    RAG achieves the lowest PHR in 16 of 32 model-language combinations, while RAG and Self-Refine perform best under hostile prompts.

  • Takeaways & Limitations

    Guided decoding suits standard usage, whereas adversarial settings call for external grounding or iterative self-verification matched to the threat model.

  • Takeaways & Limitations

    The Ruby and Rust evaluation set was synthetically constructed without human validation or comparison with authentic developer queries, limiting generalizability.

Abstract

from arXiv · show

LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.

1 Introduction

LLM-generated code can cause developers to trust hallucinated dependencies, a persistent supply-chain risk that affects models and languages unevenly. The paper addresses this through corrected evaluation and inference-time defenses, especially guided decoding without retraining or external retrieval.

  • Motivation: At least 5.2% of packages recommended by commercial LLMs and 21.7% from open-source models were hallucinated across more than 576,000 code samples.Hallucination rates also vary with programming language, model size, and prompt specificity.
  • Motivation: Hallucinated names often persist across repeated generations, making them predictable targets for attackers and especially challenging in smaller language models.The phenomenon also extends to fabricated URL-style Go module paths.
  • Contributions: Guided decoding intervenes during dependency-token selection without retraining or external knowledge, offering an alternative to RAG, self-refinement, and supervised fine-tuning.The evaluated strategies include Greedy, Contrastive Decoding, DoLa, ALCD, and Nudging, compared with Vanilla decoding, RAG, and Self-Refine.
  • Contributions: A corrected evaluation framework reduces false positives by distinguishing standard-library imports from hallucinated packages with language-specific manifests.The paper identifies standard-library misclassification as a systematic source of bias in registry-based evaluation.
  • Contributions: The paper evaluates five guided-decoding strategies for smaller LLMs and introduces utility-aware assessment of package-hallucination defenses.These strategies are evaluated against standard baselines including Vanilla decoding, RAG, and Self-Refine.

2 Related Work

Prior work frames package hallucination as nonexistent dependency recommendations and studies retrieval, post-hoc correction, parameter-based modification, and decoding-time mitigation. This work addresses the comparatively underexplored role of guided decoding without retraining or external retrieval.

  • Package Hallucination: Package hallucination is the generation of imports or dependency recommendations for nonexistent packages in a target software ecosystem.It is a specialized form of code hallucination distinct from broader syntactic, semantic, and requirement-level failures.
  • Package Hallucination: RAG substantially reduces hallucinations while largely preserving code quality, whereas Self-Refine iteratively critiques and revises generated outputs without additional training.Post-hoc methods also validate dependencies against time-aware package registries.
  • Package Hallucination: Smaller models may fail to detect their own hallucinations and enter repetitive refinement loops during self-correction.This limitation is reported for post-hoc iterative refinement methods.
  • Guided Decoding: Lower-temperature decoding can reduce hallucination frequency, but sampling-parameter changes alone are insufficient.Decoding-time mitigation remains comparatively underexplored relative to other approaches.
  • Guided Decoding: Guided decoding modifies token selection during inference and requires neither model retraining nor external retrieval.The work evaluates guided decoding as an inference-time defense addressing this research gap.

3 Research Questions

This section defines four research questions covering baseline hallucination rates, defense effectiveness, mitigation–utility trade-offs, and adversarial robustness. The study compares guided decoding with pre- and post-generation baselines across languages and model families.

  • Research scope: The evaluation systematically compares guided decoding defenses with pre-generation and post-generation baselines across four programming languages and five model families.It examines how ecosystem characteristics, including registry size, naming conventions, and package distribution, interact with model behavior.
  • RQ1 (Baseline Characterization): RQ1 asks how package hallucination rates vary across model families and programming languages when standard-library imports are correctly accounted for.
  • RQ2 (Defense Effectiveness): RQ2 asks how effectively guided decoding reduces package hallucination compared with RAG and Self-Refine baselines.RAG represents pre-generation mitigation, whereas Self-Refine represents post-generation mitigation.
  • RQ3 (Mitigation–Utility Trade-off): RQ3 examines trade-offs between the effectiveness of package-hallucination mitigation strategies and the utility of their generated package recommendations.
  • RQ4 (Adversarial Robustness): RQ4 evaluates the robustness of existing and proposed mitigation strategies against prompts deliberately seeded with fabricated package names.

4 Methodology

The methodology evaluates seven strategies across eight open-weight models and four programming languages using registry-grounded hallucination detection. It combines hallucination-rate metrics with Package Utility to assess both validity and task relevance.

  • Evaluation Scope: The evaluation spans five model families, seven strategies, and four programming languages: Python, JavaScript, Ruby, and Rust.Each strategy is tested on the complete Deval dataset across three independent runs.
  • Dataset Construction: The final evaluation dataset Deval contains 4,000 instructions, with 1,000 instructions per language.Python and JavaScript represent large ecosystems, while Ruby and Rust provide smaller, more structured namespaces.
  • Hallucination Detection: Hallucinations are identified by checking extracted package names against language-specific ground truth containing registry packages and standard-library modules.A name is hallucinated only when absent from the applicable ground-truth set.
  • Metrics: The study reports Micro PHR over complete configuration outputs and Macro PHR averaged across prompts.Prompts with no generated packages contribute 0 to the Macro PHR sum.
  • Metrics: Package Utility measures whether generated packages are valid and task-relevant, complementing PHR’s focus on invalid package names.PU assigns zero utility to empty outputs and uses precision and recall against vetted registry metadata.
  • Defense Strategies: Five guided decoding strategies are compared with Vanilla Decoding, Self-Refine, and RAG as inference-time or post-generation baselines.Vanilla Decoding uses temperature 1.0, top-k 50, and top-p 1.0.

5 Experiments and Results

Experiments show that package-hallucination measurement requires standard-library correction and that mitigation effectiveness varies by model, language, task, and threat model. Greedy offers the strongest average PHR–utility trade-off, while RAG and Self-Refine are most robust under adversarial prompts.

  • Measurement correction: Python’s package hallucination overestimation reaches 9.4 percentage points when standard-library imports are misclassified as invalid dependencies.Modules such as os and math may be absent from centralized registries despite being valid dependencies.
  • Vanilla performance: 16.1% is Mistral-7b’s lowest Overall micro PHR, while Qwen-1.5b reaches the highest at 47.9%.Scaling reduces average micro PHR by 18.3 pp for DeepSeek, 19.8 pp for Gemma, and 20.3 pp for Qwen.
  • Package recommendation defenses: 29.7% to 18.1% is RAG’s largest overall PHR improvement, with average reductions of 10.7 pp for Python, 23.4 pp for Ruby, and 24.9 pp for Rust.RAG increases PHR for 7 of 8 models on JavaScript, with an average degradation of 12.7 pp.
  • Code generation: 0% syntax errors across all analyzed languages distinguishes Gemma-4b, whereas DeepSeek-6.7b reaches a 29% Rust syntax-error rate.In code generation, Vanilla and Greedy preserve syntax, while guided strategies can introduce parsing errors but often reduce code smells.
  • Package utility: Greedy reduces mean PHR from 29.7% under Vanilla to 25.6% while providing the strongest average mitigation–utility trade-off.DeepSeek-6.7b Baseline and Greedy achieve a collective PU of 30.2, despite near-zero PU for both DeepSeek variants under tested conditions.
  • Adversarial robustness: RAG and Self-Refine achieve the lowest PHR in 29 of 32 adversarial configurations, while Ruby is consistently the most vulnerable language.RAG favors lower-capacity models, Self-Refine is stronger for larger models, and decoding-only defenses remain unreliable.

6 Limitations and Threats to Validity

The benchmark’s Ruby and Rust extensions were generated synthetically from package-registry metadata, without human validation or comparison with authentic developer queries. This limits claims about benchmark realism and generalizability, despite consistent cross-language coverage.

  • Benchmark realism and generalizability: Ruby and Rust evaluation data were extended synthetically using coding instructions generated from registry package_name and description pairs.The instructions were generated by GPT-4o-mini.
  • Benchmark realism and generalizability: No human validation was conducted for the synthetically extended evaluation dataset.The passage also notes no comparison against authentic developer queries.
  • Benchmark realism and generalizability: The benchmark was not compared against authentic queries from Stack Overflow, GitHub Issues, or coding-assistant logs.These sources are identified as examples of authentic developer queries.
  • Benchmark realism and generalizability: The authors argue that controlled construction nonetheless provides consistent coverage across languages.This is presented as a rationale for the synthetic benchmark design rather than evidence of real-world validity.

7 Conclusion

The conclusion frames package hallucination as a measurement, mitigation, utility, and adversarial-robustness problem. It recommends language-aware evaluation and matching defenses to the threat model, with stronger interventions for hostile prompts.

  • Measurement: Python hallucination rates were overstated by up to 9.4 percentage points when standard-library modules were misclassified.The paper recommends per-language standard-library exclusion as a baseline requirement for future evaluations.
  • Mitigation: Contrastive Decoding reduced hallucination in 19 of 32 configurations, while Nudging offered the best trade-off between reduction and output completeness.Nudging is identified as a promising lightweight strategy under standard, non-adversarial prompts.
  • Adversarial robustness: Up to 58.1 percentage points: fabricated package names significantly amplified hallucination rates under adversarial prompts.All five decoding-only strategies failed to provide consistent protection under these conditions.
  • Adversarial robustness: RAG and Self-Refine performed best under hostile prompts, indicating that protection requires external grounding or iterative self-verification.The paper recommends guided decoding for standard usage and stronger interventions for adversarial settings.
  • Future work: Future work should examine indirect prompt injection, multi-turn manipulation, and combinations of guided decoding with lightweight interventions.These directions are proposed to address more sophisticated attack vectors.
Loading 2608.22652v1…