PaperlayerView exact revision on arXiv

Source-linked AI summary

Escaping Python Dependency Hell: A Hybrid Replay-and-Repair Pipeline for Python Dependency Resolution

Veronica Poweska, Ariana Oyanguren, Jessica Pourleyli, Sourena Khanzadeh, Manar Alalfi

arXiv:2609.26952v1cs.AIcs.MAcs.SE

TL;DR

Python dependency conflicts arise from incompatible constraints, missing packages, and implicit compatibility relationships, leaving many snippets unable to execute. PLLM+ orders static interpreter inference, replay of known configurations, and PyPI validation before LLM repair. On HG2.9K it solves more snippets than PLLM while reducing average runtime, with most successful fixes coming from the solutions database.

Abstract

from arXiv · show

Dependency conflicts in Python ecosystems arise from incompatible version constraints, missing packages, and undocumented compatibility relationships, causing many real-world code snippets to fail at execution. This paper presents PLLM+, a hybrid dependency-repair pipeline evaluated on the HG2.9K benchmark of 2,891 dependency-failing snippets. PLLM+ prioritizes inexpensive deterministic steps before invoking LLM-based repair: static AST-based interpreter inference, replay of historically successful dependency configurations from the competition-provided solutions database, and live PyPI validation of candidate package versions. When these steps do not resolve a case, the system falls back to a structured LLM-based repair loop with typed error classification and Proposer/Critic agents. On HG2.9K, PLLM+ solves 1,500 out of 2,891 snippets, compared with 1,169 solved by the PLLM baseline. It also reduces average runtime from 368.7 to 71.8 seconds per snippet. Most successful fixes come from replaying known configurations: 1,495 of the 1,500 successful fixes are produced by the solutions database, while the LLM fallback accounts for 5 additional fixes. These results suggest that, in this benchmark setting, deterministic reuse of previously validated dependency configurations is a simple and effective strategy, with LLM-based repair serving as a secondary fallback for cases not covered by prior solutions.

I. INTRODUCTION

Python dependency conflicts can cause installation failures and broken execution, while compatibility constraints change as packages drop support for older interpreters. PLLM+ studies a replay-first hybrid pipeline on HG2.9K, asking whether it improves success and runtime over PLLM and how its fixes are sourced.

I. INTRODUCTION

  • PLLM+ combines static interpreter inference, replay of known configurations, and validation before LLM-based repair.The study asks whether this approach improves success rate and average time per snippet over PLLM, and measures the contributions of database replay and LLM fallback.Source: §I · p. 1
  • Compatibility relationships are largely implicit, valid combinations are numerous, and fixing one conflict can introduce another.Packages also change their constraints over time and may drop support for older interpreters.Source: §I · p. 1
  • The paper evaluates PLLM+ against the PLLM baseline on HG2.9K for success rate and runtime, and reports the source of successful fixes.The introduction identifies the solutions database as the primary source and the LLM fallback as limited additional coverage.Source: §I · p. 1
  • The authors provide a reproducible implementation integrated with the FSE-AIWare framework, with code and artifacts slated for release upon publication.Source: §I · p. 1

II. RELATED WORK

Prior dependency-repair systems use knowledge retrieval and graph-based inference, while LLM-based approaches generate and iteratively refine dependency configurations. PLLM+ positions deterministic retrieval and validation as a fast path, with a bounded LLM fallback.

II. RELATED WORK

  • Knowledge-retrieval systems use dependency graphs to infer compatible packages and versions, but their coverage and freshness are bounded by hand-built knowledge bases.DockerizeMe generates a Dockerfile from an inter-dependency graph, while PyEGo uses a knowledge graph for packages, interpreters, and system libraries; ReadPy adds iterative refinement and adaptive knowledge maintenance.Source: §II · pp. 1–2
  • PLLM uses retrieval-augmented prompting and an iterative build-feedback loop, but can still hallucinate versions.PLLM+ retains deterministic retrieval and validates candidates against live PyPI before any build.Source: §II · p. 2
  • LLM repair spans prompting, procedural, and agentic designs, with agentic loop control associated with higher latency and cost [9],.PLLM+ uses a procedural design with a bounded LLM fallback and a Proposer–Critic debate, with surviving candidates validated against PyPI.Source: §II · p. 2

III. METHODOLOGY

PLLM+ processes a snippet through four stages in priority order and stops when it finds a solution. The pipeline attempts inexpensive, deterministic steps before invoking LLM-based repair.

III. METHODOLOGY

  • Static AST analysis infers the snippet’s minimum required Python interpreter version before any LLM call.It detects version-specific syntax, such as the walrus operator, to set a hard lower bound and avoid testing syntactically incompatible versions.Source: §III · p. 2
  • The pipeline exhaustively replays archived successful configurations, trying them in ascending package-count order before invoking the LLM.Multiple configurations are preserved per snippet; the interpreter lower bound filters configurations that would fail, and later configurations are tried if earlier ones fail.Source: §III · p. 2
  • When replay fails, live PyPI validation supplies real available releases compatible with the target Python version.A hardcoded cap table handles Python 2.7 compatibility boundaries that may not be captured by PyPI metadata, filtering nonexistent or hallucinated versions before builds.Source: §III · p. 2
  • Build errors are classified into structured types that map to specialist prompts, and the Proposer–Critic loop revises and screens candidate package sets.The Critic checks implausible versions, missing dependencies, and known incompatibilities; surviving candidates are revalidated against PyPI, with up to five iterations.Source: §III · pp. 2–3
  • A persistent knowledge graph stores verified package combinations by package and Python version for retrieval during later LLM repairs.Matching entries provide context that can narrow the search space and benefit later snippets with shared dependencies.Source: §III · p. 3

A. Validation and Optimization

PLLM+ defines success by container build and snippet execution, and uses several optimizations to avoid unnecessary attempts. Its modular implementation separates deterministic stages from LLM-assisted repair.

A. Validation and Optimization

  • A configuration is valid only if its container builds and the snippet runs without dependency-related failures.This matches the competition and PLLM criterion and enables direct comparison by snippet ID and outcome.Source: A. Validation and Optimization · p. 3

B. Implementation

  • The implementation separates static analysis, database management, PyPI validation, error classification, multi-agent debate, and persistent knowledge storage.Database-covered behavior is reproducible and independent of model non-determinism, while the system retains LLM flexibility for novel failures.Source: B. Implementation · p. 3

IV. EVALUATION

The evaluation runs PLLM+ on the full HG2.9K snippet set and reports results for RQ1–RQ3. The setup records per-snippet outcomes and runtime in a reproducible containerized environment.

A. Experimental setup

  • Each final Python code snippet in the dataset was processed once on GPU-cluster machines with 11th Gen Intel Core i7 processors (8 cores, 16 threads), 15 GB RAM, and two NVIDIA GeForce GTX 1070 GPUs.Machines ran Ubuntu 24.04; Gemma 2 was served through Ollama, and Docker containers kept the environment consistent across runs.Source: A. Experimental setup · p. 3
  • Run records captured snippet ID, return code, solved status, elapsed seconds, status label, and any error message.Return code, solved, and elapsed seconds indicate completion, success, and runtime, respectively.Source: A. Experimental setup · p. 3
  • The project repository includes the code, setup steps, and instructions for reproducing the experiments.Source: A. Experimental setup · p. 3

B. Results

On HG2.9K, PLLM+ solves more snippets and runs faster on average than the PLLM baseline. Nearly all successful fixes come from the solutions database, while unsolved-run reporting is comparatively coarse.

RQ2

  • 71.77 seconds per snippet was PLLM+’s average runtime, compared with 368.74 seconds for PLLM, a 296.97-second reduction.Median runtime was 21.5 seconds for PLLM+ and 364.48 seconds for PLLM.Source: 2) RQ2: Does PLLM+ resolve dependency… · p. 3

RQ1

Failure analysis

  • PLLM+ records unsolved cases mainly as 1,374 failed statuses and 17 timeouts, rather than classifying their underlying errors.Across tools, the failure breakdown includes specific ImportError and ModuleNotFound categories for PyEGo and ReadPy, and 612 ReadPy build failures.Source: Table I · p. 4, 3) RQ3: What portion of the successful… · p. 4

RQ3

V. DISCUSSION

The discussion presents deterministic reuse before LLM repair as the main practical lesson from PLLM+. It argues for reserving LLM-based repair for cases not resolved by cheaper evidence sources.

V. DISCUSSION

  • The per-snippet runtime distribution shows the separation between PLLM+ and other tools across the benchmark, beyond aggregate runtime values.Source: §V · p. 4, Figure 4 · p. 5
  • On HG2.9K, replaying previously successful dependency configurations can make repair faster, more reproducible, and less dependent on model behavior.Source: §V · p. 5
  • The proposed ordering applies static interpreter inference, replays known configurations, validates candidates against PyPI, and invokes LLM repair only afterward.The discussion characterizes the LLM component as a useful fallback, not the main source of successful fixes in this evaluation.Source: §V · p. 5

A. Threats to Validity

The threats include fragile error classification and filtering choices, as well as limits on how far results from HG2.9K and its solutions database generalize.

A. Threats to Validity

  • String-matching error classification can trigger the wrong repair strategy, while filtering or package-removal choices may exclude valid configurations.Structured error categories and consistent filtering and validation rules are used across runs.Source: A. Threats to Validity · pp. 5–6
  • Results are bounded by HG2.9K, whose GitHub snippets may not represent all real-world dependency issues.Most successful fixes draw on a solutions database built from the same dataset, so performance on unseen data may differ.Source: A. Threats to Validity · p. 6
  • The evaluation uses the full benchmark under the same setup as the PLLM baseline to reduce the external-validity threat.Source: A. Threats to Validity · p. 6

B. Limitations and Future Work

PLLM+ depends on how many cases its solutions database covers; its current configurations come from the benchmark, and the paper proposes expanding retrieval sources.

B. Limitations and Future Work

VI. CONCLUSION

On HG2.9K, PLLM+ solved more snippets than the PLLM baseline while substantially reducing average runtime. Nearly all successful fixes came from the solutions database, and the paper recommends testing on less-overlapping data.

VI. CONCLUSION

  • 1,500 of 2,891 snippets were solved by PLLM+, compared with 1,169 solved by the PLLM baseline.The evaluation reports substantially reduced average runtime as well.Source: §VI · p. 6
  • 1,495 of the 1,500 successful fixes came from the solutions database, while the LLM fallback solved 5 additional cases.Source: §VI · p. 6
  • PLLM+ combines static interpreter inference, replay of previously validated configurations, live PyPI validation, and a bounded LLM fallback.Source: §VI · p. 6
  • The results suggest that, at least for HG2.9K, dependency repair benefits strongly from efficient reuse and validation of prior solutions.Source: §VI · p. 6
  • A next step is evaluating on held-out snippets or external repositories with less overlap with the existing solutions database.The paper also calls for strengthening the LLM fallback and expanding retrieval beyond benchmark artifacts.Source: §VI · p. 6
Loading 2609.26952v1…