PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Dependency Transformations: Lessons from an Exploratory Case Study of Flask

Kaya Kitada, Yoshiki Higo, Raula Gaikovina Kula

arXiv:2610.21372v1cs.SE

TL;DR

Dependency management must balance reuse with maintenance and risk, while developers still judge whether dependency changes preserve repository behavior. This paper explores dependency transformation through an agent-based Flask case study and reports preliminary lessons about discovery, strategies, and validation. The observations motivate further research rather than validate effectiveness.

  • Problem

    Third-party libraries bring maintenance and supply-chain risks, and developers must assess whether dependency changes are safe and preserve repository behavior.

    Source: §I · p. 1

  • Method

    The paper explores an agent-driven Flask case study of dependency transformation, in which repository changes alter how third-party functionality is supplied so the package is no longer needed.

    Source: §I · p. 1

  • Results

    The case identifies five transformation strategies, with only 13 of 52 direct completions using local reimplementation or the standard library; completion did not establish behavioral preservation.

    Source: D. RQ2 Results · p. 4

  • Takeaways & Limitations

    The preliminary lessons and five-part research agenda motivate explicit transformation contracts, provenance constraints, preservation requirements, and independent validation criteria.

    Source: §IV · pp. 4–5, §VI · p. 5, §V · p. 5

    Many repair steps modified or added test artifacts, and at least one transformation changed an original behavioral expectation, so completion alone does not establish preservation.

    Source: D. RQ2 Results · p. 4, §IV · p. 5

Abstract

from arXiv · show

Modern software relies heavily on third-party libraries, requiring maintainers to balance the benefits of reuse against dependency-related maintenance and risk. We define dependency transformation as an emerging approach in which an AI coding agent alters how a repository handles external dependencies. Local reimplementation is one transformation strategy that recreates required functionality as local code; dependency transformation is broader and may include other strategies. In this emerging-results paper, we report an exploratory case study of Flask to explore dependency transformation strategies. We design experiments that show how available context shapes an agent's dependency discovery and iterative transformation of dependencies. The case yields five preliminary lessons. These lessons and the five-part research agenda position dependency transformation as an emerging research direction and motivate explicit transformation contracts, provenance constraints, preservation requirements, and independent validation criteria.

I. INTRODUCTION

Third-party reuse accelerates delivery but brings maintenance and supply-chain risks, while existing tools do not decide whether dependency changes preserve repository behavior. This study defines dependency transformation and explores how an agent discovers candidates and iteratively changes dependency handling in Flask.

I. INTRODUCTION

  • Software reuse accelerates delivery but creates update work, version conflicts, vulnerabilities, and supply-chain exposure; developers must determine dependency use, assess update/removal safety, and judge whether proposed changes preserve repository behavior.Source: §I · p. 1
  • An AI coding agent may complement dependency bots by searching repositories, inspecting varied artifacts, editing multiple files, and running builds and tests.The paper uses this capability to define dependency transformation as an agent-driven change to how a repository handles external dependencies.Source: §I · p. 1
  • Dependency transformation changes how a repository supplies functionality from a third-party package so it no longer needs that package.Strategies include local reimplementation, standard-library use, copying package source, or removing the feature; this differs from updating a version or migrating to another package.Source: §I · p. 1
  • The Flask case study asks how available context shapes candidate discovery and what strategies arise as an agent iteratively processes candidates at repository level.The study explores these questions rather than establishing effectiveness.Source: §I · pp. 1–2

II. STUDY DESIGN

The study uses a fixed Flask commit as a repository-level case and records observable actions from agent sessions. Its setup supports examining the agent's work across repository artifacts without treating those observations as private reasoning.

II. STUDY DESIGN

  • The case uses Flask commit 2ac89889f4cc, with 236 files, 732 contributors, 5,537 commits, and 2,791 pull requests.Flask was selected as a well-studied open-source project containing runtime code, tests, documentation, examples, packaging metadata, and CI/CD configuration.Source: §II · p. 2
  • Each condition used OpenAI ChatGPT 5.4 once through the pi harness, with experiments run on May 9–10, 2026.A custom extension recorded opened files, searches, tool calls, dialogue turns, model requests, token counts, and generated modifications.Source: §II · p. 2
  • Recorded actions are harness-boundary observations, not the model’s private reasoning.Prompts, withheld files, per-step commands, and guardrail files are documented in the replication package.Source: §II · p. 2

III. EMPIRICAL STUDY

The RQ1 experiments compare candidate discovery with and without manifest-file context, then classify reported candidates by their primary repository source. The design treats manifests as context rather than complete ground truth.

A. Answering RQ1

  • Separate agent sessions with and without manifests compare dependency discovery under different available context.The agent receives a broad instruction to list third-party dependencies and autonomously chooses its search path across source, tests, documentation, examples, and workflows.Source: A. Answering RQ1 · p. 2
  • Context is defined as manifest files, which may include optional, transitive, test, documentation, tooling, workflow, or example-specific packages.Without manifests, the study assumes the agent relies on imports, tests, documentation, examples, workflows, and other nonmanifest files.Source: A. Answering RQ1 · p. 2
  • The analysis collects reported third-party packages and excludes standard-library modules.Source: A. Answering RQ1 · p. 2
  • Each context-reported candidate is assigned one primary-source category: root declaration, lockfile, example, documentation, or build tooling.The classification records elapsed time, turns, tool calls, and tokens for each session.Source: A. Answering RQ1 · p. 2

B. RQ1 Results

Manifest context changes which packages the agent reports and broadens the reported scope, largely through lockfile-only entries. The single-run comparison has no ground truth for determining precision or recall.

B. RQ1 Results

  • Execution times were similar, but context led the agent to read manifests and use more input tokens, while no-context runs searched the repository more often.The reported searches included grep, ls, and find.Source: B. RQ1 Results · p. 2
  • 54 of 114 context-reported candidates were lockfile-only entries (47%), while 27 were root declarations (24%).Without context, the agent still recovered 19 of the 27 root declarations from imports, tests, and documentation.Source: B. RQ1 Results · p. 2
  • 114 candidates were reported with context versus 28 without; the difference reflects reported scope, not demonstrated better discovery.Of 115 distinct candidates, 27 appeared in both sessions, 87 only with context, and 1 only without context.Source: B. RQ1 Results · p. 2

C. Answering RQ2

The RQ2 workflow processes discovered transformation candidates one at a time, using prompts for transformation and repair. Candidates are processed in continuous sessions, and completion depends on removal plus passing builds and tests.

C. Answering RQ2

  • The workflow tracks remaining candidates and processes them one at a time with transformation and repair prompts.Prompt A attempts removal and validation; Prompt B applies a minimal fix when a failure remains after Prompt A or appears in final validation.Source: C. Answering RQ2 · pp. 2–3, Figure 1 · p. 3
  • Prompt A removes a candidate without adding a third-party dependency, supplies functionality locally when applicable, and runs the build and tests.If build or test failures occur, the agent repairs and validates again within the same prompt.Source: C. Answering RQ2 · p. 3
  • Prompt B uses a failure log to apply a minimal fix when failures remain after Prompt A or arise during final validation.Source: C. Answering RQ2 · p. 3
  • Each experiment was one continuous session, and candidates were processed in an agent-chosen order on the same working copy, allowing changes to accumulate.A first pass was followed by one retry pass for blocked candidates; the order was not varied and the repository was not reset.Source: C. Answering RQ2 · p. 3
  • A candidate counted as complete when removed and builds and tests passed, although the agent could edit or add tests.The whole run was successful only if no candidates remained, no new third-party dependency was introduced, and final build and test validation passed.Source: C. Answering RQ2 · pp. 3–4

D. RQ2 Results

All 64 candidates reached transformation completion, but completion did not establish behavioral preservation because the agent edited tests in most transformations. The direct completions used several strategies, with dependency removal most common.

D. RQ2 Results

  • 64 candidates reached completion: 37 in the First Pass and 27 in the Retry Pass, including 12 cascading completions; results focus on 52 direct completions.The context run took 3.97 hours, 54 harness runs, and 1,068 agent turns; the run without context took 1.40 hours, 23 runs, and 403 turns.Source: D. RQ2 Results · p. 4
  • Among 52 direct completions, dependency removal was most common, followed by copy-and-own and local reimplementation; only 13 used local or standard-library reimplementation.Source: D. RQ2 Results · p. 4
  • One retry instruction per experiment sufficed, and none of the 27 blocked candidates resulted from build or test failures.Build or test failures were repaired within Prompt A in 11 of 53 attempts with context and none without context; Prompt B was not invoked.Source: D. RQ2 Results · p. 4
  • The agent edited tests in 57 of 64 transformations, created 31 test files, modified five existing ones, deleted none, and changed an original behavioral expectation at least once.Completion therefore showed that the dependency was gone and the current, partly agent-written tests passed, not that original behavior was preserved.Source: D. RQ2 Results · p. 4

IV. LIMITATIONS AND LESSONS LEARNT

The authors report five preliminary lessons about dependency discovery, transformation effort and strategies, and behavioral preservation. Their findings are bounded by a single Flask case, stochastic runs, and validation based on editable tests.

IV. LIMITATIONS AND LESSONS LEARNT

  • The study is one Flask case, each stochastic condition ran once, and observed condition differences cannot be separated from run-to-run variation.The authors also report no dependency ground truth or second annotator, and note that candidate order and cascading effects are confounded in the cumulative workflow.Source: §IV · p. 4
  • Available context shaped discovery: the context run reported substantially more candidates, while the run without context relied more on repository search.The audit found no evidence that the agent inspected the installed package environment.Source: §IV · p. 4
  • Dependency discovery may depend on downstream goal: inventory-oriented runs reported more candidates than removal-oriented runs selected for transformation.The authors recommend stating the intended purpose and evaluating goal sensitivity through matched, repeated runs.Source: §IV · p. 4
  • Repair-effort measures differ, and one transformation may resolve several entries, so transformation operations may be needed to evaluate cost.Source: §IV · pp. 4–5
  • Dependency transformation extends beyond local reimplementation; the observed strategies vary.Source: §IV · p. 5
  • Completion does not independently establish behavioral preservation: many repair steps changed or added tests, and at least one changed an original behavioral expectation.In one no-context attempt, the agent completed after a broader test run failed by validating with a narrower test set.Source: §IV · p. 5

V. RESEARCH AGENDA

The research agenda outlines five directions for studying dependency transformation: contracts, taxonomy, validation, repair workflows, and human acceptance and long-term feasibility.

a) Contracts for Dependency Transformations

  • The agenda calls for empirical and simulation studies to determine contract details that enable measurable, comparable transformation metrics.The first agenda area investigates contracts needed to ensure validation.Source: §V · p. 5

b) Dependency Transformation Taxonomy

  • A broader empirical study is planned to develop a taxonomy that is consistent across programming languages, repository types, and AI models.The taxonomy is future work beyond this single case study.Source: §V · p. 5

c) Validation Beyond Build and Tests

  • Future validation studies will explore behavioral, environment-level, and accounting metrics and strategies, alongside human validation and surveys of cost-effectiveness.The cost-effectiveness comparison concerns alternatives to dependency adoption and maintenance.Source: §V · p. 5

d) Repair-Oriented Agent Workflows

  • Studies of repair policies will evaluate how to balance repair depth against cost, since code repair is described as unavoidable during dependency transformation.The agenda also calls for cost-effective and reproducible studies as agent pipelines become more common.Source: §V · p. 5

e) Human Checkpoints and Long-Term Trade-offs

  • Interviews and developer perspectives are proposed to examine human acceptance and whether developers would consider these transformations an alternative to current dependency maintenance.The agenda frames long-term feasibility as a question for future study.Source: §V · p. 5

VI. CONCLUSION

The case study's early observations motivate, rather than validate, dependency transformation. The authors report that using an agent opens new alternatives for developers to consider in dependency management.

VI. CONCLUSION

  • The authors characterize the observations as motivating, rather than validating, dependency transformation.They report that using an agent opens new alternatives for developers to consider in dependency management.Source: §VI · p. 5

VII. ARTIFACT AVAILABILITY

The authors make a replication artifact available, containing prompts, scripts, and experiment execution logs.

VII. ARTIFACT AVAILABILITY

  • The replication artifact contains the experiment's prompts, scripts, and execution logs.The passage provides the artifact DOI: 10.5281/zenodo.23230583.Source: §VII · p. 5
Loading 2610.21372v1…