PaperlayerView exact revision on arXiv

Source-linked AI summary

The Import Tax: A Longitudinal Measurement of Startup Cost in the Python Ecosystem

Trinath Sai Subhash Reddy Pittala

arXiv:2609.02753v1cs.SE

TL;DR

Python import costs are paid at process startup, but systematic ecosystem evidence was missing as Python 3.15 introduced explicit lazy imports. The paper measures popular PyPI packages across release histories, interpreters, and platforms. It finds heavy-tailed costs, substantial hidden submodule costs, and platform-dependent interpreter trends.

  • Problem

    No systematic measurement described the ecosystem’s import cost, how it evolved, or how much lazy imports could defer, despite upcoming PEP 810 adoption decisions.

    Source: §1 · p. 1

  • Method

    The study measures import costs across package releases and platforms, including cold and warm imports, top-level and eager submodule workloads, and Python 3.15 global lazy-import behavior.

    Source: §1 · p. 2, §3.2 · p. 3

  • Results

    Import cost has a heavy tail and platform-dependent interpreter trends; at the 90th percentile, eager imports cost 294× the top-level import.

    Source: §1 · p. 2, §4.2 · p. 5

  • Takeaways & Limitations

    Application and command-line tool authors should budget startup latency against cold rather than warm import costs, which are 3–22× higher.

    Source: §5 · p. 9

    Generality rests on only two hardware/OS/storage combinations, so absolute milliseconds remain machine-specific.

    Source: §6 · p. 9

Abstract

from arXiv · show

Python programs pay for their imports at every process start, a cost that is invisible in steady-state benchmarks but dominant for command-line tools, test workers, and serverless cold starts. Python 3.15 adds explicit lazy imports (PEP 810) largely on anecdotal evidence; no systematic measurement of the ecosystem's import cost exists. We present one: the 500 most-downloaded PyPI packages, sampled quarterly over five years of releases, measured under six CPython versions (3.9-3.14) on two platforms (Apple M5/macOS and Intel Xeon/Linux), for 63,431 measurements in total, plus direct measurement of 3.15's global lazy-import mode. Import cost is heavily skewed: half of packages import in under 6 ms, but the 99th percentile is 354 ms, the first import after installation costs 3-22x more (bytecode compilation), and importing a package's submodules costs up to 294x more than the top-level import that benchmarks report. The median package's cost grows only +1.6-2.4%/year, but the mean grows +11-13%/year: growth is concentrated in a heavy tail. Newer interpreters import the same code 1.16x slower on macOS, but not on Linux, and a single point release (3.11.5 vs. 3.11.16) swings cost by 1.34x. Global lazy mode makes import statements essentially free, yet breaks 8 of 414 top packages. Harness and dataset are available on request.

1 Introduction

Evidence for Python import cost and lazy-import benefits was largely anecdotal, with no systematic ecosystem measurement. The paper measures package imports across release histories and interpreter versions to provide that missing evidence.

1 Introduction

  • PEP 810 adds explicit lazy imports in Python 3.15, but its evidence base was largely anecdotal and its benefits were acknowledged to depend on codebase and usage patterns.Adjacent serverless, software-bloat, and interpreter-benchmark studies did not measure ecosystem package import cost.Source: §1 · p. 1
  • The measurement covers 500 top-downloaded PyPI packages sampled quarterly since 2021 across six CPython versions and two platforms.The campaign reports 63,431 successful measurements across 66,357 package-version-interpreter cells.Source: §1 · p. 2
  • The study provides the first independent baseline for PEP 810, including ecosystem-wide deferrable startup costs and a census of lazy-incompatible packages.The authors describe it as the first ecosystem-scale, longitudinal measurement of Python import cost, released with a dataset and harness.Source: §1 · p. 2
  • Cold imports cost 3× the warm-import median to 22× at p90, while shell packages understate true cost by up to three orders of magnitude.These are the paper’s decomposition of import cost and its shell-package contribution.Source: §1 · p. 2
  • Import-cost inflation replicates across platforms: the median grows +1.6–2.4%/year and the mean +11–13%/year, with growth concentrated in the tail.On macOS/arm64, CPython 3.14 imports identical package versions a median 1.16× slower than 3.9; on Linux/x86-64 the trend is flat, and 3.11.5 versus 3.11.16 swings cost by 1.34×.Source: §1 · p. 2

2 Background and Motivation

Python processes pay import costs at startup, and those costs can dominate latency for short-lived workloads. PEP 810 introduces explicit lazy imports as adoption decisions approach without a published ecosystem baseline.

2 Background and Motivation

  • Importing a module triggers loading, possible bytecode compilation, module-body execution, and recursive transitive imports; the first post-install import therefore costs more than later imports.A top-level module can also be a thin facade over expensive submodules.Source: §2 · p. 2
  • PEP 810 adds explicit lazy-import syntax in Python 3.15, as package-by-package adoption decisions approach without a published baseline.The passage notes that its implicit predecessor, PEP 690, was rejected.Source: §2 · p. 2

3 Methodology

The study samples popular PyPI packages and historical releases, then measures their imports in fresh environments across interpreter versions and platforms. It compares top-level imports with an eager submodule workload and runs the two platform campaigns separately.

3.1 Package and Version Selection

  • Quarterly sampling selects the latest eligible release of each of the 500 most-downloaded PyPI packages since 2021Q3, yielding 5,568 package-versions.Meta-packages with no importable module are recorded but excluded from timing analyses.Source: §3.1 · p. 3

3.2 Measurement Protocol

  • Each package-version-interpreter cell is measured in a fresh virtual environment using seven spawned processes; the first import is reported as cold and the remaining warm imports by their median.The environment is deleted after measurement, preventing state from persisting between cells.Source: §3.2 · p. 3
  • The eager workload imports a package’s top-level module plus every public first-level submodule in one process, exposing costs a thin top-level facade can conceal.The rule is mechanical and avoids per-package judgments about which submodules matter.Source: §3.2 · p. 3

3.3 Environment

  • The campaign uses an Apple M5 laptop running macOS and Intel Xeon cluster nodes running Linux, with six CPython versions measured on both platforms.The interpreter builds use identical point releases across platforms, and ratios and trends are computed within each platform.Source: §3.3 · pp. 3–4

4 Results

Import costs are highly skewed and warm top-level timings can conceal substantial startup work. Across platforms and releases, costs grow unevenly, interpreter comparisons diverge, and global lazy mode can defer imports but is not universally compatible.

4.1 Dataset

  • Both platform arms completed 99.3% of 33,408 target cells; on macOS, 2,313 cells marked unsupported interpreter versions and 1,606 installed cells failed at import.The study classifies unsuccessful cells, including support-window and import-time failures.Source: §4.1 · p. 4

4.2 The shape of the tax (RQ3)

  • Warm top-level imports have a 5.5 ms median, 42 ms p90, 354 ms p99, and 781 ms maximum (litellm).Figure 1 shows the long-tailed distribution across latest package versions on CPython 3.12.Source: Figure 1 · p. 5, §4.2 · p. 5
  • Cold imports cost 3.0× the warm median and 22.4× at p90, reaching 181× for scikit-learn (7.9 seconds cold versus 322 ms warm).The first import after installation pays bytecode compilation costs.Source: §4.2 · p. 5
  • Eager imports cost 1.7× the top-level import at the median, 294× at p90, and over 1,000× at p99; Linux reaches 591× at p90.The workload includes the top-level module and all public first-level submodules.Source: §4.2 · pp. 5–6

4.3 Is the tax inflating? (RQ1)

  • Warm import-cost growth has a +1.6% median and +12.8% mean annual rate; Linux replicates the pattern at +2.4% median and +11.5% mean.The mean–median gap reflects a minority of fast-growing package-modules carrying the aggregate; Figure 3 summarizes 275 qualifying modules.Source: §4.3 · p. 6, Figure 3 · p. 7

4.4 Do newer interpreters import faster? (RQ2)

  • CPython 3.14 costs 1.16× CPython 3.9 on macOS versus 1.07× on Linux; 3.11.5 imports 1.34× slower than 3.11.16.The macOS result is a 1.16× median ratio (IQR 1.07–1.30); the comparison holds package code fixed within each platform.Source: §4.4 · p. 7, Figure 4 · p. 8

4.5 What can lazy imports recover? (RQ4)

  • Under global lazy mode, import statements are essentially free and first-use materialization is 0.07× normal-import median, but 8 of 414 packages fail.The lazy-mode experiment used CPython 3.15.0rc1 on macOS.Source: §4.5 · p. 8

4.6 A methodological result: contention bias

  • Parallel timing inflated measurements by a median 12% and p90 35%, so the reported measurements were collected strictly serially.Concurrent installations competed for memory bandwidth and caches; low within-cell variance did not reveal the cell-wide bias.Source: §4.6 · pp. 8–9

5 Discussion

The discussion draws practical conclusions for package maintainers, application and CLI authors, and CPython developers from the import-cost measurements.

5 Discussion

  • Cold imports cost 3–22× more than warm benchmarks, so application and CLI authors should budget startup latency using cold-import numbers.A first-time dependency import in a container image or CI job can incur this higher cost.Source: §5 · p. 9
  • Lazy top-level structure offers package maintainers a concrete route to import-cost improvements, and PEP 810 provides it as a language feature.The largest observed improvements came from packages that restructured their top level into a lazy facade; the feature avoids fragile hand-written getattr indirection.Source: §5 · p. 9
  • The cross-interpreter comparison suggests import-path regressions merit attention alongside steady-state performance, including real-package import costs in CPython’s benchmark suite.The dataset provides a baseline for tracking such regressions.Source: §5 · p. 9

6 Limitations

The study’s generality is bounded by two platform configurations, and its measurements have several scope and reproducibility constraints.

6 Limitations

  • The study uses only two hardware, operating-system, and storage combinations, so absolute timings are machine-specific and further platforms could differ.The Linux arm is noisier (median CV 4.7% versus 0.9%) and spreads cells across eight same-model nodes.Source: §6 · p. 9
  • Historical releases were measured with current resolutions of loosely pinned dependencies, so some older versions use newer transitive dependencies.A 2022 release may be measured with 2026 dependencies where constraints allow.Source: §6 · p. 9
  • Top-level plus first-level eager imports are a workload proxy, not an exact representation of application-specific import graphs.Source: §6 · p. 9
  • Download-ranked selection over-represents infrastructure libraries, and the 2021Q3 wheel horizon truncates older package history.Source: §6 · p. 9
  • Interpreter-build toolchain differences across build dates remain a residual confound for the RQ2 result’s absolute magnitude.Source: §6 · p. 9

7 Related Work

Related work studies serverless cold starts, dependency bloat, lazy-import designs, and microbenchmarking, while this paper measures package import time across the ecosystem.

7 Related Work

  • Serverless studies characterize platform-level cold starts, while Tariq et al. [1] identify imports as dominant in Python cold starts from 81 reports.This study quantifies and generalizes that observation beyond serverless, and tracks it longitudinally.Source: §7 · p. 10
  • Dependency-bloat analyses measure unused code volume, whereas this study measures time; the two are complementary views of the same waste.Source: §7 · p. 10
  • Prior lazy-import work documents the design space and provides corporate-deployment and CLI case-study evidence; this paper adds an ecosystem-wide baseline and census of incompatibilities.The passage cites PEP 690, PEP 810, and CLI case studies.Source: §7 · p. 10
  • The study’s variance-control protocol follows established noisy-environment microbenchmarking practice, simplified by its single-tenant settings.The cited practice is attributed to.Source: §7 · p. 10

8 Conclusion

The conclusion presents a two-platform, longitudinal measurement of Python import costs and a baseline for the transition to lazy imports in Python 3.15.

8 Conclusion

  • Heavy tails, cold-start multipliers, and shell-package illusions replicate across the two measured platforms, while interpreter regressions appear on macOS/arm64 but not Linux/x86-64.Source: §8 · p. 10
  • Point-release swings exceed major-version effects in the measured import costs.Source: §8 · p. 10
  • The study provides a directly measured baseline for the lazy-import transition beginning with Python 3.15.The measurement harness, dataset, and analysis code are available on request.Source: §8 · p. 10
Loading 2609.02753v1…