PaperlayerView exact revision on arXiv

Source-linked AI summary

An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems

Alexandre Decan, Tom Mens, Philippe Grosjean

arXiv:1710.04936v1cs.SE

TL;DR

Packaging ecosystems face scale, complexity, and dependency-update challenges, but comparative evidence across ecosystems is limited. This paper uses libraries.io to quantitatively compare seven packaging ecosystems with novel metrics for network evolution. The ecosystems generally grow over time, while transitive dependencies contribute to substantial potential fragility and a small group of packages accounts for much updating and reuse.

  • Problem

    Comparative evidence about how dependency networks evolve across packaging ecosystems is limited, despite challenges from their scale, complexity, and dependency updates.

  • Method

    The paper quantitatively analyzes seven packaging ecosystems using libraries.io data and novel Changeability, Reusability, and Impact metrics.

  • Results

    Dependency networks generally grow over time, while package updates and reverse dependencies are concentrated among minorities of packages and transitive dependencies create substantial potential impact.

  • Takeaways & Limitations

    Dependency-management tools and ecosystem health dashboards should account explicitly for transitive dependencies and socio-technical network metrics.

  • Takeaways & Limitations

    Dependency constraints can generate many potential dependency trees, while conflicting releases can make identifying satisfiable combinations complex.

Abstract

from arXiv · show

Nearly every popular programming language comes with one or more package managers. The software packages distributed by such package managers form large software ecosystems. These packaging ecosystems contain a large number of package releases that are updated regularly and that have many dependencies to other package releases. While packaging ecosystems are extremely useful for their respective communities of developers, they face challenges related to their scale, complexity, and rate of evolution. Typical problems are backward incompatible package updates, and the risk of (transitively) depending on packages that have become obsolete or inactive. This manuscript uses the libraries.io dataset to carry out a quantitative empirical analysis of the similarities and differences between the evolution of package dependency networks for seven packaging ecosystems of varying sizes and ages: Cargo for Rust, CPAN for Perl, CRAN for R, npm for JavaScript, NuGet for the .NET platform, Packagist for PHP, and RubyGems for Ruby. We propose novel metrics to capture the growth, changeability, resuability and fragility of these dependency networks, and use these metrics to analyse and compare their evolution. We observe that the dependency networks tend to grow over time, both in size and in number of package updates, while a minority of packages are responsible for most of the package updates. The majority of packages depend on other packages, but only a small proportion of packages accounts for most of the reverse dependencies. We observe a high proportion of fragile packages due to a high and increasing number of transitive dependencies. These findings are instrumental for assessing the quality of a package dependency network, and improving it through dependency management tools and imposed policies.

1 Introduction

Packaging ecosystems are large, evolving networks whose dependency complexity creates maintenance risks. This paper compares seven ecosystems and introduces metrics for analyzing their growth, changeability, reuse, and fragility.

  • Software ecosystems contain tens to hundreds of thousands of packages and potentially an order of magnitude more dependencies.Changing dependencies can burden developers, discourage upgrades, and expose systems to outdated or vulnerable packages.
  • Comparative studies are needed to understand how dependency networks differ across ecosystems and evolve over time.Such comparisons may support ecosystem-specific software analysis tools for managing complexity and fragility.
  • Prior work found important structural differences among CRAN, PyPI, and npm, suggesting that ecosystem-specific findings do not necessarily generalize.Differences may partly reflect standard-library functionality and other ecosystem-specific characteristics.
  • The study compares Cargo, CPAN, CRAN, npm, NuGet, Packagist, and RubyGems using the libraries.io dataset.It introduces Changeability, Reusability, and P-Impact indices to compare ecosystems despite differences in age and size.
  • The paper examines network growth, package-update frequency, dependency extent, and the resulting fragility across four research questions.The article then discusses threats to validity, ecosystem policies, existing dependency-management limitations, and future research.

2 Related Work

Existing research has examined package dependencies across several ecosystems, but comparative evidence remains limited. Prior studies include ecosystem-specific analyses, qualitative comparisons, and empirical comparisons of only a few ecosystems.

  • The paper focuses on technical dependencies across projects at the ecosystem level, rather than dependencies among components within individual projects.
  • Most package-dependency studies have been limited to a single ecosystem, including npm, CRAN, Ruby, and Apache-related ecosystems.These studies examined dependency characteristics, trivial-package risks, errors, ecosystem interconnection, or dependency evolution.
  • Some prior work addressed ecosystem visualization, but did not focus on computing or visualizing ecosystem metrics.
  • Very little research compares dependency and maintainability issues across different packaging ecosystems.One three-ecosystem study used qualitative interviews to examine how community values, tools, and policies affect breaking changes.
  • Prior comparative work on npm, RubyGems, and Rust empirically confirmed findings about ecosystem fragility and vulnerability to transitive dependencies.

3 Methodology

The study analyzes seven packaging ecosystems using package-release and dependency metadata from libraries.io. It combines network measures with survival analysis, Lorenz curves, and Gini indices while selecting ecosystems with sufficiently complete metadata.

  • Survival analysis is used to study the time until package releases cease to survive in packaging ecosystems.The technique accounts for censored observations and is applied here to package-release survival.
  • Lorenz curves and Gini indices measure statistical dispersion and inequality in distributions of ecosystem activity.The Gini index ranges from 0 for perfect equality toward maximal inequality as one individual accounts for the distribution.
  • The study compares ecosystems differing in programming language, network size, and age, with statistical analyses generally covering each ecosystem’s lifetime through 1 January 2017.
  • On 1 April 2017, the selected ecosystems contained 5,812k releases, more than 830k packages, and 20,509k dependency relationships.
  • The analysis uses libraries.io metadata extracted statically from package manifests and selects seven ecosystems after excluding small or incomplete candidates.The selected ecosystems are Cargo, CPAN, CRAN, npm, NuGet, Packagist, and RubyGems.

4 RQ1: How do package dependency networks grow over time?

All seven dependency networks grow over time, but their growth patterns and complexity changes differ. Some ecosystems show linear or exponential growth in packages and dependencies, while the dependency-to-package ratio increases in four ecosystems.

  • All seven dependency networks continue to grow over time in both package and dependency counts.Package counts are shown in Figure 1, while dependency counts use monthly snapshots of the latest available package releases.
  • Cargo and CPAN exhibit linear growth for both size metrics, with R2 ≥0.97 in every reported case.
  • CRAN and npm exhibit exponential growth for both size metrics, with R2 ≥0.92 in every reported case.
  • NuGet grows exponentially in dependencies but linearly in packages, whereas Packagist and RubyGems show the opposite pattern.
  • The dependency-to-package ratio remains stable for CPAN, Packagist, and RubyGems but increases for Cargo, CRAN, npm, and NuGet.The increase suggests rising complexity relative to package counts in those four ecosystems.
  • Growth differences do not appear to depend on ecosystem size or age, while popularity and contributor activity are proposed as possible external factors for future study.

5 RQ2: How frequently are packages updated?

Package updates remain stable or grow across ecosystems, but are unevenly distributed: a minority of packages accounts for most updates, while required and younger packages are updated more frequently.

  • Update frequency: Package updates either remain stable or tend to grow over time, with the largest growth in npm, NuGet, and Packagist.Cargo, CPAN, and CRAN remain more stable, while RubyGems shows a slight increase.
  • Update frequency: Disregarding CRAN, a package release has a probability higher than 50% of being updated within two months, regardless of whether it is required.CRAN releases show greater resilience to new updates, which the authors relate to its stricter update policy.
  • Update frequency: Required packages are updated significantly more often than non-required packages, with statistical significance at α = 0.01.Across ecosystems, required-package survival curves are consistently lower than those for non-required packages.
  • Distribution across packages: Between 26% and 33% of packages were never updated, 35% to 45% were updated 1–4 times, and 27% to 36% were updated at least 5 times, excluding CPAN.The higher proportion of updated CPAN packages is attributed to its older age and longer time to receive updates.
  • Distribution across packages: A minority of packages is responsible for more than 80% of updates in every ecosystem except CRAN, where 60% of packages are required to reach 80%.The responsible minority ranges from 27% of packages for NuGet to 45% for Cargo.
  • Package age: With the exception of CPAN and CRAN, most 2016 updates involved packages up to 12 months old, including 62% of npm updates for packages younger than three months.For CPAN and CRAN, 58% and 59% of updates respectively involved packages older than two years.

6 RQ3: To which extent do packages depend on other packages?

Dependencies are widespread and increasingly concentrated: most packages are connected through dependencies, while a small minority of required packages accounts for most reverse dependencies.

  • Connectedness: From 62% of NuGet packages to 79% of CRAN packages were connected in January 2017, and smaller ecosystems showed higher, increasing connectedness.Connected packages are either dependent, required, or both.
  • Connectedness: Between 89% of connected NuGet packages and 99% of connected CPAN packages belong to the largest weakly connected component.The component was computed to assess whether connectedness is spread across the ecosystem.
  • Connectedness: Most connected packages depend on other packages, while the proportion of dependent packages increases over time and the proportion of required packages remains stable.A majority of packages depends on a small minority of other packages.
  • Dependency concentration: Only 6% of required packages in npm and RubyGems, versus 17% in NuGet, concentrate more than 80% of all reverse dependencies.The inverted Lorenz analysis demonstrates strong inequality among required packages.
  • Dependency concentration: The normalized Gini index increases continuously across ecosystems, ranging from 0.77 for NuGet to 0.87 for npm on 1 January 2017.These values indicate a very unequal distribution of dependent packages.
  • Reusability: The Reusability Index increases over time in all ecosystems, exponentially for npm and NuGet and linearly for the others.Linear growth coefficients are higher for Packagist, Cargo, and RubyGems than for CPAN and CRAN.
  • Reusability: JavaScript’s small standard library is offered as an explanation for npm’s higher and faster-growing Reusability Index.npm contains many basic-functionality packages on which other packages depend.

7 RQ4: How prevalent are transitive dependencies?

Transitive dependencies are widespread and deepen dependency trees, increasing the number of packages that may be affected by failures or removals. The paper measures this hidden reuse and ecosystem fragility across seven packaging ecosystems.

  • Prevalence: Most dependent packages have few direct dependencies but many more transitive dependencies.In Cargo, npm, and NuGet, half of dependent packages have at least 41, 21, and 27 transitive dependencies, respectively.
  • Network evolution: The transitive-to-direct dependency ratio stayed stable in CPAN, CRAN, Packagist, and RubyGems but increased in Cargo, npm, and NuGet.By January 2017, the ratio was 2 to 3 times higher for Cargo, npm, and NuGet than for the other ecosystems.
  • Dependency depth: For npm, co had 2,507 direct dependents and 51,497 indirect dependents, with more than half of its indirect dependents at depth 5 or greater.
  • Dependency depth: More than half of top-level packages had dependency-tree depth of at least 3 across the ecosystems.For npm, more than half reached depth 5; for Cargo, more than half reached depth 6 and 25% reached depth 10.
  • Fragility: The 5-Impact Index increased continuously in Cargo, npm, and NuGet, while remaining nearly stable in Packagist and RubyGems.Cargo reached 99 in January 2017 despite having only 7,421 packages; four of seven ecosystems restrained fragility, whereas npm, NuGet, and Cargo showed the highest impact and growth.

8 Threats to Validity

The validity of the analysis is constrained by metadata coverage, dependency extraction, release ordering, local data anomalies, and the scope of ecosystems studied. The authors describe these limitations and argue that several are unlikely to alter the large-scale findings.

  • Data coverage: Removed packages may be absent when ecosystems do not preserve historical data after removal.The package and release data were up to date through April 2017.
  • Dependency extraction: The networks include explicitly declared manifest dependencies but omit vendored and dynamically defined dependencies.The authors therefore consider collected dependencies an underestimate of actual reuse, while arguing this threat does not affect their results.
  • Temporal reconstruction: Chronological release ordering can diverge from logical version ordering for packages maintained on multiple parallel branches.The authors regard this as unlikely to affect findings given the scale of the analyses.
  • Data anomalies: Local phenomena and extraction-quality problems can distort some analyses, such as RubyGems’ August 2014 release-import peak.That peak represented 25,000 releases with incorrect creation dates.
  • Scope: The findings are not claimed to generalize beyond package dependency networks resembling the main package managers for programming languages.The authors expect different results for higher-level, end-user-oriented ecosystems such as WordPress, Eclipse, or Atom.

9 Discussion

The discussion links observed ecosystem differences to maintenance policies and examines the limits of dependency-update support. It argues that ecosystem health analysis should combine dependency metrics with social information.

  • 9.1 Why Policies Matter: CRAN has lower changeability, reusability, transitive-to-direct dependency ratios, and top-level dependency depths than most considered ecosystems.The authors associate these differences with CRAN’s stricter maintainer policy.
  • 9.1 Why Policies Matter: CRAN requires packages to remain up to date with dependencies and uses daily continuous integration checks, archiving packages that fail to resolve problems before the next major release.
  • 9.1 Why Policies Matter: The authors found no comparable package-update or dependency policies in the other studied ecosystems and note that adopting CRAN’s process would add substantial maintainer workload.
  • 9.1 Why Policies Matter: Package-removal policies can reduce or increase the risk of breaking transitive dependents.Cargo and NuGet prevented removal, npm adopted such a policy after left-pad, while RubyGems still allowed easy removal in May 2017.
  • 9.2 Limitations of Existing Support for Package Dependency Updates: Dependency constraints and semantic versioning can support compatible updates while preventing backward-incompatible ones, but maintainers may violate versioning semantics.
  • 9.2 Limitations of Existing Support for Package Dependency Updates: Transitive-dependency monitoring is computationally expensive because constraints can generate many possible dependency trees and release conflicts.Some systems prohibit installing two distinct releases of the same package, and SAT-solver-based solutions can be difficult and costly to use.
  • 9.2 Limitations of Existing Support for Package Dependency Updates: The authors recommend combining package-management policies, dependency-update techniques, and disciplined maintainer practices because no perfect solution exists.
  • 9.3 Towards Ecosystem-Level Health Analysis Dashboards: Ecosystem-level dependency metrics could complement existing software-health dashboards, but holistic health analysis also requires social aspects of developer interaction.

10 Future Work

The paper proposes future research on ecosystem-level evolution laws, complex network properties, and socio-technical dynamics. Its current evidence concerns technical and structural aspects, while organisational and social aspects remain open.

  • Laws of software ecosystem evolution: Initial evidence supports Continuing Growth at ecosystem level, measured by package and dependency counts.
  • Laws of software ecosystem evolution: Partial evidence supports Continuing Change because package updates remain stable or grow, while Changeability Index values increase for most ecosystems.
  • Laws of software ecosystem evolution: Partial support for Increasing Complexity comes from rising dependency-to-package and transitive-to-direct-dependency ratios, alongside increasing ecosystem fragility.
  • Scope boundary: The study provides no initial evidence for organisational and social laws because those aspects were outside its empirical scope.
  • Future research directions: Future work should test complex-network properties and combine technical dependency data with contributors’ social networks to study ecosystem health.
  • Complex network perspective: Unequal connectivity distributions suggest power-law or Pareto behaviour: 20%–30% of packages, or fewer in RubyGems, are required, and few concentrate most reverse dependencies.

11 Conclusion

The paper empirically compares seven packaging ecosystems using the libraries.io dataset and novel cross-ecosystem metrics. It finds growth, persistent or increasing changeability and complexity, concentrated reuse, and increasing fragility, while motivating dependency tools that account for transitive dependencies.

  • The study compares package dependency networks across seven ecosystems spanning different programming languages, sizes, and ages.
  • The libraries.io dataset enables comparison using dependency metadata from several million open-source libraries across dozens of package managers.
  • Growth, changeability, reusability, and fragility analyses find expanding networks, stable or increasing updates, abundant dependencies, and concentrated reverse dependencies.
  • An increasing number of packages can affect many others because of transitive dependencies.
  • Novel Hirsch-index-inspired metrics quantify ecosystem changeability, reuse, and fragility for cross-ecosystem comparison.
  • Because transitive dependencies are prevalent and potentially highly impactful, dependency-management tools should explicitly account for them.
Loading 1710.04936v1…