PaperlayerView exact revision on arXiv

Source-linked AI summary

AROMA+: A Study of Factors Affecting Reproducible Builds in the Maven Ecosystem

Mehdi Keshani, Amirhossein Rahmati, Mohammad Hossein Aref, Abbas Heydarnoori

arXiv:2608.27125v1cs.SE

TL;DR

Software supply-chain reuse requires ways to verify component origin and consistency, yet Maven reproducibility is understudied and RC’s manually maintained coverage is limited. The paper introduces AROMA+ to recover source and release-environment information automatically, achieving broad recovery accuracy and enabling reproduction attempts across Maven Central.

  • Problem

    Maven reproducibility is understudied, while Reproducible Central’s manually curated list is limited and challenging to maintain.

  • Method

    AROMA+ uses heuristics on Maven artifacts, metadata, and source repositories to recover source versions and original build-environment information for reproduction.

  • Results

    32% of Maven Central packages could potentially be attempted automatically, and 12% of attempted packages were fully reproducible.

  • Takeaways & Limitations

    AROMA+ recovered up to 99.8% of manually maintained information and contributed three newly identified packages accepted into Reproducible Central.

  • Takeaways & Limitations

    The approach is limited to releases for which a basic set of features can be recovered, and heuristic extraction may introduce errors or overlook corner cases.

Abstract

from arXiv · show

Modern software engineering establishes software supply chains and relies on tools and libraries to improve productivity. However, reusing external software in a project presents a security risk when the source of the component is unknown or the consistency of a component cannot be verified. Reproducible builds present a mitigation strategy, as they can confirm the origin and consistency of reused components. A large reproducibility community has formed for Debian, but the reproducibility of the Maven ecosystem, the backbone of the Java supply chain, remains understudied in comparison. Reproducible Central is an initiative that curates a list of reproducible Maven libraries, but the list is limited and challenging to maintain due to manual efforts. Our research aims to support these efforts in the Maven ecosystem through automation. We investigate the feasibility of automatically finding the source code of a library from its Maven release and recovering information about the original release environment. Our tool, AROMA+, can obtain this critical information from the artifact and the source repository through several heuristics and we use the results for reproduction attempts of packages on Maven Central. Overall, our approach achieves an accuracy of up to 99.8% when compared field-by-field to the existing manual approach. In some instances, we even detected flaws in the manually maintained list, such as broken repository links. We reveal that automatic reproducibility is feasible for 32% of the packages on Maven Central using AROMA+, and 12% of these packages are fully reproducible. We demonstrate our ability to successfully reproduce new packages and have contributed some of them to the Reproducible Central repository. Additionally, we highlight actionable insights, outline future work in this area, and make our dataset and tools publicly available.

1 Introduction

Maven Central underpins software supply chains, but reproducibility efforts remain limited and difficult to scale. AROMA+ investigates automating source and build-environment recovery to expand reproducibility coverage.

  • Maven Central hosts libraries and build plugins that form the backbone of many software supply chains.
  • Reproducible builds compare independently compiled binaries bit-for-bit to verify consistency with the same source code.
  • Maven reproducibility remains limited: RC reports 8,660 rebuild attempts for 914 projects versus more than 10M releases in Maven Central.
  • Reproduction is manual, difficult to scale, error-prone, and often requires recovering source code and distinguishing meaningful differences from trivial variability.
  • AROMA+ explores automated reproduction without relying on project maintainers by recovering source repositories, commits, and original build environments.
  • Up to 99.8% of manually maintained information was recovered, while three newly identified packages were accepted into RC.

2 Experimental Setup

The study builds a representative Maven Central dataset and reconstructs release conditions needed for reproduction. It focuses on metadata, source repositories, build tools, timestamps, and other variability sources.

  • 2.1 Dataset Creation: The study uses Maven and GitHub data to process Maven Central releases through the methodology shown in Figure 1.
  • 2.1 Dataset Creation: One random version is sampled from each unique project to avoid bias toward frequent releases and include older packages.
  • 2.1 Dataset Creation: Figure 2 compares total and selected Maven Central releases by year, showing that the sampling distribution mirrors the overall release distribution.
  • Reproduction must account for build-time variability, including timestamps, file ordering, line endings, build tools, and commands.
  • The study investigates automatically generating RC .buildspec files from package metadata using heuristics that recover required reproduction information.

3 RQ1: Can We Recreate The Link To The Source Code Repository And Commit Of An Artifact?

AROMA+ recovers Maven source repositories and release commits using URL validation and tagging heuristics. It identifies both for 57.2% of packages, with identified tags matching Reproducible Central’s suggestions in 93.4% of cases, while inconsistent release practices remain a challenge.

  • Repository links: 74.09% of links in scm.url were validated, compared with 37.7% in url.The fields are frequently used interchangeably, although scm.url is intended for the source repository and url for the project homepage.
  • Comparison with Reproducible Central: 93.4% of identified tags were identical to Reproducible Central’s suggestions, and 97.5% of identified URLs aligned with its referenced repositories.Manual inspection found mismatches involving commit choices, tag names, broken links, and project-specific release practices.
  • Limitations: Inconsistent documentation and release practices make some repository and commit mappings difficult to interpret or automate.Examples include unrelated commits, submodule-specific versioning, and differing tag conventions.
  • Repository links: 354,948 packages had at least one valid repository URL, enabling source-repository recovery for a substantial portion of the dataset.The study also found that 79.2% of packages had at least one verifiable repository link overall.
  • RQ1 result: 57.2% of packages had both a recreated repository link and an associated commit identified by AROMA+.This directly answers RQ1 for the combined repository-and-commit recovery task.

4 RQ2: Can The Original Build Environment Be Reconstructed After The Fact?

AROMA+ reconstructs key Maven build-environment information using artifact metadata, repository files, and heuristics, with strong agreement against Reproducible Central. The analysis identifies JDK, line-ending, profile, and build-tool patterns that support reproduction attempts.

  • JDK versions: 57% (116,607) of Maven packages use Java 8, while LTS versions Java 8, 11, 17, and 21 comprise 68.5% of archives.Only 52 packages use Java 20, and one artifact appears compiled with an early-access Java 21 version.
  • JDK versions: Java 8 became the most popular version by 2016 at 26.6% and remained dominant after newer LTS releases.Non-LTS versions generally show quick growth followed by prolonged decline, unlike Java 8’s sustained dominance.
  • JDK versions: Approximately 21K packages still use Java versions that are no longer officially supported, although this estimate is constrained by limited end-of-support information.The study also found several compilations using earlier early-access Java versions.
  • JDK versions: 99.5% of identified JDK versions matched Reproducible Central’s versions, despite limited transparency about how RC determined its values.The authors used repository workflow files to investigate some differences because projects lacked manifest files.
  • Build profiles: 54.5% of packages include at least one build profile, with 41.6% inheriting profiles from parent POMs and 12.9% defining them locally.Across all levels, packages contain 2.64 profiles on average, while only 0.22 are defined directly in the package’s own POM.
  • Build profiles: 45.98% (158,730) of 345,186 unique profiles contain activation blocks, including 14.1% with JDK-based activation and 13.5% active by default.These activation conditions can affect which configuration is used during reproduction.
  • Build profiles: 23,039 packages (4.87%) use Maven Release Plugin profiles, making release-activated profiles a useful reproduction heuristic.The extracted profiles match RC’s profiles in 70% of projects and appear among RC’s selected profiles in another 6%.

5 RQ3: To What Extent Is It Possible To Automatically Reproduce Maven Artifacts?

AROMA+ automates build-information recovery and file-level reproducibility analysis, successfully reproducing packages across Maven experiments and identifying practical causes of non-reproducibility. The results show that automation can expand reproducibility coverage, although complete reproduction sometimes depends on release profiles and artifacts beyond binaries.

  • Approach: AROMA+ follows Maven repository links, recovers source connections, extracts build information, and generates a .buildspec file automatically.Its file-level comparison computes MD5 checksums to identify which generated files differ from the reference archive.
  • Maven packages: 100 packages were successfully built by AROMA+, compared with 96 using RC’s buildspecs; one RC buildspec contained a broken repository URL.For the remaining packages, AROMA+ achieved full reproducibility for 10 and reproduced all released files except sources.jar for 37.
  • Maven packages: AROMA+ achieved performance very close to RC while remaining fully automated, with no observed inherent limitation in the experiment.Both approaches reproduced binaries for packages whose only unreproduced artifact was sources.jar, while full artifact reproduction depended on release profiles.
  • Near misses: 100 packages lacking project.build.outputTimestamp were investigated for near-miss reproducibility opportunities; 39 builds failed, while eight nearly reproduced binaries and POM files.Timestamp differences were identified as straightforward fixes in some cases, including stripping or assigning a fixed timestamp.
  • Release profiles: 32 of 100 packages improved when release profiles were activated, and no package became less reproducible.For 27 of these 32 packages, the only additional reproducible artifacts were sources.jar files generated by release-profile plugins.
  • Release profiles: Release profiles are unnecessary for most projects or mainly required for sources.jar, but a small fraction cannot be reproduced without them.In five packages, profiles affected additional artifacts beyond sources.jar, including additional JAR files.

6 Discussion

The discussion identifies practical ways to improve Maven reproducibility, explains remaining ecosystem and methodological constraints, and outlines automation opportunities. It also reports that AROMA+ can substantially expand automated reproduction coverage.

  • Awareness: Defining Maven’s project.build.outputTimestamp property substantially increases automated reproducibility, motivating build-tool warnings or automatic defaults.The discussion proposes analogous Gradle defaults for preserveFileTimestamps and reproducibleFileOrder.
  • Awareness: Reproducibility should become a package quality attribute, supported by warnings about non-reproducible dependencies, standardized .buildspec files, and richer build metadata.The proposed practices aim to give maintainers incentives and improve package-selection information.
  • Awareness: Standardized release-tagging conventions would facilitate automated processing and improve ecosystem transparency.The authors recommend following established community conventions rather than inventing project-specific styles.
  • Sorry State of Maven Central: Maven projects lacking accessible resources cannot always be reproduced, and broken packages impede maintenance efforts.The authors suggest investigating whether broken artifacts can be removed to restore a more self-contained repository.
  • Relevance: 154K of 479K packages (32%) could potentially be attempted automatically; among 600 attempts, 72 (12%) were fully reproducible and 236 (39%) at least partially reproducible.These results could focus manual effort on more challenging packages and potentially expand the Reproducible Central dataset.
  • Future Directions: Release profiles are typically required to reproduce sources.jar files, while some main binaries also require profiles.Full ecosystem reproducibility therefore requires automatic profile extraction, although profile activation remains complex.
  • Internal Validity: Manual inspections, reused rebuilding tools, and implementation heuristics may introduce errors, overlook corner cases, or inherit tool flaws.The authors mitigated these risks through dual inspections, testing, documentation, comparison with a manual baseline, and public release of code and data.
  • External Validity: The study’s heuristics focus on git-based packages, excluding other version-control systems such as Subversion and Mercurial.The authors characterize this as mainly affecting older releases and encourage replication in ecosystems such as NPM or PyPI.

7 Related Work

Prior work covers Maven ecosystems, build tooling, and reproducibility across other ecosystems, but automatic reproduction of Maven libraries remains unaddressed. This study positions itself as filling that gap.

  • Software Ecosystems: Studies of Maven ecosystems have examined updates, library life cycles, vulnerability propagation, dependency datasets, and scalable call-graph generation.These works provide broader empirical and analytical foundations for studying Maven software.
  • Software Builds: Build research has analyzed Maven archetypes, repaired Maven and Gradle configuration issues, and applied static analysis to build scripts.Related tools target schema patterns, configuration repair, build speed, or maintenance rather than automatic artifact reproduction.
  • Reproducibility: Reproducibility research spans trusting-trust defenses, diverse double compiling, unified verifiable builds, issue localization, and build repair.These studies established techniques and tools for detecting or fixing reproducibility problems.
  • Reproducibility: Research across JavaScript, Python, Ruby, and six major ecosystems reports challenges linking distributed artifacts with their source code and varying reproducibility levels.These findings provide cross-ecosystem context for the Maven case.
  • Research Gap: No current research, to the authors’ knowledge, automatically reproduces Maven libraries.The study therefore targets automatic reproduction and a deeper understanding of library reproducibility in Maven Central.

8 Conclusions

The paper addresses the relative neglect of Maven reproducibility and develops automation to find reproducible packages and their build environments. AROMA+ closely matches manually crafted specifications while discovering additional reproducible libraries.

  • Conclusion: Maven reproducibility remains relatively understudied, while Reproducible Central’s library list is limited and difficult to maintain.The paper frames automation as a way to bridge this gap.
  • Conclusion: AROMA+ automatically finds Maven package sources and build environments, rebuilds libraries, and compares outputs with Maven-hosted files.The approach is limited to releases for which a basic set of features can be recovered.
  • Conclusion: 99.8% similarity to manually crafted .buildspec files was achieved, while previously missing reproducible libraries were found and contributed to Reproducible Central.The dataset and tools were also made publicly accessible.

Funding

The research received no external funding.

  • The research received no external funding.

Ethical approval

The study did not involve human participants or animals.

  • The study did not involve human participants or animals.

Informed consent

No human subjects were involved in this research.

  • No human subjects were involved in this research.

Data Availability Statement

The replication package provides the code, data, and essential materials needed to reproduce the paper’s results.

  • The replication package includes code, data, and essential materials for reproducing the reported results.

Clinical Trial Number

No clinical trial number applies to this study.

  • The clinical trial number is not applicable.
Loading 2608.27125v1…