PaperlayerView exact revision on arXiv

Source-linked AI summary

The Astropy Project: Building an inclusive, open-science project and status of the v2.0 core package

The Astropy Collaboration, A. M. Price-Whelan, B. M. Sipőcz, H. M. Günther, P. L. Lim, S. M. Crawford, S. Conseil, D. L. Shupe, M. W. Craig, N. Dencheva, A. Ginsburg, J. T. VanderPlas, L. D. Bradley, D. Pérez-Suárez, M. de Val-Borro, T. L. Aldcroft, K. L. Cruz, T. P. Robitaille, E. J. Tollerud, C. Ardelean, T. Babej, M. Bachetti, A. V. Bakanov, S. P. Bamford, G. Barentsen, P. Barmby, A. Baumbach, K. L. Berry, F. Biscani, M. Boquien, K. A. Bostroem, L. G. Bouma, G. B. Brammer, E. M. Bray, H. Breytenbach, H. Buddelmeijer, D. J. Burke, G. Calderone, J. L. Cano Rodríguez, M. Cara, J. V. M. Cardoso, S. Cheedella, Y. Copin, D. Crichton, D. DÁvella, C. Deil, É. Depagne, J. P. Dietrich, A. Donath, M. Droettboom, N. Earl, T. Erben, S. Fabbro, L. A. Ferreira, T. Finethy, R. T. Fox, L. H. Garrison, S. L. J. Gibbons, D. A. Goldstein, R. Gommers, J. P. Greco, P. Greenfield, A. M. Groener, F. Grollier, A. Hagen, P. Hirst, D. Homeier, A. J. Horton, G. Hosseinzadeh, L. Hu, J. S. Hunkeler, Ž. Ivezić, A. Jain, T. Jenness, G. Kanarek, S. Kendrew, N. S. Kern, W. E. Kerzendorf, A. Khvalko, J. King, D. Kirkby, A. M. Kulkarni, A. Kumar, A. Lee, D. Lenz, S. P. Littlefair, Z. Ma, D. M. Macleod, M. Mastropietro, C. McCully, S. Montagnac, B. M. Morris, M. Mueller, S. J. Mumford, D. Muna, N. A. Murphy, S. Nelson, G. H. Nguyen, J. P. Ninan, M. Nöthe, S. Ogaz, S. Oh, J. K. Parejko, N. Parley, S. Pascual, R. Patil, A. A. Patil, A. L. Plunkett, J. X. Prochaska, T. Rastogi, V. Reddy Janga, J. Sabater, P. Sakurikar, M. Seifert, L. E. Sherbert, H. Sherwood-Taylor, A. Y. Shih, J. Sick, M. T. Silbiger, S. Singanamalla, L. P. Singer, P. H. Sladen, K. A. Sooley, S. Sornarajah, O. Streicher, P. Teuben, S. W. Thomas, G. R. Tremblay, J. E. H. Turner, V. Terrón, M. H. van Kerkwijk, A. de la Vega, L. L. Watkins, B. A. Weaver, J. B. Whitmore, J. Woillez, V. Zabalza

arXiv:1801.02634v2astro-ph.IM

TL;DR

Astronomical research depends on software, yet common functionality and coordinated open development require shared infrastructure. This paper reviews Astropy’s organization, version 2.0 core package, and ecosystem infrastructure, finding continued growth alongside improved stability, breadth, and reliability.

  • Problem

    Astronomical research relies on software, creating a need for shared, general-purpose functionality and coordinated development across packages.

  • Method

    The paper reviews Astropy’s community organization, version 2.0 core-package features, and infrastructure supporting an interoperable ecosystem.

  • Results

    Astropy’s development and ecosystem continue growing while improving in stability, breadth, and reliability for astronomers’ daily tasks.

  • Takeaways & Limitations

    Astropy supports trusted, reproducible astronomical work and demonstrates community-driven software practices that improve software quality, accessibility, and discoverability.

  • Takeaways & Limitations

    The article describes Astropy’s organization and current package state rather than serving as an introduction or replacement for its documentation.

Abstract

from arXiv · show

The Astropy project supports and fosters the development of open-source and openly-developed Python packages that provide commonly-needed functionality to the astronomical community. A key element of the Astropy project is the core package Astropy, which serves as the foundation for more specialized projects and packages. In this article, we provide an overview of the organization of the Astropy project and summarize key features in the core package as of the recent major release, version 2.0. We then describe the project infrastructure designed to facilitate and support development for a broader ecosystem of inter-operable packages. We conclude with a future outlook of planned new features and directions for the broader Astropy project.

1. INTRODUCTION

The Astropy Project emerged as a community-driven effort to standardize core astronomical functionality in Python and now provides an open-source, open-development core package alongside affiliated packages. The introduction frames Astropy as a mature, widely used project whose organization and current package state—not user instruction—is the article’s focus.

  • Astronomical research broadly depends on software, spanning individual scripts, collaborative packages, survey pipelines, and institutionally supported tools.
  • Astropy’s core package began as a largely community-driven effort to standardize core functionality for astronomical Python software.
  • Python’s permissive open-source licensing, broad platform availability, scientific popularity, and improving package installation have supported its adoption in quantitative research.
  • The Astropy project provides an open-source, open-development core package and affiliated ecosystem supporting specialized astronomical functionality in Python.
  • The project pursues high-quality code and documentation through tools and web services without central institutional oversight, and its package has grown considerably since its first public release.
  • The article describes Astropy’s community organization and current package state rather than serving as an introduction to the package or a replacement for its documentation.

2. ORGANIZATION AND INFRASTRUCTURE

Astropy is organized as a consensus-oriented coordination committee supporting a broad, open contributor community through pull-request development and affiliated packages. Its infrastructure includes regular releases and cross-platform testing, while development remains financially unsupported directly and lacks a long-term sustainability plan.

  • Governance: Astropy uses a coordination committee rather than a “Benevolent Dictator For Life” model, expanded from three to four members in 2016.The committee works toward consensus, serves as a tie-breaker when needed, and oversees affiliated-package applications and project roles.
  • Development workflow: Astropy develops through GitHub pull requests, generally limiting each request to one conceptual change so code review remains tractable.Bugs and feature requests are reported through the GitHub issue tracker and organized with labels.
  • Community development: As of version 2.0, the core package contained 212244 lines of code from 232 contributors across 19270 git commits.The contributor distribution had a log-log slope of −0.5, indicating development by a broad contributor base; six developers had over 1000 commits each.
  • Affiliated packages: Affiliated Packages extend Astropy’s community beyond the core package while promoting code reuse, interoperability, testing, and thorough documentation.They accommodate specialized functionality, incompatible licenses, external dependencies, and rapid development that may later feed into the core package.
  • Release infrastructure: The core package follows a six-month schedule for significant releases, with bugfix releases as needed and some releases receiving two years of long-term support.Astropy is distributed through Anaconda and PyPI and continuously tested across platforms.
  • Sustainability: As of version 2.0, Astropy had no direct financial support, and a long-term sustainability plan had not yet been established.Development and community support relied on personal time, research projects, institutional allocations, and contributions facilitated through NumFOCUS.

3. ASTROPY CORE PACKAGE VERSION 2.0 … 3.3. Coordinates

Astropy 2.0’s core package provides shared astronomical functionality and base classes, with units and quantities increasingly central across the package. Version 2.0 extends coordinates to Earth locations, velocities, and solar-system ephemerides while benchmarking transformation accuracy across tools.

  • 3. ASTROPY CORE PACKAGE VERSION 2.0: The astropy core package supplies commonly needed astronomical functionality and shared base classes, including coordinate transformations, astronomical file I/O, units, and NDData interfaces.These capabilities support a broader ecosystem of interoperable Python packages for astronomy.
  • 3.1. Units: In version 2.0, units and quantities became a key package-wide concept, underpinning nearly all coordinate functionality and being accepted or expected by most other subpackages.Quantity objects extend numpy arrays, and logarithmic relative units now support astronomical magnitudes and decibels.
  • 3.2. Constants: Version 2.0 organizes physical and astronomical constants into modules for specific value versions, with astropyconst20 as the default and astropyconst13 available for compatibility.The package also adopts fixed nominal mass parameters and radii for Earth, Jupiter, and the Sun following IAU 2015 Resolution B3.
  • 3.3. Coordinates: Astropy.coordinates separates coordinate representations from reference frames, enabling independent representation changes through vector-like Representation and Frame classes and a high-level SkyCoord interface.Most inputs are Quantity objects, and SkyCoord accepts coordinate data across frames and representations.
  • 3.3.1. Local Earth coordinate frames: EarthLocation adds geocentric Earth-position systems, including AltAz, with transformations to barycentric frames requiring a Time specification.This enables Earth-location-specific coordinate transformations.
  • 3.3.2. Proper motion and velocity transformations: Frame classes now support proper-motion and radial-velocity components, and coordinate transformations can propagate velocity data.Velocity components use pm-prefixed names aligned with the positional longitude and latitude components.
  • 3.3.3. Solar System Ephemerides: Astropy can compute major solar-system-body ephemerides using ERFA analytic approximations or downloaded JPL ephemerides and return the results as coordinate objects.JPL ephemerides require the jplephem13 optional dependency and an internet connection.
  • 3.3.4. Accuracy of coordinate transformations: Coordinate-transformation benchmarks compare tools pairwise rather than assuming ground truth; for FK4-to-Galactic conversion, astropy, Kapteyn, and PyTPM agree within sub-milliarcsecond differences over 1000 tested pairs.The benchmark includes Astropy and several other astronomy packages, accommodating interpretation differences in standards such as Galactic coordinates.

3.4. Time

The astropy.time subpackage provides astronomical time scales and formats for tasks including barycentric corrections and sidereal-time calculations. It is built on the BSD-licensed ERFA library and supports correcting eclipse or transit timings for light-travel-time differences using source, observatory, and time information.

  • Time: astropy.time supports astronomical time scales such as UTC, TAI, and UT1, together with formats including Julian date and modified Julian date.These capabilities support calculations such as barycentric corrections and sidereal times.
  • Time: The subpackage is built on ERFA, a three-clause BSD-licensed C library that replicates SOFA.ERFA provides the underlying implementation for astropy.time.
  • Time: Detailed eclipse and transit timing accounts for light-travel-time differences caused by Earth’s motion by converting times to the Solar System barycenter or heliocenter.Corrections use a SkyCoord source location, an EarthLocation observatory location, and Time values.

3.5. Data containers · 3.6. io

Astropy 2.0 expands data containers for astronomical datasets, tables, and interoperable array-like columns, while strengthening I/O through enhanced ASCII, HTML, FITS, and unified format support. FITS loading is lazy by default, and ASCII processing gains a faster engine with broad compatibility.

  • 3.5.1. nddata: NDData extensions support arithmetic, limited error propagation, coordinate-aware slicing, and masking, errors, and units, while CCDData reads and writes FITS files.These classes extend base storage functionality for specialized data use cases.
  • 3.5.1. nddata: Image-cutout utilities link cutout and original pixels, convert world and pixel coordinates, overlay cutouts, and enlarge or reduce images.The utilities operate on plain numpy arrays or astropy.nddata classes.
  • 3.5.2. Tables: Table objects fully support grouping, aggregation, and filtering based on user-defined criteria.Groups can be combined by operations such as means or standard deviations.
  • 3.5.2. Tables: Tables can be stacked vertically or horizontally, and concatenated or joined when they share columns.Vertical stacking combines rows with identical columns, while horizontal stacking combines distinct columns across matched rows.
  • 3.5.2. Tables: Table columns now support array-valued Quantity, SkyCoord, Time, and other user-defined array-like objects, enabling catalogs and time series.This combines specialized object benefits with table row, column, grouping, and combination operations.
  • 3.6.1. ASCII: ECSV preserves table metadata when writing and reading ASCII files, while remaining human-readable and compatible with simple CSV readers.Preserved metadata includes comments, keywords, column data types, units, and descriptions.
  • 3.6.1. ASCII: 4 to 5 times faster on average, the new Cython/C ASCII engine supports common formats, additional exponential notation styles, and optional parallel parsing.Compatible formats use the fast engine by default, with automatic fallback to pure Python when needed; HTML tables can also be read and written.
  • 3.6.2. FITS: FITS files load lazily by default, and command-line tools provide HDU summaries and human-readable headers.Lazy loading avoids reading data into memory until requested and can speed access to early HDUs in files containing many HDUs.

3.7. Modeling · 3.8. Convolution

Astropy 2.0’s modeling framework supports composable, extensible analytical models, parameter fitting, and unit-aware evaluation, motivated partly by flexible WCS needs. Its convolution subpackage implements normalized convolution that handles missing image data during reconstruction.

  • 3.7. Modeling: The modeling framework represents analytical models, evaluates them, and fits parameters, while enabling arbitrary model combinations for GWCS.It was designed to support the Generalized World Coordinate System package.
  • 3.7. Modeling: Because FITS WCS cannot represent arbitrary distortions, unit propagation also makes astropy.modeling useful for astrophysical modeling and fitting.The framework supports representing and fitting models within data-analysis tools.
  • 3.7.1. Single Model Definition and Evaluation: Models define named, parameterized behavior with defaults, units, and constraints including fixed, tied, and bounded values.Parameter values and constraints can be updated by assignment.
  • 3.7.2. Model Sets: Model sets efficiently evaluate one model form across many parameter-value sets, such as differing PSF positions and amplitudes in photometry.This supports repeated evaluation for objects sharing the same functional model form.
  • 3.7.3. Compound Models: Models can be combined into further models using arithmetic, join, and composition operators subject to compatible inputs and outputs.Supported operators include +, -, *, /, **, &, and |.
  • 3.7.4. Fitting Models to Data: Astropy 2.0 provides extensible fitting through NumPy/SciPy-based fitters, parameter constraints, and external-fitter plugins.Available optimizers include Levenberg–Marquardt, Simplex, SLSQP, and LinearLSQFitter; SABA bridges Sherpa fitters into astropy.modeling.
  • 3.7.5. Creating New Models: New model classes can be created by converting simple functions into full-featured models or extending the built-in model class with arbitrary code.This supports models beyond arithmetic combinations of existing classes.
  • 3.7.6. Unit Support: Quantity-aware modeling supports representing, evaluating, and fitting models with units, including unit-bearing fitted parameters and equivalent-unit inputs.The blackbody model BlackBody1D is given as an example.

3.9. Visualization

Astropy’s visualization tools support image-value transformations, coordinate-aware astronomical plotting, flexible histograms, and RGB composite creation. The package integrates these capabilities with matplotlib and supports advanced visualization workflows.

  • Image transformations: The visualization framework transforms image and array values through normalization and stretching for display.Normalization maps values to [0, 1] using lower and upper limits, while stretching remaps [0, 1] using linear or non-linear functions.
  • Image transformations: Normalization and stretching can be integrated with matplotlib through normalization objects.Astropy provides a normalization class that wraps interval and stretch objects into a form matplotlib understands.
  • Coordinate-aware plotting: wcsaxes creates coordinate-aware figures from image arrays and WCS objects, accommodating world coordinates that do not align with pixel axes.Supported world coordinates include right ascension, declination, velocity, wavelength, frequency, and time.
  • Histograms: The histogram function generalizes matplotlib’s interface with four automatic bin-selection methods: “blocks”, “knuth”, “scott”, and “freedman”.It retains matplotlib-compatible syntax except for the more flexible bins parameter.
  • RGB images: Astropy provides a convenience function and alternate-scaling classes for creating RGB composite images from three separate high-dynamic-range arrays.This functionality was contributed by developers from the Large Synoptic Survey Telescope (LSST).

3.10. Cosmology · 3.11. Statistics

Astropy 2.0 provides a cosmology subpackage with standard cosmological representations and calculations, alongside a growing statistics package tailored to astronomical applications. Its statistical tools include robust, circular, periodic, and breakpoint analyses, plus adaptive histogram binning.

  • 3.10. Cosmology: The cosmology subpackage represents different cosmologies and calculates commonly used quantities such as look-back time and distance, using Planck Collaboration et al. (2016) values by default.The default cosmology applies to astropy version 2.0.
  • 3.11. Statistics: The astropy.stats package offers astronomy-focused statistical tools that extend functionality in scipy and statsmodels, while remaining a growing rather than complete collection.Its functionality is used across many different disciplines in astronomy.
  • 3.11.1. Robust Statistical Estimators: Robust statistical functions mitigate outlier effects through sigma clipping, median absolute deviation, and biweight estimators used for galaxy-cluster velocity dispersion.These methods provide reliable basic-statistics estimates for complex distributions.
  • 3.11.2. Circular Statistics: Circular statistical estimators compute circular means, variances, and moments for numpy.ndarrays in radians and Quantity objects, with tests for Rayleigh Test and vtest.The estimators are based on Jammalamadaka & Sengupta (2001).
  • 3.11.3. Lomb-Scargle Periodograms: Astropy.stats implements efficient Lomb-Scargle periodograms and generalizations supporting floating means, truncated Fourier models, and heteroscedastic uncertainties.These tools target periodic analysis of unevenly-spaced astronomical time series and use fast, scalable computations.
  • 3.11.4. Bayesian Blocks and Histogram Binning: Bayesian Blocks analyzes break-points in non-periodic astronomical time series and determines optimal histogram binnings, including unequal bin sizes.Figure 7 illustrates an astropy histogram with irregularly-spaced bins computed using Bayesian Blocks, which can represent features at various scales more accurately than regular bins.

4. INFRASTRUCTURE FOR ASTROPY AFFILIATED PACKAGES

The Astropy Project maintains general-purpose infrastructure packages that support the maintenance of Astropy and affiliated packages. These tools streamline package setup, continuous integration, and documentation workflows across the ecosystem.

  • Infrastructure packages: Astropy maintains general-purpose infrastructure packages to assist maintenance and upkeep of the core package and affiliated packages.The project describes these as its most widely used infrastructure packages.
  • Package template: The package-template repository provides ready-to-go packaging, testing, installation, and Sphinx documentation-build infrastructure for Python packages.Any Python package can use this layout and setup, originally developed for Astropy’s core and affiliated packages.
  • Continuous integration: The ci-helpers repository lets maintainers configure testing and installation processes for continuous-integration services through environment variables.Current tools support Travis CI and Appveyor CI, while usage is described as extremely widespread.
  • Documentation infrastructure: Astropy-developed Sphinx extensions facilitate automatic API documentation generation from reStructuredText documentation workflows.Sphinx transforms plain-text RST files into HTML, PDF, or LATEX documents.

5. THE FUTURE OF THE ASTROPY PROJECT

The Astropy Project’s future plans include Astropy core version 3.0, subsequent optimization and documentation improvements, broader infrastructure and affiliated-package development, and expanded educational resources through Learn Astropy. Version 3.0 will remove Python 2 support while the project develops spectroscopy tools, LSST integration, HEALPIX support, and learning materials for the wider ecosystem.

  • Core and project development: Development of Astropy core version 3.0 has begun alongside plans to overhaul educational materials and generalize infrastructure utilities for the community.These efforts extend beyond planned core-package changes to support the broader Astropy ecosystem.
  • Core and project development: Python 2 support will be removed in version 3.0, which will support Python 3.5 or higher and is scheduled for January 2018.The change enables Python 3-only features, simplifies the code base, and reduces testing overhead.
  • Core and project development: The release after version 3.0, scheduled for mid-2018, will prioritize algorithm optimization and documentation improvement through testing, evaluation, and performance monitoring.Less new functionality may be introduced as a trade-off for better performance.
  • Education and learning: Learn Astropy will provide accessible entry points to the core package and broader ecosystem through documentation, examples, tutorials, and conceptual guides.The resource is intended for users and learning environments for which existing reference-oriented documentation is not an appropriate starting point.

6. CONCLUSION

The Astropy project and ecosystem continue to grow while improving stability, breadth, and reliability. Its mature core package supports astronomers’ daily work, reproducible results, and community-driven software-development practices.

  • Project and ecosystem: Astropy continues significant growth while improving stability, breadth, and reliability across its package ecosystem.Several core subpackages have reached stability with rich functionality for astronomers’ daily tasks.
  • Project and ecosystem: Reliability and high coding standards help users trust astropy calculations and publish reproducible results.The core package supports tasks including planning observations, analyzing data or simulations, and writing publications.
  • Community practices: Astropy spreads community-driven software-development best practices by demonstrating how git version control and CI testing can help astronomers.The project addresses a gap in formal computer-science and software-development training among practicing astronomers.
  • Community and infrastructure: The project depends on external organizations and web services, including Google’s GSoC, NumFOCUS, the Python Software Foundation, GitHub, and CI and documentation platforms.GSoC has funded several students per year, many of whom become long-term contributors.

APPENDIX · A. LIST OF AFFILIATED PACKAGES

The appendix lists affiliated Astropy packages in registries, identifying package names, PyPI names, maintainers, citations, and stability status. It also includes a separate registry of provisionally accepted affiliated packages.

  • APPENDIX: Table 1 presents a registry of affiliated packages with package names, stable-status indicators, PyPI names, maintainers, and citations.The table header specifies these fields, while the appendix also references a second registry for provisionally accepted packages.
  • A. LIST OF AFFILIATED PACKAGES: The listed packages include python-cpl, pyspeckit, poliastro, marxs, photutils, and PyDL, with stable statuses and maintainer information recorded.The entries identify python-cpl, poliastro, photutils, and PyDL as No, and pyspeckit and marxs as Yes.
  • A. LIST OF AFFILIATED PACKAGES: Additional entries are naima, pyregion, omnifit, PyVO, and spectral-cube, each accompanied by a stable-status indicator, PyPI name, and maintainer.The entries mark naima, pyregion, omnifit, and spectral-cube as Yes, while PyVO is marked No.
  • A. LIST OF AFFILIATED PACKAGES: The registry also includes sncosmo, specutils, stingray, regions, and reproject, with associated maintainers and stable-status values where shown.sncosmo and stingray are marked Yes and No respectively; specutils, regions, and reproject are marked No.
  • A. LIST OF AFFILIATED PACKAGES: The appendix distinguishes the affiliated-package registry from a registry of provisionally accepted affiliated packages.Table 2 is explicitly titled “Registry of provisionally accepted affiliated packages.”
Loading 1801.02634v2…