Source-linked AI summary
astroquery: An Astronomical Web-Querying Package in Python
Adam Ginsburg, Brigitta M. Sipőcz, C. E. Brasseur, Philip S. Cowperthwaite, Matthew W. Craig, Christoph Deil, James Guillochon, Giannina Guzman, Simon Liedtke, Pey Lian Lim, Kelly E. Lockhart, Michael Mommert, Brett M. Morris, Henrik Norman, Madhura Parikh, Magnus V. Persson, Thomas P. Robitaille, Juan-Carlos Segovia, Leo P. Singer, Erik J. Tollerud, Miguel de Val-Borro, Ivan Valtchanov, Julien Woillez, the Astroquery collaboration
TL;DR
Astronomers need reproducible access to heterogeneous remote data services that may expose only websites or inconsistent interfaces. astroquery provides Python modules built on requests and Astropy, using service-like interfaces and asynchronous support to create reproducible workflows. The package is openly developed and adapts as remote services change, though some services cannot support the common API and testing depends partly on service availability.
Problem
Astronomical data-access tools are heterogeneous, often web-based, and difficult to track across archives and specialized services, limiting straightforward reproducible retrieval.
Method
astroquery provides independent Python modules with common query interfaces, requests-based HTTP communication, Astropy outputs, and low-level asynchronous access for large results.
Results
astroquery provides a toolkit that supports reproducible workflows from data acquisition through publication and is developed through broad community contributions.
Takeaways & Limitations
astroquery gives astronomers a community-maintained, scriptable way to access diverse remote services while preserving familiar service-specific interfaces.
Takeaways & Limitations
The common API is not required for services that cannot support positional queries, and remote testing depends on the stability of external services.
Abstract
from arXiv · showhide
astroquery is a collection of tools for requesting data from databases hosted on remote servers with interfaces exposed on the internet, including those with web pages but without formal application program interfaces (APIs). These tools are built on the Python requests package, which is used to make HTTP requests, and astropy, which provides most of the data parsing functionality. astroquery modules generally attempt to replicate the web page interface provided by a given service as closely as possible, making the transition from browser-based to command-line interaction easy. astroquery has received significant contributions from throughout the astronomical community, including several significant contributions from telescope archives. astroquery enables the creation of fully reproducible workflows from data acquisition through publication. This paper describes the philosophy, basic structure, and development model of the astroquery package. The complete documentation for astroquery can be found at http://astroquery.readthedocs.io/.
1. INTRODUCTION
Astronomical data are abundant but distributed across heterogeneous archives, surveys, websites, and specialized services. astroquery addresses this fragmentation with scriptable Python access that supports reproducible workflows.
- Many astronomical data-access tools are heterogeneous and available only through graphical interfaces or websites.
- A common scripted interface can make distributed data more accessible and make analyses from data download through publication repeatable.
- Astronomical data are hosted across observatory archives, individual surveys, custom websites, and specialized services such as bibliographic and ephemeris databases.
- astroquery arose to provide scriptable Python command-line access to these databases, enabling reproducible analysis scripts and pipelines with minimal user interaction.
- The paper overviews astroquery’s software layout, development model, and documentation.
2. THE SOFTWARE
astroquery organizes independent service modules around shared Python interfaces for HTTP querying, table or image results, asynchronous access, caching, authentication, and testing. The common API remains adaptable because services differ in supported operations and reliability.
- 2.1. The Software: astroquery modules use a common framework based on Python requests for HTTP communication with web services.
- 2.1. The Software: New modules typically provide a core class with query_* methods, especially query_region, and return query results as Astropy Tables.
- 2.1. The Software: The suggested API is not mandatory because non-positional services, including atomic, molecular, and Solar System databases, cannot support query_region.
- 2.3. The API: The _async interface returns response or lazy file objects, allowing large results to be streamed or metadata accessed without downloading corresponding data.
- 2.4. Common Query Functionality: Caching reduces duration and network load for repeated queries, while authentication support provides secure login and credential storage.
- 2.6. Testing: Testing separates remote tests that depend on service stability from local tests using patched requests and local files.
3. DEVELOPMENT HISTORY AND STATUS
astroquery is openly developed within the Astropy ecosystem, with contributions from individuals, institutions, and funded programs. Its directions are driven by contributors and data providers while its architecture remains stable and independently maintained.
- Community development: Anyone can contribute modules, upgrades, and corrections, with maintainers committed to helping developers meet astroquery’s requirements.
- Project structure: astroquery remains separate from astropy because remote services require faster adaptation than astropy’s stability-oriented core can provide.
- Community development: 77 people had contributed to astroquery by August 2018, while two maintainers carried the primary maintenance burden and contributors implemented most modules independently.
- Institutional contributions: Institutional contributors included ESA, ADS, and STScI developers supporting modules such as Gaia, ESASky, ADS, MAST, and VO Cone Search.
- Funded contributions: NASA-supported sbpy work contributed the JPLHorizons module and planned further Solar System services.
- Development model: Open development lets contributors and data providers drive new directions and update modules as remote data sources change.
- Relation to the Virtual Observatory: Unlike the Virtual Observatory’s unified access point, astroquery offers multiple access points without requiring hosting services to provide a specific API.
4. DOCUMENTATION AND REFERENCES
astroquery modules are documented online, with detailed examples and a gallery supporting use of individual interfaces and query workflows.
- Online documentation provides module guidance, a detailed Appendix A example, and additional examples in the gallery.
- Several authors have independently published worked examples of astroquery modules, a practice the paper encourages.
- Published examples cover Cosmosim downloads, Vizier and SIMBAD queries, the Open Astronomy Catalog API, and MAST and GAIA interfaces.
5. SUMMARY
Astroquery is a community-developed toolkit within the astropy affiliated package system, supporting reproducible astronomical workflows and evolving through broad contributions and infrastructure.
- Astroquery provides Python access to remotely hosted astronomical data and supports reproducible workflows.The package is described as part of the astropy affiliated package system.
- The package’s development model is community-driven, with contributions, issue reports, feedback, and project support shaping its future.
- Astroquery relies on external infrastructure and Python dependencies including GitHub, Travis CI, Appveyor, Read the Docs, requests, beautifulsoup, and keyring.
- The project received additional support through Google Summer of Code, which funded two students’ work in 2013 and 2014.
- Individual contributions were supported by NSF, NASA PDART, the DIRAC Institute, and related philanthropic and research organizations.
- The software stack includes Astropy, numpy, requests, keyring, beautifulsoup4, html5lib, matplotlib, APLpy, pyregions, and regions.
A. EXAMPLE
The example combines SkyView and Vizier with astropy coordinate, unit, table, and WCS tools to create a finder chart. It retrieves a 2MASS-J image, overlays two catalogs, and reproduces the Figure 1 visualization.
- A. EXAMPLE: The example demonstrates combining multiple astroquery modules with astropy’s table, coordinate, unit, and WCS tools.
- A. EXAMPLE: SkyView retrieves a 2MASS-J image of the Orion KL nebula region for the finder chart background.
- A. EXAMPLE: The returned image list is reduced to one FITS HDUList, whose image header supplies the WCS used for the plot.
- A. EXAMPLE: Vizier queries retrieve two catalogs within a 5 arcmin region, and their RA/Dec entries are converted into astropy coordinates.
- A. EXAMPLE: The two catalogs are overplotted in FK5 coordinates with blue star markers for one catalog and red circle markers for the other.
- A. EXAMPLE: Figure 1 uses the older, less complete catalog for blue stars and the newer, more complete catalog for red circles.