Source-linked AI summary
Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking
Joseph C. Jacob, Daniel S. Katz, G. Bruce Berriman, John Good, Anastasia C. Laity, Ewa Deelman, Carl Kesselman, Gurmeet Singh, Mei-Hui Su, Thomas A. Prince, Roy Williams
TL;DR
Astronomical surveys require mosaics that reconcile heterogeneous images and data distributed away from available computing resources. Montage addresses this with a portable, modular toolkit and grid portal for custom, science-grade mosaics, using reprojection, background correction, and coaddition. Its grid workflows provide comparable performance for reasonably large problems while adding operational advantages, although some fast reprojection methods are limited to small areas and timing comparisons were affected by system load.
Problem
Astronomical images differ in coordinates, projections, sizes, and resolutions, while survey archives are often remote from available computational resources.
Method
Montage combines modular reprojection, background modeling, and coaddition algorithms with a TeraGrid portal and parallel execution strategies.
Results
For reasonably large problems, the Pegasus grid approach has real-world performance comparable to MPI, while offering fault tolerance and multi-machine execution for a single large job.
Takeaways & Limitations
Montage supports custom science-grade mosaicking across varied computational environments without changing the underlying code base.
Takeaways & Limitations
The rapid plane-to-plane reprojection approach excludes many common projections and is essentially limited to small areas of a few square degrees.
Abstract
from arXiv · showhide
Montage is a portable software toolkit for constructing custom, science-grade mosaics by composing multiple astronomical images. The mosaics constructed by Montage preserve the astrometry (position) and photometry (intensity) of the sources in the input images. The mosaic to be constructed is specified by the user in terms of a set of parameters, including dataset and wavelength to be used, location and size on the sky, coordinate system and projection, and spatial sampling rate. Many astronomical datasets are massive, and are stored in distributed archives that are, in most cases, remote with respect to the available computational resources. Montage can be run on both single- and multi-processor computers, including clusters and grids. Standard grid tools are used to run Montage in the case where the data or computers used to construct a mosaic are located remotely on the Internet. This paper describes the architecture, algorithms, and usage of Montage as both a software toolkit and as a grid portal. Timing results are provided to show how Montage performance scales with number of processors on a cluster computer. In addition, we compare the performance of two methods of running Montage in parallel on a grid.
1 INTRODUCTION
Montage addresses the need for arbitrary, science-grade astronomical mosaics across heterogeneous surveys and distributed archives. The paper presents its toolkit, algorithms, grid portal, and execution strategies.
- Motivation: Wide-area surveys use varying coordinates, projections, sizes, and resolutions, while many sources exceed individual image extents.These differences motivate a general image mosaic engine for combining survey data.
- Distributed data: Distributed archives often place sky-survey data remotely from available computational resources.The portal therefore uses grid technologies, including the TeraGrid infrastructure.
- Virtual observatories: The National Virtual Observatory and International Virtual Observatory Alliance provide infrastructure for locating, retrieving, and analyzing globally hosted astronomical data.Montage is being developed as a science application portal for the National Virtual Observatory.
- Paper scope: The paper describes Montage’s modular toolkit, algorithms, TeraGrid portal architecture, two execution strategies, and their performance comparison.
2 MONTAGE COMPONENTS
Montage provides portable modules for building custom, science-grade FITS mosaics from user-specified imaging parameters. Its staged workflow and modular execution support standalone, MPI, and grid-based processing.
- Montage goal: Montage lets users specify WCS projection, coordinate system, mosaic size, image rotation, and spatial sampling rate.The resulting FITS mosaics preserve input calibration and astrometric fidelity.
- Processing workflow: Mosaic construction reprojects images, rectifies backgrounds to a common flux scale and level, and coadds the results.
- Modular design: These tasks are implemented as independent, portable ANSI C modules, enabling testing, maintenance, and user-defined processing flows.Users can reproject and co-register images, substitute background removal, or script another workflow.
- Usage scenarios: Montage modules can run standalone, process multiple images sequentially or in parallel via MPI, or execute mosaics in parallel on a computational grid.The same core compute modules operate across these computational environments.
3 MONTAGE ALGORITHMS
Montage constructs mosaics through separate reprojection, background-rectification, and coaddition stages. Its algorithms preserve input energy and astrometric information while accelerating common tangent-plane reprojections and supporting optional drizzle processing.
- Montage separates mosaic construction into image reprojection, background rectification, and coaddition modules.
- General image reprojection: Projecting input and output pixels onto the celestial sphere reduces reprojection to spherical polygon-overlap calculations.The intersection polygon is formed on the sphere and its area is calculated using Girard's Theorem.
- General image reprojection: Overlap areas determine how input-pixel energy is distributed to output pixels and provide a natural weighting mechanism for combining images.
- Rapid image reprojection: A novel direct coordinate transform makes common tangent-plane reprojection about 30 times faster than the original two-step process.The faster method uses linear equations between input and output pixel coordinates and avoids the more expensive general mapping sequence.
- Rapid image reprojection: The plane-to-plane method is limited to tangent-plane projections and small regions, while many projections and all-sky images require the slower plane-to-sky-to-plane approach.
- Background rectification: Background rectification assumes non-sky backgrounds are simple, low-spatial-frequency functions and uses overlap pixels in least-squares fitting to adjust neighboring images.
- Drizzle: Montage supports drizzle by shrinking input pixels by a user-defined factor before mapping them onto the output mosaic.The implementation uses interior pixel corners to model flux as coming from a smaller centered box.
4 MONTAGE GRID PORTAL ARCHITECTURE
The Montage grid portal accepts mosaic parameters through a web interface, automates workflow generation and execution, and notifies users when results are available. Its distributed components support flexible deployment across cluster and grid environments.
- Portal components: The portal distributes five main services across sites: User Portal, Abstract Workflow Service, 2MASS Image List Service, Grid Scheduling and Execution Service, and User Notification Service.
- Deployment flexibility: The design supports mosaic processing on different cluster and grid environments, including Condor pools, TeraGrid clusters, and multiple sites without shared storage.
- User interaction: Users submit mosaic parameters through a web portal, after which data access and processing proceed automatically without further intervention.Parameters include the region, size, data source, and user identification; completion is reported by email with a download URL.
- Workflow construction: The abstract workflow service returns a DAG and required input files, encoding job dependencies that identify which processing tasks can run in parallel.
- Grid execution: The grid scheduling service authenticates users, schedules workflows with Pegasus, and executes them with Condor DAGMan.
5 GRID-ENABLING MONTAGE VIA MPI PARALLELIZATION
Montage can be grid-enabled through MPI parallelization closely resembling traditional cluster execution. Its executives divide file-based work among processes, while parallel mAdd assigns distinct mosaic rows to processes.
- MPI approach: MPI parallelization runs Montage executives and mAdd across multiple processors on grid-accessible clusters such as TeraGrid.
- Executive parallelism: Montage executives are straightforwardly parallelized because processes of each executive perform identical operations over assigned files.
- Parallel mAdd: Parallel mAdd assigns each process a unique subset of output-mosaic rows, preventing processes from overwriting one another.
- Execution procedure: The MPI implementation can be invoked by changing shell scripts to launch MPI executables, although some modules remain single-processor and launching multiple jobs adds overhead.
- Scope boundary: The MPI portal design was discussed but not implemented or tested; measured MPI processing was performed on a local cluster workflow.
6 GRID-ENABLING MONTAGE WITH PEGASUS
Pegasus enables Montage workflows to be mapped onto distributed grid resources by converting abstract workflows into concrete executable workflows. It uses grid information and data catalogs to guide placement and reduce redundant computation.
- Workflow mapping: Pegasus maps Montage abstract workflows onto executable forms for hosts, Condor pools, clusters, and TeraGrid resources.
- Abstract representation: Montage abstract workflows represent logical transformations, data dependencies, and parallelizable execution structure without fixing resources.
- Concrete workflows: Pegasus adds resource assignments, data transfers, registration operations, and staging steps when constructing concrete workflows.
- Grid information: Pegasus queries replica, transformation, and monitoring services to locate data, executable programs, and available compute resources.
- Optimization: Scheduling combines software and resource information to place computation near needed data and reuse registered products, eliminating redundant computation.
- Execution and recovery: DAGMan and Condor-G execute the resulting submit files while enforcing dependencies, retrying failed jobs, and generating rescue workflows when needed.
7 COMPARISON OF GRID EXECUTION STRATEGIES AND PERFORMANCE
The paper compares MPI and Pegasus using a benchmark mosaic of large 2MASS data on a TeraGrid cluster. The comparison distinguishes MPI’s shared-filesystem requirement from Pegasus’s broader resource and data-transfer flexibility.
- Comparison: The study quantifies the advantages and disadvantages of MPI and Pegasus as two approaches to running Montage on the grid.
- Benchmark: The benchmark constructs a 6 x 6 degree M16 mosaic from 1,254 2MASS images totaling about 657 megapixels and approximately 5 GB.
- Benchmark: The benchmark output is a 3.7 GB FITS file containing a 21,600 x 21,600 pixel data segment.
- Evaluation setting: Timing measurements were performed on the Phase 2 TeraGrid cluster at NCSA, whose nodes used dual Itanium 2 processors and at least 4 GB of memory.
- Parallelism: The benchmark workflow contains file-based parallelism for independent files or tiles and module-based parallelism for operations such as mAdd.
- Strategy scope: MPI requires processors sharing one filesystem, whereas Pegasus can transfer files between systems and is therefore more general; the comparison nevertheless uses one TeraGrid processor set.
7.3 Data and code stage-in
Montage supports both MPI-based cluster execution and Pegasus-based grid workflows, with similar stage-in needs but different scheduling and execution behavior. MPI achieved substantial speedup, while shared filesystems, failures, I/O, and variable cluster load constrained performance.
- Stage-in: Data and executable stage-in are required in both MPI and Pegasus approaches, although Pegasus can locate and stage them automatically through catalogues.MPI requires the user or portal to know executable locations, while Pegasus uses the Replica Location Service and Transformation Catalogue.
- MPI execution: MPI generates a shell script whose sequential and parallel commands run after one queue delay, while actual filenames are discovered during execution.The script contains directories rather than individual data files; sequential commands inspect directories and inform parallel jobs of filenames.
- Pegasus execution: Pegasus performs more complex workflow preparation but uses Condor DAGMan and a processor pool to keep compute resources filled.Condor-Glidein associates allocated processors with a Condor pool, after which DAGMan schedules the jobs.
- MPI performance: 453 minutes on one processor fell to 23.5 minutes on 64 processors, producing a speedup of 19 for MPI Montage.The parallel code is almost entirely non-sequential, but I/O prevents linear scaling.
- Performance caveats: MPI timings varied with other users’ activity, and mBgExec was best run on 16 processors regardless of the processor count used elsewhere.Measured serial-module times ranged from 0.7 to 1.4 minutes, while some module timings increased with more processors.
- Performance caveats: MPI execution requires one processor pool with shared filesystem access, and a failure in one module or submodule can fail the remaining job.The paper presents Pegasus as an approach that can overcome these limitations.
7.6 Pegasus timing results
Pegasus timing measurements include workflow construction and execution for a 6 x 6 degree mosaic. Increasing processor and cluster counts eventually made Condor and DAGMan scheduling overhead dominant, while sequential workflow sections and machine contention limited efficiency.
- Measurement setup: Pegasus measurements included mDAG and workflow creation plus runtime for constructing a 6 x 6 degree mosaic.Resources were acquired with Condor Glidein, and the reported figure excluded input-data and output-mosaic transfer time.
- Measurement setup: DAGMan execution time was measured in minutes across different processor counts, with workflow clusters matched to processor counts.The number of clusters at each workflow level increased with the number of processors.
- Scheduling overhead: Approximately 1 second per cluster in DAGMan submission, plus Condor scheduling delay, made overhead dominant as processor counts increased.Consequently, adding processors did not always reduce total workflow execution time.
- Scheduling overhead: The 64-processor case performed worse than the 32-processor case, likely because other jobs affected the shared test machine.The authors expected better performance if the case were rerun on a dedicated machine.
- Parallel efficiency: Sequential workflow sections limited overall parallel efficiency in the Pegasus timing results.This limitation remained alongside Condor and DAGMan scheduling overhead.
7.7 Timing discussion
MPI and Pegasus performance depended on problem size and execution overheads: MPI was faster for a small mosaic, while larger runs were comparable and Pegasus offered operational advantages. The comparison remains sensitive to I/O contention and limited experimentation on dedicated hardware.
- Overall comparison: Pegasus was faster than MPI in most benchmark comparisons except at 64 processors, where MPI was faster.The authors considered the measured difference for large jobs not significant because of I/O contention from other jobs.
- Problem size: MPI was almost 3 times faster than Pegasus for a small 1-degree square mosaic on 8 processors.Pegasus built its DAG in advance, while MPI generated inputs on the fly and ran mImgtbl three times instead of once.
- Problem size: For the 36 square degree test problem on 64 processors, overall MPI and Pegasus times were comparable.Pegasus reduced mAdd time by opening only relevant files, while MPI incurred larger I/O overhead but shorter startup time.
- Implementation differences: Pegasus had substantial mDAG and Pegasus preparation times but a much shorter mAdd phase than MPI for the larger problem.The difference arose because Pegasus knew tile coverage in advance, whereas MPI opened all files before discarding nonintersecting ones.
- Experimental boundary: Additional experiments on a large dedicated machine could clarify performance differences between MPI and Pegasus.The reported comparison was affected by the limited experimental setting and I/O contention.
- Operational trade-offs: The authors found no performance difference outweighing Pegasus advantages such as fault tolerance and using multiple machines for one large job.This conclusion was drawn despite the heavily loaded TeraGrid cluster used for testing.
8 CONCLUSION
Montage provides a general toolkit for producing accurate astronomical mosaics and supports execution from single processors through grid environments. Its grid workflows broaden platform flexibility while retaining performance comparable to MPI for reasonably large problems, and the software has supported multiple NASA projects.
- Montage combines modules for reprojection, overlap calculation, background matching, image modification, and coaddition while preserving input photometric and astrometric accuracy.
- Pegasus provides more general cross-platform execution than MPI without changing the underlying code base, with comparable real-world performance for reasonably large problems.
- Montage can run through a simple shell script on a single processor, while grid alternatives address the long runtimes of large mosaics.
- The toolkit, user guide, and support system are available online, and Montage has supported NASA science-data generation, quality assurance, mission planning, and outreach projects.
- Figure 10 compares MPI and Pegasus module timing for creating a 1 x 1 degree mosaic on 8 processors.
WEBSITES
The paper’s websites section collects links to astronomical surveys, archives, grid-computing systems, image standards, and related scientific-infrastructure projects.
- The references include survey and astronomy resources such as 2MASS, COSMOS, DPOSS, GLIMPSE, IRSA, and SDSS.
- The listed grid-computing resources include Condor, Condor-Glidein, the Distributed Terascale Facility, and GriPhyN.
- The section also links to FITS, IVOA, the Simple Image Access specification, Montage, and the National Virtual Observatory.