PaperlayerView exact revision on arXiv

Source-linked AI summary

MegaPipe: the MegaCam image stacking pipeline at the Canadian Astronomical Data Centre

Stephen. D. J. Gwyn

arXiv:0710.0370v1astro-ph

TL;DR

Archival MegaCam data are difficult to use because multiple large, distorted exposures require substantial processing and precise alignment. MegaPipe calibrates their astrometry and photometry, resamples and stacks the images, and produces catalogues, achieving 0.04-arcsecond internal astrometric uncertainty and about 0.2-arcsecond external uncertainty.

  • Problem

    Archival MegaCam data require combining multiple large exposures, while focal-plane distortion and limited initial astrometric accuracy make processing and alignment difficult.

  • Method

    MegaPipe performs astrometric and photometric calibration, resamples calibrated images to an undistorted grid, and combines them into median stacks while extracting catalogues.

  • Results

    0.04 arcseconds internal and about 0.2 arcseconds external astrometric uncertainty are achieved, with external checks showing residuals generally near 0.2 arcseconds.

  • Takeaways & Limitations

    The resulting stacked images and catalogues provide processed MegaCam products for archival use through the Canadian Astronomical Data Centre.

  • Takeaways & Limitations

    Internal repeatability tests can remain small even when systematic errors are inherited from the reference image used to register other bands.

Abstract

from arXiv · show

This paper describes the MegaPipe image processing pipeline at the Canadian Astronomical Data Centre. The pipeline combines multiple images from the MegaCam mosaic camera on CFHT and combines them into a single output image. MegaPipe takes as input detrended MegaCam images and does a careful astrometric and photometric calibration on them. The calibrated images are then resampled and combined into image stacks. The astrometric calibration of the output images is accurate to within 0.15 arcseconds relative to external reference frames and 0.04 arcseconds internally. The photometric calibration is good to within 0.03 magnitudes. The stacked images and catalogues derived from these images are available through the CADC website:

1. INTRODUCTION

MegaPipe addresses the processing burden that limits use of archival MegaCam data by calibrating, stacking, and cataloguing multiple exposures into usable products.

  • Archival MegaCam programs often require multiple exposures to increase depth and remove image defects, so the images must be combined.
  • A typical program uses 5 or more exposures of about 0.7Gb each, making retrieval, resampling, CPU use, and storage substantial barriers.MegaCam focal-plane distortion requires resampling, which can require 32-bit copies of the images.
  • Elixir detrending is effective, but its 0.5-1.0 arcsecond astrometry is insufficient for aligning images to better than one pixel.MegaPrime focal-plane distortion makes higher-accuracy alignment difficult for users to devise independently.
  • MegaPipe aims to remove these barriers by combining MegaCam images into stacks and extracting catalogues.
  • The pipeline procedure includes image grouping, astrometric calibration, photometric calibration, image stacking, catalogue generation, quality checks, and distribution.

2. IMAGE GROUPING

MegaPipe groups only usable exposures, applying calibration prerequisites, visual and automatic quality control, spatial grouping, and filter-specific exposure thresholds.

  • Each input image must contain enough distinct, well-centred sources per CCD for astrometric calibration.Short exposures under 50 seconds, nebular images, and images taken under poor transparency cannot be used.
  • Photometric calibration requires either a photometric night or photometric standards from SDSS or previously processed MegaPipe images.
  • A trained operator visually inspects one CCD per exposure and discards exposures with asymmetric PSFs or major seeing, focus, or transparency defects.The inspection takes about 1 second per image and is semiautomated.
  • Automatic pixel-statistics tests detect dead CCDs and completely saturated exposures, which are then discarded.
  • Quality-controlled images are merged when group centres lie within 0.1 degrees, with merged centres weighted by group image counts.The grouping converges after 3 or 4 iterations and is stable for typical MegaCam observing patterns.
  • Groups retain a filter only when they contain at least five images in that filter, while the lower limit of four exposures balances defect rejection against data retention.

3. ASTROMETRIC CALIBRATION

MegaPipe calibrates MegaCam astrometry by detecting reliable sources, matching external and internal catalogues, and modelling focal-plane distortion before registering bands.

  • Source detection uses SExtractor with a 5-sigma threshold across at least 5 contiguous pixels, followed by removal of cosmic rays and extended objects.
  • Initial astrometry matches observed catalogues to USNO A2.0 by maximizing catalogue matches over RA and Dec offsets.CCDs with fewer than 10 matches or fewer than half the average matches of other CCDs are flagged as failed.
  • The WCS separates linear CCD tilt, rotation, and offset from higher-order focal-plane distortion represented with polynomial terms.
  • The distortion model is fitted by nonlinear least squares while minimizing average astrometric residuals.
  • The global distortion is translated into a local 10-parameter, third-order model for each CCD, with translation error below 0.001 arcseconds.
  • SDSS replaces USNO where available, and an internal MegaPrime catalogue further improves registration across images and bands.
  • 0.04 arcseconds is the final internal astrometric uncertainty, while external uncertainty is about 0.2 arcseconds.

4. PHOTOMETRIC CALIBRATION

MegaPipe calibrates MegaCam photometry against SDSS and related standards, using filter-transformation relations and alternative procedures for data outside the SDSS footprint.

  • SDSS DR5 provides the basis for calibration because MegaCam and SDSS filters differ and require color-term transformations.
  • The transformation-relation residuals are σu = 0.07, σg = 0.02, σr = 0.06, σi = 0.03, and σz = 0.07 magnitudes where intrinsic photometric noise does not dominate.
  • Within the SDSS footprint, at least 1000 usable sources per square degree reduce random calibration error to effectively zero, with systematic photometric errors of about 0.02 magnitudes.Individual CCDs can be calibrated using about 30 standards each.
  • For groups outside SDSS, MegaPipe modifies Elixir photometric keywords after comparing Elixir zero-points with SDSS-based calibrations.The comparison finds systematic offsets, especially in the u∗ band, with epoch-dependent variation.
  • Photometric-night data generate in-field standards for calibrating non-photometric data, but groups with no photometric-night exposures cannot be processed.

5. IMAGE STACKING

MegaPipe coadds calibrated MegaCam images after sky subtraction, photometric scaling, distortion correction, and resampling. It uses median combination to reject defects, while producing large full-mosaic stacks and weight maps for downstream analysis.

  • Image combination: SWarp subtracts the sky, applies photometric scaling, resamples images onto an undistorted tangent-plane grid, and median-combines the pixels.Median combination is noisier than averaging but better rejects cosmic rays and bad columns.
  • Image combination: Mask images assign weight 1 to good data and 0 to pixels with known defects, and an inverse-variance map accompanies each output image.
  • Output stacks: About 20,000 × 20,000 pixels, or roughly 1° × 1°, the full-mosaic stacks are approximately 1.7 Gb in size.The dimensions depend on the input dither pattern.
  • Caveat: Sky subtraction sets the image sky level to zero counts but can remove smoothly varying extended emission or create peculiar results near sharp emission features.

6. CATALOGUE GENERATION

MegaPipe runs SExtractor on each output stack using its weight map to generate single-band catalogues. The approach works in many cases but is not optimal for crowded fields and does not provide multi-band catalogues.

  • Catalogue generation: SExtractor is run on each output stack with the weight map, producing catalogues restricted to a single band.
  • Catalogue generation: The catalogue-generation approach works well in many cases but is probably not optimal for some situations, including highly crowded fields.SExtractor was originally designed for sparse, high-galactic-latitude fields.

7. CHECKS ON ASTROMETRY

MegaPipe checks internal, cross-group, and external astrometric accuracy, as well as whether stacking degrades image quality. Internal and repeatability residuals are typically about 0.06 arcseconds, while external uncertainty is estimated at about 0.15 arcseconds.

  • Internal accuracy: Internal astrometric residuals have a 68%-tile radial value of 0.040 arcseconds in a representative case, with about 0.025 arcseconds error in each coordinate.More typically, residuals are 0.06 arcseconds.
  • Internal accuracy: Cross-band residuals can remain small even when systematic errors persist, because later images are mapped to the reference image’s potentially erroneous positions.
  • Repeatability: 0.06 arcseconds is the typical astrometric repeatability across independent groups, making cross-group comparisons more stringent than within-group comparisons.Independent group calibrations enable this stronger test.
  • External accuracy: About 0.15 arcseconds is the estimated external astrometric uncertainty after accounting for reference-catalogue uncertainties.The estimate applies when SDSS is used as the external reference; USNO gives a similar result after accounting for its larger uncertainties.
  • Image quality: Output image quality matches the median input image quality, while mosaic corners are about 5% worse than the centre.The checks indicate that astrometric errors and stacking do not affect image quality.

8. CHECKS ON PHOTOMETRY

MegaPipe’s photometric checks compare calibrated stacks with SDSS, independent groups, and stellar color loci, while several limiting-magnitude tests characterize detection performance. These diagnostics show small photometric offsets but also identify boundaries for color-based and crowded-field assessments.

  • Systematic Errors: Photometry tied directly to SDSS has effectively nil systematic error relative to SDSS, whose quoted systematic errors are 2–3%.Each field contains approximately 1000 standards.
  • Systematic Errors: 0.02 to 0.03 magnitudes is the typical night-to-night scatter, while the combined estimated systematic error is 0.035 magnitudes.The total combines 0.025-magnitude SDSS systematic error with 0.025 magnitudes from transferring primary to secondary standards.
  • External Comparisons: Less than 0.005 magnitudes is the typical relative photometric offset between mosaic centers and corners for stacks calibrated with SDSS.Stacked-image magnitudes agree very well with SDSS-transformed magnitudes, with differences attributed to calibration-equation residuals and random errors.
  • External Comparisons: 0.03 magnitudes is the typical residual when stacks use only Elixir zero-points and are compared with SDSS.This agrees with residuals between individual non-stacked images and SDSS.
  • Internal Consistency: 0.015 magnitudes is the typical offset between independently reduced overlapping groups, below the 0.03-magnitude night-to-night Elixir zero-point variation.The test uses common sources in group pairs with more than 100 objects.
  • Star Colors: Stellar-color diagnostics cannot reliably reveal shifts below 0.05 magnitudes and are affected by the metallicity of the stellar population.They remain applicable to groups outside the SDSS footprint, where direct photometric checks are unavailable.
  • Limiting Magnitudes: The limiting-magnitude tests include number-count turnover, magnitude-error thresholds, and recovery of artificial galaxies.The magnitude-error method uses a 5-sigma threshold corresponding to a 0.198-magnitude error; one example gives limits of 25.5 and 26.5 magnitudes by different methods.
  • Limiting Magnitudes: Extended objects become harder to detect at brighter magnitudes, whereas stellar objects remain detectable roughly a magnitude fainter.Limiting-magnitude plots are less useful in crowded fields because blending can hide objects, especially stellar ones.

9. PRODUCTION AND DISTRIBUTION

MegaPipe is operational at the Canadian Astronomical Data Centre, where it has generated hundreds of groups and stacks. The CADC distributes the products through searchable previews and cutout access.

  • Production: Each group currently takes 10 minutes multiplied by the number of input images to produce on CADC processing nodes.The production rate depends directly on the number of input images.
  • Production: More than 700 groups and about 1500 stacks comprising 12000 input images have been generated.The stated plan is to process all MegaCam images as they become public.
  • Distribution: CADC users can search images by position, name, or input-image properties such as image count and total exposure time.A preview facility supports rapid panning and zooming without downloading the full science images.
  • Distribution: The CADC distribution service also provides cutouts, allowing users to retrieve small subsections of MegaPipe images.The paper identifies the Canadian Astronomical Data Centre as the distribution site.
Loading 0710.0370v1…