PaperlayerView exact revision on arXiv

Source-linked AI summary

Scanamorphos: a map-making software for Herschel and similar scanning bolometer arrays

Hélène Roussel

arXiv:1205.2576v3astro-ph.IM

TL;DR

Scanamorphos addresses low-frequency noise and artefacts in Herschel scan maps, where preserving both compact and extended emission is important. It uses observational redundancy rather than an assumed noise model, combining drift estimation, artefact masking, and map projection; the method also applies with minimal adjustment to suitable other arrays and was tested on P-Artemis. The paper reports reduced drift-induced photometric errors and successful processed-map flux recovery, while noting scope limits from scan patterns and non-unique drift solutions.

  • Problem

    Herschel scan observations contain low-frequency noise that creates stripes, reduces faint-source sensitivity, and can alter photometric and morphological properties.

  • Method

    Scanamorphos empirically uses spatial redundancy to estimate and subtract low-frequency drifts, mask high-frequency artefacts, and project data into maps without assuming a noise spectrum.

  • Results

    Scanamorphos reduced drift-induced photometric errors from as much as 20σ to residual errors within 3σ in the reported small-scale tests.

  • Takeaways & Limitations

    The software preserves compact and extended emission while providing an adaptable map-making approach for redundant bolometer-array scan observations.

  • Takeaways & Limitations

    The method’s long-timescale drift subtraction must be modified for scan patterns such as rasters or spirals, and other instruments may require additional thermal-noise terms.

Abstract

from arXiv · show

Scanamorphos is one of the public softwares available to post-process scan observations performed with the Herschel photometer arrays. This post-processing mainly consists in subtracting the total low-frequency noise (both its thermal and non-thermal components), masking high-frequency artefacts such as cosmic ray hits, and projecting the data onto a map. Although it was developed for Herschel, it is also applicable with minimal adjustment to scan observations made with some other imaging arrays subjected to low-frequency noise, provided they entail sufficient redundancy; it was successfully applied to P-Artemis, an instrument operating on the APEX telescope. Contrary to matrix-inversion softwares and high-pass filters, Scanamorphos does not assume any particular noise model, and does not apply any Fourier-space filtering to the data, but is an empirical tool using purely the redundancy built in the observations -- taking advantage of the fact that each portion of the sky is sampled at multiple times by multiple bolometers. It is an interactive software in the sense that the user is allowed to optionally visualize and control results at each intermediate step, but the processing is fully automated. This paper describes the principles and algorithm of Scanamorphos and presents several examples of application.

1. Introduction

Scanamorphos is an interactive yet automated map-making tool developed for Herschel scan observations, with broader applicability to other redundant bolometer-array scans. It removes low-frequency drifts and high-frequency artefacts before projecting calibrated data into maps.

  • Scope: Scanamorphos processes Herschel PACS and SPIRE photometer scan observations and is designed to apply more broadly.Its broader applicability depends on the characteristics and redundancy of the data.
  • Motivation: Low-frequency noise produces scan-parallel stripes, reduces sensitivity to faint sources, and alters their photometric and morphological properties.The map-making stage is suited to this task because temporal and spatial information jointly characterize brightness drifts.
  • Operation: Users can choose map parameters and inspect intermediate results, while the processing itself remains fully automated.The code is written in IDL to facilitate programming and multidimensional array processing.
  • Capabilities: The software removes thermal and flicker-noise drifts, masks cosmic-ray and other brightness discontinuities, and projects data onto spatial grids.It also produces associated error and weight maps; other instrumental corrections occur beforehand in the pipeline.
  • Paper organization: The paper presents Scanamorphos principles, algorithm, simulations, flight-data tests, and practical distribution details.Applications include PACS simulations and visual examples from Herschel flight data.

2. Principles and prerequisites

Scanamorphos exploits spatial redundancy in scan observations to separate astronomical emission from low-frequency detector drifts without prescribing a noise spectrum. Its effectiveness depends on adequate coverage, scan configuration, and assumptions about additive and stable instrumental effects.

  • Redundancy: Each sky position is sampled by multiple bolometers at multiple epochs, providing redundancy for reconstructing brightness drifts without chopped observations.Even the minimal redundancy of standard Herschel scans is reported as sufficient for accurate drift reconstruction.
  • Noise treatment: Scanamorphos makes no assumption about the low-frequency-noise power spectrum, unlike matrix-inversion methods that require calibrated statistical noise models.The paper notes that inaccurate, time-varying, or preprocessing-altered noise properties can impair matrix-inversion results.
  • Astronomical signal: Scan observations can contain compact and extended structures on arbitrary spatial scales, motivating a method that preserves their brightness distribution.In space observations, detector and telescope noise varies more slowly than atmospheric emission and can be separated from sky signal.
  • Background level: Herschel maps retain an unavoidable global offset because Herschel is not an absolute photometer, so users must estimate the background for their application.This choice is especially application-dependent in complex fields such as Galactic star-formation regions.
  • Observing strategy: The method requires scan coverage and redundancy, with scan direction, scan speed, sampling rate, array geometry, and leg spacing affecting the available redundancy.At least two scans with directions separated by at least 20 degrees are generally combined for efficient noise separation.
  • Adaptability: Scanamorphos requires minimal instrumental knowledge and was successfully tested on the ground-based P-Artemis array, although other instruments may require additional noise terms.P-Artemis data helped test and refine the code before Herschel’s launch.
  • Instrumental assumptions: The drift model assumes brightness drifts are additive and multiplicative effects such as gains and flatfields remain stable.PACS lacks blind detectors, making map-based removal of correlated noise necessary for that instrument.

3. Algorithm

The algorithm decomposes detector timelines into sky signal, average drift, individual drift, and high-frequency noise, then estimates and subtracts these components using redundant samples. It uses beam-scale spatial assumptions, coarse grids, iterative corrections, and glitch masking while preserving source structure.

  • Noise decomposition: Scanamorphos represents low-frequency noise as an average time-dependent drift plus an individual drift for each bolometer.This algorithmic decomposition replaces the physical thermal/flicker distinction with shared and detector-specific time functions.
  • Processing sequence: The processing sequence includes coordinate mapping, scan-speed tagging, baseline subtraction, drift subtraction, glitch masking, and final projection.Average and individual drifts are removed on timescales below the scan-leg duration, with repeated residual glitch masking.
  • Redundancy-based estimation: At a fixed sky position, differences among samples from different bolometers and epochs estimate low-frequency drifts because the astronomical signal is shared.High-frequency noise, including glitches and white noise, remains as a separate component.
  • Signal model: The signal model divides the recorded bolometer signal by its gain to obtain sky brightness plus average drift, individual drift, and high-frequency noise.The decomposition can ignore constant differential gains between signal and noise.
  • Spatial assumptions: Drift removal treats the beam FWHM as the spatial scale over which the sky signal is invariant and protects compact sources during estimation.This choice balances source preservation against limited sampling statistics and execution time.
  • Grids: The algorithm uses mapping and coarse drift-removal grids, with the coarse temporal step defined by Tc = ls / vscan.The spatial grid is automatically enlarged when sampling provides fewer than six samples per stability length.
  • Grid orientation: Near a 45° grid orientation relative to a scan direction, drift subtraction can leave residual short-timescale striping.Processing therefore uses an orientation selected by the code, while the user’s grid is applied only to the final projection.

3.3. High-frequency noise

Scanamorphos quantifies high-frequency noise per bolometer and scan to benchmark low-frequency noise and set data weights and detection thresholds. It masks compact sources and glitches before spectral-density estimation, then derives white-noise and threshold-noise measures for subsequent processing.

  • Noise estimation: High-frequency noise is measured for each bolometer and scan from binned spectral-density averages over instrument-specific frequency ranges.The averages use 2.5–5 Hz for SPIRE and 3–5 Hz for PACS.
  • Noise estimation: The white-noise standard deviation benchmarks low-frequency noise amplitude and weights data by its inverse square.
  • Preprocessing: Compact sources and glitches are detected from short-timescale signal differences, masked over six samples, and interpolated before Fourier analysis.Artificial Gaussian noise is added to the interpolated samples because Fourier transforms require continuous time series.
  • Preprocessing: The removed-signal maps show that this masking and interpolation procedure effectively removes both bright compact sources and glitches.
  • Detection thresholds: A threshold-noise estimate accounts for high-frequency spectral-density behavior and defines detection thresholds for compact sources, steep gradients, and glitches.For SPIRE, the estimate uses the 5–10 Hz range because some pipeline modules amplify high-frequency noise beyond 5 Hz.
  • Detection thresholds: For PACS, quantization noise is separately estimated from the brightness quantization step when it contributes significantly to high-frequency noise.

3.5. Short-timescale drift subtraction

Scanamorphos estimates short-timescale drifts empirically from redundant bolometer crossings of the same sky pixels, then reconstructs and subtracts average and individual drift series. Iteration, map protection, and convergence handling address non-uniform sky emission and non-unique drift solutions.

  • Two-stage subtraction: The average drift is first estimated as a single array-wide series, while deviations from that average are treated as uncorrelated individual drifts.This separates coherent thermal contributions from bolometer-specific components before the second subtraction step.
  • Drift differences: Bolometer crossings of the same pixel provide brightness differences that estimate drift differences while cancelling the astronomical signal under local uniformity.The algorithm rejects crossings affected by compact sources, steep gradients, or excessive high-frequency noise, switching to finer pixels when statistics permit.
  • Drift reconstruction: Drift differences are coadded in a weighted time matrix, whose entries are converted into an absolute drift series by repeated scans until convergence.The matrix uses a coarse time grid and symmetric inverse-white-noise weighting; one time is assigned zero drift because the absolute zero is undefined.
  • Redundancy: 5% of the average amplitude of individual drifts is the estimated coaddition noise level, while residual weighted-mean terms become negligible with sufficient redundancy.For the Rosette example, the average redundancy was approximately 650 bolometer crossings per coarse pixel.
  • Non-uniqueness: The drift-difference solution is non-unique because an oscillatory excess drift can produce the same map for successive scans.Scanamorphos maps each scan separately, extracts the component common to the maps, simulates its time series, and removes it before drift subtraction.
  • Operational scope: The algorithm can remove some low-frequency noise from a single scan, enabling data-quality and depth assessment before multi-scan observations are complete.Combining two or more scans remains desirable, but is not required for partial drift removal.

3.6. Detection of other artefacts

Scanamorphos detects and masks brightness discontinuities, residual glitches, and transient sources that can contaminate maps after pipeline processing. It combines temporal statistics with spatial coherence to distinguish instrumental artefacts from real sky structure.

  • Brightness discontinuities: Brightness discontinuities affecting entire PACS array rows or individual bolometers can arise from glitches or electronic instabilities and leave bright or dark map bands.These high-frequency changes are followed by slow decay or a plateau and were not corrected in the pipeline described.
  • Brightness discontinuities: The discontinuity module uses 16-pixel row averages, individual bolometer series, spatial information, and temporal tests to make detections more robust.A candidate is confirmed only after diagnostics rule out compact sources or steep emission gradients.
  • Glitches: Residual glitches are permanently masked rather than interpolated, whether or not a prior HIPE deglitching module was run.Glitch detection is integrated with the high-frequency-noise and drift-estimation modules.
  • Transient sources: Asteroids and other transient sources are identified through coherent signals across bolometers and can be separated from glitches during deglitching.When detected, their affected samples are projected into a dedicated transient-signal map to estimate location and brightness.

3.7. Projection

The final projection produces sky, weight, error, and low-frequency-noise maps in a FITS cube. For SPIRE, a drizzle-like projection reduces pixel-center bias and sky variation while slightly broadening the PRF.

  • Final products: The final projection stores sky, weight, error, and low-frequency-noise maps together in a FITS data cube.Signal weights use inverse-square white-noise values, and the projection uses a gnomonic geometry.
  • SPIRE projection: SPIRE nearest-pixel projection can bias source positions relative to pixel centers by up to a/2.The bias varies with source position relative to the pixel grid.
  • SPIRE projection: A drizzle-like SPIRE projection uses a beam-centered uniform disk, eliminating projection-center bias and reducing sky standard deviation relative to HIPE maps.The reduction exceeds 30% in some configurations when low-frequency noise is perfectly removed.
  • SPIRE projection: The SPIRE projection increases the FWHM by 1.5% and the beam area by 3%, while leaving the beam area used for unit conversion unaffected.The projection changes the PRF profile, not the beam area used to convert Jy per beam to Jy per pixel.
  • Error estimation: The Scanamorphos error map estimates weighted mean-brightness errors but does not propagate uncertainties from earlier processing steps.It is intended to estimate random photometric errors and help filter remaining glitches.
  • Diagnostics: A clean map may diagnose low-redundancy artefacts by excluding noisy scans, but it is not intended for scientific use.It is generally produced for observations with two or three scans and serves as an aid for masking bad data.

3.8. Spatial slicing

Scanamorphos can divide deep, wide-field observations into partially overlapping spatial blocks to reduce memory and computation demands. Each block uses its available redundancy independently before the maps are stitched.

  • Spatial slicing: Deep wide-field observations can be sliced into partially overlapping sub-fields to reduce memory and computing-time requirements.This is suitable when very extended bright emission does not need to be contained within a single sub-field.
  • Spatial slicing: Each sub-field is processed using all available redundancy as though it were a complete observation, and the resulting maps are stitched by matching brightness.
  • Resource requirements: The slicing functionality was unnecessary with 48 GB of memory but was used for some deep observations on an 8 GB machine.
  • Drift computation: Spatial blocking can also address drift-matrix limits that otherwise force a longer minimum timescale for average-drift computation than for individual-drift computation.The method can increase the time resolution available for the relevant drift calculation.

4. Simulations of PACS observations

Simulations of NGC 6334 and a nearby galaxy test whether Scanamorphos removes low-frequency noise while preserving extended emission and compact-source photometry. The processed maps substantially reduce drift-induced errors, recover diffuse structure, and conserve global flux within the reported uncertainties.

  • 4.1. Star formation region: The NGC 6334 simulation used real-observation-based sky structure, PACS blue-band data, and two scans providing minimum recommended redundancy.Noise series combined datasets totaling 5.91 hours, with calibration glitches and brightness discontinuities retained in the simulation.
  • 4.1. Star formation region: The processed NGC 6334 map showed no significant brightness gradient or source imprint in the difference map, indicating satisfactory drift removal and preserved extended emission.Residual low-level structure from imperfect drift subtraction remained within the photometric errors.
  • 4.1. Star formation region: Errors initially reaching 20σ were reduced to within 3σ, while faint-source bias remained about −0.5σ below 170 mJy and disappeared at higher detection significance.The authors conclude that Scanamorphos conserves flux and reduces noise even on small spatial scales.
  • 4.2. Nearby galaxy: For the galaxy simulation, processed-minus-ideal errors stayed below Scanamorphos photometric uncertainties, residual artefacts were mostly below 2σ, and diffuse emission improved beyond about 150″ radius.The comparison used signal maps, error maps, and azimuthally averaged surface-brightness profiles.
  • 4.2. Nearby galaxy: 58.06 ± 0.69 Jy was recovered in the processed galaxy map versus 58.81 ± 0.05 Jy in the ideal map, while the noisy map was 3.7–2.4Σtot below the true flux.The processed-map flux difference corresponded to 1.2–0.8Σtot, or 1.4% of the true flux; the noisy-map deficit was 4.4%.

5. Examples and visualization step by step

The examples apply Scanamorphos to varied Herschel observations, visualizing processing stages and showing that redundancy-based subtraction preserves sky structure while reducing low-frequency noise and artefacts.

  • Examples and visualization step by step: The examples span diverse structures, brightnesses, observing modes, scan speeds, and processing conditions, with time-series and map visualizations at important stages.The observations include parallel-mode and nominal datasets, and the processing summary also records field sizes, options, drift amplitudes, and indicative processing times.
  • Rosette nebula: Residual baselines derived from redundancy are required to remove the large-scale striping left after simple baseline subtraction in Rosette observations.Simple baselines are only the first step and are insufficient to eliminate striping caused by long-timescale drifts.
  • Drift computation: Scanamorphos detects non-uniformities within stability lengths and rejects contaminated samples to prevent sources and glitches from biasing drift calculations.The method automatically identifies compact sources and steep brightness gradients during this process.
  • Atlas field: In the Atlas field, empirical drift subtraction makes overlapping scans agree almost perfectly despite abrupt thermal-drift changes and minimal redundancy.Residual thermal-drift effects remain only at map edges covered by a single scan.
  • NGC 6822: For NGC 6822, baselines and average and individual drifts are robustly derived despite complex large-scale cirrus emission.The example separates residual nonlinear average drift from complex sky signal.
  • Comparisons: Scanamorphos preserves diffuse emission and compact sources under unfavorable PACS sampling conditions, while pipeline polynomial baselines or Fourier filtering strongly distort extended emission.The comparison with PhotProject and MADmap further reports that Scanamorphos recovers more diffuse emission than the alternatives.

6. Final remarks

Scanamorphos removes low-frequency noise by deriving drifts directly from scan redundancy rather than imposing a noise model. Simulations and Herschel examples show preserved compact and extended emission, while limited redundancy leaves low-significance correlated residuals and constrains some scan patterns.

  • Core method: Scanamorphos derives thermal and non-thermal drifts from scan and array redundancy without assuming a noise model.The method uses the data themselves to subtract low-frequency noise in bolometer-array scan observations.
  • Drift timescales: Long-timescale and short-timescale drifts require distinct subtraction methods because the long-timescale component is inadequately sampled within the map.The boundary between the two regimes is set by the scan-leg crossing time T_leg.
  • Redundancy: A useful fiducial redundancy is 10 samples per pixel per scan pair for pixels of size FWHM/4, although coverage can be inhomogeneous.The examples span 75–530 samples per square FWHM per scan pair, depending on instrument and observing parameters.
  • Results: Simulations and Herschel applications reduce noise while preserving the flux and morphology of compact sources and extended emission, with robustness to small pointing and calibration errors.The authors report no dataset that failed to produce a scientifically correct map in the presented context.
  • Limitations: Limited redundancy can create correlated artificial sky fluctuations, but these remain below the 3σ significance level.The software is also described as adaptable to additional instrumental corrections as they become understood.
  • Scope boundary: The long-timescale subtraction currently depends on scan legs spanning the map and would require modification for rasters or spirals.Other instruments may also require adaptations for more complex thermal-noise behavior.

7. Distribution

The distribution section presents visualization and batch-use options alongside figures documenting Scanamorphos processing stages, drift grids, glitch masking, and final maps across several observations.

  • Distribution and operation: Scanamorphos can be run in visualization, non-interactive, or batch mode, with batch processing recommended only after users become familiar with the software.Installation, usage instructions, example scripts, and IDL commands are included with the distribution.
  • Processing illustrations: The Rosette 250 µm figure traces processing from raw data through source interpolation, masking, baseline subtraction, residual-baseline derivation, and subsequent data products.The displayed sequence is intended to show successive processing stages.
  • Drift grids: Drift-mode grids show that finer spatial grids are enabled at 350 µm when signal dispersion is high, but not at 250 µm because too few samples fall within a stability length.The 350 µm fine grid is coded by value 3.
  • Glitch masking: Glitch maps distinguish amplified trails and simultaneous whole-array glitches from residual glitches detected during later average-drift subtraction.The figures illustrate two successive masking stages.
  • Application outputs: The Atlas figures compare pipeline temperature-drift correction with Scanamorphos subtraction and show scan-difference maps, subtracted drifts, and spectral densities.The NGC 6822, NGC 6946, and PACS comparison figures extend the documentation to final maps, weight maps, profiles, and alternative map-makers.
  • PACS comparisons: PACS comparison figures present PhotProject, MADmap, and Scanamorphos maps at 70 and 160 µm together with azimuthally averaged surface-brightness profiles.The figures use logarithmic displays and specified pixel sizes to compare the map products.

A. Gain corrections for SPIRE

SPIRE gain corrections address bolometer-response differences for extended sources, which point-source calibration and uniform-beam corrections do not capture. The procedure derives relative gains from science observations and reduces photometric errors without changing global extended-source fluxes.

  • SPIRE bolometer gain corrections compensate for non-uniform beam coupling to extended sources that point-source calibration does not capture.The standard extended-source corrections assume a unique beam and neglect bolometer-to-bolometer variations.
  • Relative gains are derived by fitting each bolometer’s brightness series to a map-simulated series after baseline subtraction, then iterating slope corrections.Slopes are determined independently for each scan, averaged, and multiplied across iterations to obtain each bolometer’s gain.
  • Global extended-source fluxes are unchanged by the gains, while small-scale amplitudes can depend on source position relative to the array.The gains are a non-random function of bolometer position, so local effects may be non-negligible.
  • Photometric errors were found to decrease significantly after applying the gain corrections.
  • Figure 26 plots relative gains against bolometer index for the 250, 350, and 500 µm arrays, with error bars showing field-to-field standard deviations.The calibration set uses the Rosette nebula and 16 other bright complex fields.
  • Figure 27 maps the relative gains by bolometer position for the 250, 350, and 500 µm arrays in the scan-direction orientation.

B. Options

Scanamorphos exposes its inputs and processing options in a summary table and provides a separate online user guide for setup and outputs.

  • Table 2 summarizes the inputs and options available to Scanamorphos.
  • The online distribution includes a full user guide covering input setup and output structure.
  • The detailed descriptions of inputs and options follow the summary table.

B.1. Main options

Main options support PACS discontinuity masking and visualization of average drift subtraction, with user review available in interactive mode.

  • The /jumps pacs option detects PACS brightness discontinuities and masks affected samples.Very unstable rows or bolometers are automatically detected and pre-filtered by the same module.
  • Interactive mode requires the user to visually confirm or reject detected PACS discontinuities.
  • Visualization mode allows inspection of the average drift series and selection of whether to apply its correction.
  • When nonlinear thermal drift is negligible, subtracting a very small average drift can introduce noise.The recommended check is to compare final maps processed with and without average-drift subtraction.

B.2. Non-default astrometry

Scanamorphos normally derives astrometry and the spatial grid from input scans, while allowing reference headers, scan offsets, and field restriction through user-specified options.

  • By default, Scanamorphos determines astrometry and the spatial grid from the input scans.
  • A reference FITS header can replace the input-derived astrometry and spatial grid, while restricting processing to part of the field is also supported.With a reference header, only one array should be processed at a time, and the option is usable in batch mode.
  • Per-scan right-ascension and declination offsets can be supplied as angular distances in arcseconds.
  • Interactive mode allows users to reject false detections, while field-of-view slicing is intended only for specific cases.
Loading 1205.2576v3…