PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast and Accurate Retrieval of Methane Concentration from Imaging Spectrometer Data Using Sparsity Prior

Markus D. Foote, Philip E. Dennison, Andrew K. Thorpe, David R. Thompson, Siraput Jongaramrungruang, Christian Frankenberg, Sarang C. Joshi

arXiv:2003.02978v1eess.IVcs.DCphysics.ao-phstat.AP

TL;DR

Identifying methane point sources matters for mitigating anthropogenic emissions. This paper adds sparsity and albedo correction to matched-filter retrieval, reducing background standard deviation in real and simulated data to 36.3%-47.1% of the reference robust matched filter.

  • Problem

    Identifying and mitigating anthropogenic methane point sources matters because they dominate anthropogenic methane sources and emissions reduction offers potential for rapidly reducing global radiative forcing.

    Source: §I · p. 1

  • Method

    The algorithm applies a sparse prior and albedo correction to matched-filter retrieval of local methane concentration enhancements from imaging spectrometer data.

    Source: §I · p. 2, §V · p. 11

  • Results

    Background standard deviation was reduced to 36.3%-47.1% that of the reference robust matched filter in both real and simulated data.

    Source: §IV · p. 10

  • Takeaways & Limitations

    Under computational constraints, albedo correction alone can improve retrieval accuracy while preserving Gaussian retrieval-noise statistics for uncertainty assessment and propagation.

    Source: §IV · p. 10

    Unaccounted variation in geometry and atmospheric conditions across flightlines limits retrieval accuracy; the unit absorption spectrum must be tuned for individual images.

    Source: §IV · p. 10

Abstract

from arXiv · show

The strong radiative forcing by atmospheric methane has stimulated interest in identifying natural and anthropogenic sources of this potent greenhouse gas. Point sources are important targets for quantification, and anthropogenic targets have potential for emissions reduction. Methane point source plume detection and concentration retrieval have been previously demonstrated using data from the Airborne Visible InfraRed Imaging Spectrometer Next Generation (AVIRIS-NG). Current quantitative methods have tradeoffs between computational requirements and retrieval accuracy, creating obstacles for processing real-time data or large datasets from flight campaigns. We present a new computationally efficient algorithm that applies sparsity and an albedo correction to matched filter retrieval of trace gas concentration-pathlength. The new algorithm was tested using AVIRIS-NG data acquired over several point source plumes in Ahmedabad, India. The algorithm was validated using simulated AVIRIS-NG data including synthetic plumes of known methane concentration. Sparsity and albedo correction together reduced the root mean squared error of retrieved methane concentration-pathlength enhancement by 60.7% compared with a previous robust matched filter method. Background noise was reduced by a factor of 2.64. The new algorithm was able to process the entire 300 flightline 2016 AVIRIS-NG India campaign in just over 8 hours on a desktop computer with GPU acceleration.

I. INTRODUCTION

Identifying methane point sources matters for mitigation, but existing retrieval approaches face limits in spatial resolution or computational practicality. The paper introduces a sparsity- and albedo-corrected matched-filter algorithm and evaluates it on real and synthetic data.

I. INTRODUCTION

  • Anthropogenic methane emissions are dominated by point sources, whose identification and mitigation offer potential for rapid reductions in radiative forcing.Methane accounts for 20% of radiative forcing from anthropogenic emissions despite representing 4% of carbon mass flux.Source: §I · p. 1
  • Satellite instruments have kilometer-to-tens-of-kilometers resolution, whereas airborne instruments typically provide less than 10-meter resolution suited to individual point sources.Airborne imaging spectrometers can map entire methane plumes.Source: §I · p. 1
  • IMAP-DOAS has not been practical for full-scene concentration estimates or real-time methane mapping because of its high computational requirements.Matched-filter and band-ratio methods can process large datasets, but accurate methane concentration retrieval required a unit absorption spectrum.Source: §I · pp. 1–2

II. METHODS

  • The paper applies sparsity and albedo correction to matched-filter retrievals, with evaluation on real AVIRIS-NG data and synthetic images for validation.The methods section describes matched filters, sparsity and spatial conformity, target spectra, background statistics, and experimental and validation datasets.Source: §I · p. 2, §II · p. 2

A. Matched Filter

The matched filter estimates methane enhancement by modeling radiance changes against a methane target spectrum and background noise. Its likelihood minimizer is the robust matched filter.

A. Matched Filter

  • Sensor radiance is modeled as methane absorption following Beer-Lambert behavior, with additive zero-mean colored Gaussian noise characterized by covariance C.The target spectrum is based on methane absorption, and the model is linearized with a first-order Taylor expansion.Source: A. Matched Filter · p. 2
  • The target spectrum depends on ambient radiance, approximated by the mean at-sensor radiance μ, and the model uses negative target-spectrum values for the single gas.The radiance-dependent target can be generated from an absorption spectrum or radiative-transfer simulations.Source: A. Matched Filter · p. 2
  • The log-likelihood minimizer is the robust matched filter used for hyperspectral signature detection.The empirical covariance C and mean μ are estimated as described in Section II-E.Source: A. Matched Filter · p. 2

B. Sparsity Prior

The sparsity prior reflects the expectation that enhanced pixels are rare within large imaging scenes. Reweighted ℓ1 optimization provides a tractable iterative approach to sparse, nonnegative enhancement estimates.

B. Sparsity Prior

  • A typical AVIRIS-NG scene contains millions of pixels, while enhanced pixels may occupy only a few thousand, motivating a sparse prior on the matched-filter optimization.The ℓ0 count directly represents the number of enhanced pixels, but its nonconvexity makes optimization impractical because it requires exhaustive combinatorial search.Source: B. Sparsity Prior · p. 2
  • Relaxing ℓ0 to ℓ1 yields a convex optimization problem with efficient solutions, and reweighting ℓ1 iteratively addresses its dependence on magnitude.The reweighted scheme applies iterative convex ℓ1 solutions with updated weights to tractably solve a relative of the nonconvex ℓ0 problem.Source: B. Sparsity Prior · pp. 2–3
  • The optimization estimates pixelwise gas enhancements α_i using empirical covariance C and mean μ, a target spectrum t(μ), and iteration-specific regularization weights.A small positive ε stabilizes the weights when enhancement is zero.Source: B. Sparsity Prior · p. 3
  • The method combines reweighted ℓ1 with ISTA, alternating weight and concentration updates while thresholding to retain only positive enhancements.ISTA is used for simplicity, and the combined iterations avoid computing an optimal ℓ1 solution at every reweighting step.Source: B. Sparsity Prior · p. 3

C. Albedo Correction

Albedo correction addresses the reduced absolute absorption signal over low-reflectance surfaces by scaling the target spectrum using each pixel’s observed albedo. The corrected optimization also scales regularization by albedo.

C. Albedo Correction

  • Low-albedo surfaces reduce the absolute-radiance absorption signal, and a single target spectrum estimated from background mean radiance may not represent each pixel accurately.A per-pixel target spectrum that accounts for local albedo could be more accurate.Source: C. Albedo Correction · p. 3
  • The method scales the target spectrum by each pixel’s observed albedo factor, calculated from the spectral mean μ and pixel radiance L_i.This scaling reflects the dependence of absorbed radiance on initial radiance without the absorber.Source: C. Albedo Correction · p. 3
  • Albedo scaling normalizes the solution and adjusts regularization, decreasing it in low-signal regions and increasing confidence in high-signal regions.The optimization is solved again with reweighted ℓ1 and ISTA.Source: C. Albedo Correction · p. 3
  • Figure 1 describes the algorithm’s component settings: results without sparsity enforce w_k = 0, and results without albedo correction set r_i = 1.These settings specify how to disable each component.Source: Figure 1 · p. 3

D. Target Spectrum Generation

The target spectrum is derived from radiative-transfer simulations of methane absorption and scaled by mean radiance at each wavelength. Retrieved enhancement is expressed as concentration-pathlength in ppm·m, reflecting an ambiguity between concentration and pathlength.

D. Target Spectrum Generation

  • The unit absorption spectrum gives the per-band change in natural-log radiance per 1 ppm·m methane enhancement, estimated from MODTRAN6 simulations.Simulations assumed 1.85 ppm background methane, a 500 m enhanced surface layer, and the AVIRIS-NG flight altitude and band response.Source: D. Target Spectrum Generation · p. 4
  • The target spectrum scales unit absorption by mean radiance at each wavelength, giving it radiance-per-ppm·m units.The unit absorption spectrum is multiplied element-wise by mean radiance.Source: D. Target Spectrum Generation · p. 4
  • Retrieved α_i represents enhancement above background in ppm·m, because low concentration over a long pathlength is radiatively equivalent to high concentration over a short one.Source: D. Target Spectrum Generation · p. 4

E. Strategies for Estimating C and µ

The background mean and covariance are estimated iteratively while removing the current gas-enhancement signal. Grouping adjacent detectors reduces computational work while balancing background sample size against detector-specific fidelity.

E. Strategies for Estimating C and µ

  • Contamination of the target spectrum in the background mean or covariance degrades matched-filter detection, especially when it enters the covariance matrix.Source: E. Strategies for Estimating C and µ · p. 4
  • The iterative mean and covariance estimates remove the latest enhancement estimates from radiance data, preventing target-signal contamination of the background model.The authors describe this iterative estimation as less computationally demanding than an exhaustive shrinkage search.Source: E. Strategies for Estimating C and µ · p. 4
  • Pooling adjacent detectors into a partition reduces repeated mean and covariance calculations, trading sample size and detector-response fidelity against computational complexity.Experiments group five adjacent detectors; the convergence experiment groups all detectors together.Source: E. Strategies for Estimating C and µ · p. 4

F. Experimental Data

The study uses AVIRIS-NG flight data over varied Ahmedabad landscapes and constructs simulated radiance images with known methane enhancements for validation. The simulations use smoothed reflectance and include both realistic plume fields and randomly assigned enhancements.

F. Experimental Data

  • More than 300 India campaign flightlines covered varied terrain at approximately 3.3–8.2 m spatial resolution; Ahmedabad flightline ang20160211t075004 was selected for its varied land cover and suspected sources.The selected scene includes urban and agricultural areas, low-albedo water, spectral confusers, and suspected petroleum, landfill, and wastewater sources.Source: F. Experimental Data · p. 5
  • The Ahmedabad flightline was processed with three matched filters using five-detector partitions and 30 iterations for iterative methods.The methods were Reference RMF, RWL1, and Albedo-Corrected RWL1.Source: F. Experimental Data · p. 5

G. Simulated Gas Enhancements

  • A plume-free adjacent flightline supplied reflectance for simulated radiance; Savitzky-Golay smoothing removed narrow features that could be confused with methane while preserving broader surface absorptions.The smoothing used a fourth-degree polynomial and 21-band width.Source: G. Simulated Gas Enhancements · p. 5, Figure 2 · p. 5
  • Simulated radiance had 3.0% mean absolute error at 2134 nm; residual differences were attributed in part to aerosol scattering that the urban aerosol model may represent poorly in a tropical urban environment.The authors state that the residual shape from 2080 to 2450 nm should have negligible retrieval impact because methane absorption features are relatively fine.Source: G. Simulated Gas Enhancements · p. 5
  • The validation images included LES plumes from a 300 kg hr^-1 source under 4 m s^-1 wind and random enhancements in 1% of pixels spanning 0–10000 ppm·m.LES fields were resampled to the flightline’s 8.1 m spatial resolution; the same simulated radiance image was used across filters within each experiment.Source: G. Simulated Gas Enhancements · p. 6

III. RESULTS

The reweighted-ℓ1 methods reduced background variability in the Ahmedabad retrievals, while albedo correction did not improve every land-cover case. The method also retrieved known plumes and reached stable convergence within 20 iterations.

A. Implementation Details

  • The matched-filter implementations used MATLAB; Albedo-Corrected RWL1 was also implemented in GPU-accelerated PyTorch and processed on a workstation with a Quadro GV100 GPU.Flightline portions censored by the Indian military were skipped.Source: A. Implementation Details · p. 6

B. Retrieval of Known Methane Point Sources

  • 129.43 ppm·m agricultural-background standard deviation for Albedo-Corrected RWL1MF versus 294.08 ppm·m for Reference RMF, while water-background noise was lower for RWL1MF than for Albedo-Corrected RWL1MF.Water-region standard deviations were 162.32 ppm·m for RWL1MF and 245.08 ppm·m for Albedo-Corrected RWL1MF; false positives remained similar and coincided with known confuser surfaces.Source: B. Retrieval of Known Methane Point… · p. 6
  • 158.09 ppm·m urban-background standard deviation for Albedo-Corrected RWL1MF compared with 331.75 ppm·m for Reference RMF.The RWL1 method measured 182.06 ppm·m in the same urban background region.Source: Figure 3 · p. 6, B. Retrieval of Known Methane Point… · p. 6
  • Known plume examples included a 570 m wastewater-treatment plume peaking at 6391 ppm·m and petroleum-infrastructure plumes peaking at 24681 and 7255 ppm·m.The petroleum plumes extended 350 m and 250 m, respectively.Source: Figure 4 · p. 6, B. Retrieval of Known Methane Point… · p. 7
  • The optimization reached stable convergence within 20 iterations, and other experiments used 30 iterations to provide additional margin.The convergence test grouped all 598 detectors and tracked energy for 200 iterations.Source: Figure 5 · p. 7, B. Retrieval of Known Methane Point… · p. 7

C. Simulated Plume Retrievals

Simulated plumes overlaid on a real Ahmedabad image with known concentration-pathlength show that sparse retrievals reduce background noise. Albedo-corrected RWL1MF also reduces agricultural background variability and improves recovery of enhanced pixels.

C. Simulated Plume Retrievals

  • 2.64× lower agricultural background standard deviation (90.35 vs 238.8 ppm·m) was achieved by Albedo-Corrected RWL1MF versus Reference RMF.The simulation used plumes overlaid on an image without obvious methane enhancement and compared retrieved pixel values with known simulated values.Source: C. Simulated Plume Retrievals · p. 7, Figure 6 · p. 7
  • Sparse matched filter methods significantly reduced background noise in pixelwise comparisons of simulated and retrieved enhancement.Figure 7 includes pixels above the approximate 100 ppm·m background noise level.Source: Figure 7 · p. 7, C. Simulated Plume Retrievals · p. 7
  • Albedo-corrected retrievals reduced errors associated with extreme albedo factors, and the albedo-corrected filter recovered enhancements most reliably with the highest coefficient of determination.The reweighted ℓ1 methods showed a small bias toward lower concentration retrievals.Source: C. Simulated Plume Retrievals · p. 7, Figure 7 · p. 7

D. Random Simulated Enhancement Retrievals

Randomly sampled simulated enhancements tested retrieval behavior across values up to 10,000 ppm·m. The albedo-corrected sparse matched filter achieved the best all-pixel RMSE, while albedo correction and sparsity affected retrievals differently across enhanced and nonenhanced pixels.

D. Random Simulated Enhancement Retrievals

  • Randomly placed simulated enhancements ranged from 0 to 10,000 ppm·m, enabling evaluation across a broader range with uniform sampling.The proposed algorithm combines albedo correction, iterative estimation with a positivity constraint, and a sparse ℓ1 prior.Source: D. Random Simulated Enhancement… · p. 7
  • The comparison included albedo-only and iterative positivity-constrained variants, with and without albedo correction, to isolate the effects of these features.The iterative positivity constraint is applied before background mean and covariance calculations to prevent adding methane signatures to the covariance estimate.Source: D. Random Simulated Enhancement… · p. 8, Table I · p. 8
  • Albedo correction reduced retrieval variance and the dependence of retrievals on surface albedo; the albedo-corrected RWL1 filter had 85 ppm·m regression bias versus 93 ppm·m for Reference RMF.The albedo-corrected traditional matched filter had the lowest bias at 71 ppm·m, while the albedo-corrected iterative positivity-constrained algorithm had the highest R2 but 488 ppm·m bias.Source: D. Random Simulated Enhancement… · pp. 8–9
  • 93.9% of pixels with zero simulated enhancement were estimated as exactly zero by Albedo-Corrected RWL1MF.Reweighted ℓ1 sparsity also produced an order-of-magnitude decrease in low retrieved enhancements, while albedo correction generally widened the nonenhanced-pixel distribution.Source: D. Random Simulated Enhancement… · p. 9, Figure 10 · p. 9
  • 60.7% relative improvement over Reference RMF was achieved in all-pixel RMSE by the albedo-corrected sparse matched filter.The all-pixel RMSE evaluates performance across enhanced and nonenhanced pixels together.Source: D. Random Simulated Enhancement… · p. 9, Table II · p. 9
  • The combined iterative, positivity-constraint, sparsity, and albedo-correction contributions generally yielded lower enhancements than Reference RMF, with negative Reference RMF retrievals mapped to zero.The retrieval change depended on pixel albedo; sufficiently low-albedo pixels had increased retrievals.Source: D. Random Simulated Enhancement… · p. 10, Figure 11 · p. 10

E. India Dataset Processing

The algorithm processed the full India Phase 1 campaign in 8.3 hours; discussion reports retrieval and noise improvements alongside limitations in spectral assumptions and simulated-noise realism.

E. India Dataset Processing

  • 8.3 hours sufficed to process all 300 India Phase 1 flightlines and 2.4 TB of radiance files on a GPU-accelerated desktop workstation.The authors report that storage-reading speed limited completion time.Source: E. India Dataset Processing · p. 10

IV. DISCUSSION

  • Sparsity and albedo correction are complementary: their combined RMSE reduction exceeds the sum of their individual reductions, while albedo-only use preserves Gaussian retrieval-noise statistics.Albedo correction improves retrieval over low-albedo surfaces, with consistent correction reported for albedo factors as low as 0.2.Source: §IV · p. 10
  • 36.3%–47.1% was the background standard deviation relative to the reference robust matched filter in both real and simulated data.The simulated-image background standard deviation was 58–81% of that from an adjacent real flightline, and the noise model may omit noise present in real data.Source: §IV · p. 10
  • Accurate retrievals are limited by within- and between-flightline variables not accounted for, including geometry and atmospheric conditions affecting the unit absorption spectrum.The spectrum must be tuned for individual images; uniform spectra may represent airborne scenes spanning tens of kilometers, whereas future satellite missions will require spatially varying spectra.Source: §IV · p. 10
  • The target spectrum and sparse prior are gas- and wavelength-agnostic, and the improvements may also benefit methane flux estimation and automated plume detection.Reduced background noise is identified as beneficial for machine-learning-based automated plume detection.Source: §IV · pp. 10–11

V. CONCLUSION

The conclusion reports improved methane enhancement retrievals from combining albedo correction with a sparse prior, with faster convergence and reduced background noise. AVIRIS-NG and other data are available upon request.

V. CONCLUSION

  • A background standard deviation decrease by a factor of over 2.6 in the validation dataset makes point-source plumes more discernible against background noise.The approach uses a sparse prior for rare methane enhancements and albedo correction for more accurate estimates over low-reflectance surfaces.Source: §V · p. 11

Data Statement

  • AVIRIS-NG data are available upon request from JPL, and other data are available upon request from the authors.The passage provides the AVIRIS-NG website: https://aviris-ng.jpl.nasa.gov.Source: Data Statement · p. 11

Related papers

Loading 2003.02978v1…