Source-linked AI summary
Recovering Weak Signals with Normalizing Flows
Sarod Yatawatta
TL;DR
The paper addresses recovery of weak signals distorted by calibration against dominant nuisance signals. It uses normalizing flows with calibration-derived mapping information to reconstruct the weak-signal distribution, and simulated examples show improved reconstruction compared with using residuals alone.
Problem
Calibration can distort or suppress weak signals hidden beneath dominant nuisance signals, and no existing method recovers the original weak signals from the residual.
Method
The method uses a normalizing flow whose externally provided calibration Jacobian models the transformation from observed data to residuals.
Results
Simulated linear and radio-interferometric examples show reconstructed weak signals more closely matching ground truth than averaged residuals alone.
Takeaways & Limitations
The framework improves recovery of calibration-affected weak signals and can use observed data and metadata directly for training with real observations.
Abstract
from arXiv · showhide
In many scientific disciplines, weak signals of interest are obscured by dominant nuisance signals that are several orders of magnitude stronger. Recovering these weak signals requires subtracting the dominant ones; however, this calibration process inherently distorts or partially suppresses the underlying signal of interest. To address this problem, we propose the use of normalizing flow models to reconstruct calibration-affected weak signals. By leveraging the statistical invariance of the target signals and assuming minimal initial suppression, our framework effectively recovers the lost signal components. We provide a comprehensive theoretical overview of this normalizing flow-based recovery method and demonstrate its efficacy using simulated data.
I. INTRODUCTION
Weak signals can remain statistically stable while being overwhelmed by nuisance signals and distorted during calibration. The paper proposes normalizing flows to recover undistorted weak signals using calibration-derived mapping information.
- Weak scientific signals are often overwhelmed by nuisance signals several orders of magnitude stronger.
- Calibration models and subtracts nuisance signals from observed data, producing a residual presumed to retain the desired weak signal.
- No existing method recovers the original weak signals hidden in calibration-affected residuals, motivating this work.
- The proposed method uses normalizing flows to recover undistorted weak signals in the residual after calibration.
- The method incorporates a closed-form calibration Jacobian into the flow to train the base distribution and estimate mainly first-order hidden-signal statistics.
- Unlike common flow models that learn the Jacobian, this approach receives it externally from calibration and therefore is not constrained to a bijective mapping.
II. DATA MODEL
The data model treats observations as dominant parametric nuisance signals plus additive noise, then analyzes how calibration transforms the data into residuals. Because calibration can distort statistics and produce a non-invertible Jacobian, the weak-signal distribution must be recovered from the residual and Jacobian.
- Data model: Observed data consist of a parametric nuisance signal and additive noise, with calibration estimating nuisance parameters before subtraction.
- Assumptions: The nuisance signal is assumed dominant, allowing noise distributions to be specified during calibration even though the weak signal is hidden within the noise.
- Calibration mapping: The residual is not generally distributed like the original noise because calibration transforms the observed data through a mapping with a Jacobian.
- Assumptions: Under the stated assumptions, multiple observations provide parameter diversity while the base distribution remains statistically stable.
- Calibration mapping: Calibration can make the Jacobian lose rank and become non-invertible, so residual statistics cannot directly provide the desired weak-signal statistics.
- Recovery problem: The recovery problem is to extract the statistics of the original data or noise from the residual and the a-priori Jacobian.
III. NORMALIZING FLOWS
The proposed normalizing-flow model transforms a parameterized base density into the calibration-affected residual density while using the externally available Jacobian. Training combines likelihood optimization with a constraint keeping predictions close to the noise.
- Flow formulation: Normalizing flows model probability densities undergoing transformations, here representing noise density transformed by calibration into residual density.
- Flow formulation: The flow uses Jacobian eigenvalues to evaluate the log-determinant, while additional constraints address the possibility of non-invertible calibration mappings.
- Training objective: Training imposes a lower bound on the expected correlation between predicted noise and residual rather than using a penalty or equality constraint.
- Training objective: The correlation constraint uses normalized inner products averaged over minibatches and is enforced through an augmented Lagrangian.
- Training procedure: Algorithm 1 alternates flow-parameter updates with periodic Lagrange-multiplier updates, optionally preceded by pre-training.
- Training procedure: The method is applied to two examples with different nuisance models using the same training procedure.
IV. SIMULATION RESULTS
The simulations evaluate the same recovery method on a simple linear model and a radio-interferometric model with substantially different nuisance signals.
- Simulation setup: The experiments include a simple linear example and a radio-interferometric example with different nuisance models.
- Simulation setup: The same normalizing-flow method and training algorithm are applied to both simulated examples.
A. Linear model
The linear simulation models observations as a nuisance signal plus noise, then tests whether a normalizing flow can recover a weak signal suppressed in the residual. Across 3000 test samples, averaged flow predictions agree more closely with the ground truth than averaged residuals.
- The linear nuisance model uses x = Aθ + n, with known design matrix A and observation-specific parameters θ.
- The residual is computed after elastic net estimation of θ, and the DNN receives y, A, and estimated parameters as input while predicting a noise sample.
- The simulation generates 90000 training samples with D = 15 and M = 10, using Gaussian design entries and uniformly distributed nuisance parameters.
- The noise is sampled from N(µ, I), with a fixed weak-signal vector µ, while the nuisance signal is scaled to signal-to-noise ratio 1.
- The averaged residual suppresses the weak signal, whereas averaged predictions from the trained normalizing flow more closely match the ground truth over 3000 test samples.
B. Radio interferometric model
The radio-interferometric model embeds a weak diffuse sky signal in noise alongside stronger clustered sources and receiver systematics. Calibration produces residuals that suppress the desired signal, while the trained DNN prediction more accurately reconstructs it in simulated test images.
- The interferometric observation combines clustered sky sources and receiver-systematic terms with a weak signal contained in complex Gaussian noise.
- The simulation uses 14 receivers, six nuisance-source clusters, randomly selected pointing and frequency conditions, and 60000 snapshot observations.
- The diffuse sky model is fixed at the north celestial pole, while nuisance signals are scaled to a nuisance signal-to-noise ratio of 10 and calibration assumes a known six-source sky model.
- The DNN uses calibrated residuals and metadata including the nuisance sky model, calibration solutions, observed data, and baseline coordinates.
- For a separate two-hour test observation, Monte Carlo dropout predictions are averaged across time and frequency samples before imaging.
- The DNN prediction more accurately reconstructs the hidden signal than the residual and closely matches the ground truth images.
V. CONCLUSIONS
The paper presents normalizing flows for recovering weak signals degraded by calibration and demonstrates improved reconstruction using simulated training and testing data. It also describes a potential route to combining diverse real radio observations without requiring simulated training data.
- The paper uses normalizing flows to recover weak signals affected by calibration and demonstrates improved reconstruction on simulated data.
- For real observations, the authors state that the DNN can be trained directly from observed data and metadata without simulated training data.
- In radio interferometry, the method could combine observations targeting diverse science goals and observations from different radio telescopes.
- All data used in the paper are simulated, and the simulation and evaluation software is provided online.
Appendix A: Proof of (7)
The appendix derives the Jacobian of the calibration mapping by differentiating the residual relation and the stationarity condition at the estimated nuisance parameters. The derivation supports the influence-function formulation used by the method.
- The proof begins by differentiating the residual equation with respect to an element x_m of the observed data.
- At the estimated parameters, the objective has a local minimum, so its first derivative vanishes before further differentiation yields the Jacobian relation.
- The resulting Jacobian is identified with the influence function used to describe how calibration maps observations to residuals.
- The accompanying radio-interferometric images compare observed data, noiseless ground truth, calibrated residuals, and DNN predictions across Stokes I, Q, U, and V.
Appendix B: Linear model
The linear-model appendix presents influence-function terms used in the normalizing-flow formulation, including a Dirac delta function and an intermediate transformation.
- The influence function is expressed using I + A.
- The appendix applies equation (7) to obtain the next expression in the linear-model derivation.
- The notation δ(·) denotes the Dirac delta function.
Appendix C: Radio interferometric model
The radio-interferometric appendix defines station-baseline data and its vectorized representation, while noting that spectral regularization is omitted for simplicity.
- The canonical selection matrix A_p extracts the p-th 2×2 block from a 2N-dimensional vector.Only the p-th block equals I ∈ R2×2; all remaining entries are zero.
- Solutions J_kν are obtained using data from all baselines, with N(N − 1)/2 baselines per frequency.
- The derivative is taken with respect to an observed-data component x_p′q′r associated with a station baseline and matrix-component index.
- The appendix rearranges the result into vectorized form using eight real values for each complex 2×2 matrix.These values represent the real and imaginary parts for any given p, q, and ν.
- Spectral regularization is omitted from the displayed formulae for simplicity.The text states that including it would modify the formulae as in prior work.
Appendix D: Deep learning model and training details
The deep-learning appendix describes a configurable dense DNN used in both examples and summarizes the training setup, including optimizer and learning-rate scheduling.
- The DNN uses dense linear layers with SiLU activations, except in the last layer.
- D, H, M_x, and T denote input dimension, hidden dimension, metadata dimension, and the number of sequential linear blocks.
- The architecture can vary metadata handling, depth, dropout, and residual connections across examples.Dropout may be inserted at two locations with rate 0.1, and a residual connection links the input to the post-concatenation output.
- AdamW training uses learning-rate warmup followed by cosine decay.Additional training details are provided in Table I.