ANNz2 - photometric redshift and probability distribution function estimation using machine learning

1.1. Photometric redshifts

• Photometric surveys will observe billions of galaxies, making spectroscopy infeasible at that scale and high-quality photo-z measurements essential.A cited LSST benchmark calls for systematic redshift bias precision of about 0.002 · (1 + z).Source: §1.1 · p. 1
• ANNz2 introduces photo-z PDFs and a way to contrast and combine machine-learning techniques, extending the earlier ANNz1 code.The paper also notes that PDFs have been shown to improve cosmological-measurement accuracy.Source: §1.1 · pp. 1–2

1.2. Methodology for photo-z estimation

• Photo-z estimation includes template fitting and training-based machine learning; ANNz2 derives two primary PDF types, one novel and one similar to PDFs from other machine-learning codes.Training-based methods learn the mapping from photometric observables to redshift using examples with precise redshifts.Source: §1.2 · pp. 2–3

1.3.1. Basics of machine learning

• Supervised machine learning learns a mapping from labeled examples, which are divided into training, validation, and independent testing samples.Training derives the mapping, validation checks convergence, and testing independently assesses performance.Source: §1.3.1 · p. 3

1.3.2. Available methods in TMVA

• ANNs map inputs to outputs through weighted neuron responses; ANNz2 trains multilayer perceptrons by adjusting weights with back propagation.ANN training must guard against over-training, which can appear as better training performance but worse validation performance.Source: §1.3.2 · p. 4
• Decision trees split examples by input-variable cuts into target-value or class leaves, while boosting combines trees trained with differently weighted data.Boosting is used because a small change in one tree node can affect the tree structure that follows.Source: §1.3.2 · p. 5

1.3.3. Method selection and parameter tuning

• Because machine-learning methods trade off training and evaluation speed, ANNz2 automates deriving multiple methods and parameter choices to help select an estimator.The text contrasts generally faster BDT training with generally shorter ANN evaluation time than large random forests.Source: §1.3.3 · pp. 5–6

• The example dataset contains galaxy and star observations from SDSS DR10, including measurements from BOSS.The package includes this dataset as a toy analysis.Source: §2 · p. 6
• Photo-z inference uses ugriz Pogson magnitudes calculated from fluxes for roughly 180,000 catalogue objects.The magnitude relation is m = 22.5 − 2.5 log10(f).Source: §2 · p. 6

• Galaxy-level bias is zphot − zspec, scatter is the standard deviation of that bias, and σ68 is the half-width enclosing its peak 68th percentile.Source: §3 · p. 6
• The outlier fraction f(ασ) counts objects whose bias exceeds α times σ or σ68; f(2, 3σ68) averages the 2σ68 and 3σ68 fractions.Metrics may be calculated in redshift bins or averaged across bins to summarize the sample.Source: §3 · p. 6
• Npois compares bin-wise differences between estimated and true redshift distributions relative to Poisson fluctuations, while KS measures their cumulative-distribution distance.Unlike Npois, KS does not depend on the redshift-distribution binning; the statistics compare estimators rather than establish absolute significance.Source: §3 · p. 6

4.1. Photo-z PDF derivation

• ANNz2 combines multiple MLM estimators and their individual uncertainties to form PDFs that can expose multi-peak redshift degeneracies and reduce dependence on any single model choice.Different estimators can respond differently to ambiguous photo-z/redshift relations, and the ensemble can reveal poorly performing configurations.Source: §4.1 · p. 8

4.2. Single regression and uncertainty estimation

• ρNN widths were 0.27 for ANNz1 and 1.04 for ANNz2, with both means consistent with zero to better than 3%; ANNz2’s uncertainty estimates were more reliable.The expected width for representative uncertainty estimates is 1.Source: §4.2 · p. 10, Figure 6 · p. 12

4.3. Randomized regression PDF

• Randomized regression trains and preselects varied MLMs, folds their predictions with individual uncertainties, and chooses the candidate PDF whose CDF is closest to uniform.The best-performing preselected MLM also supplies the single-value estimate zbest.Source: §4.3 · pp. 10, 12, 13

4.4. Representativeness and completeness of the training sample

• Training weights match the training-sample density to a reference sample, but absent training examples leave photo-z estimates and uncertainties unreliable, so ANNz2 flags those objects.The density matching uses a kd-tree; weights are also propagated to PDF optimization.Source: §4.4 · pp. 13–14
• The nominal nmin_NN is 100 and is user-adjustable; the QNN distribution’s steepness depends on this choice and the dataset.Source: §4.4 · p. 14
• The stacked PDF’s overall fit to the true redshift distribution is better than that of the single-value solutions.Source: Figure 9 · p. 15
• As QNN increases, photo-z bias and 68th-percentile scatter improve; a conservative example rejects galaxies with QNN < 0.8.The appropriate cut depends on the case, including the fraction of objects excluded and the relative performance improvement.Source: §4.4 · pp. 14–15, Figure 8 · p. 14

5.1. Toy analysis

• The tested PDF ensemble combined 50 ANNs and 50 BDTs with randomized input selections and transformations.The single-value zbest solution used one ANN; the ensemble varied model options and training inputs.Source: §5.1 · p. 15
• The full PDF has larger scatter by construction because it includes distribution tails, while its uncertainty estimate is closer to 1 than the average-PDF estimator.The increased scatter represents a more realistic uncertainty estimate, not a disadvantage.Source: §5.1 · pp. 15–16
• The full PDF better describes the underlying redshift distribution, with improved estimates around zspec ∼0.35 and above 0.7.The single-value solutions are less precise in those regions.Source: §5.1 · p. 16

5.2. Other applications of ANNz2

• The reported qualification covers photo-z metrics and compatibility with the redshift distribution, not a cosmological analysis.A cosmological analysis involving photometric redshifts is stated to be beyond the work's scope.Source: §5.2 · p. 16
• In a DES comparison, ANNz2 was compatible with other codes, although assigning galaxies to SkyNet-defined redshift bins may introduce selection bias.A separate DES study found ANNz2 minimized photo-z distribution variations under degraded input data and reduced outliers.Source: §5.2 · p. 16

6. Summary

• ANNz2 had been used in the first DES weak-lensing analysis and included in the experiment's first public data release.The paper illustrates its performance using spectroscopic data.Source: §6 · p. 16
• ANNz2 combines artificial neural networks and boosted decision/regression trees to produce single-value photo-z estimates or full PDFs.Generating PDFs is a new feature relative to the previous version.Source: §6 · p. 16
• Randomized regression selects the best-performing single solution and combines an ensemble of solutions into a PDF using KNN-derived uncertainty distributions and dynamic weights.The alternative binned-classification approach is implemented but not discussed in this paper.Source: §6 · p. 16
• ANNz2 addresses non-representative training samples with object weights and incomplete samples with per-object reliability flags.The flags indicate whether a derived photo-z solution is reliable given sample completeness.Source: §6 · pp. 16–17
• The full stacked PDF has larger scatter than the single-value estimators but better fit statistics for the underlying redshift distribution.The three solutions have comparable bias; the figure reports Npois and KS-test statistics as fit measures.Source: Figure 10 · p. 17

A. Quick-start guide

• The guide covers randomized regression for version 2.1.2 and points readers to online documentation for further, current instructions.Source: A. Quick-start guide · p. 19

A.1. Work-flow

• The four workflow stages are data processing, training, optimization, and evaluation.The guide represents the example script using Python pseudo-code and job-option parameters.Source: §A.1 · p. 19

A.2. Data processing

• Without separate training and validation input files, ANNz2 randomly splits the complete sample.Users may instead define both samples explicitly.Source: §A.2 · p. 19
• A user may supply a reference sample representing the dataset to be evaluated, enabling calculation of training weights.Training weights are calculated as described in Sect. 4.4.Source: §A.2 · p. 19
• The KNN search can use any functional combination of parameters defined in both input-variable lists; setting useWgtKNN to True activates training-weight calculation.The example uses magnitudes, magnitude errors, and colours in the search.Source: §A.2 · p. 19
• Training weights are propagated to model training, optimization, performance plots, and output, and can also serve as representativeness weights for other codes.Weights are output for objects in both training and validation samples.Source: §A.2 · p. 20

A.3. Training

• The training target is spectroscopic redshift, and inputs may be functional combinations of available variables, including mappings to effective magnitude limits that may prevent training on noisy data.The target's allowed range is set with minValZ and maxValZ.Source: §A.3 · p. 20
• ANNs, BDTs, and KNNs are illustrated, while all TMVA machine-learning methods are available through ANNz2; the authors report ANNs and BDTs perform best for photo-z inference.The example uses 20 randomized models and gives example ANN, BDT, and KNN configurations.Source: §A.3 · pp. 20–21
• An empty userMLMopts configuration prompts randomized ANN and BDT configurations, and variable transformations can be applied within the pipeline and automatically reused for evaluated objects.Transformations include normalization, decorrelation, and principal-component decomposition.Source: §A.3 · p. 21
• Users can vary cuts and weights across randomized models to compare training subsets, including survey selections and magnitude-error constraints.User-defined weights supplement representativeness weights when those have been calculated.Source: §A.3 · p. 21

A.4. Optimization

• Users can define PDF bins explicitly, specify an equal-bin count, or set a bin width for a dynamically determined number of bins.The examples show 90 equal-width bins or a width of 0.01.Source: §A.4 · p. 21
• The ensemble is used to derive the optimal solution zbest and a PDF, with individual models optionally excluded when their bias, 68th-percentile scatter, or combined outlier fraction exceed set limits.By default, derived models are combined to form the PDF.Source: §A.4 · p. 21
• By default, ANNz2 outputs zbest and the PDF, while MLMsToStore can request additional model estimators by their IDs.The permitted ID range is 0 ≤ id < nMLMs.Source: §A.4 · p. 21

A.5. Evaluation

• The evaluation stage calculates photo-z estimators for a user-defined dataset and can calculate QNN, a quality parameter for incomplete training, on request.QNN is added to output when addInTrainFlag is set to True.Source: §A.5 · p. 22
• Evaluation input variables need not match the earlier file structure, but must include every variable used to train the models.Source: §A.5 · p. 22
• For QNN calculation, users specify variables for the KNN search and may set the minimum number of objects in a volume.Source: §A.5 · p. 22