PaperlayerView exact revision on arXiv

Source-linked AI summary

MOSFiT: Modular Open-Source Fitter for Transients

James Guillochon, Matt Nicholl, V. Ashley Villar, Brenna Mockler, Gautham Narayan, Kaisey S. Mandel, Edo Berger, Peter K. G. Williams

arXiv:1710.02145v2astro-ph.IMastro-ph.HE

TL;DR

Time-domain astronomy faces reproducibility and data-access challenges that can slow modeling of unique transients. MOSFiT addresses these challenges with an open, reproducible fitting package that uses Bayesian parameter inference, while its sampling approach retains important computational limitations.

  • Problem

    Incomplete observational metadata and limited transient identification can hinder reproducible modeling and reduce the utility of expanding time-domain surveys.

  • Method

    MOSFiT is an open Python package that models transients using Bayesian posterior inference and parallel-tempered ensemble Monte Carlo sampling.

  • Results

    MOSFiT is designed to make light-curve analysis more reproducible and accessible through standardized, publicly accessible transient models.

  • Takeaways & Limitations

    The package provides a common basis for applying and improving transient models across the time-domain astronomy community.

  • Takeaways & Limitations

    Ensemble Monte Carlo sampling can leave low-posterior regions of parameter space unexplored, limiting exact evaluation of total model evidence.

Abstract

from arXiv · show

Much of the progress made in time-domain astronomy is accomplished by relating observational multi-wavelength time series data to models derived from our understanding of physical laws. This goal is typically accomplished by dividing the task in two: collecting data (observing), and constructing models to represent that data (theorizing). Owing to the natural tendency for specialization, a disconnect can develop between the best available theories and the best available data, potentially delaying advances in our understanding new classes of transients. We introduce MOSFiT: the Modular Open-Source Fitter for Transients, a Python-based package that downloads transient datasets from open online catalogs (e.g., the Open Supernova Catalog), generates Monte Carlo ensembles of semi-analytical light curve fits to those datasets and their associated Bayesian parameter posteriors, and optionally delivers the fitting results back to those same catalogs to make them available to the rest of the community. MOSFiT is designed to help bridge the gap between observations and theory in time-domain astronomy; in addition to making the application of existing models and creation of new models as simple as possible, MOSFiT yields statistically robust predictions for transient characteristics, with a standard output format that includes all the setup information necessary to reproduce a given result. As large-scale surveys such as LSST discover entirely new classes of transients, tools such as MOSFiT will be critical for enabling rapid comparison of models against data in statistically consistent, reproducible, and scientifically beneficial ways.

1. INTRODUCTION

Time-domain astronomy benefits from physically self-consistent transient modeling, but fragmented, incomplete, and unevenly documented data and models impede reproducible analysis. MOSFiT addresses this gap as an open, reproducible package for fitting transient models to cataloged observations.

  • Motivation: Incomplete metadata for older transient observations can make reproducing data with models require contacting original researchers, slowing scientific exploration.Missing information may include the bandset, instrument, or magnitude system used for an observation.
  • Motivation: Future surveys will greatly expand transient populations, but limited spectroscopic follow-up may make many events difficult to identify and characterize.LSST is expected to produce approximately 20 TB of imaging data daily, while transient-modeling products remain comparatively manageable.
  • Existing infrastructure: Open astronomy catalogs aggregate transient data from publications, private communications, and public resources, enabling comparison and combination across researchers.These catalogs help observers compare new data with published work and identify similar transients.
  • Motivation: Publicly accessible transient models remain scarce, and reproducing existing models often depends on study-specific data products and incomplete supporting information.Prior studies commonly focused on small data subsets or particular scientific motivations.
  • Contribution: MOSFiT is a Python package released under the MIT license that produces publicly accessible and reproducible transient models.The paper presents MOSFiT as a platform for connecting transient observations with physical modeling.
  • Paper scope: The paper describes MOSFiT’s reproducibility principles, data input, model definition, shared products, performance assessment, shortcomings, and future directions.It is intended as a descriptive guide to the initial version 1.0 release.

2. END-TO-END REPRODUCIBILITY

End-to-end reproducibility in time-domain astronomy requires open data, pipelines, models, and result sharing because each transient and its observing conditions are effectively unique. MOSFiT addresses this challenge through modular, reproducible modeling, Bayesian parameter exploration, efficient computation, and community-accessible outputs.

  • Reproducibility rationale: Each transient is unique and can be observed only once, making open access to data especially important for reproducible time-domain research.Even nearly identical transients generally differ in observing conditions.
  • Reproducibility barriers: Distributed pipelines and inconsistent data exchange can leave results irreproducible unless all processing components are open or cooperative.Pipeline fragments may be exchanged through publications, private communications, or public repositories.
  • Reproducibility caveat: Stochastic sampling need not be bit-for-bit reproducible, but convergence should be checked because such methods may fail to converge in finite time or become trapped in local minima.Repeated runs and mixing diagnostics can strengthen convergence assessment.
  • MOSFiT principles: MOSFiT aims to make transient-data analysis reproducible and publicly available through an open-source Python platform.Its design is guided by the astronomy software ecosystem and packages such as astropy, astroquery, and emcee.
  • MOSFiT principles: MOSFiT supports rapid construction and modification of semi-analytical models so scientists can respond to newly discovered transients.The platform is intended both for developing new models and for applying vetted models to new events.
  • MOSFiT principles: MOSFiT seeks to simplify ingestion of historical and contemporary observations by reducing the need to scrape, annotate, and convert data manually.This addresses the labor involved in assembling data from multiple sources into consistent inputs.
  • MOSFiT principles: MOSFiT evaluates fits with scoring metrics related to total model evidence, supporting comparisons beyond simple goodness-of-fit tests.Its Bayesian approach maps plausible parameter combinations rather than selecting only one best fit.
  • MOSFiT principles: MOSFiT distributes transient modeling and enables user fits to be shared rapidly with the broader community.The platform targets public availability of fits potentially within hours after data becomes available.

3. DATA INPUT

MOSFiT standardizes heterogeneous transient observations through catalog access and conversion tools, reducing manual preparation while preserving metadata needed for reproducible fitting. It also supports data selection, filter-response association, and sharing of converted datasets.

  • Catalog-based input: MOSFiT relies on Open Astronomy Catalogs to provide sanitized, homogeneously formatted transient data, optionally supplemented by users.This addresses the difficulty of converting datasets from multiple sources into a common input format.
  • Catalog-based input: Public data can be accessed directly by transient name, after which MOSFiT prompts the user to select an available model.The example downloads all data for PS1-11ap from the catalogs.
  • Catalog-based input: Catalog access eliminates substantial labor in locating literature, collecting fittable observations, and combining them into a common format.Independent users also receive the same catalog data, although later reductions can change the best available dataset.
  • Private and arbitrary input: MOSFiT’s Converter class reads diverse ASCII formats, including delimited tables, fixed-width CDS files, and LaTeX tables, then feeds converted data into the Transient module.The conversion process asks for metadata not inferable from the input, such as data source, instrument, and zero point, and writes Open Astronomy Catalog format.
  • Private and arbitrary input: A converted dataset can be fitted by other MOSFiT users and optionally uploaded to the Open Astronomy Catalogs for public use.The resulting file preserves a common representation for subsequent fitting and sharing.
  • Response functions and selection: MOSFiT associates observations with filter response functions using filter, instrument, telescope, and photometric-system information, interfacing with the SVO service.This accounts for differences between filters sharing the same letter designation.
  • Response functions and selection: Users can restrict fits by time range, bands, instruments, photometric systems, or sources, and such subsets receive distinct data hashes.Only fits with identical model and data hashes are directly compared by scoring metrics.

4. DEFINING MODELS

MOSFiT defines transient models through JSON-described module dependencies and parameters, then constructs efficient execution schedules from those dependencies. Its modular architecture supports reusable components, model modification, and built-in transient models while retaining computational and representational limits.

  • Model structure and execution: Each MOSFiT model uses separate JSON files for model structure and problem parameters, defining module interactions, inputs, outputs, and priors.The structure file specifies how data become outputs such as light curves and likelihood scores.
  • Model structure and execution: MOSFiT constructs call trees and combines call stacks so shared modules are executed only once while producing requested outputs.The automatically determined order avoids repeated computation across dependent outputs.
  • Built-in modules: Built-in modules cover data import, physical engines, energetics, transformations, photospheres, SEDs, observables, objectives, outputs, constraints, arrays, and utilities.Examples include radioactive or fallback engines, ejecta-velocity energetics, photometric observables, and likelihood objectives.
  • Built-in models: Default models favor computational simplicity, with many transients represented by simple one-zone models and some models serving only as basic-property placeholders.Specialty software may provide superior representations for certain transient classes.
  • Modifying and creating models: MOSFiT models can be extended by changing priors or replacing modules, such as switching from a blackbody SED to a custom spectral-energy-distribution module.Related models can inherit shared code and setup through JSON modifications.
  • Computational constraints: MOSFiT’s shipped models have sub-second execution times, whereas models involving differential-equation integrations may take minutes per realization.Because sampling and optimization repeatedly execute model realizations, model complexity directly affects runtime.

5. INTERPRETING OUTPUTS

MOSFiT makes model outputs shareable and reproducible by publishing fit data, model descriptions, realizations, and hashes, while acknowledging practical limits from stochastic sampling and implementation differences.

  • Sharing fits: MOSFiT extends the Open Astronomy Catalogs to publish complete model descriptions, Monte Carlo parameter combinations, and realizations that users can reload and modify.Published fits can be regenerated with different cadences, photometric bands, or inferred-parameter variations.
  • Model uniqueness: Model outcomes can differ when datasets, model implementations, parameter bounds, or external package versions differ, limiting exact reproducibility across analyses.These differences can alter parameter inferences, model scores, or numerical outputs.
  • Model uniqueness: Three hashes identify the input data, model dictionary, and invoked Python code, helping users verify identical fits and avoid inadvertent cross-model comparisons.A changed photometric magnitude produces a unique data hash, while external package-version changes remain unaddressed.
  • Choosing a Sampler and a Minimizer: Monte Carlo methods may fail to converge or accurately represent posteriors within the allotted time, especially when walkers become trapped in local minima.MOSFiT uses multiple techniques to increase the chances of obtaining a converged solution.
  • MOSFiT’s approach: MOSFiT uses parallel-tempered emcee after burn-in, while a Gibbs-like stretch move varies selected dimensions during pre-burn to improve early exploration.The post-burn-in sampler preserves detailed balance; the pre-burn variant is used for agility before reverting to the ensemble algorithm.

6. ASSESSING MODEL PERFORMANCE

MOSFiT assesses model performance by comparing plausible parameter combinations with likelihood-based and Bayesian methods, while accounting for correlated errors, noise-dominated solutions, model evidence, and convergence.

  • Error models: Goodness-of-fit metrics compare physical parameter combinations with transient observations, but reduced χ2 assumes Gaussian uncorrelated noise that may poorly represent magnitude-space and systematic errors.
  • Bayesian fitting: Bayesian analysis maps all parameter combinations by posterior probability rather than selecting only a single best-fit solution.
  • Gaussian processes: Gaussian-process modeling represents correlated errors through time and wavelength lengthscales, while the default squared-exponential kernel approaches the simpler error model as both lengthscales approach zero.
  • Gaussian processes: When observations are sparse relative to model parameters, noise-dominated Gaussian-process solutions can fit variation across the transient and sometimes score better than physical models.
  • Model evidence: Model evidence requires integrating likelihood across the full parameter space, so walker likelihood scores cannot simply be summed when ensemble Monte Carlo leaves regions unexplored.
  • Convergence: The autocorrelation-time algorithm in emcee can fail on edge cases, leaving users without information about their distance from convergence.
  • Convergence: MOSFiT uses the maximum parameter PSRF to assess convergence, with PSRF 1.1 strongly suggesting that the Monte Carlo chain has converged.

7.1. Stress Testing

MOSFiT produced fits without error for all approximately 16,000 Open Supernova Catalog supernovae with at least five photometric measurements, despite the dataset’s heterogeneity.

  • ∼16,000 supernovae with 5+ photometric measurements were all fit without error, demonstrating robustness across a heterogeneous dataset.

7.2. Performance

MOSFiT’s inference runtime depends on model and dataset size, with converged posterior estimation typically requiring thousands of emcee iterations and parallel execution reducing wall time.

  • A few ten thousand emcee iterations and roughly ten times as many walkers as free parameters are typically required for converged posterior distributions.
  • Runs take from a few hours for events with dozens of detections to a few days for events with thousands of detections, with parallel execution reducing wall time.

7.3. Synthetic Photometry

MOSFiT generates synthetic photometric observations either from physical-parameter priors or from an existing fit, allowing users to customize instruments, bands, epochs, and extrapolation.

  • Users can generate synthetic photometry for any instrument and band combination from posterior parameter ensembles or reasonable physical-parameter priors.
  • Existing MOSFiT results can be loaded to generate additional light-curve data, including brightnesses in bands or epochs absent from the original fit.
  • The -S and -E options add intermediate epochs or extrapolate observations before or after the transient, respectively.

7.4. Future versions

Future versions of MOSFiT could improve spectral and photometric modeling, error models, survey simulation, and evidence estimation. The paper also identifies practical limitations in current approximations and model-comparison diagnostics.

  • Flux and magnitude model matching: Switching model outputs from magnitudes to fluxes could improve upper limits and enable a less approximate Gaussian-process error model with symmetrical errors.MOSFiT currently compares modeled magnitudes with observed magnitudes and errors.
  • Improved spectral modeling: MOSFiT currently approximates spectral energy distributions with blackbodies, an assumption that breaks down for cooler photospheres such as type Ia supernovae.Deep absorption lines can produce large systematic color errors in broadband magnitude estimates.
  • Generative modeling: MOSFiT can generate synthetic light curves from priors or posteriors, resample them by survey cadence and limiting magnitude, and simulate transient populations for surveys such as LSST.These simulations could assess model comparisons for unknown transients and MOSFiT’s utility for real-time classification.
  • Error models: Future error-model improvements include additional kernel families and faster-inversion choices for large datasets with O(10^4) points.MOSFiT currently provides white-noise and Gaussian-process error models, while users can create new error-model modules.
  • Model comparison: WAIC is useful for disfavoring clearly underperforming models, but similarly scoring models should not be rank-ordered confidently because its approximation error is not measured.Nested sampling, particularly dynamic sampling, could provide direct evidence estimates with measured errors and more posterior samples.

7.5. Concluding remarks

The paper presents MOSFiT as a reproducible and accessible framework for light-curve analysis in time-domain astronomy. It highlights innovations across data input, processing, and output while acknowledging opportunities for future improvement and external contribution.

  • Concluding remarks: MOSFiT is intended to make light-curve analysis in time-domain astronomy more reproducible and accessible.The authors welcome external contributions to improve its performance, breadth, accuracy, and accessibility.
  • Concluding remarks: The paper highlights MOSFiT’s innovations in data input, processing, and output.
Loading 1710.02145v2…