Source-linked AI summary
Interpreting Multi-band Galaxy Observations with Large Language Model-Based Agents
Zechang Sun, Yuan-Sen Ting, Yaobo Liang, Nan Duan, Song Huang, Zheng Cai
TL;DR
Interpreting astronomical observations requires domain expertise to sift through hypotheses, while AI-assisted astronomy has often concentrated on individual downstream tasks. mephisto combines multi-agent SED fitting with learning from prior fits, and tests on JADES data show improved search efficiency and multiple Little Red Dot explanations. Its current per-source computational cost limits scaling to billions of sources.
Problem
Astronomical AI often focuses on individual downstream tasks, while interpreting limited, noisy observations requires expertise to sift through hypotheses that cannot feasibly be searched exhaustively.
Source: §1 · pp. 1–2Method
mephisto uses multiple agents to iteratively fit galaxy SEDs with CIGALE and distills and validates insights from fitting histories for use in future fits.
Source: §2 · pp. 2–3Results
On 32 SOM-selected JADES sources, accumulated experience improved search efficiency, and tests on 31 Little Red Dots yielded two scenarios with derived properties consistent with.
Source: §3 · p. 4Takeaways & Limitations
The authors present mephisto as an end-to-end agentic astronomy demonstration that develops efficient multi-step reasoning through interactions with real data.
Source: §4 · p. 5At about 100K tokens, or approximately 1 US dollar per source, mephisto's reasoning is infeasible for billions of sources.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Astronomical research traditionally relies on extensive domain knowledge to interpret observations and narrow down hypotheses. We demonstrate that this process can be emulated using large language model-based agents to accelerate research workflows. We propose mephisto, a multi-agent collaboration framework that mimics human reasoning to interpret multi-band galaxy observations. mephisto interacts with the CIGALE codebase, which includes spectral energy distribution (SED) models to explain observations. In this open-world setting, mephisto learns from its self-play experience, performs tree search, and accumulates knowledge in a dynamically updated base. As a proof of concept, we apply mephisto to the latest data from the James Webb Space Telescope. mephisto attains near-human proficiency in reasoning about galaxies' physical scenarios, even when dealing with a recently discovered population of "Little Red Dot" galaxies. This represents the first demonstration of agentic research in astronomy, advancing towards end-to-end research via LLM agents and potentially expediting astronomical discoveries.
1 Introduction
Astronomical AI has often targeted individual downstream tasks, while interpreting limited, noisy observations still requires researchers to sift through hypotheses that cannot feasibly be searched exhaustively. The paper asks whether AI can learn from experience to reason about physical models and applies that question to galaxy SED fitting.
1 Introduction
- Focusing AI on individual downstream tasks limits its potential impact on astronomical research.Examples include classifiers and brokers, emulation of hydrodynamical simulations, and generative models for statistical inference.Source: §1 · p. 1
- Reasoning through hypotheses for limited, noisy observations requires experience because exhaustive search is computationally infeasible.The passage describes this filtering and reflection as a distinctive part of human research.Source: §1 · p. 2
- The paper asks whether AI can learn from experience to reason about physical models against astronomical observations without exhaustive search.It introduces mephisto and focuses on fitting broadband photometric SEDs, described as extremely low-resolution galaxy spectra.Source: §1 · p. 2
- Interpreting galaxy SEDs is time-intensive and requires domain expertise, yet is fundamental to galaxy physics and cosmology.The paper aims to emulate and accelerate this process using LLM-based agents, with a proof-of-concept application to JWST's JADES survey.Source: §1 · p. 2
2 Method
mephisto iteratively proposes and evaluates SED model variants through CIGALE, combining tree search with temporal memory and a dynamically updated knowledge base. Its inputs and evaluation are designed to support human-like judgment of fit quality, while validated lessons from prior fits inform future searches.
2 Method
- mephisto iteratively builds increasingly complex SED models through CIGALE, using temporal memory and a dynamically updated knowledge base to improve proposal diversity and efficiency.The models can combine star-formation histories, stellar populations, dust, nebular emission, and active galactic nuclear emission.Source: §2 · p. 2
- Its input state combines wavelength-flux observations, SED model specifications, and fitting results, including χ2 and well-fitted-band information.The evaluation uses human-like fuzzy judgment of global fit quality and signal-to-noise rather than relying only on a single statistic.Source: §2 · p. 2
- At each reasoning step, mephisto generates four hypotheses and uses tree search to select promising model variants for further fitting.It evaluates children using cumulative residuals, records refinement and fit-quality changes in temporal memory, and differs from optimizing parameters within one fixed model.Source: §2 · pp. 2–3
- A distillation agent extracts lessons from fitting histories, a validation agent tests them, and verified knowledge is added for future fits.The knowledge base can also incorporate astronomers' intuition, and its updated context is intended to make later tree-search actions more direct.Source: §2 · p. 3
3 Result
Tests on JWST JADES data show that accumulated experience improves mephisto's search efficiency, while its Little Red Dot fits produce two physical scenarios with properties consistent with prior work. The study also evaluates multiple LLM backbones, with GPT-4o used primarily.
3 Result
- The evaluation primarily used GPT-4o and also assessed LLaMA-3.1-405B and GLM-4-0520.The codebase is designed to integrate different LLMs.Source: §3 · pp. 3–4
- Accumulated experience improved search efficiency on 32 SOM-selected JWST sources, allowing similar fit quality at lower inference depth on subsequent similar spectra.The paper reports improvement with both inference depth within a search and run depth across previous fits.Source: Figure 2 · p. 4, §3 · p. 4
- For 31 Little Red Dots, mephisto proposed dusty star-forming galaxies without AGN and dust-free galaxies with AGN, with derived properties consistent with.The authors note that these sources were discovered after GPT-4o's training cutoff; the two scenarios were also proposed for a representative source in Figure 2.Source: §3 · p. 4, Figure 2 · p. 4
4 Broader Impact
The authors present mephisto as an early end-to-end example of agentic research in astronomy, using real data to develop knowledge and efficient multi-step reasoning. Scaling the approach to billions of sources remains constrained by per-source computational costs.
4 Broader Impact
- The study presents what the authors describe as the first end-to-end agentic research in astronomy, with mephisto developing solver knowledge through interactions with real data.The SED task offers many plausible models from limited wavelength information, making exhaustive search impractical.Source: §4 · p. 5
- The authors see mephisto as a step toward analyzing survey sources more broadly and potentially uncovering unknown phenomena or competing explanations.They note that most observed sources are rarely subjected to detailed human reasoning and may instead be triaged using simple χ2 criteria.Source: §4 · p. 5
- At about 100K tokens, or approximately 1 US dollar per source, mephisto's reasoning is infeasible for billions of sources.The authors identify computational costs as the current limit to scaling up to the entire sky.Source: §4 · p. 5
A mephisto’s Input State
mephisto represents each fitting problem as a structured input combining numerical observations, a CIGALE SED model, and fit outcomes. The agent uses these data and band-level fit assessments to guide model changes.
A mephisto’s Input State
- The input state combines redshift and numerical photometry with the CIGALE model configuration and fitting results.Photometry includes effective wavelengths, bandwidths, fluxes, and signal-to-noise ratios; results include parameter estimates, residuals, computational cost, and fit quality.Source: A mephisto’s Input State · p. 5
- Photometric observations are supplied as numerical wavelength-flux tuples because current AI systems have limitations interpreting scientific visuals quantitatively.This differs from researchers’ visual analysis of observations.Source: A mephisto’s Input State · p. 5
- The agent receives per-band qualitative fit labels—good, overestimate, or underestimate—to guide physical model changes and reduce numerical hallucinations.For example, poor optical and UV fits can prompt changes to dust attenuation models.Source: A mephisto’s Input State · p. 5
B mephisto’s Reasoning Process
mephisto iteratively proposes and evaluates CIGALE SED modifications using tree search, temporary memory, and an evolving knowledge base. Its prompt structures the proposals and constrains model choices and parameter grids.
B mephisto’s Reasoning Process
- Temporary memory captures information from the current search, while the knowledge base supplies accumulated expertise from earlier interactions.Search results update the knowledge base so later analyses can reuse learned insights alongside current-session suggestions.Source: B mephisto’s Reasoning Process · p. 6
- The prompt directs mephisto to choose CIGALE modules and parameter grids that target the bands with the poorest fit.Each proposed modification includes a reasoning step, module, choice, and parameter grid; the prompt also specifies mandatory modules and configuration rules.Source: B mephisto’s Reasoning Process · pp. 6–7
- A hybrid search alternates depth-first and breadth-first strategies, balancing exploration of deep branches with coverage of shallow configurations.DFS prioritizes recently added states; BFS explores states in insertion order.Source: B mephisto’s Reasoning Process · p. 8
- At each search step, the reasoning agent generates model states, an evaluator assesses them, and the search updates its tree and memory before revising the knowledge base.The search is bounded by maximum state and depth limits.Source: B mephisto’s Reasoning Process · p. 8
C mephisto’s Learning Process
mephisto learns from logged interactions by extracting candidate CIGALE insights, checking their applicability and impact, and integrating beneficial knowledge. This process is intended to refine future SED fitting across observations.
C mephisto’s Learning Process
- A learning agent processes interaction histories to extract insights intended to improve future CIGALE model designs.The reasoning process logs suggestions and their fit assessments for this learning step.Source: C mephisto’s Learning Process · p. 9
- A validation agent converts each candidate insight into a logical expression and tests its applicability to observation states using eval_state.The function checks whether a multi-band observation meets a specified feature and fit-quality pattern.Source: C mephisto’s Learning Process · pp. 9–10
- Candidate knowledge is assessed for impact across multiple sources, and knowledge with zero or negative impact is filtered out.The stated process retains knowledge judged beneficial before integration.Source: C mephisto’s Learning Process · p. 9
- An optimization agent integrates retained knowledge by reconciling it with existing entries, adding, revising, or merging information as needed.The integration prompt calls for concise, informative knowledge.Source: C mephisto’s Learning Process · pp. 9, 11
- The paper describes this iterative extraction and integration cycle as a way to refine the knowledge base and improve SED fits over time.The knowledge is intended to help navigate the hypothesis space of SED models.Source: C mephisto’s Learning Process · p. 9
D Using Different LLM Backbones for mephisto
Across 32 SOM-selected JWST sources, GPT-4o consistently benefits from the extracted knowledge base, whereas LLaMA-3.1-405B and GLM-4-0520 use it less effectively on average. The preliminary comparison also highlights the cost and task-specific nature of evaluating LLM backbones.
D Using Different LLM Backbones for mephisto
- mephisto costs approximately one dollar per source for detailed reasoning and checking, and the authors propose it as a potential benchmark for astronomical research reasoning.Their findings also differ from prior astronomy question-answering comparisons: LLaMA-3.1-405B loses efficacy here, while GLM-4-0520 performs comparably to it.Source: D Using Different LLM Backbones for… · pp. 11–12
- The comparison shares GPT-4o-distilled knowledge with all agents to reduce differences in internal domain knowledge, and the authors note that this natural-language knowledge need not be redistilled for each model.The comparison is preliminary rather than a comprehensive LLM benchmark.Source: D Using Different LLM Backbones for… · p. 11
- Only GPT-4o consistently improves inference with depth by using the extracted knowledge base; LLaMA-3.1-405B and GLM-4-0520 perform worse on average, though they sometimes find good proposals.The study averages results over 32 SOM-selected JWST sources; Figure 3 compares median χ2 across proposals and the best χ2 by inference depth.Source: D Using Different LLM Backbones for… · pp. 11–12, Figure 3 · p. 12
E SED Fitting Result on Little Red Dots
For the five representative JADES Little Red Dots, mephisto identified two distinct, physically plausible SED interpretations: a dusty star-forming galaxy without an AGN and a relatively dust-free galaxy hosting an AGN. These alternatives highlight the degeneracies in interpreting high-redshift galaxies with limited photometric data.
E SED Fitting Result on Little Red Dots
- For each LRD, mephisto identified two physically plausible solutions: a dusty star-forming galaxy without an AGN and a relatively dust-free galaxy hosting an AGN.The appendix presents results for five representative LRDs.Source: E SED Fitting Result on Little Red Dots · pp. 12–13
- Mephisto reached these solutions through reflection and model changes, rather than simply through hyperparameter search.The paper presents this as evidence of its capacity to propose and evaluate multiple hypotheses for challenging observations.Source: E SED Fitting Result on Little Red Dots · p. 13
- The contrasting solutions highlight degeneracies in SED fitting for high-redshift objects with limited photometric data, while mimicking the consideration human experts apply to such cases.The proposed physical scenarios range from nearly dust-free to heavily obscured and from purely star-forming to AGN-dominated.Source: E SED Fitting Result on Little Red Dots · p. 13
- The results illustrate mephisto’s potential for exploratory data analysis and hypothesis generation traditionally performed by human experts.The figures present derived parameters including dust attenuation, stellar mass, and AGN fraction.Source: E SED Fitting Result on Little Red Dots · p. 13
- For JADES LRD 79803, Figure 4 contrasts a dust-free solution with fracAGN = 0.99 and a star-forming solution with Av = 0.92; both fit the observations well.The alternatives exemplify the ongoing debate about Little Red Dots’ nature.Source: Figure 4 · p. 14