A Multi-Agent LLM System for Automated Tuning of Specialized Neural Network Models

1. Introduction

• Manual selection of specialized-model components is labor-intensive and poorly reproducible, and no universal selection rule exists across tasks.Relevant choices include input/output representation, decoding internal activity, and balancing loss-function terms.Source: §1 · p. 2
• The paper addresses an understudied use of LLM agents: automatically tuning individual functional blocks while leaving the network architecture and training procedure fixed.Unlike systems that build a network, its interaction pattern modifies predefined blocks and does not depend on the application task.Source: §1 · p. 3
• Accuracy increases by 13% on MNIST and 6% on scikit-learn digits; GNLSE relative error falls about 2500-fold, selected-run conservation error about 325-fold, and KS mass drift 14-fold.For the GNLSE soliton with a Gaussian perturbation, conservation error falls more than 100-fold on average over repeated initializations.Source: §1 · p. 3
• The contributions include applying the same framework to five tasks with different information representations and showing that detailed prompts improve less capable LLMs’ performance.The detailed prompt brings performance closer to that of more advanced models.Source: §1 · p. 3

2. Related Work

• Agent meta-optimization relies on role separation, logged memory, and sandboxed execution to separate hypothesis formulation from implementation, avoid repeating rejected variants, and run generated code safely.These are identified as essential components for agents optimizing neural-network models.Source: §2 · p. 3
• Existing agent systems cover programming, automated ML, scientific research, and robot control, but the cited examples do not incorporate domain-specific knowledge of PINNs or SNNs.The passage gives MLE-Bench medal-rate examples and names scientific-cycle and robot-control systems.Source: §2 · p. 3
• PINN tuning lacks a unified automatic procedure, while the closest cited agent approach constructs networks from task descriptions rather than selecting functional blocks for physical problems.For SNNs, cited automatic block-selection approaches rely on evolutionary algorithms and reinforcement learning.Source: §2 · p. 3
• Unlike neural architecture search, the proposed LLM approach locally modifies a selected module without searching a fixed variant space and logs hypotheses in natural language.That history lets later iterations use substantive analysis of prior hypotheses rather than only their numerical results.Source: §2 · p. 4
• The authors report that, to their knowledge, no open-literature work uses LLM agents as a general tool to tune individual network blocks without redesigning the network.MLE-STAR is described as a nearby system, but it processes the entire ML script and uses more than ten agents.Source: §2 · p. 4

3. Method: Multi-agent LLM framework

• The Human Researcher specifies the task, editable block, target metric, software contract, and baseline; the rest of the task pipeline remains fixed.Applying the framework to a new task requires new task-specific settings, but the agent roles and execution loop remain unchanged.Source: §3 · pp. 4–5
• At each iteration, the Analyst proposes a block-level hypothesis, the Coder implements it under a software contract, and the sandbox tests and evaluates the candidate before the Recorder logs successful results.The structured experiment log serves as external memory for subsequent iterations.Source: §3 · p. 4

3.1. Run protocol

• The log retains errored records, but filters them from the Analyst prompt; only successfully evaluated OK records count toward the run budget.The Analyst therefore receives the baseline and successful evaluations only.Source: §3 · p. 4

3. Method: Multi-agent LLM framework

• The detailed-prompt figure depicts the Analyst using an experiment log to produce a specification, followed by contract-constrained coding, sandbox evaluation, and Recorder logging.The architecture and loop remain unchanged across tasks; task prompts, contract, editable block, and metric vary.Source: Figure 1 · p. 5
• Detailed prompts add a method-class taxonomy, a structured reasoning procedure, and an updated coverage map that steers hypotheses toward underexplored classes without restricting search to finite implementations.Standard prompts instead supply the role, problem, metric, constraints, specification format, and experiment log.Source: §3 · p. 6

3.1. Run protocol

• A run consists of a task-specific number of successful iterations, each testing one hypothesis through specification, implementation, functional testing, evaluation, and logging.Invalid code can be retried up to 10 times; unresolved errors do not count toward the run budget.Source: §3.1 · p. 6

4. Experimental setup

• Across five tasks, the framework tunes an SNN decoder or PINN input/output transformations while leaving the other task components fixed.The tasks comprise two spiking-classification problems, two GNLSE boundary-value problems, and one Kuramoto–Sivashinsky boundary-value problem.Source: §4 · p. 6, §4.2 · p. 7

4.1. SNN task: Correlation Classwise Network decoder

• For MNIST and Digits, only the decoder of an STDP-trained CCN is modifiable; input encoding, network weights, and recorded output activity remain fixed.The decoder may fit parameters from training activity and labels, but cannot modify the trained SNN or use error backpropagation; test sets are unavailable during candidate selection.Source: §4.1 · pp. 6–7, Figure 2 · p. 7

4.2. PINN tasks: GNLSE and Kuramoto–Sivashinsky

• For the PINN tasks, agents modify input and output transformations, while the MLP architecture, loss functional, and training procedure remain fixed.The GNLSE problems predict two real components; the Kuramoto–Sivashinsky problem predicts one scalar field. The perturbed GNLSE and KS problems lack complete analytical solutions, so evaluation relies on boundary-value residuals and physical diagnostics.Source: §4.2 · pp. 7, 9, 10

4.3. Task-specific prompt configurations

• PINN prompts specify the equation, modifiable transformations, software contract, and physical target metric; detailed SNN prompts add a 12-class decoder taxonomy, structured analysis, and a coverage map.The PINN prompts use only the standard mode, whereas SNN experiments compare simple and detailed prompts.Source: §4.3 · p. 10

4.4. Search baselines and budget: SNN decoder on MNIST

• On MNIST, the framework is compared with random decoder sampling and two TPOT search modes using the same catalog and 250 evaluated candidates per run.Each configuration was run ten times; framework and Random Choice use training-derived validation scores, while TPOT selects by cross-validation and evaluates its retrained pipeline on the test set.Source: §4.4 · p. 11

4.5. Evaluation protocol

• Evaluation uses test accuracy for classification and task-specific error or physical diagnostics for PINNs, with 50 successful-iteration budgets for PINNs and 250 for SNNs.PINN comparisons include identity-map baselines; the KS comparison is limited to Claude Opus 4.6 and its baseline. The single-soliton metric averages three internal seeds, while other PINN tasks use one seed per hypothesis.Source: §4.5 · p. 12

4.6. Statistical protocol

• Ten matched MNIST run pairs showed significant differences after Holm correction for all six prompt tests; perturbed-GNLSE repeat runs assessed invariant preservation but not joint-criterion success rates.Perplexity Sonar’s detailed prompt exceeded 90% in every run, while its simple prompt stayed at baseline; the GNLSE repeated-run series were unpaired because seeds were unrecorded.Source: §4.6 · pp. 12–13

5.1. Summary across five tasks

• The best agent configuration outperformed the baseline on all five task metrics; repeated Gaussian-perturbation GNLSE runs showed 116× and 103× reductions in Rel_i.The reductions were for configurations selected by Resbvp and Rel_i, respectively.Source: §5.1 · p. 13

5.2. SNN classification results

• MNIST detailed prompts lifted Sonar and Qwen above baseline, while Claude's mean maximum rose from 91.13 ± 0.44 % to 92.15 ± 0.62 %.On Digits, prompt effects varied: Sonar's mean dipped slightly, while Claude and Qwen's means increased.Source: §5.2 · p. 14

5.3. Cost of the agent search

• $0.07 for Random Choice versus $5.85 for the Qwen detailed-prompt agent bought comparable accuracy, while stronger proprietary analysts exceeded Random Choice's best accuracy.Random Choice maxima were 89.48–89.93 %; the Qwen agent's were 82.28–89.16 %, and neither reached 90 %.Source: Table 4 · p. 16, §5.3 · pp. 16–17

5.4. PINN results: single soliton

• Rel_h fell 2491-fold, from 5.51 ⋅10^-2 to 2.21 ⋅10^-5, in Claude's best run on the analytically solvable single-soliton problem.The best configurations from all three analysts exploited the known soliton structure in both representation and reconstruction.Source: Table 5 · p. 17, §5.4 · p. 17

5.5. PINN results: soliton with gaussian perturbation

• Rel_i fell from 1.28 ⋅10^-1 to 6.61 ⋅10^-4 (194×) for Claude's conservation-error winner; a different Claude configuration reached Resbvp = 2.08 ⋅10^-5.Across ten additional trainings, the configurations selected by Resbvp and Rel_i reduced mean invariant error by 116× and 103×; repeated runs did not report Resbvp.Source: §5.5 · pp. 17–18, Table 6 · p. 18

5.6. PINN results: Kuramoto–Sivashinsky

• Δ(u)_m fell approximately 14-fold, from 9.31 ⋅10^-2 to 6.56 ⋅10^-3, for BoundedMultAdd_SechZoom_DualPhase; a separate hypothesis reduced Resbvp 9.3-fold.The mass-drift winner bounds its multiplicative correction with tanh; Figure 5 shows it retaining a localized pulse while the baseline loses roughly half its amplitude by t = 10.Source: Table 7 · p. 18, §5.6 · p. 18, Figure 5 · p. 19

6.1. Simple and detailed prompts

• Claude Opus 4.6’s detailed prompt raised mean maximum MNIST accuracy from 91.13 ± 0.44 % to 92.15 ± 0.62 % across ten matched runs.The detailed prompt yielded higher accuracy in eight of ten pairs; the mean paired difference was 1.02 ± 0.90 percentage points.Source: §6.1 · p. 19
• Perplexity Sonar’s detailed prompt reached a mean maximum MNIST accuracy of 91.76 ± 0.56 %, exceeding 90 % in all ten runs; Qwen3-30B-A3B-Instruct-2507 averaged 85.88 ± 2.42 % and never exceeded 90 %.With the simple prompt, neither model beat the 79.88 % baseline in any of ten runs.Source: §6.1 · p. 19
• Qwen3’s detailed-prompt mean maximum accuracy was 85.88 ± 2.42 %, with a best result of 89.16 %; its run-to-run variability indicates an unstable search.The simple prompt stayed at the 79.88 % baseline in all ten runs.Source: §6.1 · p. 20

6.2. Coverage-map ablation

• Without the coverage map, mean maximum accuracy fell by 9.42 percentage points for Perplexity Sonar and 5.70 percentage points for Qwen3, while Claude Opus 4.6 remained above 90 % in all runs.For Claude Opus 4.6, removing the map yielded 91.02 ± 0.25 %, only 1.14 percentage points below the detailed prompt.Source: §6.2 · p. 20

7. Discussion

• Detailed prompts let Perplexity Sonar exceed 90 % MNIST accuracy in all ten runs, while Qwen3’s best result rose from 79.88 % to 89.16 % but stayed below 90 %.Claude Opus 4.6 reached a comparable best-result cost to Sonar despite its higher per-run cost; Qwen3 also remained below the target threshold over the full run.Source: §7 · p. 20
• The agent found physics-specific techniques across the tasks, and in the GNLSE case with a Gaussian perturbation and no known analytical solution, the best configuration improved conservation of integrals of motion by more than an order of magnitude.The KS improvement concerned mass drift; the reported framework handled all five experiments without internal-structure changes.Source: §7 · p. 20

8. Limitations

• The agent can modify only a fixed catalogue of editable blocks; structural changes such as adding auxiliary networks or replacing the training paradigm require an extended action space.The catalogue includes encoders, neuron models, and connectivity schemes for SNNs, and loss weights, activation functions, and sampling strategies for PINNs.Source: §8 · p. 20
• The flat chronological experiment log lacks a hypothesis graph, so the Analyst must rederive experiment relations and sustained multi-step reasoning is limited.The Recorder stores runs as a chronological JSON log.Source: §8 · p. 20
• Run caps of 250 iterations for SNNs and 50 for PINNs mean reported accuracy and relative-error values are lower bounds on what larger budgets could reach.The bounds were chosen to keep wall-clock cost reproducible within the reported budgets.Source: §8 · p. 21
• Claude Opus 4.6’s GNLSE and Kuramoto–Sivashinsky configurations use one seed each; a full multi-seed sweep across every LLM/task pair remains future work.Source: §8 · p. 21
• Two evaluated models rely on commercial APIs whose availability, pricing, and behavior may change; the open-weights Qwen3 run is only a partial mitigation.Its cost/quality trade-off is not directly comparable to the proprietary systems.Source: §8 · p. 21
• The paper lacks a same-task, same-budget head-to-head comparison with LLM-driven neural-architecture-search systems.Reported baselines are random choice and state-of-the-art results from cited references.Source: §8 · p. 21

9. Conclusion

• For tasks judged by multiple physical invariants or structural requirements, the paper argues that a different class of framework is needed; its analyst, programmer, and recorder framework uses a sandbox and experiment log as external memory.The framework is reported to generalize across tasks without changes to its internal structure.Source: §9 · p. 21
• The unchanged framework improved results across five tasks: MNIST accuracy rose from 79.88 % to 92.83 %, digits accuracy from 90.83 % to 96.67 %, and selected PINN errors or diagnostics fell by task-specific factors.Reported PINN changes include an approximately 2491-fold relative amplitude error reduction for the single-soliton problem, approximately 325-fold conservation-error reduction for a selected Gaussian-perturbation run, and 14-fold lower KS mass drift.Source: §9 · p. 21
• Future work includes more complex PINN regimes without analytical solutions, joint tuning of SNN decoder and plasticity blocks, and models that account for neuromorphic-system constraints.Source: §9 · p. 21

• The work received ongoing institutional funding, with no additional grants obtained to carry out or direct the research.Source: Funding · p. 22

• Perplexity AI assisted with language editing and clarity; the authors reviewed and edited the manuscript and take full responsibility for its content.Source: Declaration of generative AI and… · p. 22
• The LLMs used in experiments also served as research tools, distinct from their manuscript-preparation use.Source: Declaration of generative AI and… · p. 22

CRediT authorship contribution statement

• The contribution statement assigns conceptualization and methodology across several authors, with roles also including software, investigation, data curation, analysis, validation, supervision, visualization, project administration, and writing.Source: CRediT authorship contribution statement · p. 22

A. Baseline PINN model

• The GNLSE setups include a single soliton with known analytical reference and a Gaussian-perturbed soliton whose analytical solution is unknown.The perturbed case uses a larger domain and a deeper baseline network with adaptive collocation sampling; a prior PINN field is retained as an auxiliary amplitude reference.Source: A. Baseline PINN model · pp. 22–23
• The GNLSE baseline uses a fully connected MLP with two 32-neuron hidden layers and sine activation, with identity-mapped input and output blocks available for tuning.Its loss has equally weighted PDE, initial-condition, and boundary-condition terms; derivatives are computed by TensorFlow automatic differentiation.Source: A. Baseline PINN model · p. 23

A.1. Kuramoto–Sivashinsky baseline hyperparameters

• The Kuramoto–Sivashinsky baseline scales the residual by 10 before squaring, lowers the learning rate to 5 ⋅10−5, and clips gradient norm at 1.Without these measures, many initializations diverge to non-finite loss within the first few hundred iterations; the boundary term penalizes both u and u_xx.Source: §A.1 · pp. 23–24

A.2. Kuramoto–Sivashinsky field shift and PINN residual

• The KS field shift removes the constant background; the resulting negative u_xx coefficient and added 2u_x term follow from that shift, not independent source-equation changes.The implementation uses the residual obtained after combining the two u_xx terms.Source: §A.2 · pp. 24–25

B. Baseline SNN model

• The CCN baseline keeps its architecture, encoding, supervision, and plasticity rule fixed; agent modifications target only the output-spike-activity decoder.GRF input expansion and bagging were not used after preliminary MNIST experiments showed no appreciable gain.Source: B. Baseline SNN model · p. 25
• The SNN tasks use MNIST and Digits with ten classes, ten output LIF neurons, and fully connected trainable input-to-output connections.Pixel inputs become correlated spike sequences; supervised currents and a local multiplicative STDP rule adjust weights.Source: B. Baseline SNN model · p. 25

• MNIST decoder search uses a stratified 80:20 split of the 60,000-sample training set, reserving the standard 10,000-sample test set from agent evaluation and selection.The highest-validation-accuracy decoder is retrained on the full original training set and evaluated once on the standard test set.Source: §B.1 · p. 25
• Digits uses a fixed stratified five-fold split of 1,437 development samples; mean fold accuracy is logged for decoder search, while 360 samples remain held out.The selected decoder is retrained on all development samples and tested once; test accuracy is not recorded in the search log.Source: §B.1 · p. 26

C. Analysis of the best PINN hypotheses

• Rel_h=2.21⋅10^-5 for Claude Opus 4.6, a 2491-fold reduction from its identity baseline of 5.51⋅10^-2.The configuration uses seven analytical features and time-gated exponential amplitude and additive phase corrections.Source: C. Analysis of the best PINN hypotheses · p. 26
• Sonar's best configuration gives Rel_h=1.97⋅10^-4, a 238-fold reduction from its identity baseline; Qwen's gives 8.86⋅10^-3, a 5.48-fold reduction.Both configurations use analytical-solution information in their input or output transformations.Source: C. Analysis of the best PINN hypotheses · p. 26
• Because all three best configurations exploit the analytical solution, these results support discovering useful physics-informed representations, not performance on problems without a known solution.Source: C. Analysis of the best PINN hypotheses · pp. 26–27

D. Repeated PINN runs and statistical analysis

• Both agent-designed configurations reduce invariant error by more than two orders of magnitude on average across ten successful retrainings.The original invariant-selected result was lower than any of its ten retrainings, whereas the residual-selected original result was close to its repeated-run mean.Source: D. Repeated PINN runs and statistical… · p. 27, Table 11 · p. 27

E. Additional classification results

• The best Claude detailed-prompt decoder achieved 92.83% using class-averaged centroids and Mahalanobis distance with class-specific covariance; without a coverage map, Perplexity’s best accuracy fell from 92.70% to 83.18%.The Perplexity coverage-map run focused on hybrid ensembles and discriminant analysis, leaving correlation, margin-based, and argmax classes uncovered.Source: E. Additional classification results · p. 27, Table 12 · p. 28
• Coverage maps raised mean maximum accuracy by 5.70 points for Qwen, 9.42 for Perplexity, and 1.14 for Claude; all three paired comparisons remained significant after Holm correction.All ten paired differences were positive for each model.Source: E. Additional classification results · pp. 28, 30, Table 14 · p. 29, Table 16 · p. 30
• Detailed prompts increased mean maximum accuracy by 1.02 percentage points for Claude, 11.89 for Perplexity, and 6.00 for Qwen; all six prompt and map tests remained significant after Holm correction.The per-model prompt comparisons used ten matched pairs; the adjusted p-values were below 0.05.Source: Table 13 · p. 28, Table 15 · p. 29, E. Additional classification results · pp. 30–31
• On digits, Claude Opus 4.6 with a detailed prompt reached 96.67% accuracy versus the 90.83% argmax baseline, using a KDE classifier in a PCA subspace.The decoder used a Gaussian kernel and Bayes rule, with hyperparameters selected by 5-fold cross-validation.Source: E. Additional classification results · p. 31, Table 19 · p. 31

E.1. Decoder search dynamics

• Search dynamics depended on analyst and prompt: Claude reached a plateau by iterations 5–10, detailed-prompt Perplexity gained at 50, 120, and 170, and simple-prompt Perplexity and Qwen never beat baseline.For digits, Claude’s best-run gain occurred almost entirely in the first 10 iterations.Source: §E.1 · pp. 31–32, Figure 6 · p. 32, Figure 7 · p. 33

E.2. Decoder catalog and TPOT configuration

• Classic and sequential TPOT modes and Random Choice each evaluated 250 candidates per run; Random Choice averaged 347 model calls to record those candidates.The TPOT modes differed in search strategy, not in their admissible solution space.Source: §E.2 · p. 32

F.2. Analyst prompt (detailed)

• The analyst is instructed to propose a new decoder, prioritize underexplored method classes, analyze prior failures, and use firing-rate vectors because spike-sequence timing is unstable.The SNN must remain unchanged, and fitting may use only training data.Source: §F.2 · pp. 36–38

F.3. Coder prompt

• The coder prompt specifies a NumPy-only Decoding class that learns from training firing rates and labels and returns integer predictions for test samples.Any tunable decoding parameters must be fitted on training data and stored in self.decoding_params.Source: §F.3 · p. 39

F.4. Recorder prompt

• The recorder prompt requests a short method description, experiment name, and decoding parameters in a specified JSON structure.Source: §F.4 · p. 39