PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search

Zhixuan Gao, Ke Xue, Rongxi Tan, Ming Chen, Chao Qian

arXiv:2609.34281v1cs.LG

TL;DR

Agentic Bayesian optimization has been evaluated mainly on low- and moderate-dimensional problems, leaving its transfer to high-dimensional optimization underexplored. HERA uses task context, optimization feedback, and structural diagnostics to revise hypotheses and guide numerical search. It matches strong classical methods and outperforms evaluated LLM-based and agentic baselines on synthetic benchmarks, while achieving the best mean final objective on most real-world tasks.

  • Problem

    Agentic BO has mainly been evaluated on low- and moderate-dimensional problems, leaving transfer to HDBO underexplored, where limited evaluations must guide both optimization and strategy choice.

    Source: §1 · pp. 1–2

  • Method

    HERA maintains revisable structural hypotheses and selects numerical strategies, while PRISM executes the chosen strategy and generates high-dimensional candidates.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    HERA matches strong classical HDBO methods and outperforms evaluated LLM-based and agentic baselines on synthetic benchmarks, and has the best mean final objective on most real-world tasks.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The results support an approach in which the LLM manages the evolving search strategy rather than directly generating high-dimensional candidates.

    Source: §5 · p. 10

    HERA is bounded by PRISM’s available diagnostics and numerical strategies, and its working hypotheses do not guarantee recovery of the objective’s true structure.

    Source: §5 · p. 10

Abstract

from arXiv · show

High-dimensional Bayesian optimization (HDBO) seeks sample-efficient optimization when the number of variables is large relative to the evaluation budget. Recent LLM-based and agentic BO methods incorporate task knowledge and adapt search decisions during a run, but have primarily been evaluated on low- and moderate-dimensional problems. We ask whether this paradigm can transfer to the higher-dimensional regime. Our experiments show that these methods do not remain reliable in the high-dimensional regime, where the challenge is not only where to evaluate, but also which modeling assumption and search geometry to use when the objective's useful structure is unknown. We therefore introduce HERA, a Hypothesis- and Evidence-guided Research Agent that uses task context, optimization feedback, and structural diagnostics to revise search hypotheses, select and configure HDBO strategies, and determine their execution length. PRISM, its numerical optimization engine, generates and evaluates candidates sequentially within each search block, updating numerical models after each observation. HERA remains competitive with strong numerical HDBO baselines and outperforms the evaluated LLM-based and agentic methods on four metadata-free synthetic functions. Across eight real-world tasks, HERA achieves the best mean final objective among all evaluated systems on most benchmarks. Further analyses show that structural diagnostics change strategy use, metadata effects vary across tasks, and adaptive search blocks reduce inference cost.

1 INTRODUCTION

HERA places an agent above candidate generation to revise search hypotheses and manage bounded numerical experiments, addressing uncertainty about which HDBO strategy fits an objective. It matches strong classical HDBO methods, outperforms the evaluated LLM-based and agentic baselines, and achieves the best mean final objective on most real-world tasks.

Introduction

  • HDBO transfer remains underexplored: sparse observations complicate modeling and search, while coordinate sparsity, embeddings, additivity, and locality impose different structural biases.Carefully designed full-dimensional BO can also be competitive, so no single strategy is an obvious default (Hoang et al., 2025; Letham et al., 2020; Kandasamy et al., 2015; Eriksson et al., 2019; Hvarfner et al., 2024; Papenmeier et al., 2025b).Source: §1 · pp. 1–2
  • Direct transfer is difficult because generic full-dimensional backends can be fragile, fixed specialized backends may mismatch objectives, and LLM-generated proposals can have poor coverage.These limitations motivate assigning the agent control over revisable hypotheses and bounded experiments while specialized numerical methods generate concrete candidates (Papenmeier et al., 2025b; Yu et al., 2026; Schwanke et al., 2026).Source: §1 · p. 2
  • HERA uses task context, progress, and structural diagnostics to choose whether to continue, reconfigure, restart, or switch strategy; PRISM validates interventions and searches through the selected HDBO method.Their shared observation history supports an observe–hypothesize–intervene–evaluate–reflect loop, with one agent decision governing several numerical evaluations before reassessment.Source: §1 · p. 2
  • HERA matches strong classical HDBO methods, outperforms evaluated LLM-based and agentic baselines, and achieves the best mean final objective on most of eight real-world tasks.The evaluation also covers four metadata-free synthetic functions and examines strategy selection, structural diagnostics, task metadata, decision frequency, model family, and inference cost.Source: §1 · p. 2

2 BACKGROUND

The background moves from standard surrogate-and-acquisition Bayesian optimization to the challenge of choosing useful structure and control in high-dimensional optimization.

High-dimensional Bayesian optimization

  • HDBO methods use variable selection, embeddings, low-dimensional decomposition, trust regions, or full-dimensional search as inductive biases for objectives with unknown structure.These approaches address the difficulty of surrogate fitting and acquisition optimization when the dimension is large relative to the evaluation budget.Source: §2 · p. 3

Large language models for black-box optimization

  • LLM-based BBO either enriches prescribed BO components or generates evaluations more directly, but its intervention location and form remain algorithm-specified.Examples include LLM-designed kernels (Suwandi et al., 2025), LLM-generated proposals in OPRO (Yang et al., 2024), and LLAMBO’s fixed intervention interface (Liu et al., 2024b).Source: §2 · p. 3

Agentic BBO

  • Agentic BBO gives models broader authority in reason–act–observe loops, while Sara (Brunzema et al., 2026) is the closest prior system to HERA.Because HDBO requires numerical candidate generation under unknown useful search bias, HERA chooses and configures biases while PRISM generates consecutive candidates before reassessment.Source: §2 · p. 3

3 HERA: HYPOTHESIS- AND EVIDENCE-GUIDED RESEARCH AGENT

HERA assigns the LLM a strategy-level role: it maintains and revises search hypotheses, while PRISM performs high-dimensional candidate generation and numerical optimization. Structural diagnostics and block outcomes inform strategy choice and execution length.

Agentic HDBO

  • HERA controls the structural search process through state queries, one initialization action, strategy interventions, block-length choices, and stopping; PRISM retains authority over candidate generation.Unlike point-level Agentic BO, HERA does not propose candidates after initialization; it can switch, reconfigure, or restart a numerical strategy.Source: §3 · p. 4

HDBO diagnostics and working hypotheses

  • PRISM summarizes search geometry, surrogate credibility, and evidence for sparsity, low-dimensional variation, and decomposable interactions to support HERA’s revisable working hypotheses.HERA combines these signals with block improvement, the active strategy, and remaining budget; hypotheses connect supporting and conflicting evidence to an intervention and reassessment condition without claiming to recover true structure.Source: §3 · p. 5
  • HERA can test hypotheses using Vanilla BO for full-dimensional search, LassoBO for axis-aligned sparsity, ALEBO-inspired search for low-rank structure, Add-GP-UCB for additive interactions, and TuRBO for local regularity.The broader sparse and embedding families include SAASBO and BAxUS; these numerical strategies let HERA test and revise hypotheses as evidence accumulates (Hvarfner et al., 2024; Hoang et al., 2025; Letham et al., 2020; Kandasamy et al., 2015; Eriksson et al., 2019; Eriksson & Jankowiak, 2021; Papenmeier et al., 2022).Source: §3 · p. 5

Agentic HDBO

  • HERA may propose one task-semantics-informed initial design, which PRISM validates and queues for the next block; after a search block begins, all subsequent candidates come from numerical strategies.The queued design is charged to the shared evaluation budget when the next block evaluates it.Source: §3 · pp. 4–5

Search blocks

  • Within each search block, PRISM fits models, optimizes acquisitions, validates complete candidates, and can update its backend after every observation; all observations remain available to later strategies.A block ends at its evaluation horizon, budget exhaustion, or evaluator failure, after which updated context returns to HERA for reassessment.Source: §3 · pp. 4, 6

4 EXPERIMENTS

Across 100-evaluation, 10-seed experiments, HERA remains competitive with numerical HDBO methods on metadata-free synthetic tasks and leads the evaluated systems on most real-world benchmarks. Ablations show that diagnostics, metadata, decision frequency, and agent-model choice affect behavior or performance in task-dependent ways.

4.1 SYNTHETIC BENCHMARKS

  • No search bias wins across all four functions: full-dimensional or sparse search leads on Hartmann, HERA on L´evy, and local trust-region search on Rosenbrock.HERA stays close to the best numerical methods without relying on hidden task construction or a prescribed structural assumption.Source: §4.1 · p. 6
  • Across four metadata-free synthetic tasks, LLM/agentic baselines trail strong numerical HDBO; HERA ranks first on L´evy, second on Ackley, and third on Hartmann and Rosenbrock.The synthetic suite extends Hartmann6 to 100-D, Ackley10 to 200-D, L´evy15 to 300-D, and Rosenbrock10 to 300-D, with task identities anonymized and metadata cards empty.Source: §4.1 · p. 6, Figure 2 · p. 7

4.2 HEURISTIC AND BANDIT BASELINES

  • HERA beats heuristic and UCB controllers on Hartmann, L´evy, and Rosenbrock, while UCB is slightly better on Ackley; UCB allocations are nearly task-invariant, unlike HERA’s task-dependent patterns.Both controllers use HERA’s numerical portfolio, but HERA can also configure or restart a backend and choose its execution horizon.Source: Figure 3 · p. 7, §4.2 · p. 7

4.3 REAL-WORLD BENCHMARKS

  • On real-world tasks, HERA beats the strongest classical baseline on 6/8 and has the best mean versus LLM/agentic methods on 7/8, with Median Molecules 2 the exception.Classical methods use only evaluated inputs and outputs, while HERA and LLM/agentic baselines share task information, so these are complete-system comparisons.Source: Figure 4 · p. 8, §4.3 · p. 8

Structural evidence

  • Removing structural diagnostics changes strategy use and concentrates backend choices, especially toward Vanilla BO on MOPTA and Mazda, while final performance stays similar on most tasks.Variable-relevance and low-dimensional-subspace readings become more informative as observations accumulate, but interaction grouping remains unreliable; the diagnostics are provisional evidence, not recovered ground truth.Source: §4.4 · p. 9

Task metadata

  • Removing task metadata degrades performance on most tasks, especially Rover and Ant, but slightly improves a few others, making metadata a task-dependent prior.Removal almost eliminates initial-solution-tool use; metadata can affect initialization and later reasoning, so this intervention does not isolate strategy selection.Source: §4.4 · p. 9

Decision frequency

  • With context compaction, per-evaluation decisions increase inference by roughly an order of magnitude without consistent paired-run gains; without compaction, improvements are also inconsistent across tasks.Adaptive blocks let numerical backends accumulate evidence before deliberation while avoiding the large inference overhead.Source: §4.4 · p. 9

Model family

  • Kimi-K3 has the best final mean on HalfCheetah, while GPT-6-astra leads on MOPTA and Mazda; no model dominates across tasks.Model comparisons replace DeepSeek-V4.1-Flash on the same three tasks.Source: §4.4 · p. 9

5 CONCLUSION · A FURTHER RELATED WORK · B EXPERIMENTAL SETTINGS AND TASK DEFINITIONS

HERA pairs an LLM that revises structural hypotheses and search strategies with PRISM’s numerical candidate generation, and the paper reports strong results while noting limits from the diagnostics and strategies available to the framework. The appendices document experimental settings and task definitions, while related work situates the approach among classical HDBO, LLM-assisted BO, and agentic optimization methods.

5 CONCLUSION

  • HERA remains competitive with strong HDBO methods and improves over evaluated LLM-based and agentic baselines on metadata-free synthetic functions, while performing strongly across eight real-world tasks.Ablations show that structural evidence changes strategy use, metadata supplies task-dependent priors, and adaptive decision intervals control inference cost.Source: §5 · p. 9
  • HERA manages evolving search strategies rather than directly generating high-dimensional candidates, but its hypotheses cannot guarantee recovery of the objective’s true structure.The framework is bounded by PRISM’s available diagnostics and numerical strategies; future work could expand the tool and strategy space, calibrate diagnostic reliability, and transfer experience across related tasks.Source: §5 · p. 10

AI USE STATEMENT

  • The authors used generative AI for translation and language assistance, but not for research formulation, methodology, implementation, data work, or results analysis.They reviewed AI-assisted revisions for intended meaning and technical accuracy and accept responsibility for the final work.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The appendices specify task representations, opening designs, evaluation budgets, backend adaptations, scheduler rules, intervention scopes, and sample counts; all objective evaluations count toward the same budget.No results are inferred from unexecuted experiments, and code is provided in supplementary material.Source: REPRODUCIBILITY STATEMENT · p. 11

A FURTHER RELATED WORK

B.1 TASKS AND REPRESENTATIONS · B.2 BASELINES AND IMPLEMENTATION DETAILS · C TOOL DETAILS

The appendix specifies task representations, evaluation conditions, baseline comparisons, and how HERA interacts with PRISM. It also clarifies what anonymization does and does not conceal.

B.1 TASKS AND REPRESENTATIONS

  • Each task–method pair uses seeds 0–9, a budget of 100, and a shared five-point scrambled-Sobol opening design; optimization uses normalized coordinates and minimization-oriented rewards.Molecular tasks optimize continuous latent representations of GuacaMol median-molecule objectives, and the reported dimension is the latent representation’s dimension.Source: §B.1 · p. 15
  • Synthetic embeddings use block averages in which all ambient coordinates contribute, while shifted tasks apply a fixed coordinatewise wrap-around shift before evaluation.The shift is (x + δ) mod 1.Source: §B.1 · p. 15
  • Anonymized synthetic runs use opaque identifiers and empty metadata cards, limiting access to named formulas without preventing agents from inferring properties from observations.The optional initial-solution tool remains available.Source: §B.1 · p. 15

B.2 BASELINES AND IMPLEMENTATION DETAILS

  • The fixed numerical comparators are the five portfolio members and Sobol search; the strongest classical comparator is chosen per task by mean final objective, not as a deployable oracle.This selection is an evaluation reference, not an additional numerical method.Source: §B.2 · p. 16
  • LLAMBO, Centaur, and Sara share task wrappers, budgets, and opening designs; the principal real-world comparison uses DeepSeek-V4.1-Flash and archived task cards.HERA uses DeepSeek-V4.1-Flash at temperature zero; model ablations use Kimi-K3 at temperature one or GPT-6-astra at temperature zero through a separate relay.Source: §B.2 · p. 16

C TOOL DETAILS

  • HERA calls PRISM through the in-process PrismToolbox interface rather than shell commands, with the observed dataset authoritative and diagnostics, scoreboards, and hypotheses as derived views.Switching starts a new backend-specific episode while preserving the complete observation history.Source: C TOOL DETAILS · p. 16

D SYNTHETIC RESULTS AND SCHEDULER DEFINITIONS · D.1 HEURISTIC AND BANDIT POLICIES

The section compares heuristic and UCB scheduler behavior with HERA and specifies how the heuristic routes search and how UCB scales its response. The tested policies show distinct allocation patterns, but the findings characterize these policies rather than learned portfolio selection in general.

D.1 HEURISTIC AND BANDIT POLICIES

  • The heuristic starts with full-dimensional BO (Vanilla BO) and reassesses its backend after every five-evaluation block.Source: §D.1 · p. 17
  • It activates sparse search below 0.1 active fraction, the low-dimensional subspace backend at dimension ratio ≤ 0.02, grouped-GP search at partition score ≥ 0.9, and local search above 0.05 normalized observation-tour length.Source: §D.1 · p. 17
  • When multiple heuristic triggers fire, a seeded random choice resolves them; with no trigger, the active backend is retained.The geometry rule reflects the search distribution, not a statistical test of local smoothness.Source: §D.1 · p. 17
  • UCB treats each block as one play and selects each untried backend before maximizing its UCB criterion.Source: §D.1 · p. 17
  • UCB scales its response using 1.4826 times the median absolute deviation, with interquartile-range and standard-deviation fallbacks, making its bonus comparable across objective scales.Source: §D.1 · p. 17
  • These measurements characterize the tested policies rather than learned portfolio selection in general.Source: §D.1 · p. 17
  • Figure 5 shows heuristic routing often saturating on grouped-GP search, while UCB is close to HERA on several tasks but largely task-invariant; HERA uses task-dependent allocations and longer blocks.HERA commits longer blocks to a smaller subset of backends, whereas UCB allocations are driven by recent normalized improvement rather than structural evidence.Source: Figure 5 · p. 17, §D.1 · p. 17

E DETAILED REAL-WORLD RESULTS … G.1 HDBO DIAGNOSTIC STATE

HERA’s real-world results are mixed across tasks, while withholding explicit structural evidence changes backend use without a clear aggregate performance effect. PRISM exposes geometric, surrogate-credibility, and structural diagnostics to guide HDBO search.

E DETAILED REAL-WORLD RESULTS

  • Across real-world tasks, HERA has the best mean on Chip Design, but full-dimensional BO beats it on MOPTA and Mazda, while Centaur and LLAMBO beat it on MM2.HERA’s Chip Design margin over full-dimensional BO is small relative to seed variation; the small MM1 gap does not establish a substantial improvement, and large control-task gains are variable, particularly on HalfCheetah.Source: E DETAILED REAL-WORLD RESULTS · pp. 17–18, Table 5 · p. 18, Table 6 · p. 18

F STRUCTURAL-EVIDENCE INTERVENTION

  • The evidence-removal intervention withheld three explicit structural channels while retaining task cards, tools, opening designs, the main prompt, and backend-reported state.The intervention therefore did not remove every possible source of structural information.Source: F STRUCTURAL-EVIDENCE INTERVENTION · p. 18
  • Withholding structural evidence concentrated strategy use on several tasks, especially in longer Vanilla BO episodes on MOPTA and Mazda, while aggregate performance curves remain difficult to distinguish on most tasks.Backend diversity and switching do not directly measure hypothesis correctness, and evidence removal may alter initialization or earlier actions, so later trajectory differences cannot be attributed to an isolated switch.Source: F STRUCTURAL-EVIDENCE INTERVENTION · p. 18, Figure 6 · p. 19, Figure 7 · p. 19

G.1 HDBO DIAGNOSTIC STATE

  • PRISM derives its HDBO diagnostic state from normalized observations and objective values without spending evaluations, targeting space coverage, surrogate credibility, and structural-bias selection.The quantities are derived control signals, not observations of the unknown objective structure.Source: §G.1 · p. 18, Table 7 · p. 20
  • PRISM’s tour-length and observation-entropy signals provide complementary geometric readings, but irrelevant coordinates can still affect both.The tour length is based on a cheapest-insertion tour through normalized observations, and entropy uses a cumulative k-nearest-neighbor estimator.Source: §G.1 · p. 20
  • PRISM assesses surrogate credibility before structural interpretation by fitting a diagnostic GP on normalized inputs and standardized outcomes, summarizing posterior standard deviation over 256 scrambled-Sobol probes, and conducting leave-one-out-style validation.All observations are held out once when n ≤12; otherwise, at most 12 deterministically spaced folds are used. PRISM reports residual summaries, empirical 1σ and 2σ coverage, and RMSE normalized by the observed response standard deviation, and attaches n/D to every structural block.Source: §G.1 · p. 20

G.2 SPARSITY AND LOW-DIMENSIONAL VARIATION · G.3 INTERACTION GROUPING AND FRAGMENTATION

The diagnostics summarize coordinate-support recovery, sparsity, low-dimensional variation, and interaction grouping. They are interpreted as imperfect structural indicators rather than guarantees of recovered structure.

G.2 SPARSITY AND LOW-DIMENSIONAL VARIATION

  • Across coordinates, support recovery is measured with Jaccard overlap between the estimated active set and known support in the offline audit.Source: §G.2 · p. 21
  • The active fraction is estimated support size divided by ambient dimension, providing a compact sparsity reading but not confirming coordinate correctness.Source: §G.2 · p. 21
  • The low-dimensional subspace diagnostic uses surrogate posterior-mean gradients at probe locations to assess whether response variation concentrates in a few directions.Source: §G.2 · p. 21
  • With M = 64 probes, the effective energy dimension is not an exact algebraic dimension or basis recovery and should be read alongside surrogate-credibility diagnostics.Source: §G.2 · p. 21

G.3 INTERACTION GROUPING AND FRAGMENTATION

  • The Differential Grouping-inspired diagnostic detects coordinate links from normalized mixed differences of the surrogate mean, aggregates them over anchors, and forms connected components.Source: §G.3 · p. 21
  • The fragmentation score is the fraction of coordinate pairs in different components; high values may indicate missed interactions, not additive structure.Source: §G.3 · p. 21

H METADATA

Strict metadata removal generally worsens performance on several tasks, but gains and reversals vary by task, indicating no uniform semantic advantage. The ablation withholds task identity, semantic descriptions, reference-design hints, and their prompt instructions while preserving the optimization budget and interface.

H METADATA

  • The main ablation withholds task identity, semantic descriptions, reference-design hints, and corresponding prompt instructions while preserving the optimization budget and interface.This is broader than deleting one semantic card alone.Source: H METADATA · p. 21
  • Strict metadata removal generally worsens performance on several tasks, with the largest effects concentrated in particular tasks.Figure 9 reports the strict-removal condition.Source: H METADATA · p. 21
  • Gains and reversals under strict removal support task dependence rather than a uniform semantic advantage.Source: H METADATA · p. 21

I AGENT MODEL CONFIGURATIONS · J TOOL-USE TRAJECTORIES · K BLOCK SIZE AND CONTEXT COMPACTION

Across these analyses, agent-model rankings vary by task and seed, while tool-use patterns and decision-block trade-offs are also task-dependent. Context compaction reduces inference cost while preserving broadly comparable optimization behavior, but the evidence does not establish a universally best decision interval.

I AGENT MODEL CONFIGURATIONS

  • The model comparison used Kimi-K3 and GPT-6-astra against DeepSeek-V4.1-Flash on three tasks, with ten completed seeds per configuration and the same interface and evaluation budget.Kimi required temperature one rather than zero, and GPT used a separate relay, so the comparison reflects deployed configurations rather than isolated model capability.Source: I AGENT MODEL CONFIGURATIONS · p. 21
  • Kimi has the best mean on HalfCheetah, while GPT has the best means on MOPTA and Mazda; DeepSeek’s HalfCheetah median beats Kimi’s, preventing a uniform model ranking.Substantial seed variation also means backend behavior alone does not explain final-objective changes.Source: I AGENT MODEL CONFIGURATIONS · p. 21, Table 8 · p. 22

J TOOL-USE TRAJECTORIES

  • Tool trajectories share a common initialization pattern, then show task- and seed-dependent control, from frequent strategy interventions to stable backends over extended search episodes.This pattern is consistent with adaptive outer-loop control, but does not establish hypothesis correctness or a causal benefit from more frequent tool use.Source: J TOOL-USE TRAJECTORIES · p. 23, Figure 11 · p. 23
  • Tool-call, assistant-response, and provider-request timelines describe intervention timing, not whether hypotheses were correct or whether more frequent tool use caused better outcomes.These accounting levels are distinct.Source: J TOOL-USE TRAJECTORIES · p. 23

K BLOCK SIZE AND CONTEXT COMPACTION

  • The block-size experiment compared adaptive decision blocks with and without context compaction, plus block-size-one execution under compaction and an exploratory uncompacted matched-subset arm.Compaction keeps recent tool results detailed, summarizes older results, and preserves assistant reasoning; block size one returns after each evaluation.Source: K BLOCK SIZE AND CONTEXT COMPACTION · p. 23
  • Context compaction substantially reduces inference cost while preserving broadly comparable optimization behavior, but more frequent decisions do not establish a universally best interval.More frequent decisions can improve individual task trajectories, but their added cost and the limited exploratory comparison support a task-dependent trade-off between control frequency and inference efficiency.Source: Figure 12 · p. 23, K BLOCK SIZE AND CONTEXT COMPACTION · p. 23

L TOKEN ACCOUNTING

The accounting distinguishes token volume, request frequency, and billed cost: HERA used the most total tokens despite fewer requests, while its repeated context yielded a 94.0% prompt-cache hit rate.

L TOKEN ACCOUNTING

  • 94.0% was HERA’s prompt-cache hit rate, versus 16.4% for LLAMBO, 27.3% for Centaur, and 36.2% for Sara, while HERA used the most total tokens despite fewer requests.Figure 13 shows per-task totals and cached-input, uncached-input, and output components.Source: L TOKEN ACCOUNTING · p. 24, Figure 13 · p. 24
  • The cost study covers 320 complete campaigns across four LLM methods, eight tasks, and ten seeds, with 100 evaluations per campaign.Total tokens include prompt and completion tokens; prompt tokens are divided into provider-reported cache hits and misses.Source: L TOKEN ACCOUNTING · p. 24
  • Cache hits still count as tokens and may incur cost, so neither total nor uncached tokens alone identifies the cheapest method; fixed HDBO uses no LLM tokens but incurs numerical-computation costs.Monetary costs require prices in consistent token units.Source: L TOKEN ACCOUNTING · p. 24

Related papers

Loading 2609.34281v1…