From Pareto to Preference: Personalized Test-Time Scaling via Amortized Agentic Policy Discovery

1 INTRODUCTION

• Test-time scaling allocates additional inference computation to improve reasoning, but extended exploration incurs substantial overhead.Efficiency approaches often reduce redundant computation through adaptive sampling, early stopping, or selective branch pruning.Source: §1 · p. 1
• Personalized Test-Time Scaling discovers executable controllers to maximize joint satisfaction of user-specific accuracy, latency, and inference-cost requirements.Different requirements can favor different controllers even on the same problem set.Source: §1 · pp. 1–2
• PersonTTS reuses a requirement-matched source controller and distilled procedural guidance to initialize discovery, while evaluating each candidate under the target profile.A discovery agent proposes executable controllers using target requirements, replay feedback, and accumulated search history.Source: §1 · pp. 2–3
• PersonTTS substantially outperforms strong TTS baselines in joint requirement satisfaction on unseen user profiles and held-out problems.Under the same candidate-evaluation budget, cross-user experience reuse further improves policy quality while substantially reducing discovery-agent time and monetary cost.Source: §1 · p. 3

Efficient Test-Time Scaling

• Efficient TTS methods reduce unnecessary exploration through adaptive sampling, early stopping, difficulty-conditioned allocation, pruning, and search-control strategies, but largely target an accuracy–cost or accuracy–latency frontier.Examples include adaptive sampling and early stopping [Aggarwal et al., 2023; Li et al., 2024], and branch pruning and adaptive search control [Wang et al., 2025b; Huang et al., 2025; Zheng et al., 2026a;b].Source: §2 · p. 3

Agentic Discovery

• AutoTTS frames TTS strategy design as executable controller synthesis for the accuracy–cost frontier, whereas PersonTTS targets joint user-specific requirements and amortizes discovery across profiles.LLM discovery agents iteratively propose, evaluate, and refine executable artifacts using textual feedback or execution histories [Liu et al., 2024; Agrawal et al., 2026; Lee et al., 2026].Source: §2 · p. 3

Experience Reuse in LLMs

• Experience-reuse work includes distilled reflections, search experience, reusable skills, and workflow selection; PersonTTS instead reuses source controllers and procedural experience across user requirement profiles.It retains target-profile evaluation for controller selection.Source: §2 · p. 3

3.1 PROBLEM FORMULATION

• A user profile specifies an accuracy floor and latency and inference-cost ceilings; an executable policy maps that profile and observation history to controller actions.Actions include spawning branches, continuing or refining them, pruning, and finishing with an answer or plurality voting; observations exclude reference answers and correctness labels.Source: §3.1 · p. 4

3.2 USER-CONDITIONED AGENTIC POLICY DISCOVERY

• The joint satisfaction rate is the fraction of replay seed batches meeting all three requirements, so candidates are evaluated under the target profile.An LLM agent revises executable controller logic using cached trajectories and replay feedback; initialization is evaluated separately, and only strict JSR improvements replace the incumbent.Source: §3.1 · p. 4, §3.2 · pp. 4–5

3.3 CROSS-USER EXPERIENCE REUSE

• The Policy Experience Bank stores source profiles, candidate code, evaluation feedback, execution traces, and selected policies for controller retrieval and procedural guidance.A compatible, requirement-similar source policy is replayed under the target profile to establish its initial JSR and provide diagnostics, rather than relying on its source score.Source: §3.3 · p. 5
• A shared Guide distills source-candidate comparisons into rules for when revisions apply, which changes to prioritize, and how to evaluate resulting policies.When enabled, the frozen Guide informs every proposal, while target feedback updates search evidence rather than the Guide itself.Source: §3.3 · p. 5

3.4 SEPARATING REQUIREMENT CHANGES FROM EXECUTION CHANGES

• The JSR decomposition separates threshold changes on fixed outcomes from the effects of requirement-conditioned execution.The target-satisfaction bound is exact under zero execution deviation; more generally, it depends on execution deviation rather than profile distance alone, so target execution remains necessary for candidate evaluation and selection.Source: §3.4 · p. 6

4.1 EXPERIMENTAL SETUP

• Experiments use six Qwen3 models on AIME and HMMT, with separate discovery and held-out problem sets and independently sampled source and target profiles.The main comparison uses 100 source profiles and 20 target profiles; the experience bank and Guide use source-profile histories, excluding target records and held-out outcomes. Baselines include scalar-objective AutoTTS, Self-Consistency, and Parallel-Probe.Source: §4.1 · p. 6

4.2 MAIN RESULTS

• Personalized controller discovery substantially outperforms strong TTS baselines across benchmarks and held-out problems, with cross-user reuse further improving target-profile policies under the same evaluation budget.The no-reuse variant preserves most of the advantage over external baselines; the comparison uses joint satisfaction rate (JSR).Source: Table 1 · p. 7, §4.2 · p. 7

4.3 ABLATION STUDY

• Warm-start mainly improves the initial search point, while the Guide informs later revisions and provides more consistent held-out gains across benchmarks.Their gains are not uniformly additive: retrieved controllers act as target-dependent initialization priors, while procedural guidance is less dependent on a particular source controller.Source: §4.3 · p. 7, Figure 2 · p. 8
• Guide-only discovery reduces agent-call time by approximately 46% and cost by approximately 36% versus no reuse across both benchmarks.Warm-start alone mainly shortens elapsed time and can slightly increase cost.Source: §4.3 · p. 7, Table 2 · p. 8

4.4.1 DISCOVERY-TO-HELD-OUT GENERALIZATION

• Discovery JSR generally rises alongside held-out policy quality, but the persistent gap means it is informative for search rather than a direct estimate of out-of-sample satisfaction.PersonTTS finishes above independent discovery on both benchmarks.Source: §4.4.1 · p. 8, Figure 3 · p. 9

4.4.2 EXPERIENCE BANK SCALING

• Larger experience banks consistently improve discovery JSR, but held-out effects are not monotonic: they help on AIME and reverse on HMMT.The analysis varies nested banks of 20 and 60 source-profile–policy pairs from the full 100-pair bank while keeping the Guide and target-side evaluation budget fixed.Source: §4.4.2 · pp. 8–9, Table 3 · p. 9

5 CONCLUSIONS

• PersonTTS combines target-conditioned controller search with requirement-matched source controllers and procedural guidance, retaining target-profile evaluation for candidate selection.Experiments on AIME and HMMT report gains over strong TTS baselines on unseen profiles and held-out problems, with reuse improving policy quality and reducing discovery overhead under the same candidate-evaluation budget.Source: §5 · p. 9
• Future work could use calibrated coverage or uncertainty and request frequency to decide when to reuse matched controllers or trigger background discovery.The proposed deployment distinction is between reliable bank coverage and frequent or poorly covered profiles whose search costs may be amortized over future requests.Source: §5 · p. 10

AI USE STATEMENT

• Generative AI tools were used for manuscript polishing and readability improvements, and the authors reviewed the polished text.The statement says the tools were not used for other tasks with required disclosure.Source: AI USE STATEMENT · p. 10

• The evaluation uses offline replay with simulated operational requirement profiles and mathematical reasoning problems.Source: ETHICS STATEMENT · p. 10
• The study involves no human participants and does not measure subjective satisfaction.Source: ETHICS STATEMENT · p. 10
• Joint satisfaction is defined by accuracy and resource constraints in replay; deployment requires validation on the intended workload and resource accounting.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The reproducibility statement documents replay-pool construction, profile sampling, discovery prompts, Guide distillation, measurements, and proofs for conditional transfer analysis.Source: REPRODUCIBILITY STATEMENT · p. 10

A USER-CONDITIONED POLICY DISCOVERY ALGORITHM

• Candidate generation receives the environment, target requirements, initial policy, accessible history, calibration estimates, and optional Guide.Source: A USER-CONDITIONED POLICY DISCOVERY… · p. 14
• Algorithm 1 evaluates an initial policy, generates and validates B candidates using requirements, history, calibration traces, and an optional Guide, then returns the highest-JSR policy.Source: A USER-CONDITIONED POLICY DISCOVERY… · p. 14

B.1 OFFLINE REPLAY POOL CONSTRUCTION

• Each problem–model pair contributes 128 offline rollouts with answer and resource records, up to five refinement stages, and checkpoints every 500 thinking tokens.Source: §B.1 · p. 14

B.2 REPLAY EVALUATION AND COST ACCOUNTING

• Methods share profiles, replay pools, 1,000 seeds, and resource accounting; latency is maximum per-question replay latency, while weighted compute sums token use weighted by model size.Source: §B.2 · p. 14

B.3 REQUIREMENT PROFILE SAMPLING

• Profile sampling uses 1,260 anchor configurations per benchmark, stratifies candidates by feasibility ratio, and selects independent target profiles without shared threshold triples.Source: §B.3 · p. 15

B.4 DISCOVERY SETTINGS

• PersonTTS outperforms strong TTS baselines on unseen profiles and held-out problems, using the same candidate-evaluation budget for its discovery comparisons.Source: §B.4 · p. 15
• Discovery scores an initialization plus five candidates, selects the highest discovery-problem JSR, and freezes that policy for held-out evaluation.Source: §B.4 · p. 15

C CONTROLLER AND DISCOVERY CONFIGURATION

• Controller execution, candidate generation, and external evaluation are separated; static validation limits code access, and gold answers and held-out results are withheld.Source: C CONTROLLER AND DISCOVERY CONFIGURATION · p. 16

D.1 CROSS-BENCHMARK GENERALIZATION

• Requirement-matched controllers remain useful across benchmark shifts without target-benchmark discovery, but perform substantially below target-native PersonTTS.The reported cross-benchmark evaluation averages JSR over 20 target profiles with 1,000 replay seeds each.Source: §D.1 · p. 16, Table D.1 · p. 16

E.1 TARGET SATISFACTION BOUNDS

• When source–target execution deviation is zero, rethresholding determines target JSR exactly; otherwise, the bound depends on execution deviation rather than profile distance alone.Source: §E.1 · pp. 16–17

E.2 COMPARING PROGRAMS ON FIXED SOURCE RECORDS

• For fixed source records, program comparisons are constant within regions induced by threshold hyperplanes, with equality faces treated separately.Source: §E.2 · p. 17
• Relaxing requirements makes each fixed-record score nondecreasing, but does not necessarily preserve the difference between programs’ scores.Source: §E.2 · pp. 17–18

E.3 IDENTICAL MARGINALS CAN CONCEAL A RANKING REVERSAL

• Identical full marginals can conceal a joint-satisfaction ranking reversal: relaxing only latency changes the programs’ scores from (0, 1/3) to (2/3, 1/3).Paired dominance is sufficient for aggregate ordering, but not necessary.Source: §E.3 · p. 18, Table E.1 · p. 18

E.4 RECLASSIFICATION VERSUS TARGET EXECUTION

• Target execution changes may add to threshold reclassification: source records alone do not determine target performance when a program’s behavior depends on the user profile.The decomposition separates threshold changes on fixed records from profile-conditioned execution changes; unchanged paired outcomes are a special case, not an assumption for the source bank.Source: §E.4 · pp. 18–19

E.5 WHAT CONSTRAINT DIAGNOSTICS IDENTIFY

• For each constraint i, H_i has sharp bounds from the marginal pass rates and joint rate, with every allowed grid value attainable by abstract panels.When the other two pass sets are preserved, the joint-rate change lies between −J and H_i; this is an outcome-level bound, not an executable repair rule.Source: §E.5 · p. 19
• Equal marginal pass rates and JSR can still leave different isolated-failure fractions, so summaries may not identify which constraint repair could help.The illustrative three-seed arrays have identical rates and JSR but different (H_A, H_L) values; they are not measured cache outcomes.Source: §E.5 · p. 19

F PROMPTS

• The discovery prompt directs the agent to inspect authorized evidence and implement a candidate within the supplied workspace and data boundaries.It instructs personalized runs to use all three explicit requirements directly, without inventing a scalar beta.Source: F PROMPTS · p. 20

Public Policy API

• The public API exposes the current profile’s accuracy, latency, and weighted-compute constraints alongside model availability and controller actions.Personalized evaluation has no beta field; the interface includes spawning, continuing, refining, pruning, and finishing policies.Source: Public Policy API · p. 21

H META-PROMPT FOR STRATEGY GUIDE GENERATION

• The strategy-guide procedure distills how to search from authorized experience, using joint profile satisfaction as the objective rather than copying historical policies.It calls for preserving evaluator definitions and analyzing actual code and execution traces while distinguishing satisfaction changes from progress relative to the incumbent.Source: H META-PROMPT FOR STRATEGY GUIDE… · p. 22

I GENERAL METHODOLOGY FROM TRAINING-SET DISCOVERY TRACES

• The workflow is distilled from executed training-set policies and separates replay contracts, local calibration, search heuristics, and failure diagnosis.Its design is intended for new profiles with profile-specific re-estimation; the training evidence should not be treated as universal laws.Source: I GENERAL METHODOLOGY FROM… · p. 23, Figure 7 · p. 24
• Use aggregate summaries before bounded representative traces, keeping raw questions, gold answers, private caches, and unexecuted checkpoints outside the evidence.The methodology recommends conservative policy updates and failure diagnosis based on training-trace evidence.Source: I GENERAL METHODOLOGY FROM… · p. 23, Figure 7 · p. 24
• Calibrate phase latency and continue-action token estimates from trace summaries, while accounting for cohort-max latency and summed weighted compute.These estimates are heuristic because scoring uses maximum per-question latency within each seed batch and action-level variation.Source: I GENERAL METHODOLOGY FROM… · p. 23