PaperlayerView exact revision on arXiv

Source-linked AI summary

Compiling Learning Problems into Adaptation Programs for Language Models

Rebecca Ramnauth, Brian Scassellati

arXiv:2609.37371v1cs.CLcs.AI

TL;DR

Fixed adaptation recipes may not suit learning problems with different update needs and behavioral tradeoffs. Adaptation compilation predicts the outcomes of candidate programs from prior episodes and selects a program before adaptation; across objectives, programs vary in their episode-level value, and selection improves over fixed policies on Llama.

  • Problem

    Adaptation configurations are commonly fixed before optimization even though effective update locations can vary across learning objectives and behavioral goals.

    Source: §1 · p. 1

  • Method

    Adaptation compilation learns from prior episodes to predict candidate programs’ behavioral outcomes and selects a program before adaptation begins.

    Source: §1 · p. 2

  • Results

    On Llama, compiler-selected programs substantially close the gap to exhaustive search while outperforming global and objective-specific defaults; preferred programs vary across episodes.

    Source: §11 · p. 8

  • Takeaways & Limitations

    Prior adaptation episodes can inform future program selection, but predicted advantages should be reliable enough to justify departing from strong defaults.

    Source: §11.4 · p. 10

    The study uses four LoRA configurations across five controlled objectives, and compilation amortizes rather than eliminates search because prior candidate programs must still be executed.

    Source: §11.4 · pp. 9–10

Abstract

from arXiv · show

Model adaptation is typically governed by a fixed recipe, even though different update programs can produce substantially different behavioral outcomes. We introduce adaptation compilation, which reframes where, how, and to what extent a model should adapt as a joint prediction and decision problem. Rather than searching over candidate programs anew for each learning episode, a compiler learns from prior adaptations to predict a vector-valued counterfactual response surface over candidate programs---their expected effects on acquisition, transfer, boundedness, and preservation---and selects a program before adaptation begins. Because this predicted geometry captures multiple behavioral consequences rather than a single winner or scalar score, it can be reused under different downstream priorities without retraining. Across five learning types, preferred programs vary meaningfully across episodes, and this variation is predictable from pre-adaptation information. On Llama-3.1-8B, compiler-selected programs approach exhaustive search while outperforming global and objective-specific defaults. Replication on Gemma-2-9B preserves program heterogeneity and selection headroom, but shows that exploiting this headroom requires accounting for uncertainty when departing from strong defaults. Together, these results show that adaptation search can be amortized across related learning problems, turning prior adaptation experience into a basis for deciding how future learning should occur.

1 INTRODUCTION

Fixed adaptation architectures can mismatch the distinct demands of different learning problems. Adaptation compilation predicts the behavioral consequences of candidate programs from prior episodes and selects a program before adaptation begins.

1 INTRODUCTION

  • Different learning problems can favor different update locations and trade off acquisition, generalization, and boundedness, motivating adaptation compilation as a decision about where and how to adapt.Source: §1 · pp. 1–2

2 BACKGROUND AND RELATED WORK

  • Prior adaptive PEFT methods adjust structure within a task or predefined scheme, whereas adaptation compilation reuses prior episodes to predict candidate-program consequences for a new episode.Source: §1 · p. 1, §2 · p. 2

3 A FRAMEWORK FOR ADAPTATION COMPILATION

  • Adaptation compilation amortizes empirical search by learning to predict alternative programs’ consequences from prior adaptation episodes before a new episode is adapted.Source: §3 · p. 3
  • An adaptation configuration specifies adaptable modules and their capacities; modules outside that set remain frozen, and the selected program is trained through ordinary PEFT optimization.Source: §3 · p. 3
  • The predicted adaptation geometry describes expected acquisition, transfer, boundedness, and preservation across programs, and a utility specification determines which program is selected.Source: §3 · p. 3

4 EXPERIMENTAL SETUP

The experiments evaluate whether adaptation programs create selection headroom, whether pre-adaptation signals predict their outcomes, and whether that prediction supports selection and generalizes. The shared setup uses five learning objectives, four LoRA programs, and pre-adaptation episode–model features.

4 EXPERIMENTAL SETUP

  • Five experiments assess selection headroom, pre-adaptation prediction, program selection, informative signals, and generalization beyond represented learning families.Source: §4 · p. 3
  • The benchmark uses five learning objectives and 600 latent-specification episodes split into 400 training, 100 validation, and 100 test episodes; objective identity is withheld from the predictor.Source: §4 · p. 4
  • The primary backbone is Llama-3.1-8B-Instruct, and the candidate programs are four approximately budget-matched LoRA configurations spanning early, middle, late, and full-stack adaptation.Source: §4 · p. 4
  • Utility variants reweight transfer, boundedness, or preservation, while the optimization schedule is calibrated on meta-training episodes and fixed across programs and splits.Source: §4 · p. 4
  • Pre-adaptation features include frozen hidden states, target-token loss statistics, and module-level probes; test outcomes remain unavailable before compiler selection.Source: §4 · p. 4
  • The predictor estimates four behavioral outcomes for each episode–program pair, with regression models and hyperparameters selected by validation MAE averaged across those outcomes.Source: §4 · p. 4

5 EXPERIMENT 1: IS ADAPTATION COMPILATION NECESSARY?

On held-out episodes, the episode oracle outperforms fixed policies, and oracle-optimal programs vary across episodes and learning objectives. The remaining selection headroom is concentrated in some objectives and nearly absent in procedural reasoning.

5 EXPERIMENT 1: IS ADAPTATION COMPILATION NECESSARY?

  • On 100 held-out episodes, oracle utility is 0.618 versus objective-fixed 0.600 and global-fixed 0.569; objective-fixed selection is oracle-optimal on 64% of episodes.Source: §5 · p. 4, Figure 1 · p. 5
  • A unique utility-maximizing program occurs in 99% of episodes, and all four candidate programs are oracle-optimal for substantial subsets.Source: §5 · p. 4
  • Headroom is largest for lexical binding and factual association, smaller for causal and behavioral learning, and nearly absent for procedural reasoning.Source: §5 · p. 5, Figure 1 · p. 5

6 EXPERIMENT 2: CAN ADAPTATION GEOMETRY BE PREDICTED?

Pre-adaptation episode–model signals predict candidate programs’ behavioral geometry and within-episode rankings; compiler selection improves on objective-fixed defaults, while its performance depends on available signals and represented learning families.

6 EXPERIMENT 2: CAN ADAPTATION GEOMETRY BE PREDICTED?

  • Geometry MAE is 0.040 versus 0.238 for configuration-mean and 0.060 for objective-conditioned prediction; ranking reaches Spearman 0.803 and 77%/93% top-1/top-2 oracle recovery.The objective-conditioned predictor is a diagnostic given the true learning objective.Source: §6 · p. 5, Figure 2 · p. 6

7 EXPERIMENT 3: DOES PREDICTION YIELD BETTER PROGRAMS?

  • Compiler selection reaches 0.607 mean utility versus 0.600 for objective-fixed, with 0.011 versus 0.018 oracle regret (oracle: 0.618).Across held-out episodes, compilation improves on objective-fixed selection in 22, degrades it in 7, and ties in 71.Source: §7 · p. 5, Figure 3 · p. 6
  • Without retraining, the same predicted geometry keeps compiler selection closer to the oracle than either fixed policy under transfer-, boundedness-, and preservation-heavy utilities.This tests whether the predicted multi-outcome geometry can be reused under different downstream priorities.Source: §7 · pp. 5–6, Figure 3 · p. 6

8 EXPERIMENT 4: WHICH PRE-ADAPTATION SIGNALS ARE NEEDED?

  • Episode-only signals nearly match the full representation (geometry MAE 0.0434 vs 0.0435; top-1 oracle recovery 83%), while module probes and behavioral statistics are weaker alone.Figure 4 also shows the advantage reverses when a learning family is withheld; on Gemma, direct geometry selection does not beat the objective-fixed policy.Source: §8 · p. 6, Figure 4 · p. 7

9 EXPERIMENT 5: DOES ADAPTATION COMPILATION GENERALIZE?

Compilation generalizes well to new episodes from represented learning families, but not reliably to entirely unseen families. Gemma preserves episode-level selection headroom, while showing that a compiler must account for uncertainty before departing from strong defaults.

9 EXPERIMENT 5: DOES ADAPTATION COMPILATION GENERALIZE?

  • Compiler utility falls to 0.533 on an unseen family, below the global-fixed policy at 0.556, as geometry prediction and oracle recovery also deteriorate.Mean geometry MAE rises to 0.339, within-episode Spearman correlation falls to 0.058, and top-1 oracle recovery drops to 14%.Source: §9 · p. 7

10 EXPERIMENT 6: DOES THE CASE FOR COMPILATION REPRODUCE ACROSS BACKBONES?

  • Gemma retains episode-level headroom despite different backbone-specific program preferences: oracle selections are 40% full, 31% middle, 23% late, and 6% early.The best fixed program is full-stack; objective-specific defaults choose full-stack for behavioral, factual, and procedural learning, middle for causal mapping, and late for lexical binding.Source: §10 · pp. 7–8
  • On Gemma, the learned compiler reaches .435 mean utility versus .461 for the oracle, with .026 regret and 42% oracle recovery, not exceeding fixed baselines.Global-fixed utility is .436 and objective-fixed utility is .438; most loss comes from procedural reasoning, with a smaller lexical-binding deficit, where small predicted margins can trigger uncertain switches.Source: §10 · p. 8

11 DISCUSSION

  • Across Llama results, predicted geometry supports selection that substantially closes the gap to exhaustive search and outperforms global and objective-specific defaults.Predicting acquisition, transfer, boundedness, and preservation also permits re-evaluating the same predictions under different behavioral priorities without retraining.Source: §11 · p. 8

11.1 ADAPTATION AS A PREDICTION AND DECISION PROBLEM

  • Compilation separates predicting outcomes for candidate programs from choosing among them according to preferred behavioral tradeoffs.Predicted geometry can be reused under different utilities, while compilation uses prior interventions to make a decision before executing only the selected program; constructing the initial geometry dataset remains expensive.Source: §11.1 · p. 8

11.2 WHAT DETERMINES AN ADAPTATION PROGRAM?

  • Useful adaptation programs vary within learning objectives, so neither one global recipe nor objective identity alone determines the episode-level choice.Frozen episode representations alone recover nearly all predictive performance of the full episode–model representation in Experiment 4; adaptation geometry also depends on the episode, model, and program, not canonical knowledge locations.Source: §11.2 · p. 9

11.3 WHEN SHOULD A COMPILER TRUST ITS PREDICTION?

  • When learning families are absent from meta-training, geometry error rises, rankings become less reliable, and compilation can underperform a fixed policy.On Gemma, small predicted differences can also make deterministic selection worse than a strong default; the study identifies confidence-aware selection as a future design requirement, not an introduced rule.Source: §11.3 · p. 9

11.4 PRACTICAL SCOPE AND LIMITATIONS

  • The study tests four LoRA configurations across five controlled objectives, while scaling requires richer program representations and more selective adaptation experience.Adaptation outcomes remain stochastic, and the method amortizes search rather than eliminating the cost of executing candidate programs on prior episodes.Source: §11.4 · pp. 9–10

APPENDICES

The appendices document optimization calibration, expanded Llama analyses, unseen-family generalization, and the Gemma replication.

APPENDICES

  • Appendices A–G cover optimization calibration, expanded Llama analyses, unseen-family generalization, and the full Gemma replication protocol and results.The Llama analyses include selection headroom, stochasticity, geometry prediction, program selection, and representation ablations.Source: APPENDICES · p. 12

A OPTIMIZATION SCHEDULE CALIBRATION

The section calibrates a shared optimization schedule, then characterizes how oracle-optimal programs vary across held-out episodes and adaptation seeds. Objective conditioning reduces selection regret, but episode-specific headroom and stochasticity remain.

A OPTIMIZATION SCHEDULE CALIBRATION

  • Gradient accumulation 2 is selected because it supports acquisition and transfer across objectives without the preservation loss seen at 1.Reducing accumulation to 1 yields negligible additional acquisition (0.896 → 0.904), slightly lower transfer (0.867 → 0.853), and substantially lower preservation (0.971 → 0.876).Source: A OPTIMIZATION SCHEDULE CALIBRATION · pp. 12–13

B EXPERIMENT 1: CHARACTERIZING SELECTION HEADROOM

  • The selection-headroom analysis uses 100 held-out episodes, with global-fixed and objective-fixed programs chosen from meta-training data and an episode oracle selecting separately for each episode.Headline test outcomes average each configuration across three adaptation seeds.Source: B EXPERIMENT 1: CHARACTERIZING… · p. 13, §B.1 · p. 13, Table 2 · p. 13

B.1 SELECTION HEADROOM BY LEARNING OBJECTIVE

  • Mean regret falls from 0.049 under the global-fixed policy to 0.018 with objective conditioning, while lexical and factual episodes retain the largest objective-to-oracle gaps.For procedural reasoning, the objective-fixed policy is oracle-optimal on 95% of held-out episodes and differs from the oracle by less than 0.001 utility on average.Source: §B.1 · p. 13

B.2 ORACLE PROGRAMS AND WINNER SEPARATION

  • Middle-depth adaptation is oracle-optimal most often across 100 test episodes, but every candidate program is preferred for a nontrivial subset.Only 1% of test episodes have multiple utility-maximizing configurations; the mean best-to-second-best utility margin is 0.054 (median 0.041).Source: §B.2 · pp. 13–14, Table 3 · p. 14, Figure 5 · p. 14

B.3 ROBUSTNESS TO ADAPTATION STOCHASTICITY

  • Across three seeds, only 40% of episodes retain the same unique oracle winner, yet objective-to-oracle regret remains 0.016 on the robustness subset.The tie-aware criterion finds a common oracle configuration across all seeds for 55% of episodes; the objective-fixed policy is oracle-optimal on 60%.Source: §B.3 · p. 14, Figure 6 · p. 15

B.4 CONFIGURATION SENSITIVITY ACROSS BEHAVIORAL OUTCOMES

Alternative programs produce tradeoffs across four behavioral outcomes, and the predictor’s reliability varies by learning objective. Compiler selection generally reduces oracle regret, with lexical binding the clearest exception to improvement over the objective-fixed policy.

B.4 CONFIGURATION SENSITIVITY ACROSS BEHAVIORAL OUTCOMES

  • Alternative programs induce distinct tradeoffs in acquisition, transfer, boundedness, and preservation, rather than sensitivity limited to aggregate utility.Figure 7 groups seed-averaged outcomes by learning objective and budget-matched program.Source: §B.4 · p. 15

C EXPERIMENT 2: PREDICTING ADAPTATION GEOMETRY

  • The same predictor’s geometry error and program-ranking fidelity vary across objectives: prediction and ranking are stronger for behavioral, causal, and procedural episodes, while lexical and factual episodes are harder to rank.Lexical binding contains the largest-error episodes; the predictor is evaluated without access to objective identity.Source: C EXPERIMENT 2: PREDICTING ADAPTATION… · p. 16, Table 4 · p. 16, Figure 8 · p. 16

D EXPERIMENT 3: EVALUATING PROGRAM SELECTION

  • The compiler reduces oracle regret relative to the global-fixed policy for behavioral, causal, factual, and lexical episodes, and approaches the oracle most closely for behavioral and causal learning.Procedural reasoning has little episode-conditioned headroom because it already admits a strong fixed structural default.Source: §D.1 · p. 17, Table 5 · p. 17

D.1 SELECTION BY LEARNING OBJECTIVE

  • Lexical binding is the clearest exception: the compiler matches the objective-fixed policy’s 65% oracle-optimal rate but has lower mean utility, 0.746 versus 0.751.For factual association, regret falls from 0.027 to 0.006 with episode-conditioned selection.Source: §D.1 · p. 17

D.2 PAIRED COMPARISON WITH OBJECTIVE-FIXED SELECTION

Compiler gains over the objective-fixed policy are positive on average, and the predicted geometry can be reused under different utility priorities without retraining. Restricted representations show objective-dependent strengths, while leave-one-family-out evaluation degrades selection.

D.2 PAIRED COMPARISON WITH OBJECTIVE-FIXED SELECTION

  • The compiler improves utility on 22 of 100 held-out episodes, performs worse on 7, and ties on 71; the mean paired gain is 0.0072 with a 95% interval of [0.0011, 0.0136].Departures from the objective-level default are more often beneficial than harmful.Source: §D.2 · pp. 17–18, Table 6 · p. 18

D.3 RECOMPILATION UNDER ALTERNATIVE UTILITY SPECIFICATIONS

  • Changing transfer or boundedness weights switches the selected program for approximately 14% and 13% of episodes, respectively, while preservation weighting switches 1%.The compiler remains closer to the oracle than the objective-fixed policy under every tested utility specification, without retraining the geometry predictor.Source: §D.3 · p. 18

E EXPERIMENT 4: ABLATING PRE-ADAPTATION INFORMATION

  • No restricted representation dominates across objectives: episode features work particularly well for behavioral and factual episodes, while full representations yield lowest geometry error for lexical and procedural episodes.Prediction error and selection quality can diverge; module probes achieve zero procedural oracle regret despite not minimizing geometry MAE.Source: E EXPERIMENT 4: ABLATING… · p. 18, Table 7 · p. 18, Figure 10 · p. 19

F EXPERIMENT 5: GENERALIZING TO UNSEEN LEARNING FAMILIES

  • Leave-one-family-out evaluation substantially increases geometry error and degrades selection; macro compiler regret rises from .062 under the global-fixed policy to .085 under LOFO selection.LOFO selection reduces regret for behavioral and factual episodes, is neutral for causal mapping, and increases regret for lexical and procedural episodes.Source: F EXPERIMENT 5: GENERALIZING TO UNSEEN… · p. 19

G EXPERIMENT 6: CROSS-BACKBONE REPLICATION ON GEMMA

The Gemma replication reuses the learning setup but retrains its compiler on Gemma adaptation records. Its calibration rule is inconclusive under the prespecified boundedness gate, and the chosen schedule is fixed for later adaptations.

G.1 REPLICATION PROTOCOL

  • The Gemma replication preserves the learning episodes, data split, outcomes, utility, and conceptual program library, while retraining the compiler from Gemma records alone.The protocol uses 400 training, 100 validation, and 100 test episodes; test geometry averages three seeds.Source: §G.1 · pp. 19–20, G EXPERIMENT 6: CROSS-BACKBONE… · p. 19

G.2 BACKBONE-SPECIFIC OPTIMIZATION CALIBRATION

  • No candidate met the prespecified boundedness gate of .80, leaving automatic calibration formally inconclusive; gradient accumulation 1 was then selected after inspecting the tradeoff.The schedule is fixed for subsequent Gemma adaptations, but calibration does not guarantee every objective is equally learnable under every primary candidate.Source: §G.2 · pp. 20–21

G EXPERIMENT 6: CROSS-BACKBONE REPLICATION ON GEMMA

  • Under leave-one-family-out evaluation, ranking fidelity declines across all five held-out objectives, with particularly large degradation for factual, lexical, and procedural episodes.The figure compares Spearman correlation and pairwise ranking accuracy for represented versus unseen learning families.Source: Figure 11 · p. 20, Figure 12 · p. 21

G.3 GEMMA ADAPTATION GEOMETRY

Gemma retains episode-level variation in preferred adaptation programs, leaving measurable headroom beyond fixed policies. The compiler captures some of this variation but does not outperform the fixed defaults on average.

G.3 GEMMA ADAPTATION GEOMETRY

  • Held-out test-set profiles vary across learning objectives and programs, and a family’s highest-mean-utility program can differ from the episode-wise oracle-optimal program.Profiles average three adaptation seeds across 20 episodes per objective; A, T, B, and P denote acquisition, transfer, boundedness, and preservation, while U is their balanced mean.Source: §G.3 · p. 21, Table 13 · p. 23

G.4 SELECTION HEADROOM

  • Across 100 test episodes, the episode-wise oracle achieves .461 mean utility versus .436 for global-fixed and .438 for objective-fixed selection.Regrets are .025 and .023, respectively; on causal episodes, middle is oracle-optimal more often than full-stack (.55 versus .30), while full-stack has slightly higher mean utility (.388 versus .387).Source: §G.4 · p. 22, Table 14 · p. 23

G.5 ORACLE-PROGRAM HETEROGENEITY

  • Oracle-optimal programs are distributed across full, middle, late, and early adaptation at 40%, 31%, 23%, and 6%, with only 2% exact top ties.The best-versus-second-best utility gap averages .039 and has a .023 median.Source: §G.5 · p. 22

G.6 COMPILER SELECTION ON GEMMA

  • The compiler reaches .435 mean utility versus .461 for the episode-wise oracle, .438 for objective-fixed selection, and .436 for the global-fixed policy.It recovers an oracle-optimal program on 42% of episodes and an oracle top-two program on 73%.Source: §G.6 · pp. 22–23

G.7 WHERE DOES GEMMA SELECTION FAIL?

Gemma preserves episode-specific selection headroom, but direct argmax selection can respond to small predicted advantages and underperform strong defaults. Across backbones, useful compilation therefore depends on both available headroom and reliable discrimination among programs.

G.7 WHERE DOES GEMMA SELECTION FAIL?

  • Procedural reasoning accounts for approximately 79% of the compiler’s utility deficit versus objective-fixed selection, while its lexical deficit is partly offset by causal-episode gains.The compiler selects middle adaptation on 12 of 20 procedural episodes, reducing mean utility from .309 under the fixed full-stack policy to .297.Source: §G.7 · p. 23
  • On seven procedural episodes where middle is selected but full-stack is oracle-optimal, the predicted advantage averages .0027 while realized regret averages .050.The current rule acts on any predicted utility advantage, regardless of its magnitude.Source: §G.7 · pp. 23–24, Table 17 · p. 25

G.8 RELATIONSHIP TO THE LLAMA RESULTS

  • Gemma preserves episode-level oracle variation, but unlike Llama, its deterministic argmax compiler does not outperform fixed policies when small predicted advantages prompt costly departures from defaults.Objective-conditioned defaults differ across backbones except for causal mapping; Gemma’s results suggest compilation also requires calibrated discrimination between candidate programs.Source: §G.8 · p. 24, Table 18 · p. 25
Loading 2609.37371v1…