PaperlayerView exact revision on arXiv

Source-linked AI summary

A helps B while B hurts A: directed transfer in instruction-tuning mixture

Nima H. Siboni, Vahid Rostami

arXiv:2609.39702v1cs.AIcs.CLcs.LG

TL;DR

Choosing instruction-tuning tasks under a fixed budget is costly, while common count and similarity heuristics assume transfer is nonnegative or symmetric. The paper fits a signed transfer map from mixtures that withhold the target. Source identity predicts held-out mixture accuracy better than mixture size, and map-selected mixtures improve reasoning-target accuracy over uniform training.

  • Problem

    Under a fixed example budget, practitioners must choose task types, but testing each mixture costs a fine-tuning run and common heuristics assume transfer is nonnegative or symmetric.

    Source: §1 · p. 1

  • Method

    The transfer map estimates signed effects of sources on held-out targets from balanced mixtures generated from one corpus, then selects sources with positive effects.

    Source: §1 · pp. 2–3

  • Results

    Source identity predicts held-out mixture accuracy better than mixture size, and on Qwen3-32B the map’s mixture raises multi-hop accuracy from 22% to 34%.

    Source: §9 · p. 9

  • Takeaways & Limitations

    A mixture selected from the 32B map beats training on all source tasks at each smaller model size tested.

    Source: §9 · p. 9

    Effects are measured by automated scoring against GPT-5-mini-generated gold answers, not human judgments of answer quality.

    Source: §8 · p. 9

Abstract

from arXiv · show

Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choice costs a fine-tuning run. Common heuristics add more source tasks or pick sources similar to the target. The first assumes transfer is never negative; the second, that it is symmetric. We show that both assumptions fail: task $A$ can help task $B$ while $B$ hurts $A$, so helpfulness is a signed property of ordered source--target pairs. We introduce the transfer map, a signed estimate of how much each source helps or hurts each held-out target. We fit the map in hundreds of fine-tuning runs on Qwen3 and Mistral models from 0.6B to 32B parameters, with all sources drawn from one corpus and no training examples from the target. The map predicts a held-out target's accuracy on unseen mixtures: recorded before those runs, its predictions have less than half the error of a mixture-agnostic baseline. The map is specific to its target and corpus but transfers across model scale: a mixture selected in advance at one size beats training on all source tasks at every other size we tested. Transfer is thus a property of the data. The map selects the tasks that help and drops the one that interferes: accuracy on the reasoning targets (causal explanation, multi-hop questions and methodological critique) rises by up to 14 percentage points over training on all source tasks.

1 INTRODUCTION

Choosing instruction-tuning tasks under a fixed budget is costly to test, and common count and similarity heuristics assume transfer is nonnegative or symmetric. The paper measures signed, directed source effects and finds task identity—not simply mixture size—matters for held-out targets.

1 INTRODUCTION

  • With a fixed example budget, adding task types displaces examples of others, and testing each candidate mixture requires a fine-tuning run.Seven source types already yield 127 possible mixtures.Source: §1 · p. 1
  • The transfer map estimates each source’s signed effect on a held-out target using balanced mixtures that withhold that target.All eight task types are generated from one neuroscience corpus.Source: §1 · p. 2
  • Task pairs can help in one direction and hurt in the other, so a symmetric similarity score cannot represent transfer.On Qwen3-32B, definitions raise multi-hop accuracy by 5.3 pp, while adding multi-hop lowers definitions accuracy by 1.9 pp.Source: §1 · pp. 1–3
  • Across model families, the map selects mixtures that drop interfering tasks and improve reasoning-target accuracy over training on all source tasks.On Qwen3-32B, it doubles critique accuracy to 16% and raises multi-hop accuracy to 34%.Source: §1 · p. 3

2 RELATED WORK

Prior work studies transfer, task counts, target-directed selection, mixture optimization, and generated domain supervision. This paper distinguishes its setting by measuring directed effects among task types generated from one corpus in fixed-budget mixtures that withhold the target.

2 RELATED WORK

  • Prior transfer studies report asymmetry and even sign differences, but typically vary datasets or train sources separately rather than measuring fixed-budget mixtures from one corpus.The paper’s design aims to separate task type from data-source differences.Source: §2 · p. 3
  • Evidence that more tasks help establishes only that the average added task helps, not that every task should be included.A selected subset can outperform the full set.Source: §2 · p. 3
  • Similarity selectors measure resemblance symmetrically, whereas the transfer map estimates signed benefit from fine-tuning runs that withhold the target.LESS instead estimates example-level benefit with a signed, first-order gradient approximation.Source: §2 · pp. 3–4
  • Mixture optimization and task-similarity methods address proportions or symmetric task relations, while this paper selects sources for one withheld target.Skill-It is a directed skill graph, and prior work also studies cross-model transfer of selected data.Source: §2 · p. 4
  • Generated domain-supervision pipelines commonly train on all generated task types, while target-type generation assumes the target task is known.Dropping task types may cost ability outside the target.Source: §2 · p. 4

3 METHOD

The study generates eight task types from specialized corpora and probes fixed-budget mixtures that withhold each target. It fits an additive transfer map and uses its source effects to select mixtures.

3 METHOD

  • The study compares Qwen3-32B and Mistral-24B on the same neuroscience corpus, plus Qwen3-32B on a PubMed corpus-shift control.The neuroscience corpus contains 110 papers, split into 88 training and 22 evaluation papers.Source: §3 · p. 4
  • A structured generator renders passages into eight task types, with causal explanation, multi-hop questions, and methodological critiques as the reasoning targets.These targets are described as lying outside an untuned model’s reach.Source: §3 · p. 4
  • Balanced probe designs withhold targets and vary source mixtures; each run uses 6,000 examples divided equally among sources and excludes the target task.Across designs, each reasoning target is withheld in 53 mixtures, and all seven other tasks appear as candidate sources.Source: §3 · pp. 4–5
  • Open-task accuracy is scored with sentence-embedding similarity, while closed tasks use exact or option-set matching.The open-task rule counts an item correct when whole-answer or averaged sentence-match similarity exceeds 0.85.Source: §3 · p. 5
  • For each target, an additive regression estimates centered source-substitution effects, where positive entries help and negative entries hurt relative to an average source.Distinct mixtures are weighted once, rather than counting retrainings repeatedly.Source: §3 · p. 5
  • The selection rule keeps sources with positive uncentered coefficients and drops the rest; prediction baselines use no mixture effect or mixture size alone.Competing selectors include uniform training, single-source scores, rankers, and embedding similarity.Source: §3 · p. 5

4 THE TRANSFER MAP

The transfer map captures signed, directed effects: task-pair directions differ, and some sources strongly hurt particular targets. Across unseen mixtures, source identity predicts accuracy better than mixture size, whose apparent effect is entangled with average source effects.

4.1 TRANSFER IS DIRECTED AND SIGNED

  • In 14, 13, and 9 task pairs across the three conditions, directional differences exclude zero; opposite-sign effects with intervals excluding zero occur in 3, 4, and 0 pairs.On Qwen3-32B, DEF→MHOP is +5.27 pp, while MHOP→DEF is −1.24 pp as a centered substitution effect.Source: §4.1 · pp. 5–6
  • ERR is the most negative source for MHOP on Qwen3-32B and Mistral-24B, at −9.72 and −9.55 pp relative to an average source.These effects do not track supervised token volume.Source: §4.1 · p. 6, Figure 2 · p. 6

4.2 SOURCE IDENTITY, NOT MIXTURE SIZE, PREDICTS ACCURACY

  • Source identity explains 0.768, 0.746, and 0.557 of within-target variance, while a linear mixture-size term explains 0.114, 0.003, and 0.153.Adding all 21 pairwise source interactions improves the first two conditions but lowers the third.Source: §4.2 · p. 6
  • A mixture-size slope reveals only the average source effect, not whether every source helps; reasoning-target accuracy can fall for definitions as mixture size increases.Even where the average source helps, dropping negative-effect sources at a fixed budget can raise target accuracy.Source: §4.2 · pp. 6–7

5 PREDICTING UNSEEN MIXTURES

The transfer map predicts the effect of moving from the uniform mixture to its selected mixture on held-out targets, using predictions recorded before the test runs. It outperforms the supplied baselines, though two small predicted gains realized as slight losses.

5 PREDICTING UNSEEN MIXTURES

  • The comparison used a 6,000-example budget, and predictions were recorded before training the 54 test runs.Both mixtures were compared at three data-order seeds in each of nine cells; one cell was vacuous, leaving n = 8.Source: §5 · p. 7
  • 2.31 pp MAE for recorded predictions, versus 6.20 pp for the zero-effect baseline and 9.31 pp (linear) and 9.20 pp (quadratic) for count baselines.The best constant prediction chosen after seeing outcomes had 4.80 pp MAE.Source: §5 · p. 7
  • Six of eight realized effects were gains, reaching 13.76 pp over three seeds; two small predicted gains realized as slight, statistically unresolved losses.The two losses had 95% confidence intervals that did not exclude zero, while all six gains did.Source: §5 · p. 7
  • On critiques, the uniform mixture gained only about 1 pp over the untuned model in both neuroscience conditions, while the selected mixture doubled accuracy.In each non-vacuous MHOP and ERR cell, the target’s interferer carried 75 to 100% of the predicted gain.Source: §5 · p. 7

6 THE MAP AGAINST OTHER SELECTORS

The map-selected mixture generally outperforms similarity-based selection on reasoning targets, while comparisons with selectors that rank measured target-specific effects are closer. Some one-run comparisons remain unresolved.

6 THE MAP AGAINST OTHER SELECTORS

  • The similarity selector lost 7.47 to 9.68 pp on three of four MHOP and ERR cells and 3.11 pp on the fourth; the table reports map-minus-competitor gaps.Positive table entries favor the map, and competitors are compared at their own mixture sizes d.Source: §6 · p. 7, Table 1 · p. 8
  • The TaskShop-style ranker stayed within 2.73 pp of the map on six neuroscience cells, while Skill-It-style comparisons ranged from a 1.45 pp edge to a 7.91 pp deficit.Each comparison used one run per mixture, so these margins and edges were not resolved.Source: §6 · p. 7
  • The map stayed ahead of the similarity selector at d = 3, its matched d, and d = 5 on all four MHOP and ERR cells, by 3.11 to 15.91 pp.At d = 3, similarity selection still trailed by 3.41 to 9.68 pp despite training at least as many examples per source.Source: §6 · p. 8
  • A similarity score does not encode transfer sign, and at its matched d the similarity mixture included the target’s interferer in four of eight non-vacuous cells.Both large jumps in the d-sweep occurred on MHOP cells when widening to d = 5 admitted ERR.Source: §6 · p. 8

7 WHERE THE MAP TRANSFERS: TARGET, CORPUS, FAMILY, AND SCALE

The map’s selection is target-specific and transfers across model scale, but the reported cross-corpus comparison does not isolate corpus effects. Selecting for a target also trades off performance on non-target tasks.

7 WHERE THE MAP TRANSFERS: TARGET, CORPUS, FAMILY, AND SCALE

  • Across eight non-vacuous cells, the selected mixture scored 2.40 pp below the uniform mixture on the non-target macro and 2.70 pp above the single best source.The rule keeps a different set of sources for each target in the same condition.Source: §7 · p. 8
  • The Qwen3-32B and Mistral-24B maps agreed at r = 0.880, while the PubMed map agreed with them at only r = 0.279 to 0.545.The PubMed comparison also changed document granularity, pool size, and evaluation size, so it does not isolate corpus effects.Source: §7 · p. 8
  • Across Qwen3 sizes from 0.6B to 8B, maps agreed with the 32B map at r = 0.757 to 0.918; DEF helped MHOP by 4.4 to 5.8 pp and ERR hurt it by 9.3 to 11.6 pp.Eleven of 21 source-to-reasoning-target entries kept their sign across all five sizes and Mistral-24B.Source: §7 · p. 8
  • A selection recorded from the 32B map beat the uniform mixture in 11 of 12 other-size cells by a mean of +4.44 pp.The 32B map predicted these effects with 2.28 pp MAE versus 4.64 pp for the mixture-agnostic baseline; the mean gain was positive at every size.Source: §7 · pp. 8–9
  • A 24-run probe recovered 96% of the full-probe gain on Qwen3-32B and the full gain on Mistral-24B.The gains were +4.83 pp versus +5.03 pp on Qwen3-32B, and +5.88 pp versus +5.82 pp on Mistral-24B.Source: §7 · p. 9

8 LIMITATIONS

The reported transfer effects are bounded by automated evaluation, source-passage and dose confounds, a small reused-cell prediction test, and a set-based abstraction that omits training order.

8 LIMITATIONS

  • Effects are measured by automated scoring rules against GPT-5-mini-generated gold answers; human evaluation is needed to determine whether they hold beyond the rule.Restricting to single-sentence answers preserved the two largest effects but two others vanished or reversed.Source: §8 · p. 9
  • The design does not isolate passage content or source dose: task passages partly overlap, while narrower mixtures train more examples per source.Passage-matched and dose-matched controls would address these limits; supervised token volume did not rank the effects.Source: §8 · p. 9
  • Prediction tests cover few cells and reuse targets and conditions, making their intervals descriptive; scale transfer was tested within one model family.The PubMed condition had about a tenth of the evaluation items, so its cells were unresolved rather than null.Source: §8 · p. 9
  • The set-based mixture abstraction omits training order, which changed target accuracy when an interferer was among the sources.The authors treat this as a boundary of the set abstraction.Source: §8 · p. 9

9 CONCLUSION

The transfer map identifies source–target effects that can be negative and asymmetric, and its source choices predict and improve held-out target accuracy. It transfers across model scales but changes with the corpus, so each corpus needs its own probe.

  • Across both neuroscience conditions, an interfering task cancels most gains on two of three reasoning targets, and task pairs can help in one direction while hurting in the other.Source: §9 · p. 9
  • Source choice predicts accuracy, whereas task count does not because mixture size cannot be separated from average source effect.Source: §9 · p. 9
  • On Qwen3-32B, the map-selected mixture raises multi-hop accuracy from 22% to 34% and doubles critique accuracy.Source: §9 · p. 9
  • Map predictions for unseen mixtures have less than half the error of a mixture-agnostic baseline in both prediction tests.Source: §9 · p. 9
  • At smaller model sizes, mixtures selected by maps fitted from 24 runs—and by the 32B map—beat training on all source tasks.Source: §9 · p. 9
  • Transfer held across two model families and a 53-fold size range but changed with the corpus; the smaller second-corpus pools left the probe’s gain unresolved.A selection serves one target at some cost to other tasks, and a new corpus needs its own probe.Source: §9 · p. 10

AI USE STATEMENT

The recorded predictions retain within-target signal, while identification analyses show that mixture-size effects reflect average source effects and that pairwise interactions do not consistently improve selection. Transfer remains directional, with substantial asymmetries across model and corpus conditions.

AI USE STATEMENT

  • GPT-5-mini generated instruction–response pairs for all eight task types from corpus passages, and AI assistants including Claude helped with code and text drafting and editing.The authors take responsibility for the final content, claims, and artifacts.Source: AI USE STATEMENT · p. 10

A RECORDED PREDICTIONS AND WHERE THE MAP HOLDS

  • Within-target predictions track realized effects at r = 0.951 with MAE 0.79 pp, versus 2.26 pp for a per-target constant, though centering makes this check non-independent.Three targets over eight cells leave five effective degrees of freedom, and the two-cell ERR group anticorrelates by construction.Source: A RECORDED PREDICTIONS AND WHERE THE… · p. 15

B IDENTIFICATION

  • Mixture size cannot be separated from average source effect when each source has its own effect, and higher-order size components likewise alias with mean interaction terms.The data identify heterogeneous nonadditivity rather than a generic size component without an external restriction.Source: B IDENTIFICATION · p. 18
  • Pairwise source interactions improve leave-one-mixture-out prediction on both neuroscience conditions, but do not make a consistently better selector.Selecting from the pairwise fit changes out-of-fold gains from +5.03 to +5.37 pp on Qwen3-32B, +5.82 to +4.93 pp on Mistral-24B, and −0.51 to −0.28 pp on PubMed.Source: B IDENTIFICATION · p. 18, Table 6 · p. 18
  • The six-source dose curve omits the target and its design partner; for Qwen/MHOP it gives 32.1% versus 22.0% for the seven-source uniform run.The dose curve is not a ceiling for the uniform mixture; selection comparisons should match budget and candidate sources.Source: B IDENTIFICATION · p. 18
  • Across 12 cells, the mixture-size slope tracks the mean source effect at r = 0.898, with a 0.46 pp mean absolute difference.The slope can be negative, and the dose curve is an unweighted marginal over mixtures that are not balanced across sizes.Source: B IDENTIFICATION · p. 18, Table 7 · p. 19, Figure 4 · p. 19

C DIRECTEDNESS OF THE MAP

  • The study tests symmetry itself using bootstrap intervals over distinct mixtures; strict opposite-sign pairs include three Qwen3-32B pairs and four Mistral-24B pairs.On Mistral, one counted pair has small entries (−0.32 and +0.20 pp) and involves near-ceiling MATCH.Source: C DIRECTEDNESS OF THE MAP · p. 20, Table 8 · p. 20
  • The largest asymmetry is WHY→MHOP minus MHOP→WHY: −8.16 pp on Qwen3-32B, −7.09 pp on Mistral-24B, and −4.81 pp on PubMed.The corresponding 95% intervals exclude zero in all three conditions.Source: C DIRECTEDNESS OF THE MAP · p. 20, Table 9 · p. 21

D SELECTOR COMPARISON, FULL SWEEP

The full selector sweep compares map-selected mixtures with alternatives at each competitor’s own mixture size, measuring both withheld-target accuracy and a non-target retention macro. Similarity-based selection can miss helpful sources, while selected mixtures are generally narrower than the uniform default.

D SELECTOR COMPARISON, FULL SWEEP

  • The sweep reports map-minus-competitor accuracy on the withheld target and the non-target macro; the latter summarizes six non-target, non-MATCH tasks as a retention measure.Table 10 lists competitors at each trained mixture size under one seed; positive values favor the map-selected mixture.Source: D SELECTOR COMPARISON, FULL SWEEP · p. 22, Figure 5 · p. 22, Table 10 · p. 22
  • On Mistral/MHOP, embed_sim trails the map by 9.68 pp despite excluding ERR, because it omits DEF, the target’s largest positive source.On that cell, TaskShop at d = 5 also trails by 8.42 pp after adding two sources the map drops.Source: D SELECTOR COMPARISON, FULL SWEEP · pp. 22–23, Table 10 · p. 22
  • Skill-It edges the map by at most 1.45 pp on three cells but loses 17.54 pp on PubMed/WHY, where its mixture includes the target’s interferer.The comparisons use each competitor at its own mixture size.Source: D SELECTOR COMPARISON, FULL SWEEP · pp. 22–23
  • The map-selected mixture keeps 3.2, 3.0 and 4.1 sources on average across three conditions, versus seven for the uniform default.At a fixed 6,000-example budget, narrower selections provide 1.2 to 2.3× as many examples per source in eight cells.Source: D SELECTOR COMPARISON, FULL SWEEP · p. 22

E TRAINING-ORDER EXPERIMENT

The experiment tests whether the transfer map predicts target accuracy changes from training the better source last. The predicted direction is confirmed in three of seven rows, but the recorded magnitude law fails and the map does not generally predict order effects.

E TRAINING-ORDER EXPERIMENT

  • The experiment compares two-source blocks in both orders using bit-identical examples, budgets, steps and seeds, isolating order within each paired contrast.The realized effect is better-source-last accuracy minus better-source-first accuracy; the recorded prediction is half the map-effect difference between sources.Source: E TRAINING-ORDER EXPERIMENT · p. 24, Table 11 · p. 24
  • Three of seven rows are confirmed, three intervals cover zero, and one is reversed; the three confirmations are one source pair repeated at three model sizes.For P1, training DEF last gains +7.75 pp on MHOP over the shuffled default at the same training cost.Source: E TRAINING-ORDER EXPERIMENT · p. 24, Table 11 · p. 24
  • The magnitude law is refuted: five pairs’ intervals exclude their predicted values, and the map does not generally predict training-order effects.Under a constant learning rate, the P1 effect grows from +9.55 to +15.12 pp; the authors report that cosine decay damps recency rather than causing the effect.Source: E TRAINING-ORDER EXPERIMENT · p. 24
  • The order study uses one corpus, two-source blocks and three seeds; only P1 was tested at multiple model sizes or learning-rate schedules.The authors expect finer interleaving to produce smaller effects.Source: E TRAINING-ORDER EXPERIMENT · p. 24

F EXPERIMENTAL DETAILS

The experimental details describe probe designs, training and data conditions, then audit the scoring rule and output-format sensitivity. The reported accuracy calculations reproduce from stored generations, while alternative scoring and format checks qualify how several effects should be interpreted.

F EXPERIMENTAL DETAILS

  • Four balanced probe designs withhold disjoint target pairs and vary the remaining six sources across mixture sizes d = 1, . . . , 6; each condition trains 94 distinct mixtures.Across designs, 53 mixtures withhold each reasoning target, with all seven other tasks appearing as sources.Source: F EXPERIMENTAL DETAILS · p. 25
  • The study rests on 751 fine-tuning runs, each training on 6,000 examples plus evaluation over the full held-out split.Training uses response-masked causal language modeling; each d-source run contributes 6000/d examples per source.Source: F EXPERIMENTAL DETAILS · pp. 25–26
  • The PubMed condition repeats examples from at least one source in every run, and 11.4% of unique validation DEF questions appear verbatim in training, though no scored DEF run trains on DEF.The overlap bounds what DEF means as a source diagnostic; it cannot reach a withheld DEF target.Source: F EXPERIMENTAL DETAILS · p. 26

G THE SCORING RULE, AUDITED

  • Recomputing effects from retained per-item outputs reproduces all 54 reported accuracies to within 7 × 10^-15 pp.The effects are the scoring rule applied to stored generations, not a rerun.Source: G THE SCORING RULE, AUDITED · p. 26
  • Under rescoring at τ = 0.80 or τ = 0.90, recorded predictions have MAE 2.81 and 2.12 pp, versus 6.82 and 3.82 pp for the mixture-agnostic baseline.At τ = 0.85, the comparison is 2.31 against 6.20 pp.Source: G THE SCORING RULE, AUDITED · p. 26
  • With answers restricted to one sentence, six of eight effects keep their sign and the two largest remain +11.44 and +13.22 pp, against +11.89 and +13.76 pp.Mistral/ERR changes from +8.56 to −0.10 pp and PubMed/MHOP from +3.10 to −2.96 pp; in both, one run answers fewer than 6% of items in one sentence.Source: G THE SCORING RULE, AUDITED · p. 27

Related papers

Loading 2609.39702v1…