PaperlayerView exact revision on arXiv

Source-linked AI summary

Soft Strategy Selection for Batch-Mode Active Learning

Rushil Gupta, Romain Lopez

arXiv:2610.05544v1cs.LGcs.AI

TL;DR

Active-learning strategy performance varies across settings, and existing selection methods choose one strategy per round, a poor fit for few-round, large-batch experiments. FractAL allocates each batch across strategies using influence-based rewards and online mirror descent. Across seven setups, it is the only method whose worst-case performance exceeds random sampling.

  • Problem

    Strategy performance varies across domains, datasets, and model architectures, while existing selection methods choose a single strategy at each round.

    Source: §1 · pp. 1–2

  • Method

    FractAL allocates each batch across multiple strategies, estimates strategy rewards through influence-based attribution, and updates budget shares with online mirror descent.

    Source: §1 · p. 2

  • Results

    Across seven setups, FractAL is the only method whose worst-case performance exceeds random sampling; every baseline falls below random sampling on at least one setup.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Across classification, regression, and genetic perturbation screens, FractAL up-weights the strongest strategies and down-weights poorly performing ones.

    Source: §5 · p. 9

    Influence functions are known to be fragile in deep networks, and alternative data-valuation methods may stabilize rewards under more complex surrogates.

    Source: §5 · p. 10

Abstract

from arXiv · show

Real-world deployment of active learning typically forces practitioners to choose an acquisition strategy before any data is labeled. This is a daunting task: strategy performance varies widely across settings (e.g. datasets, surrogate models) and cannot be assessed without deployment. Existing strategy selection methods explore one strategy from a portfolio at each round and identify the optimal one using bandit feedback or model retraining. Many acquisition rounds are therefore spent exploring strategies rather than collecting the most informative data. Such overhead is a major barrier to AL-driven design of high-throughput experiments, such as genetic perturbation screens and directed evolution, where AL runs consist of only a few rounds with large batch sizes. This regime permits a natural alternative: acquiring data using multiple AL strategies within a single batch. We refer to this as soft strategy selection and introduce FractAL, a method specifically designed for this task. FractAL infers a per-strategy reward using influence-function-based data attribution, which requires no additional retraining, and then computes budget shares for each strategy in the portfolio using online mirror descent. We benchmark FractAL across 7 setups spanning classification, regression, and genetic perturbation effect prediction. The results highlight that strategy selection is a hard problem: every existing method performs worse than random sampling on at least one setup. FractAL, however, matches or outperforms every baseline, including random sampling, on all 7 setups. Its allocations concentrate budget on the strongest strategies in the portfolio while pruning the weakest. FractAL is therefore a reliable choice for real-world deployments, where the optimal strategy is unknown in advance, an important step towards making AL practical for high-throughput experiments and modern scientific discovery.

1 INTRODUCTION

Active-learning strategies perform inconsistently across domains, datasets, and model architectures, while high-throughput experiments make committing a large batch to one strategy risky. FractAL instead allocates each batch across strategies and updates those shares using influence-based rewards; the paper reports that it outperforms random selection across all evaluated setups.

1 INTRODUCTION

  • Strategy performance varies across domains, datasets, and model architectures, so no single strategy performs well across settings.Strategies target uncertainty, batch diversity, pool representativeness, or combinations of these.Source: §1 · p. 1
  • In high-throughput biological experiments with few rounds and large batches, relying on a single strategy per round is described as unnecessarily risky.These assays can provide results for thousands of inputs at once, while experimental time remains dominated by biological timescales.Source: §1 · p. 2
  • Soft strategy selection allocates fractions of each round’s budget across portfolio strategies, forming strategy-specific sub-batches.Table 1 compares related work on batch sampling, fractional budget allocation, and operation without additional model training.Source: §1 · p. 2, Table 1 · p. 2
  • FractAL estimates strategy-wise rewards by attributing validation-risk changes with influence functions, then updates budget shares using online mirror descent.The approach is designed for soft strategy selection and avoids additional retraining for reward estimation.Source: §1 · p. 2
  • Across the evaluated datasets, FractAL is the only strategy-selection method reported to outperform random selection on all of them.Its allocations also concentrate budget on the strongest strategy in each portfolio while pruning the weakest.Source: §1 · p. 2

2 BACKGROUND

The background reviews adaptive strategy selection and influence functions. Existing strategy-selection approaches include bandit, retraining, and differentiable-search methods, while influence functions provide a basis for retrospective, strategy-wise reward estimates.

2 BACKGROUND

  • Figure 1 presents performance gain over random sampling for six adaptive strategy-selection methods across seven setups.Source: Figure 1 · p. 2

2.1 ADAPTIVE STRATEGY SELECTION FOR ACTIVE LEARNING

  • Adaptive strategy selection methods fall into bandit-based, retraining-based, and differentiable-search paradigms.Source: §2.1 · p. 2
  • Bandit methods select one strategy per iteration, which in batch active learning commits the round to one strategy and prevents within-batch strategy-level reward attribution.ALBL uses Exp4.P with importance-weighted accuracy rewards.Source: §2.1 · p. 3
  • SelectAL selects a strategy based on risk increases after retraining, and evaluating K strategies over T rounds requires O(KT) model retrainings.Its original specification is limited to K = 2 strategies, though the selection principle generalizes to larger portfolios.Source: §2.1 · p. 3
  • AutoAL uses bi-level optimization with two co-trained auxiliary networks, adding hyperparameters and computational overhead per iteration.Source: §2.1 · p. 3

2.2 INFLUENCE FUNCTIONS

  • Influence functions estimate how removing a training point would change validation loss by approximating parameter perturbations and propagating them with a first-order Taylor expansion.Earlier forward-looking active-learning uses require pseudo-labels for unmeasured inputs; this work instead uses retrospective influence on measured training points to estimate strategy rewards.Source: §2.2 · p. 3

3 FRACTAL: FRACTIONAL ACTIVE LEARNING

FractAL attributes validation-risk changes to strategies’ sub-batches and uses those rewards to update fractional budget allocations. Its allocation procedure smooths rewards, applies online mirror descent, and ensures every strategy retains a nonzero share.

3 FRACTAL: FRACTIONAL ACTIVE LEARNING

  • FractAL has two components: influence-based strategy-wise reward attribution instead of retraining, followed by online mirror descent to update allocations on the probability simplex.Source: §3 · p. 3

3.1 STRATEGY-WISE INFLUENCE ATTRIBUTION

  • Each strategy’s sub-batch is assessed through its contribution to validation risk after the model is trained on the augmented data.The strategy-wise influence vector differentiates validation risk with respect to reweighting acquired points by strategy; its component gives the first-order risk change under upweighting that strategy’s sub-batch.Source: §3.1 · pp. 3–4
  • The influence-based reward’s sign indicates whether a strategy’s sub-batch reduced or increased validation risk, and its magnitude reflects the effect size.The stated decomposition assumes a twice continuously differentiable loss, a training-risk minimizer, and a positive-definite Hessian; for deep networks, the Hessian is computed on the last layer and damped.Source: §3.1 · p. 4

3.2 FRACTIONAL PORTFOLIO ALLOCATION

  • FractAL smooths noisy rewards with an exponentially weighted moving average, then uses a KL-regularized update and mixes the result with a uniform allocation.The mixing sets p_min = 1/B so every strategy has a nonzero allocation; strategies select disjoint sub-batches sequentially in decreasing allocation order.Source: §3.2 · p. 4

4 EXPERIMENTS

Across seven classification, regression, and genetic perturbation setups, FractAL is the top-ranked method and the only one with positive worst-case gain over random. The experiments also examine how well methods allocate budget, alongside ablations and runtime.

4 EXPERIMENTS

  • Experiments cover seven classification, regression, and genetic perturbation setups, comparing adaptive methods and Equal Split while assessing pure-strategy performance and allocation diagnostics.Performance is averaged over five data splits and ten seeds per split.Source: §4 · pp. 4–5

4.1 CLASSIFICATION BENCHMARK: CIFAR-10

  • FractAL reaches 70.2 ± 0.2 test accuracy in C1 and 69.8 ± 0.2 in C2, leading adaptive methods in both and uniquely surpassing Random in C1.In C1, it shifts budget from CoreSet toward ProbCover; in C2, it significantly exceeds Random and prunes CoreSet to zero.Source: §4.1 · pp. 6–7, Table 2 · p. 6, Table 4 · p. 7

4.2 REGRESSION BENCHMARKS

  • FractAL has the lowest test MSE among selection methods on all three regression datasets and leads both allocation diagnostics on KEGG and SARCOS, tying on Diamonds.Strategy rankings vary: CoreSet is among the strongest across these regression setups despite performing worst on CIFAR-10, and ProbCover varies across the regression datasets.Source: Table 3 · p. 6, §4.2 · p. 7

4.3 DESIGN OF HIGH-THROUGHPUT GENETIC PERTURBATION SCREENS

  • FractAL attains the highest test AUPRC among adaptive methods on both perturbation tasks and steadily concentrates budget on CoreSet while downweighting Random.On B2, it is the only method to significantly outperform chance on Worst-Set Overlap.Source: §4.3 · p. 8, Table 5 · p. 8, Figure 3 · p. 9

4.4 AGGREGATE RESULTS, ABLATIONS, AND RUNTIME

  • FractAL achieves the highest mean gain (0.689), the only positive worst-case gain (0.222), and the best mean rank (1.14 of 7), ahead of SelectAL (3.14).Every other method falls below Random on at least one setup.Source: §4.4 · pp. 8–9
  • Soft fractional allocation beats hard allocation on five of seven setups and ties on two; mirror descent outperforms an additive update, while influence-based rewards match retraining-based rewards (Spearman’s ρ = 0.7).Performance is robust to the smoothing parameter α.Source: §4.4 · p. 9
  • FractAL completes a CIFAR-10 active-learning run 2.5× faster than SelectAL and 5× faster than AutoAL, with a 0.5-minute allocation update versus 19.8 minutes for SelectAL.Its single inverse-Hessian-vector product per round does not scale with portfolio size K, unlike retraining-based attribution.Source: §4.4 · p. 9

5 DISCUSSION

FractAL’s allocation advantage is clearest in iteration-scarce, large-batch settings, and its worst-case performance across seven setups exceeds random sampling. The discussion also flags influence-function fragility in deep networks as a direction for further investigation.

5 DISCUSSION

  • Figure 4 reports runtime for an AL run on C2.Source: Figure 4 · p. 9
  • Across seven setups, FractAL is the only method whose worst-case performance exceeds random sampling; every baseline falls below it on at least one setup.Its allocation advantage is clearest in the iteration-scarce, large-batch regime.Source: §5 · pp. 9–10
  • Influence functions may be fragile in deep networks, motivating investigation of alternative data-valuation frameworks.The discussion names Shapley values, gradient similarity, and Banzhaf values as possible alternatives.Source: §5 · p. 10

APPENDICES

The appendices collect proofs, implementation and experimental details, and supplementary results for FractAL. They also list the source datasets and provide runtime and allocation-dynamics material.

APPENDICES

  • The appendix overview also lists source links for CIFAR, KEGG, SARCOS, Diamonds, and a genome-scale T-cell perturb-seq dataset.Source: APPENDICES · p. 14
  • Appendices A and B cover the strategy-wise influence-score derivation, practical influence estimation, and construction of the composite batch.Source: APPENDICES · p. 14
  • Appendices C–F describe the strategy portfolio, selection baselines, allocation diagnostics, and setup-specific datasets, surrogate models, and hyperparameters.Source: APPENDICES · p. 14
  • Appendices G–I provide aggregate method results, per-setup results with pure strategies, and ablations of FractAL components.Source: APPENDICES · p. 14
  • Appendices J and K report wall-clock runtime by method phase and budget-allocation dynamics across methods and setups.Source: APPENDICES · p. 14

A PROOF OF PROPOSITION 1

The proof derives the strategy-wise influence component by adapting the standard influence-function derivation to a portfolio-level perturbation. It first obtains parameter sensitivity, then applies the chain rule to validation risk.

A PROOF OF PROPOSITION 1

  • The derivation assumes a twice continuously differentiable loss, a training-risk minimizer, and a positive-definite Hessian; these assumptions do not hold exactly for deep networks trained by SGD with early stopping.The paper points to Appendix B for practical workarounds.Source: A PROOF OF PROPOSITION 1 · p. 15
  • The proof adapts standard influence-function analysis by assigning one perturbation component to each strategy rather than each training point.Source: A PROOF OF PROPOSITION 1 · p. 15
  • The first step uses the implicit function theorem and a Taylor expansion around the fitted parameter to express its shift as a function of the perturbation.Each strategy’s perturbation scales that strategy’s gradient contribution.Source: A PROOF OF PROPOSITION 1 · p. 15
  • The second step differentiates validation risk with respect to each perturbation component and applies the chain rule to obtain the strategy-wise influence component.The resulting expression matches Equation 2.Source: A PROOF OF PROPOSITION 1 · pp. 15–16

B.1 PRACTICAL ESTIMATION OF INFLUENCE SCORES

This material explains practical influence-score estimation, composite-batch construction, the strategy portfolio and baselines, and allocation diagnostics. It also describes the assumptions and approximations behind the influence scores.

B.1 PRACTICAL ESTIMATION OF INFLUENCE SCORES

  • Influence-score estimation departs from the idealized proposition: training parameters are not exact risk minimizers, the Hessian may not be positive definite, and computation is restricted to the last layer.The authors compare the resulting scores with retraining-based rewards and report consistent strategy rankings (Spearman correlation 0.7–0.8 on CIFAR-10) and comparable downstream performance across seven setups.Source: §B.1 · p. 17

B.2 FORMING THE COMPOSITE BATCH

  • The composite batch combines strategy-specific sub-batches, with allocation-based ordering resolving duplicate proposals so each acquired point is attributed to one strategy.On CIFAR-10 C2, overlap resolution affected an average of 3 of 500 acquired points per round (0.6%).Source: §B.2 · p. 17

C ACTIVE LEARNING STRATEGIES

  • The portfolio includes general-purpose sampling strategies and classification-specific methods, including CoreSet, ProbCover, TypiClust, BADGE, LeastConf, and MaxEnt.CoreSet selects distant points, ProbCover targets distribution coverage, and TypiClust samples typical points from clusters.Source: C ACTIVE LEARNING STRATEGIES · p. 18

D.1 ALBL: ACTIVE LEARNING BY LEARNING; D.2 UPPER CONFIDENCE BOUND (UCB); D.3 SCRIBLE ALGORITHM

  • ALBL, UCB, and SCRiBLe choose strategies or allocations using bandit feedback, with rewards based on validation performance or loss.ALBL applies an exponential weight update, UCB uses reward means and uncertainty, and SCRiBLe explores the allocation simplex.Source: §D.1 · p. 19, §D.2 · p. 20, §D.3 · pp. 20–21

D.4 SELECTAL

  • SelectAL evaluates candidate strategies by removing a small strategy-selected subset of labeled points and measuring the resulting change in model performance.The removed subset is max(⌊0.05 · |Dt train|⌋, 1); this implementation evaluates all portfolio strategies.Source: §D.4 · p. 21

D.5 AUTOAL

  • AutoAL combines per-point strategy scores through a learned weighting, unlike FractAL’s fractional allocation into disjoint strategy sub-batches.Its bilevel optimization uses two co-trained networks and adds hyperparameters and per-round computational overhead.Source: §D.5 · pp. 21–22

E ALLOCATION DIAGNOSTICS

  • Allocation diagnostics compare methods’ allocations with sets of the best- and worst-performing pure strategies, allowing for multiple strategies with comparable performance.Set size is determined using overlap among mean ± one standard deviation performance bands; the authors describe this as a practical heuristic, not a formal hypothesis test.Source: E ALLOCATION DIAGNOSTICS · p. 23

F EXPERIMENTAL SETUPS

The experimental setups cover classification, tabular regression, and high-throughput perturbation prediction, with configurations summarized across datasets and methods. Each task uses task-specific inputs, surrogate models, and acquisition portfolios.

F EXPERIMENTAL SETUPS

  • The setup summary reports surrogate models, dataset split sizes, and batch sizes, with all methods and pure strategies sharing each setup’s configuration.The passage points to Table 7 for the configuration summary.Source: F EXPERIMENTAL SETUPS · p. 24, Table 7 · p. 24

F.1 CLASSIFICATION DATASET

  • CIFAR-10 experiments use a ResNet-18 surrogate and fixed 512-dimensional SimCLR embeddings for embedding-based strategies.The model is trained for 50 epochs per active-learning iteration, and the embeddings remain fixed throughout the run.Source: §F.1 · p. 24

F.2 REGRESSION DATASETS

  • KEGG, SARCOS, and Diamonds use a three-layer MLP, with processed input features serving as embeddings for CoreSet, ProbCover, and TypiClust.The regression portfolio excludes logit-based strategies such as MaxEnt, MinMargin, and LeastConf.Source: §F.2 · pp. 24–25

F.3 HIGH-THROUGHPUT PERTURBATION SCREENS

  • The perturbation setups use processed gene-expression measurements and 70-dimensional input embeddings to predict perturbation-associated log-fold changes.For CD4+ T cells, 8-hour log-fold changes form the input and 48-hour changes the target; both perturbation setups use Random, CoreSet, ProbCover, and TypiClust.Source: §F.3 · p. 25

G PERFORMANCE ACROSS SETUPS

Because the seven setups use different metrics, the paper compares methods using relative gain over Random and summarizes performance across setups. FractAL has the highest mean gain and the only positive worst-case gain.

G PERFORMANCE ACROSS SETUPS

  • The seven setups use accuracy, MSE, and AUPRC, so averaging raw performance differences across them would be meaningless.The paper reports mean, worst-case, and rank summaries instead.Source: G PERFORMANCE ACROSS SETUPS · p. 26, Table 8 · p. 26
  • Relative gain is scaled between Random and the best pure strategy: g = 0 matches Random, g = 1 matches the portfolio’s best pure strategy, and g < 0 falls below Random.The gap between g and 1 represents the cost of not knowing in advance which strategy is best.Source: G PERFORMANCE ACROSS SETUPS · p. 26
  • FractAL has the highest mean gain (0.689) and the only positive worst-case gain (0.222) across seven setups.Its mean rank is 1.14 of 7 (median 1), compared with SelectAL’s mean rank 3.14 (median 3).Source: G PERFORMANCE ACROSS SETUPS · p. 26, Table 8 · p. 26

H PER-SETUP RESULTS WITH PURE STRATEGIES PERFORMANCE

Across setups, FractAL achieves the highest test performance among strategy-selection methods and matches the best pure strategy or ranks second. The ablations examine smoothing and allocation-update choices.

H PER-SETUP RESULTS WITH PURE STRATEGIES PERFORMANCE

I ABLATION STUDIES

  • The ablations test sensitivity to EWMA smoothing, the mirror-descent update, influence-based versus retraining-based rewards, and fractional allocation.The listed ablations concern components of the proposed FractAL method.Source: I ABLATION STUDIES · p. 29

I.1 SMOOTHING PARAMETER

  • FractAL’s performance is similar across most tested EWMA smoothing values, while no smoothing performs particularly worse on CD4+ T cells and Diamonds.The sweep covers all seven setups and averages results over five splits and ten runs per split.Source: §I.1 · p. 29, Table 12 · p. 29

I.2 ALLOCATION OPTIMIZATION ALGORITHM

  • The mirror-descent update is never worse than the additive update with clipping and renormalization, and is better on five of seven datasets.The comparison is reported across all setups in Table 13.Source: §I.2 · p. 29, Table 13 · p. 29

I.3 INFLUENCE REWARD ATTRIBUTION

Influence-function attribution provides rewards comparable to retraining at lower runtime cost, and soft fractional allocation improves on hard allocation in most setups. The section also reports agreement between influence and retraining reward rankings in CIFAR-10 (C1).

I.3 INFLUENCE REWARD ATTRIBUTION

  • The comparison replaces influence rewards with validation-performance changes after retraining, measured with the fractional allocation framework unchanged.The retraining reward compares validation performance with and without each strategy’s latest selected sub-batch; results average five splits and ten seeds per split.Source: §I.3 · pp. 29–30
  • Influence rewards trail retraining by at most 0.4 accuracy points on CIFAR-10, match or outperform on four of five other tasks, and make the AL run 2.5× faster overall.Table 14 compares performance across setups; the runtime comparison is reported separately.Source: §I.3 · p. 30, Table 14 · p. 30
  • In CIFAR-10 (C1), reward-ranking correlations range from 0.7 to 0.8, and best- and worst-strategy agreements are generally above 70%.The authors report these results as evidence that influence estimates reliably proxy strategy utility.Source: §I.3 · p. 30, Table 15 · p. 30

I.4 SOFT FRACTIONAL ALLOCATION

  • Soft allocation improves on hard allocation in five of seven setups and is comparable on the other two.The hard variant gives the entire round budget to the strategy with the highest reward; Table 16 compares the variants across setups.Source: §I.4 · p. 30, Table 16 · p. 31

J DETAILED RUNTIME ANALYSIS

The runtime analysis decomposes a complete CIFAR-10 (C2, K = 8) AL run into acquisition and allocation-update phases. Additional plots show how strategy allocations evolve across classification, regression, and genetic perturbation setups.

J DETAILED RUNTIME ANALYSIS

  • A complete CIFAR-10 (C2, K = 8) run is timed in acquisition and allocation-update phases, with runtime reported in minutes for different methods.Acquisition queries portfolio strategies and assembles the batch; allocation update computes the reward signal and updates allocations for the next round.Source: J DETAILED RUNTIME ANALYSIS · p. 32, Table 17 · p. 32

K ADDITIONAL BUDGET ALLOCATION PLOTS

  • Across setups, FractAL often increases allocations to empirically best strategies and reduces shares for suboptimal ones; plots show rounds t = 2 through T − 1.The plots omit initialization at t = 0 and the equal-split round at t = 1.Source: K ADDITIONAL BUDGET ALLOCATION PLOTS · p. 33
  • On CIFAR-10 (K = 3), FractAL concentrates on ProbCover and drives CoreSet to zero; with K = 8, it increases ProbCover and decreases CoreSet, while in KEGG it concentrates on CoreSet and suppresses Random and TypiClust.The KEGG caption identifies CoreSet as the second-strongest strategy.Source: Figure 5 · p. 33, Figure 6 · p. 34, Figure 7 · p. 34
  • In SARCOS, FractAL avoids SelectAL’s lowest allocation to TypiClust; in Diamonds, its allocations align with empirical strategy rankings by the final rounds; in BMDM, it drives three strategies close to zero.In BMDM, both FractAL and SelectAL upweight CoreSet, while SelectAL retains substantial allocation to Random.Source: Figure 8 · p. 35, Figure 9 · p. 35, Figure 10 · p. 36
  • For CD4+ T cell perturbation prediction, FractAL upweights CoreSet and reduces the other three strategies to similar low allocations.SelectAL temporarily reduces CoreSet’s allocation in rounds 3–5 and keeps ProbCover proportions high.Source: Figure 11 · p. 36
Loading 2610.05544v1…