PaperlayerView exact revision on arXiv

Source-linked AI summary

ATLAS-AL: Adaptive Trust-Region for Latent Adversarial Searches via Active Learning

Marsalis Gibson, Claire Tomlin, Shankar Sastry

arXiv:2610.07323v1cs.LGcs.CRcs.CV

TL;DR

ATLAS addresses the challenge of finding diverse sets of failures when high-dimensional search spaces make exhaustive exploration infeasible. It uses calibrated active level-set estimation and local-global sampling to recover adversarial regions and construct sets; the paper reports greater empirical coverage than optimization-based baselines, with attack success rates on par or better. The conclusion presents these sets as a foundation for automated red-teaming tools.

  • Problem

    High-dimensional inputs make exhaustive search infeasible, and adversarial regions may occupy only a small fraction of the space, while fixed attack collections do not adaptively discover diverse failures.

    Source: §1 · p. 1

  • Method

    ATLAS uses active level-set estimation, calibrated predictions, and boundary and saturation trust-region sampling to learn an adversarial value function and construct an adversarial set.

    Source: §1 · p. 1, §3 · p. 3, §4 · p. 3

  • Results

    ATLAS recovers more empirical adversarial-region coverage than popular optimization-based methods, with attack success rates on par or better than the baselines, and constructs more diverse adversarial sets.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    ATLAS’s more diverse adversarial sets provide a useful foundation for building automated red-teaming tools.

    Source: §7 · p. 8

    The formulation assumes a bounded image space, limited black-box query access, and access to target output probabilities.

    Source: §3 · p. 2

Abstract

from arXiv · show

Security evaluation of learning-based systems requires more than just testing the system against a fixed collection of attacks. It requires adaptive mechanisms that can efficiently discover \textit{sets} of inputs that induce model failure. We introduce ATLAS (Adaptive Trust-Regions for Latent Adversarial Searches), which is a query-based framework that discovers adversarial input sets for black-box learning systems. ATLAS casts attack generation as an active learning level set estimation problem then combines calibrated approximations with a local-global sampling architecture to find regions of the input space that contain adversarial examples. Once discovered, ATLAS is designed to sample points within these adversarial regions to build adversarial sets that accurately represent the state of robustness of the target model. When applied on toy experiments, we find that ATLAS is able to recover more of the adversarial region under a limited query budget than does previous work. When applied to standard and adversarially trained MNIST, CIFAR, and ImageNet model targets, ATLAS produces better representative attacks than other query-based black-box attacks (NES, SignHunter, BayesOpt). ATLAS represents an automated red-teaming framework that can be used for both analyzing the robustness of learning-based systems under development and continuous auditing to see how the robustness of a system changes over time.

1 INTRODUCTION

ATLAS reframes adversarial discovery as finding diverse sets of failures rather than isolated successful examples. It combines active learning with calibrated predictions and dual sampling to recover such sets, outperforming optimization-based baselines on empirical coverage while matching or exceeding their attack success rates.

1 INTRODUCTION

  • High-dimensional inputs make exhaustive search infeasible, while adversarial regions may occupy only a small fraction of the search space.The goal is a widespread set that supports auditing where and how a system fails, beyond testing a fixed collection of known attacks.Source: §1 · p. 1
  • ATLAS combines level-set estimation, Bayesian optimization, and adversarial generation to discover regions and build both an adversarial classifier and an associated dataset.It uses calibrated model predictions and dual sampling; Figure 1 depicts the learned input-to-adversity mapping leading to a set of attacks.Source: §1 · p. 1, Figure 1 · p. 2
  • ATLAS recovers more empirical coverage of adversarial regions than popular optimization-based methods, with attack success rates on par or better than the baselines.The comparison tests whether repeated baseline attacks can recover diverse sets or whether a method tailored to set construction offers an advantage.Source: §1 · p. 2
  • The paper frames adversarial generation as active level-set estimation and reports that optimization-based attacks recover less of the adversarial set even when all their queries are retained.Its contributions also include residual-model calibration, saturation trust regions, and evaluation on non-convex benchmarks and image classifiers.Source: §1 · p. 2

2 RELATED WORKS

Prior image-attack methods include gradient-based, generative, and query-based approaches. ATLAS is closest in adaptive sampling to BayesOpt, but learns a global adversarial mapping to recover sets rather than optimizing for one attack.

2 RELATED WORKS

  • Image-attack methods include gradient-based approaches, generative models, and query-based methods such as Bayesian optimization, SquareAttack, SignHunter, and NES.The passage also mentions LM-evolutionary methods and generic try-and-check methods.Source: §2 · p. 2
  • BayesOpt uses adaptive sampling but decomposes inputs into lower-dimensional subspaces and optimizes an adversarial objective for one specific attack.ATLAS instead decomposes its input space into local regions of the same dimension; the passage describes the approaches as complementary.Source: §2 · p. 2
  • ATLAS learns an adversarial mapping globally to recover sets, so its adaptive sampling targets model error rather than generic adversarial maximization.This contrasts with methods designed to find individual attacks.Source: §2 · p. 2

3 THREAT MODEL AND PROBLEM FORMULATION

The formulation treats attack generation as recovering the super-zero level set of an adversarial value function under limited black-box queries. ATLAS combines boundary discovery with interior sampling so the recovered region is both classified and represented by a set of attacks.

3 THREAT MODEL AND PROBLEM FORMULATION

  • The target is a black-box classifier on a bounded image space, and the evaluator has limited query access while observing output probabilities.The adversarial objective scores the target output, and successful attacks exceed threshold ϕ.Source: §3 · pp. 2–3
  • ATLAS represents the adversarial set as the super-zero level set of a learned value function, V(x), that maps candidate inputs to adversarial scores.The target set is A = {x ∈ X : fadv(x) ≥ ϕ}.Source: §3 · p. 3
  • Sampling only near the zero level set can classify adversarial points but leaves the set interior under-sampled; ATLAS therefore ranks both boundary and interior candidates.Direct interior sampling is difficult because surrogate errors can send queries outside the set or into already well-sampled regions; calibrated predictions support the complementary rankings.Source: §3 · p. 3, Figure 2 · p. 3

4 ATLAS: ADAPTIVE TRUST-REGION FOR LATENT ADVERSARIAL SEARCH

ATLAS alternates global model fitting, calibration, trust-region selection, local sampling, and refitting. Its design pairs boundary discovery with saturation sampling to classify the adversarial region and curate a representative set.

4 ATLAS: ADAPTIVE TRUST-REGION FOR LATENT ADVERSARIAL SEARCH

  • ATLAS combines a calibrated value function with local placement-sampling in boundary and saturation trust regions, then repeats global fitting, calibration, region selection, local sampling, and refitting.Boundary regions target the zero level set; saturation regions sample within discovered super-level sets to produce the adversarial set.Source: §4 · p. 3

4.1 CALIBRATION

  • Global calibration uses a uniformly sampled held-out set to train a residual model, while local models use leave-one-out cross-validation on their training points to conserve query budget.Fresh hold-out sets for temporary, moving local regions would consume too much of the sampling budget.Source: §4.1 · pp. 4–5

4.2 ACQUISITION DESIGN FOR PLACEMENT AND SAMPLING

  • ATLAS selects local trust regions first and then samples within them; each region is a hypercube centered at a selected point.This global-local architecture has an outer loop for choosing centers and an inner loop for selecting dataset samples.Source: §4.2 · p. 5
  • Boundary and saturation acquisitions select zero-level and super-level candidates, respectively, prioritizing high model error to balance classification and adversarial-set diversity.These objectives can conflict: improving classification favors high-error points, while diverse adversarial sampling may favor regions already known to be adversarial.Source: §4.2 · p. 5

5 TOY EXPERIMENTS: FINDING ADVERSARIAL REGIONS IN HIGHLY NON-CONVEX FUNCTIONS

The toy experiments test whether ATLAS can recover adversarial super-level sets of non-convex functions under a limited query budget. They examine calibration and the roles of boundary and saturation sampling.

5 TOY EXPERIMENTS: FINDING ADVERSARIAL REGIONS IN HIGHLY NON-CONVEX FUNCTIONS

  • The experiments compare ATLAS with AP2L and TRLSE on 50K super-level samples and a 1-million-point mixed test set, using F1, recall, precision, spread, margin, and penetration measures.Experiments use 30 seeds and a total budget of 650; the super-level-set volume is set to 5% of the domain.Source: §5 · pp. 5–6
  • At 100D, adding 50 calibration samples raises classification performance from .163, .003, and .100 to .322, .288, and .457 for Ack, Rast, and Rosen, respectively.Calibration samples come from the fixed budget of 650, leaving the remainder for sampling; the benefit grows with dimension.Source: Table 1 · p. 6
  • Without calibration, high precision comes from predicting few super-level points, but recall suffers—a poor trade-off for a security analysis tool meant to catch adversarial inputs.Source: §5 · p. 6
  • In 100D, only the combined 1:1 boundary-to-saturation sampling is robust across all landscapes; saturation harvests adversarial points, while boundary sampling gives the widest spatial spread.ATLAS-s has F1 ≈0.01 on Rastrigin and Rosenbrock, while ATLAS-b finds almost no adversarial points on Rastrigin.Source: Table 2 · p. 7

6 IMAGE EXPERIMENTS: MAPPING ADVERSARIAL REGIONS THROUGH LATENT-SPACE SEARCH

The image experiments search bounded latent spaces of fixed class-conditional generators for targeted adversarial regions. ATLAS produces more representative sets than the compared query-based baselines, while trading off some dispersion and time to first success.

6 IMAGE EXPERIMENTS: MAPPING ADVERSARIAL REGIONS THROUGH LATENT-SPACE SEARCH

  • The search evaluates 20 source–target class pairs per dataset–model setting within a bounded region around the fixed generator’s latent prior mean.Candidate adversaries are natural images decoded from the generator’s latent samples.Source: §6 · p. 7
  • ATLAS is evaluated against NES, SignHunter, BayesOpt, and uniform random sampling; the authors frame finding diverse failures rather than repeating known attacks as more useful for security evaluation.Source: §6 · pp. 7–8
  • At 650 queries, ATLAS recovers the most representative adversarial set and distributes its points better than the baselines, despite returning fewer raw points than a couple of them.Uniform sampling has Spacing = 0.88–0.93 but finds 1–19 adversarial points; NES and SignHunter find more points but cluster more or spread less.Source: §6 · p. 7
  • ATLAS can be slower than BayesOpt to first success (Qfirst = 83–229 vs. 36–102 for BayesOpt), but the authors report that it spends its budget finding more adversarial regions.Source: §6 · p. 7
  • Recall was best in 9 of 9 settings and F1 (Mix) in 8 of 9; ATLAS also found the most super-level points per unit budget in 8 of 9, with highest margin spread and depth in every experiment.ATLAS used 112 calibration samples and a 1:1 boundary-to-saturation sampling split at B = 538, versus baselines at B = 650.Source: Table 3 · p. 8
  • Sampling deeper inside the super-level set sacrifices average dispersion, but gives access to adversarial scores unavailable from the other baselines or uniform sampling.At 50D and 100D, ATLAS recovers scores higher than every sample in the super-level test set.Source: §6 · p. 8

7 CONCLUSION

ATLAS constructs diverse adversarial sets, a useful foundation for automated red-teaming tools. In the reported image examples, the sets reveal shared target-model failure patterns that single attacks cannot expose.

7 CONCLUSION

  • ATLAS constructs more diverse adversarial sets, providing a useful foundation for automated red-teaming tools.The conclusion notes variance in performance across function landscapes and identifies evaluation on more targets as future work.Source: §7 · p. 8
  • ATLAS matches the best attack success rate while returning the most representative adversarial set, and uniquely achieves the best Margin J with good spatial spread and competitive set size.Table 4 averages results over 20 source–target class pairs per target model, using 650 oracle queries per experiment.Source: Table 4 · p. 9
  • For stingray-to-great-white-shark targets, every recovered image shows open ocean rather than shark features; goldfish-target sets include flower petals and feathers.The figure caption says these patterns point to areas needing further study and analysis of the target model.Source: Figure 3 · p. 9

A FULL TOY EXPERIMENT EVALUATION

Across toy benchmarks, ATLAS combines calibration with boundary and saturation sampling to recover adversarial sets under a fixed query budget. Its results show strong recall and depth, alongside trade-offs in calibration allocation, dispersion, and weight selection.

A FULL TOY EXPERIMENT EVALUATION

  • The toy evaluation covers Ackley, Rastrigin, and Rosenbrock at 10D, 50D, and 100D, using 650 queries, four trust regions, and a 1:1 saturation-to-boundary ratio.Dataset sizes and calibration-set sizes are held equal across methods, and baseline implementations are not altered.Source: A FULL TOY EXPERIMENT EVALUATION · p. 12, Figure 4 · p. 14
  • Rastrigin 50D uses w=40 rather than a larger scale-matched weight because raising w shifted budget from boundary sampling and reduced F1 from 0.54 to 0.20.The authors state that output units vary substantially across benchmarks and grow with dimension, so no single w balances the saturation acquisition terms across settings.Source: A FULL TOY EXPERIMENT EVALUATION · p. 12
  • A calibration set of 50–100 samples (8–15% of B) gives best or near-best F1 in every experiment and higher recall than no calibration in all nine.The main experiment uses 112 calibration samples (17%); no calibration can yield high precision by predicting few super-level points, at the cost of recall.Source: Table 6 · p. 13, A FULL TOY EXPERIMENT EVALUATION · p. 13
  • A 1:1 boundary–saturation split is the only tested variant robust across landscapes in model estimation and adversarial-set quality.Saturation wins nSuper/B in 7 of 9 cells, often at about 90%, while boundary-only sampling generally has better spatial spread and precision.Source: Table 7 · p. 14
  • With default weights, ATLAS achieves the best recall in 8 of 9 experiments, best F1 (Mix) in 7 of 9, and highest depth (mV) in all 9, with dispersion within 30% of the best.The default setting is c=2, w=40; per-benchmark weights mainly improve Rastrigin 100D and adversarial-set metrics.Source: Table 8 · p. 15
  • Sampling deeper inside the super-level set lowers average dispersion and coverage, but recovers adversarial scores unavailable to other baselines or uniform sampling.At 50D and 100D, ATLAS recovers adversarial scores higher than every sample in the super-level test set.Source: A FULL TOY EXPERIMENT EVALUATION · p. 15

B FUNCTIONS USED IN TOY EXPERIMENTS

The toy experiments use Ackley, Rastrigin, and Rosenbrock functions, each defined with its parameters and global minimum.

B.1 ACKLEY FUNCTION

  • The Ackley function is defined over x ∈ R^d with a=20, b=0.2, and c=2π; its global minimum is 0 at x=0.Source: §B.1 · p. 15

B.2 RASTRIGIN FUNCTION

  • The Rastrigin function is defined over x ∈ R^d with A=10; its global minimum is 0 at x=0.Source: §B.2 · p. 15

B.3 ROSENBROCK FUNCTION

  • The Rosenbrock function is defined over x ∈ R^d with a=1 and b=100; its global minimum is 0 at x=(1, ..., 1).Source: §B.3 · p. 16

C IMPLEMENTATION SPEED UP

ATLAS's large-dimensional runtime is concentrated in global-model training and trust-region center selection. The implementation uses analytic gradients and parallelization to speed these computations.

C IMPLEMENTATION SPEED UP

  • In large dimensions, most ATLAS runtime comes from training a new global model and selecting new trust-region centers with L-BFGS-B and warm starts.Source: C IMPLEMENTATION SPEED UP · p. 16
  • Analytic gradients during optimization and parallelized calculations provide substantial speedups, especially after the model has acquired many samples.Source: C IMPLEMENTATION SPEED UP · p. 16
  • Reported timings use a CPU; depending on the model and learning algorithm, GPU resources may provide further speedups.Source: C IMPLEMENTATION SPEED UP · p. 16

D IMAGE EXPERIMENT DETAILS

The image experiments use several standard and adversarially trained targets, with model-specific image generators and different baseline configurations.

D IMAGE EXPERIMENT DETAILS

  • The targets include LeNet5 at 99% accuracy, standard ResNet50 at 93.65% on CIFAR and 76.13% on ImageNet, and adversarially trained models at 60.75% and 54.53%.The adversarially trained targets are WRN-28-10 with DDPM for CIFAR and ResNet50 trained on L∞ perturbations for ImageNet.Source: D IMAGE EXPERIMENT DETAILS · p. 16
  • MNIST uses a conditional DCGAN, CIFAR uses BIGGAN, and ImageNet uses BIGGAN-256 as image generators.BIGGAN and the conditional DCGAN were retrained to take 32 dimensions and a source class; BIGGAN-256 uses StudioGAN implementation and weights.Source: D IMAGE EXPERIMENT DETAILS · p. 16
  • NES and SignHunter run to the full query budget, while BayesOpt uses early stopping and is reported only for ASR and query-to-success.BayesOpt’s run-to-budget super-level sets, required for dispersion statistics, were computationally infeasible to obtain.Source: D IMAGE EXPERIMENT DETAILS · p. 16

E INTERPRETING ATLAS IMAGE RESULTS

At a matched 650-query budget, ATLAS produces better-distributed adversarial sets than query-based baselines, while recovered images reveal model-specific classification cues and targeted failure regions, including for adversarially trained models.

E INTERPRETING ATLAS IMAGE RESULTS

  • MNIST and CIFAR sets show benign-looking source-class images misclassified as targets, while ImageNet sets expose cues such as ocean context or flower-like structures.The authors suggest the ImageNet examples reveal features that may shape class boundaries, including ocean context for shark classification and non-goldfish structures accepted when goldfish cues are present.Source: E INTERPRETING ATLAS IMAGE RESULTS · pp. 16–17, Figure 5 · p. 18
  • At B=650, ATLAS is best on Margin J in 4 of 5 targets, with Spacing 0.55–0.65 versus SignHunter’s 0.24–0.31.ATLAS returns fewer raw points than query-based attacks but distributes them better; the table reports comparable attack success.Source: Table 9 · p. 17
  • Two of 20 standard ImageNet BayesOpt batches hit the ~12-hour cap after ~490 and ~585 of 650 queries without finding an adversarial point.The slowest ATLAS batch took 37 minutes, while no NES or SignHunter batch exceeded 2 minutes.Source: E INTERPRETING ATLAS IMAGE RESULTS · p. 17

F SECURITY AUDIT SUMMARY

  • ATLAS characterizes failure regions in generator latent space where source-class images are misclassified as a specified target class.The audit frames this as identifying targeted adversarial attacks using a chosen image generator.Source: F SECURITY AUDIT SUMMARY · p. 19
  • With a fixed query budget, ATLAS recovered nonempty sets in every reported setting, including against adversarially trained CIFAR-10 and ImageNet models.The authors conclude that adversarial training did not eliminate targeted failure regions in those models.Source: F SECURITY AUDIT SUMMARY · p. 19
  • The sets can guide further robustness evaluation, adversarial training, or potential monitoring and security filters, while audit artifacts document repeatable security assessments.The stated artifacts include recovered sets, a trained surrogate, quantitative measurements, and the threat model and experimental settings.Source: F SECURITY AUDIT SUMMARY · p. 19

ETHICAL CONSIDERATIONS

  • The authors note that developers’ robustness tool may also be used by outside actors to attack systems they do not own.They identify future work on defenses that may hinder efficient mapping from inputs to adversarial scores, and mention limiting outside query access as a possible mitigation.Source: ETHICAL CONSIDERATIONS · p. 19

AI STATEMENT

  • AI assisted debugging experiments, parameter sweeps, scripts, figures, writing, and feedback, while authors conducted the core experimental and analytical designs.The authors also state that they reviewed and edited AI-generated artifacts.Source: AI STATEMENT · p. 19

Related papers

Loading 2610.07323v1…