Geometry Is Not Robustness: A Trajectory-Level Study of PGD Evaluation

Dhairysheel Durgule

Source-linked AI summary

Geometry Is Not Robustness: A Trajectory-Level Study of PGD Evaluation

Dhairysheel Durgule

arXiv:2608.14594v1Jun 26, 2026cs.LG

TL;DR

It remains unclear whether PGD trajectory diagnostics reliably indicate adversarial robustness. This study evaluates loss evolution, gradient alignment, and steps-to-failure across models and robustness regimes, finding that steps-to-failure separates robustness more clearly than the other trajectory metrics.

Problem
Whether trajectory-level PGD diagnostics reliably indicate robustness strength remains unclear, limiting interpretation beyond final adversarial accuracy.
Method
The study compares clean-trained and adversarially trained Fashion-MNIST CNNs using PGD trajectories to analyse loss evolution, gradient alignment, and steps-to-failure.
Results
Steps-to-failure separates robustness regimes more clearly than loss trajectories or gradient alignment, while the models show a clear robustness hierarchy under strong PGD evaluation.
Takeaways & Limitations
Trajectory diagnostics describe adversarial optimisation dynamics but should complement, rather than replace, standard robustness measurements and be interpreted in context.
Takeaways & Limitations
The findings are limited to one dataset, Fashion-MNIST, and one compact architecture, CNNSmall, rather than universal claims.

Abstract

from arXiv · show

Projected Gradient Descent (PGD) is widely used to evaluate adversarial robustness, typically via final adversarial accuracy, which does not capture model behaviour throughout the attack. Recent work proposes trajectory-level diagnostics, such as loss evolution, gradient alignment, and steps-to-failure, for deeper insight into adversarial optimisation dynamics. However, whether these diagnostics reliably indicate robustness strength remains unclear. We conduct a trajectory-level investigation of PGD attacks on convolutional neural networks trained on Fashion-MNIST. We compare clean-trained and adversarially-trained models across multiple robustness regimes, using rigorous 20-step PGD evaluations with random initialisation and multiple restarts for robustness measurement, and single-initialisation trajectory recording for diagnostics. We record full PGD trajectories across 3000 clean-correct samples per model and analyse loss evolution, gradient alignment, and failure timing across attack iterations. Our results reveal a clear robustness hierarchy across models; however, trajectory metrics do not contribute equally to its identification. Mean loss trajectories and gradient alignment patterns appear quantitatively similar across adversarially-trained models with substantially different robust accuracies. In contrast, steps-to-failure distributions provide a clearer separation of robustness regimes, directly reflecting functional resistance to adversarial perturbation. These findings indicate that trajectory-level diagnostics describe optimisation geometry but do not independently measure adversarial robustness. Their interpretability depends on robustness regime, attack strength, and multi-metric evaluation. Trajectory-level analysis should be a complementary diagnostic tool, interpreted in context, rather than a replacement for standard robustness measurements.

1 Introduction

PGD is a standard robustness evaluation method, but final adversarial accuracy can conceal how models behave across attack iterations. This study examines whether trajectory-level geometry reliably distinguishes robustness strength across model regimes.

Motivation: Adversarial examples exploit gradients to manipulate model decision boundaries, and their transferability suggests vulnerability is systematic in modern deep networks.
Evaluation background: PGD iteratively increases model loss under a perturbation constraint and is widely used to measure robustness through final adversarial accuracy and clean accuracy.
Evaluation limitation: Final adversarial accuracy hides intermediate attack behaviour, allowing models with similar final robustness to differ in how many steps they resist before failing.
Trajectory diagnostics: Trajectory diagnostics examine loss evolution and gradient alignment, but their interpretation depends on the robustness regime and may not track robustness differences reliably.
Study scope: This work records complete PGD trajectories to study loss, gradient alignment, and prediction outcomes across iterations for clean-trained and adversarially-trained models.The analysis conditions trajectories on clean-correct samples and compares models across multiple robustness regimes.
Research question: The study asks whether trajectory geometry alone distinguishes robustness strength and adopts a diagnostic perspective rather than proposing a new attack or defence mechanism.

2 Literature Review

Adversarial robustness research progressed from documenting adversarial examples and gradient-based attacks to standardised PGD-based evaluation. However, final accuracy and trajectory diagnostics have interpretability limitations, motivating this study’s focus on trajectory behaviour conditioned on clean-correct samples.

Adversarial vulnerability: Adversarial examples are small, imperceptible perturbations that can cause high-confidence misclassification in otherwise capable neural networks.Their discovery established a central trustworthiness challenge for deep learning.
Gradient-based attacks: FGSM introduced efficient single-step gradient attacks, but later work showed that they can underestimate worst-case adversarial vulnerability.PGD addressed this limitation through iterative projected gradient steps within a constrained perturbation set.
Robustness evaluation: Robust accuracy under fixed-step PGD became a standard robustness benchmark, enabling comparisons across architectures and training procedures.However, evaluations remain sensitive to attack configurations and can be distorted by gradient masking.
Trajectory-level diagnostics: Most evaluations report only final adversarial accuracy, overlooking how losses, gradients, and predictions evolve across the intermediate steps of an attack trajectory.Prior work also shows that attack dynamics and adversarial objectives can involve complicated geometry and changing gradient behaviour.
Interpretability gap: Trajectory diagnostics remain difficult to interpret because smooth behaviour can reflect degenerate failure dynamics, especially when clean-correct samples are sparse.This study addresses the gap by conditioning analysis on clean-correct samples from models with clean accuracies between 84% and 92%, rather than proposing a new attack or defence.

3 Methodology · 3.1 Dataset · 3.2 Model Architecture

The study uses Fashion-MNIST and a compact CNNSmall convolutional network to investigate PGD optimisation dynamics. Raw pixel values are used without additional normalisation, while the intentionally simple architecture limits confounding complexity and supports interpretability.

3.1 Dataset: Fashion-MNIST contains 70,000 grayscale 28 × 28 images spanning 10 clothing classes.The dataset is used as a standardised image-classification benchmark and a challenging substitute for MNIST.
3 Methodology: The methodology combines a standardised Fashion-MNIST benchmark with a compact CNN designed for image classification.This pairing provides the dataset and model setting for analysing PGD trajectories.
3.1 Dataset: Images are converted to floating-point tensors in the [0, 1] range without mean subtraction or standard-deviation scaling.Training and attack gradients are computed with respect to raw pixel values.
3.2 Model Architecture: CNNSmall uses two convolutional layers with 32 and 64 channels, each followed by ReLU activation and max-pooling.A final fully connected layer maps the learned representation to 10 output classes.
3.2 Model Architecture: The architecture is intentionally compact to clarify optimisation dynamics and reduce architectural complexity as a confounding factor.This design supports clearer observation of gradient behaviour and loss evolution during PGD attacks.
3.2 Model Architecture: The study restricts trajectory-level assessments to CNNSmall because the objective is diagnostic rather than performance-driven.Architectural simplicity is intended to improve interpretability of gradient behaviour and loss evolution during PGD attacks.

3.3 Training Procedure

Models used supervised cross-entropy training with Adam, early stopping, and up to 75 epochs. Adversarial training used PGD-20 at two perturbation budgets, combined clean and adversarial losses, and repeated experiments across three seeds.

Optimisation and stopping: 1 × 10−3 was the Adam learning rate, with training lasting up to 75 epochs and using batches of 128.Early stopping used clean validation accuracy, with patience 5 for clean-trained and 7 for adversarially trained models.
Adversarial training: PGD-20 adversarial training used separate models with ε = 0.1 and ε = 0.2, generating training examples at each model’s corresponding budget.The step size was α = 0.25ε.
Adversarial training: 0.5 weighted the clean loss and 0.5 weighted the adversarial loss in the training objective.The objective was L = 0.5 Lclean + 0.5 Ladv.
Experimental repetition: 3 random seeds, {0, 1, 2}, were used to reduce sensitivity to initialization.Experiments were repeated across all three seeds.
Evaluation metrics: Clean accuracy was measured on the unperturbed test set and used as both a baseline robustness reference and the conditioning criterion for trajectory analysis.Final clean and robust accuracies for each training regime were reported in Table 2.

3.4 Adversarial Attack: Projected Gradient Descent

The study uses 20-step untargeted PGD with ℓ∞ budgets of ε ∈ {0.1, 0.2}, while separating single-path trajectory diagnostics from multi-restart worst-case robustness evaluation. Loss trajectories saturated before iteration 20, supporting the chosen optimisation depth under the studied configuration.

Attack configuration: PGD applied gradient ascent on cross-entropy loss, projecting each update onto an ℓ∞-constrained perturbation set and clipping inputs to [0, 1].The attack was untargeted and used the true label when computing the loss.
Attack configuration: T = 20 iterations were used with perturbation budgets ε ∈ {0.1, 0.2} and step size α = 0.25ε.After every update, adversarial inputs were projected back onto the valid ℓ∞ball around the original image.
Attack configuration: Loss trajectories saturated before the maximum iteration count for both perturbation budgets, indicating that 20 steps approximated convergence in the studied configuration.The step count was chosen to balance computational tractability with sufficient optimisation depth.
Trajectory initialisation: Trajectory collection recorded one deterministic PGD path per clean-correct sample after single random initialisation within the ε-radius ℓ∞ball.The initial iterate was x0 + δ0 with δ0 ∼ Uniform(−ε, ε), followed by projection into the valid input domain; no multiple restarts were used.
Robustness evaluation: Robustness evaluation instead used PGD-20 with random initialisation, three restarts, and worst-case selection per example.This separates trajectory geometry from worst-case maximisation effects.

3.5 PGD Trajectory Representation

The study represents each PGD attack as a full input trajectory from the clean sample through the final adversarial example, enabling analysis of loss, gradients, and predictions throughout optimization rather than only at convergence.

Trajectory definition: PGD trajectories are recorded as {x0, x1, . . . , xT}, with x0 as the clean input and xT as the final adversarial example after T updates.Each xt denotes the perturbed input after t PGD updates.
Trajectory definition: Full-trajectory recording enables analysis of how loss values, gradients, and predictions evolve throughout the attack instead of only at convergence.This differs from standard evaluation practice, which considers only the final adversarial example.
Failure-step indexing: Failure-step statistics count adversarial updates only: failure at step 0 means misclassification at x1 immediately after the first PGD update, while x0 is excluded.The clean input is not counted as a failure step.

3.6 Trajectory-Level Metrics

The section defines trajectory-level metrics for characterising adversarial dynamics throughout PGD. These metrics track attack-objective progression, gradient-direction stability, and the iteration at which classification fails.

Loss evolution: Cross-entropy loss trajectories track attack-objective progression and reveal optimisation speed and stability across PGD steps.Loss is recorded on the perturbed input at each PGD iteration.
Gradient alignment: Gradient cosine similarity measures stability between successive PGD gradient directions, with values near 1 indicating stable updates and negative values indicating oscillation.The metric ranges from −1 to 1; values near 0 indicate rapidly fluctuating gradients and reflect local loss-landscape smoothness.
Failure timing: Steps-to-failure records the initial PGD iteration when the predicted label differs from the true label, providing a time-resolved robustness measure beyond final accuracy.It captures when classification failure occurs during the attack.

3.7 Data Filtering and the Clean-Correct Constraint · 3.8 Evaluation Protocol and Robust Accuracy

The study filters trajectory analysis to clean-correct samples and evaluates robustness with strong, restart-based PGD-20 attacks at two perturbation budgets. Robustness evaluation and trajectory recording use distinct protocols, with the clean-trained model achieving zero robust accuracy at both budgets.

3.7 Data Filtering and the Clean-Correct Constraint: Trajectory diagnostics are conditioned on clean-correct samples because pre-existing misclassifications can distort adversarial-behaviour statistics.This filtering ensures trajectories reflect adversarial vulnerability rather than classification errors present before perturbation.
3.7 Data Filtering and the Clean-Correct Constraint: Clean accuracy for retrained models ranged between 84% and 92%, depending on the training regime.The reported range motivated conditioning trajectory analysis on correctly classified clean inputs.
3.7 Data Filtering and the Clean-Correct Constraint: Trajectory collection used 3000 clean-correct test samples per model to balance statistical power and computational tractability.The sample count was applied separately to each model.
3.8 Evaluation Protocol and Robust Accuracy: Robust accuracy measures the proportion of test inputs remaining correctly classified under a fixed adversarial attack and is reported with clean accuracy.This pairing contextualizes robustness–accuracy trade-offs.
3.8 Evaluation Protocol and Robust Accuracy: PGD-20 robustness evaluation used ε = 0.1 and ε = 0.2, random ℓ∞ initialisation, three independent restarts, and worst-case selection per example.The configuration approximates a strong first-order white-box adversary.
3.8 Evaluation Protocol and Robust Accuracy: 0.0000 robust accuracy was achieved by the clean-trained model at both ε = 0.1 and ε = 0.2, with 0/10000 correct classifications.Robust accuracy values are reported to four decimal places throughout.
3.8 Evaluation Protocol and Robust Accuracy: Trajectory metrics use one randomly initialised path per clean-correct sample, unlike restart-based robustness evaluation, and aggregate means with standard deviations.Shaded plot regions represent inter-sample variability where applicable.

3.9 Summary of Methodology

The methodology extends standard PGD-based robustness evaluation by explicitly capturing and evaluating adversarial attack trajectories. It complements final accuracy with careful data conditioning and multiple trajectory-level indicators for a more nuanced understanding of adversarial robustness.

The methodology explicitly captures and evaluates adversarial attack trajectories alongside standard PGD-based robustness evaluation.
Trajectory-level analysis provides a more nuanced understanding of adversarial robustness beyond final accuracy alone.
The approach uses careful data conditioning and multiple trajectory-level indicators to evaluate adversarial robustness.

4 Results and Discussion

Under strong PGD evaluation, the three models form a clear robustness hierarchy, but trajectory diagnostics differ in how well they identify it. Mean loss and gradient alignment describe optimisation geometry without uniquely measuring robustness, whereas steps-to-failure more directly separates functional resistance.

Robust evaluation used 20-step PGD with random initialisation and multiple restarts, while trajectory metrics were computed on 3000 clean-correct samples per model.
The clean-trained model collapses under attack, the ε = 0.1 model shows moderate robustness, and the ε = 0.2 model retains substantial robustness at higher perturbation budgets.
At ε = 0.2, the ε = 0.1-trained model’s robust accuracy falls from 72.98% to 8.35%, demonstrating that robustness does not transfer across perturbation budgets.
Mean loss trajectories are quantitatively similar for the adversarially trained models despite different robust accuracies, with final-step losses of 0.388 and 0.440.
Gradient alignment also fails to uniquely indicate robustness: early-step means were 0.758 ± 0.208 for ε = 0.1 and 0.198 ± 0.300 for ε = 0.2, while robust accuracy was 8.35% versus 68.24%.The ε = 0.2 model showed lower initial alignment followed by stabilisation, indicating heterogeneous local optimisation geometry before convergence.
Steps-to-failure most clearly separates regimes: the clean model fails immediately, the ε = 0.1 model shows scattered delays, and the ε = 0.2 model has many samples surviving all 20 steps.Failure timing conveys functional robustness more directly than smooth optimisation metrics such as mean loss trajectories.

5 Conclusion

Trajectory-level diagnostics illuminate PGD optimisation geometry but do not independently measure adversarial robustness. Steps-to-failure more directly separates robustness strength, so trajectory metrics should supplement, not replace, standard evaluations and be interpreted within their experimental and attack context.

Diagnostic value: Steps-to-failure distributions clearly separate robustness strength by describing how long inputs endure adversarial perturbation under strong PGD evaluation.Failure timing more directly captures functional resistance to attack than optimisation-trajectory smoothness.
Diagnostic value: Gradient alignment and loss evolution characterise adversarial update geometry and broaden understanding of attack dynamics beyond final accuracy.These metrics describe how adversarial updates advance through input space when interpreted collectively.
Limits: Mean loss smoothness and gradient alignment do not uniquely determine robustness strength because models with substantially different robust accuracies can exhibit visually similar loss trajectories.Smooth, stable optimisation geometry may conceal substantial differences in adversarial resistance.
Evaluation practice: Trajectory metrics should supplement rather than replace robust accuracy and failure distributions, with multi-metric evaluation and clear attack-configuration reporting preventing over-interpretation.The recommended interpretation is alongside strong attack settings rather than in isolation.
Scope and future work: The findings are limited to Fashion-MNIST and the CNNSmall architecture, so they should be understood within this experimental environment rather than as universal claims.PGD-20 with random initialisation and multiple restarts provides a strong first-order baseline; future work may add AutoAttack and formal interpretation criteria.