Source-linked AI summary
Houdini: Fooling Deep Structured Prediction Models
Moustapha Cisse, Yossi Adi, Natalia Neverova, Joseph Keshet
TL;DR
Existing adversarial-example methods mainly target classification and are not designed to alter the true performance measure of structured prediction tasks. Houdini generates adversarial examples tailored to combinatorial or non-differentiable task losses. Across speech recognition, pose estimation, and semantic segmentation, Houdini attacks achieve higher success rates with less perceptible perturbations than traditional training surrogates.
Problem
Existing adversarial-example methods mainly target classification and cannot directly optimize many structured prediction metrics because those metrics are non-differentiable.
Method
Houdini is a surrogate combining a stochastic model-confidence margin with the task loss, while providing an analytical gradient for adversarial-example generation.
Results
Houdini attacks achieve higher success rates with less perceptible perturbations than traditional training surrogates across speech recognition, pose estimation, and semantic segmentation.
Takeaways & Limitations
Houdini extends task-loss-targeted adversarial evaluation beyond image classification to structured prediction systems with imperceptible perturbations.
Takeaways & Limitations
The method assumes a network score function and a task loss satisfying ℓ(y, y) = 0 for every target y.
Abstract
from arXiv · showhide
Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.
1 Introduction
Adversarial examples are important for evaluating and improving robustness, but structured prediction tasks use non-differentiable, combinatorial metrics that existing gradient-based attacks cannot directly optimize. Houdini is introduced to generate attacks tailored to each task loss across speech recognition, pose estimation, and semantic segmentation.
- Adversarial examples should directly alter the application-specific task loss, such as word error rate in speech recognition.
- Structured prediction metrics, including keypoint accuracy and segmentation intersection over union, are non-differentiable.
- Traditional surrogate losses lack known consistency guarantees for many structured prediction tasks and may only correlate positively with task loss.
- Direct task-loss minimization is computationally expensive because it requires loss-augmented inference for each update.
- Houdini generates adversarial examples tailored to combinatorial or non-differentiable task losses and is applied to speech recognition, pose estimation, and semantic segmentation.
2 Related Work
Prior attacks rely on differentiable losses, while structured prediction introduces non-differentiable task metrics and imperfect surrogates. Houdini addresses this gap by combining a tight relationship to task loss with an analytical gradient requiring only one inference per update.
- Adversarial examples perturb inputs slightly while causing networks to predict incorrect targets.
- Standard gradient-based attacks require a differentiable loss and therefore do not directly apply to structured task losses.
- Fast gradient sign and related iterative attacks generate perturbations from input gradients under chosen p-norm constraints.
- Direct task-loss methods can perform well but are sensitive to hyperparameters and require two inference operations per training iteration.
- Houdini is tightly related to task loss, has an analytical gradient, and requires only a single inference in its update rule.
3 Houdini
Houdini replaces difficult structured task-loss optimization with a surrogate combining model confidence and task loss. Its analytical input gradient enables efficient gradient-based attacks while retaining a tight relationship to the target metric.
- Houdini is defined for a neural network score function and a task loss that equals zero for any correct target.
- Houdini multiplies a stochastic margin, reflecting prediction confidence, by the task loss between predicted and desired targets.
- Houdini is a lower bound of the task loss and converges to it as the predicted target’s score grows without bound.
- The gradient computation differentiates Houdini with respect to network outputs and obtains the input gradient through backpropagation.
- Houdini’s analytical gradient with respect to the input can be combined with gradient-based adversarial example procedures for untargeted or targeted attacks.
4 Human Pose Estimation
Houdini attacks human pose estimators by targeting PCKh0.5, the final evaluation metric, rather than the dense pixelwise training losses. On MPII, it enables effective untargeted and targeted attacks with low-perceptibility perturbations.
- Task and metric: Pose estimation evaluates the percentage of correctly detected keypoints (PCKh), while models are trained with dense heatmap losses applied independently across pixels and keypoints.PCKh0.5 counts keypoints within half a person’s head size of their target positions.
- Untargeted attacks: Houdini perturbations were up to 50% less perceptible than those generated with the MSE training proxy for untargeted attacks optimized to convergence.Perceptibility is lower-is-better, while SSIM is reported as a complementary measure of visual similarity.
- Untargeted attacks: After 100 iterations, Houdini could completely compromise performance at the target metric, whereas MSE did not reach a similar success level after 300 iterations.The compromised predictions could still appear plausible despite failing formal evaluation, with errors including localization imprecision and left-right limb confusion.
- Targeted attacks: Targeted pose-transfer attacks forced predictions toward arbitrary poses even when the source and target ground-truth poses differed substantially.Examples include transferring between standing and horizontal poses and between standing and sitting poses.
5 Semantic segmentation
Semantic segmentation is evaluated with mIoU, while models are commonly trained with per-pixel or per-class accuracy proxies. Houdini enables targeted and untargeted attacks that achieve substantial segmentation changes with less perceptible perturbations.
- 5 Semantic segmentation: mIoU averages per-class IoU, TP/(TP + FP + FN), and favors accurate object localization with tighter masks.Compared with per-pixel and per-class accuracy, mIoU better accounts for class imbalance and false alarms.
- 5 Semantic segmentation: Table 2 compares targeted and untargeted attacks using mIoU-based success, SSIM, and perceptibility, with Houdini-based attacks reported as less perceptible.mIoU/2 denotes a 50% performance drop, while mIoUlim denotes convergence or termination after 300 iterations.
- 5 Semantic segmentation: Segmentation models are trained with per-pixel softmax or multiclass cross-entropy losses rather than directly optimizing mIoU.Earlier attacks consequently exploited the same training proxy instead of the task metric.
- 5 Semantic segmentation: Targeted attacks switched an image’s segmentation to the ground-truth map of another Cityscapes image using a small perturbation.The perturbation remained small even after zooming in on the perturbed image.
- 5 Semantic segmentation: Houdini also made segmentation systems hallucinate arbitrary target maps, such as a minion, with barely noticeable perturbations.This demonstrates targeted manipulation beyond transferring a segmentation map from another image.
6 Speech Recognition
The speech-recognition experiments apply Houdini to an end-to-end ASR model and compare it with CTC-based attacks. Houdini produces larger CER and WER increases at small perturbation levels, while targeted attacks depend on phonetic similarity and adversarial speech can transfer across models.
- 6 Speech Recognition: The ASR system is an end-to-end deep model that maps speech segments directly to transcripts without additional post-processing.The evaluated architecture uses convolutional layers, bidirectional LSTMs, and a fully connected layer trained with CTC loss.
- 6 Speech Recognition: WER and CER measure speech-recognition errors using substitutions, deletions, and insertions normalized by target length.On Librispeech, the model achieves 12% WER and 1.5% CER without additional language modeling.
- 6 Speech Recognition: 53.73% of adversarial examples were distinguishable from originals in the ABX test, not statistically significantly different from the 50% optimum.The experiment used 100 adversarial audio samples and about 100 human participants.
- 6 Speech Recognition: At ϵ = 0.05, Houdini attacks caused 2.3x larger WER and 1.8x larger CER than CTC attacks.Across tested distortion values, Houdini caused larger decreases in both CER and WER; the spectrograms were visually indistinguishable.
- 6 Speech Recognition: Targeted attacks iteratively push the model toward a selected transcription, but phonetically distant targets yield wrong outputs that are not necessarily close to the target.With phonetically close targets, the model behaves as expected; examples include exact target transcriptions.
- 6 Speech Recognition: Adversarial speech can transfer between models, and Google Voice largely fails to produce good transcriptions for adversarial examples.The black-box experiment used Google Voice predictions for original and adversarial audio files.
7 Conclusion
The conclusion presents Houdini as a task-loss-tailored method for structured prediction. Across pose estimation, semantic segmentation, and speech recognition, it fools state-of-the-art systems with imperceptible perturbations.
- 7 Conclusion: Houdini generates adversarial examples tailored to the performance measure of the task, including combinatorial and non-differentiable losses.The method extends adversarial-example generation beyond image classification to structured prediction problems.
- 7 Conclusion: Houdini fools state-of-the-art systems in pose estimation, semantic segmentation, and speech recognition with imperceptible perturbations.These applications broaden the demonstrated use of adversarial examples beyond image classification.