# Adversarial Robustness ## Vulnerability, transfer and accuracy - **Adversarial examples and transfer** - Finds small image perturbations that induce misclassification and can transfer between networks trained on different data subsets. Transfer enables attacks without differentiating the target network. [Szegedy et al. 2014]( "Intriguing properties of neural networks; ICLR"). - **Non-robust features** - Classifiers trained on adversarially relabeled images generalize to clean test data. These vision experiments show that predictive but perturbation-sensitive features can support ordinary generalization and adversarial transfer. [Ilyas et al. 2019]( "Adversarial Examples Are Not Bugs, They Are Features; NeurIPS"). - **Robustness-accuracy trade-offs** - Proves an accuracy-robustness conflict for a constructed distribution and observes a similar tension in vision experiments. The conflict depends on the data and perturbation set, so it need not hold for every classification problem. [Tsipras et al. 2019]( "Robustness May Be at Odds with Accuracy; ICLR"). ## Attack construction - **FGSM** - Takes one step in the sign of the input-loss gradient, maximizing a linearized loss under an l-infinity bound. Supplies a cheap attack and training perturbation; iterative attacks search beyond that local linearization. [Goodfellow et al. 2015]( "Explaining and Harnessing Adversarial Examples; ICLR"). - **Carlini-Wagner attacks** - Optimizes targeted misclassification under l0, l2 or l-infinity distance criteria. Defeats defensive distillation in the tested networks, showing that failure of earlier attacks had overstated that defense. [Carlini and Wagner 2017]( "Towards Evaluating the Robustness of Neural Networks; IEEE S&P"). - **Adversarial patches** - Optimizes a shared visible patch over images, locations, rotations and scales to induce a target class. Demonstrates printed attacks; its localized replacement threat differs from small changes to every pixel. [Brown et al. 2017]( "Adversarial Patch; arXiv"). - **Projected gradient attacks** - Repeatedly increases input loss and projects onto the permitted perturbation set. Random starts explore different local maxima; failure to find an adversarial input supplies no certificate of its absence. [Madry et al. 2018]( "Towards Deep Learning Models Resistant to Adversarial Attacks; ICLR"). - **GCG** - Uses gradient-guided discrete search for adversarial text suffixes. Optimizing across prompts and models transfers some attacks on refusal behavior to other language models; results concern the tested 2023 checkpoints and interfaces. [Zou et al. 2023]( "Universal and Transferable Adversarial Attacks on Aligned Language Models; arXiv"). ## Robust training and its generalization - **PGD adversarial training** - Approximates minimax training: projected-gradient attacks maximize loss within a specified input perturbation set; parameter updates minimize that loss. Reported resistance concerns the evaluated first-order attacks. [Madry et al. 2018]( "Towards Deep Learning Models Resistant to Adversarial Attacks; ICLR"). - **TRADES** - Balances clean-label loss with disagreement between predictions on clean and adversarial inputs. Derives the regularizer from natural-error and boundary-error terms, making the accuracy-robustness trade-off explicit. [Zhang et al. 2019]( "Theoretically Principled Trade-off between Robustness and Accuracy; ICML"). - **Robust self-training** - Pseudo-labels additional unlabeled images, then uses them in robust training. Extra data improves CIFAR-10 and SVHN robustness in the tested setups, connecting robust sample requirements with semi-supervised learning. [Carmon et al. 2019]( "Unlabeled Data Improves Adversarial Robustness; NeurIPS"). - **Robust overfitting** - Robust test error can rise while training continues to improve. Across the studied image datasets, early stopping matches or beats the tested regularizers; selecting the final training iterate can misrank defenses. [Rice et al. 2020]( "Overfitting in adversarially robust deep learning; ICML"). ## Certified robustness - **Convex outer adversarial polytope** - Relaxes the reachable activations of ReLU networks to bound worst-case loss under l-infinity perturbations. Optimizes a dual bound during training; a verified bound covers the specified input region. [Wong and Kolter 2018]( "Provable defenses against adversarial examples via the convex outer adversarial polytope; ICML"). - **Interval bound propagation** - Propagates activation intervals to bound worst-case outputs and trains against those bounds. Cheap interval relaxations can be loose, so failure to certify an input need not imply an adversarial example exists. [Gowal et al. 2018]( "On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models; arXiv"). - **Randomized smoothing** - Defines a classifier by majority prediction under Gaussian input noise and certifies l2 robustness from class probabilities. Finite-sample certificates use confidence bounds and may abstain; they concern the smoothed classifier. [Cohen et al. 2019]( "Certified Adversarial Robustness via Randomized Smoothing; ICML"). ## Evaluation and comparative evidence - **Obfuscated gradients** - Seven of nine studied ICLR 2018 defenses relied on gradient masking. Approximate backward passes and averaging over randomness broke six completely and one partially, separating attack failure from robustness. [Athalye et al. 2018]( "Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples; ICML"). - **Adaptive attacks** - Circumvents 13 selected defenses despite their published adaptive evaluations. Successful attacks required defense-specific objectives and treatment of preprocessing or randomness; a generic attack suite was insufficient. [Tramèr et al. 2020]( "On Adaptive Attacks to Adversarial Example Defenses; NeurIPS"). - **AutoAttack** - Combines two Auto-PGD variants with FAB and a score-based Square Attack. Its evaluation of over 50 classifiers reduced reported robust accuracy by over 10 percentage points for 13 models; threat sets remain fixed. [Croce and Hein 2020]( "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks; ICML"); [code](). - **RobustBench** - Standardizes datasets, perturbation budgets, model interfaces and attack evaluation. Reported attacked accuracy is an empirical upper bound on worst-case robust accuracy; the benchmark also accepts stronger adaptive evaluations. [Croce et al. 2021]( "RobustBench: a standardized adversarial robustness benchmark; NeurIPS Datasets and Benchmarks"); [code](). ## Related topics - [Neural Networks and Deep Learning](https://mlpapers.org/neural-nets/) - [Generalization and Learning Theory](https://mlpapers.org/generalization/) - [Model Evaluation and Comparison](https://mlpapers.org/evaluation/) - [Natural Language Processing](https://mlpapers.org/nlp/) - [Computer Vision](https://mlpapers.org/computer-vision/) - [Optimization](https://mlpapers.org/optimization/)