Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Code for NeurIPS 2021 Paper “Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks” by Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, Xingjun Ma


Robust Configurations for WideResNet (WRN-34-R)

def RobustWideResNet34(num_classes=10):
    # WRN-34-R configurations
    return RobustWideResNet(
        num_classes=num_classes, channel_configs=[16, 320, 640, 512],
        depth_configs=[5, 5, 5], stride_config=[1, 2, 2], stem_stride=1,
        drop_rate_config=[0.0, 0.0, 0.0], zero_init_residual=False,
        block_types=['basic_block', 'basic_block', 'basic_block'],
        activations=['ReLU', 'ReLU', 'ReLU'], is_imagenet=False,
        use_init=True)

Reproduce results from the paper

  • Pretrained Weights for WRN-34-R used in Table 2 available on Google Drive
  • All hyperparameters/settings for each model/method used in Table 2 are stored in configs/*.yaml files.

Evaluations of the robustness of WRN-34-R

WRN-34-R trained with TRADES

Replace PGD with other attacks [‘CW’, ‘GAMA’, ‘AA’].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades
               --load_best_model --attack PGD --data_parallel

WRN-34-R trained with TRADES and additional 500k data

Replace PGD with other attacks [‘CW’, ‘GAMA’, ‘AA’].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --load_best_model --attack PGD --data_parallel

Train WRN-34-R with 500k additional data from scratch

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --train --data_parallel

CIFAR-10 – Linf AutoAttack Leaderboard using additional 500k data

  • Note: This is not maintained, please find up-to-date leaderboard is available in RobustBench.
# paper model architecture clean report. AA
1 (Gowal et al., 2020) available WRN-70-16 91.10 65.87 65.88
2 Ours‡ + EMA available WRN-34-R 91.23 62.54 62.54
3 Ours available WRN-34-R 90.56 61.56 61.56
4 (Wu et al., 2020a) available WRN-34-15 87.67 60.65 60.65
5 (Wu et al., 2020b) available WRN-28-10 88.25 60.04 60.04
6 (Carmon et al., 2019) available WRN-28-10 89.69 62.5 59.53
7 (Sehwag et al., 2020) available WRN-28-10 88.98 57.14
8 (Wang et al., 2020) available WRN-28-10 87.50 65.04 56.29

Citation

@inproceedings{huang2021exploring,
    title={Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks},
    author={Hanxun Huang and Yisen Wang and Sarah Monazam Erfani and Quanquan Gu and James Bailey and Xingjun Ma},
    booktitle={NeurIPS},
    year={2021}
}

Part of the code is based on the following repo:

GitHub

View Github