Abstract
Deep neural networks have heen known to be vulnerable to adversarial attacks,raising lots of security concerns in the practical deployment.Popular defensive approachcs can bc formulatcd as a(distributionally)robust optimization problcm,which minimizes a"point estimate"of worst-case loss derived from either per-datum perturbation or adversary data-generating distribution within certain pre-defined constraints.This point estimale ignores potential test adversaries that are beyond the pre-defined constraints.The model robustness mighl deteriorate sharply in the scenario of stronger test adversarial data.In this work,a novel robust training framework is proposed to alleviate this issue,Bayesian Robust Learning,in which a distribution is put on the adversarial data-generating distribution to account for the uncertainty of the adversarial data-generating process.The uncertainty directly helps to consider the potential adversaries that are stronger than the point estimate in the cases of distributionally robust optimization.The uncertainty of model paramctcrs is also incorporatcd to accommodate the full Bayesian framework.We design a scalable Markov Chain Monte Carlo sampling strategy to obtain the posterior distribution over model parameters.Various experiments are conducted to verify the superiority of BAL over existing adversarial training methods.The code for BAL is available at https://tinyurl.com/ycxsaewr.