Provably Robust Adversarial Examples

This is supplementary code to the ICML 2021 submission of the 'Provably Robust Adversarial Examples' paper.

CIFAR10 and MNIST Pixel Intensity Experiments

To download and install the code for pixel intensity CIFAR10 and MNIST experiments execute the following commands:

git clone https://github.com/dimy93/symadex21
cd symadex21
./install_server.sh 
source ~/miniconda3/bin/activate
conda activate eran_symadex

To run our approach on MNIST convSmall call the run_mnist.sh script as follows:

cd tf_verify/
./run_mnist.sh

To run the baseline on MNIST convSmall call the run_mnist_baseline.sh script as follows:

cd tf_verify/
./run_mnist_baseline.sh

To run our approach on CIFAR10 convSmall call the run_cifar10.sh script as follows:

cd tf_verify/
./run_cifar10.sh

To run the baseline on CIFAR10 convSmall call the run_cifar10_baseline.sh script as follows:

cd tf_verify/
./run_cifar10_baseline.sh

To run our approach on MNIST convBig call the run_mnist_big.sh script as follows:

cd tf_verify/
./run_mnist_big.sh

To run the baseline on MNIST convBig call the run_mnist_big_baseline.sh script as follows:

cd tf_verify/
./run_mnist_big_baseline.sh

To run our approach on MNIST 8x200 call the run_mnist_ffn.sh script as follows:

cd tf_verify/
./run_mnist_ffn.sh

To run the baseline on MNIST 8x200 call the run_mnist_ffn_baseline.sh script as follows:

cd tf_verify/
./run_mnist_ffn_baseline.sh

The resulting robust adversarial examples from our approach will appear under ./tf_verify/NetworkName_ImgNum_class_AdvClass_it_Iteration. The output of our method will appear under ./tf_verify/NetworkName_ImgNum.txt.

The resulting robust adversarial examples from the baseline will appear under ./tf_verify/NetworkName_ImgNum_class_AdvClass_baseline_it_Iteration. The output of the baseline experiment will appear under ./tf_verify/NetworkName_baseline_ImgNum.txt

CIFAR10 and MNIST Geometric Experiments

To download and install the code for pixel intensity CIFAR10 and MNIST experiments execute the following commands:

git clone https://github.com/dimy93/symadex21
cd symadex21
git checkout geometric
git clone https://github.com/eth-sri/eran.git
cp * eran/
cd eran/
./install_server.sh 

To run the geometric 3D Rotation experiments on MNIST convSmall call the 3d_convSmall_rotation.sh script as follows:

cd tf_verify/
./3d_convSmall_rotation.sh

To run the geometric 3D Rotation experiments on MNIST convBig call the 3d_convBig_rotation.sh script as follows:

cd tf_verify/
./3d_convBig_rotation.sh

To run the geometric 3D Rotation experiments on CIFAR10 convSmall call the 3d_cifar_rotation.sh script as follows:

cd tf_verify/
./3d_cifar_rotation.sh

To run the geometric 3D Translation experiments on MNIST convSmall call the 3d_convSmall_translation.sh script as follows:

cd tf_verify/
./3d_convSmall_translation.sh

To run the geometric 3D Translation experiments on MNIST convBig call the 3d_convBig_translation.sh script as follows:

cd tf_verify/
./3d_convBig_translation.sh

To run the geometric 3D Translation experiments on CIFAR10 convSmall call the 3d_cifar_translation.sh script as follows:

cd tf_verify/
./3d_cifar_translation.sh

To run the geometric 4D Rotation experiments on MNIST convSmall call the 3d_convSmall_translation.sh script as follows:

cd tf_verify/
./4d_convSmall_rotation.sh

To run the geometric 4D Rotation experiments on MNIST convBig call the 3d_convBig_translation.sh script as follows:

cd tf_verify/
./4d_convBig_rotation.sh

To run the geometric 4D Rotation experiments on CIFAR10 convSmall call the 3d_cifar_translation.sh script as follows:

cd tf_verify/
./4d_cifar_rotation.sh

The output of the experiments above will appear under tf_verify/TransformationName_NetworkName_0_100.txt.

To run the c value comparison experiment execute the following:

cd tf_verify/
./3d_convBig_translation.sh
./3d_convBig_translation_075.sh
./3d_convBig_translation_085.sh

The output of the experiments above will appear under tf_verify/3d_translation_convBig_0_100_0.75.txt and tf_verify/3d_translation_convBig_0_100_0.85.txt.

To run the EoT experiment execute the following:

cd tf_verify/
./3d_convBig_rotation_EoT.sh

The output of the experiments above will appear under tf_verify/3d_rotation_convBig_0_100_eot.txt