odin-pytorch
This is a PyTorch implementation for detecting out-of-distribution examples in neural networks. The method is described in the paper Principled Detection of Out-of-Distribution Examples in Neural Networks by S. Liang, Yixuan Li and R. Srikant. The method reduces the false positive rate from the baseline 34.7% to 4.3% on the DenseNet (applied to CIFAR-10) when the true positive rate is 95%.
We used two neural network architectures, DenseNet-BC and Wide ResNet. The PyTorch implementation of DenseNet-BC is provided by Andreas Veit and Brandon Amos. The PyTorch implementation of Wide ResNet is provided by Sergey Zagoruyko. The experimental results are shown as follows. The definition of each metric can be found in the paper.
We provide four pre-trained neural networks: (1) two DenseNet-BC networks trained on CIFAR-10 and CIFAR-100 respectively; (2) two Wide ResNet networks trained on CIFAR-10 and CIFAR-100 respectively. The test error rates are given by:
Architecture | CIFAR-10 | CIFAR-100 |
---|---|---|
DenseNet-BC | 4.81 | 22.37 |
Wide ResNet | 3.71 | 19.86 |
CUDA 8.0
PyTorch
Anaconda2 or 3
At least three GPU
Note: Reproducing results of DenseNet-BC only requires one GPU, but reproducing results of Wide ResNet requires three GPUs. Single GPU version for Wide ResNet will be released soon in the future.
We provide download links of five out-of-distributin datasets:
Here is an example code of downloading Tiny-ImageNet (crop) dataset. In the root directory, run
mkdir data cd data wget https://www.dropbox.com/s/avgm2u562itwpkl/Imagenet.tar.gz tar -xvzf Imagenet.tar.gz cd ..
We provide download links of four pre-trained models.
Here is an example code of downloading DenseNet-BC trained on CIFAR-10. In the root directory, run
mkdir models cd models wget https://www.dropbox.com/s/wr4kjintq1tmorr/densenet10.pth.tar.gz tar -xvzf densenet10.pth.tar.gz cd ..
Here is an example code reproducing the results of DenseNet-BC trained on CIFAR-10 where TinyImageNet (crop) is the out-of-distribution dataset. The temperature is set as 1000, and perturbation magnitude is set as 0.0014. In the root directory, run
cd code # model: DenseNet-BC, in-distribution: CIFAR-10, out-distribution: TinyImageNet (crop) # magnitude: 0.0014, temperature 1000, gpu: 0 python main.py --nn densenet10 --out_dataset Imagenet --magnitude 0.0014 --temperature 1000 --gpu 0
Note: Please choose arguments according to the following.
args.nn: the arguments of neural networks are shown as follows
Nerual Network Models | args.nn |
---|---|
DenseNet-BC trained on CIFAR-10 | densenet10 |
DenseNet-BC trained on CIFAR-100 | densenet100 |
args.out_dataset: the arguments of out-of-distribution datasets are shown as follows
Out-of-Distribution Datasets | args.out_dataset |
---|---|
Tiny-ImageNet (crop) | Imagenet |
Tiny-ImageNet (resize) | Imagenet_resize |
LSUN (crop) | LSUN |
LSUN (resize) | LSUN_resize |
iSUN | iSUN |
Uniform random noise | Uniform |
Gaussian random noise | Gaussian |
args.magnitude: the optimal noise magnitude can be found below. In practice, the optimal choices of noise magnitude are model-specific and need to be tuned accordingly.
Out-of-Distribution Datasets | densenet10 | densenet100 | wideresnet10 | wideresnet100 |
---|---|---|---|---|
Tiny-ImageNet (crop) | 0.0014 | 0.0014 | 0.0005 | 0.0028 |
Tiny-ImageNet (resize) | 0.0014 | 0.0028 | 0.0011 | 0.0028 |
LSUN (crop) | 0 | 0.0028 | 0 | 0.0048 |
LSUN (resize) | 0.0014 | 0.0028 | 0.0006 | 0.002 |
iSUN | 0.0014 | 0.0028 | 0.0008 | 0.0028 |
Uniform random noise | 0.0014 | 0.0028 | 0.0014 | 0.0028 |
Gaussian random noise | 0.0014 | 0.0028 | 0.0014 | 0.0028 |
args.temperature: temperature is set to 1000 in all cases.
args.gpu: make sure you use the following gpu when running the code:
Neural Network Models | args.gpu |
---|---|
densenet10 | 0 |
densenet100 | 0 |
wideresnet10 | 1 |
wideresnet100 | 2 |
Here is an example of output.
Neural network architecture: DenseNet-BC-100 In-distribution dataset: CIFAR-10 Out-of-distribution dataset: Tiny-ImageNet (crop) Baseline Our Method FPR at TPR 95%: 34.8% 4.3% Detection error: 9.9% 4.6% AUROC: 95.3% 99.1% AUPR In: 96.4% 99.2% AUPR Out: 93.8% 99.1%
上一篇:odin-serializer
下一篇:odin-project
还没有评论,说两句吧!
热门资源
Keras-ResNeXt
Keras ResNeXt Implementation of ResNeXt models...
seetafaceJNI
项目介绍 基于中科院seetaface2进行封装的JAVA...
spark-corenlp
This package wraps Stanford CoreNLP annotators ...
capsnet-with-caps...
CapsNet with capsule-wise convolution Project ...
inferno-boilerplate
This is a very basic boilerplate example for pe...
智能在线
400-630-6780
聆听.建议反馈
E-mail: support@tusaishared.com