partialconv
This is the PyTorch implementation of partial convolution layer. It
can serve as a new padding scheme; it can also be used for image
inpainting.
Partial Convolution based Padding
Guilin Liu, Kevin J. Shih, Ting-Chun Wang, Fitsum A. Reda, Karan Sapra, Zhiding Yu, Andrew Tao, Bryan Catanzaro
NVIDIA Corporation
Technical Report (Technical Report) 2018
Image Inpainting for Irregular Holes Using Partial Convolutions
Guilin Liu, Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, Bryan Catanzaro
NVIDIA Corporation
In The European Conference on Computer Vision (ECCV) 2018
Installation can be found: https://github.com/pytorch/examples/tree/master/imagenet
using partial conv for padding
#typical convolution layer with zero padding nn.Conv2d(3, 16, kernel_size=3, stride=1, padding=1, bias=False) #partial convolution based padding PartialConv2d(3, 16, kernel_size=3, stride=1, padding=1, bias=False)
using partial conv for image inpainting, set both multi_channel
and return_mask
to be True
#partial convolution for inpainting (using multiple channels and updating mask) PartialConv2d(3, 16, kernel_size=3, stride=1, padding=1, bias=False, multi_channel=True, return_mask=True)
Installation: to train with mixed precision support, please first install apex from: https://github.com/NVIDIA/apex
Required change #1 (Typical changes): typical changes needed for AMP
from apex import amp #initializing model and optimizer self.model, self.optimizer = amp.initialize(self.model, self.optimizer, opt_level=args.amp_opt_level) #initializing vgg loss function/extractor self.vgg_feat_loss = amp.initialize(self.vgg_feat_loss, opt_level=args.amp_opt_level) #scale loss with amp.scale_loss(total_loss, self.g_optimizer) as scaled_loss: scaled_loss.backward()
Required change #2 (Gram Matrix Loss): in Gram matrix loss computation, change one-step division to two-step smaller divisions
change from one-step division: gram = features.bmm(features_t) / (ch * h * w)
input = torch.zeros(b, ch, ch).type(features.type()) gram = torch.baddbmm(input, features, features_t, beta=0, alpha=1./(ch * h * w), out=None)
Required change #3 (Small Constant Number): make the small constant number a bit larger (e.g. 1e-8 to 1e-6)
ResNet50 using zero padding (default padding)
python main.py -a resnet50 --data_train /path/ILSVRC/Data/CLS-LOC/train --data_val /path/ILSVRC/Data/CLS-LOC/perfolder_val --batch-size 192 --workers 32 --prefix multigpu_b192 --ckptdirprefix experiment_1/
ResNet50 using partial conv based padding
python main.py -a pdresnet50 --data_train /path/ILSVRC/Data/CLS-LOC/train --data_val /path/ILSVRC/Data/CLS-LOC/perfolder_val --batch-size 192 --workers 32 --prefix multigpu_b192 --ckptdirprefix experiment_1/
vgg16_bn using zero padding (default padding)
python main.py -a vgg16_bn --data_train /path/ILSVRC/Data/CLS-LOC/train --data_val /path/ILSVRC/Data/CLS-LOC/perfolder_val --batch-size 192 --workers 32 --prefix multigpu_b192 --ckptdirprefix experiment_1/
vgg16_bn using partial conv based padding
python main.py -a pdvgg16_bn --data_train /path/ILSVRC/Data/CLS-LOC/train --data_val /path/ILSVRC/Data/CLS-LOC/perfolder_val --batch-size 192 --workers 32 --prefix multigpu_b192 --ckptdirprefix experiment_1/
https://www.dropbox.com/sh/t6flbuoipyzqid8/AACJ8rtrF6V5b9348aG5PIhia?dl=0
The best top-1 accuracies for each run with 1-crop testing. *_zero, *_pd, *_ref and *_rep indicate the corresponding model with zero padding, partial convolution based padding, reflection padding and replication padding respectively. *_best means the best validation score for each run of the training. Average represents the average accuracy of the 5 runs. Column diff represents the difference with corresponding network using zero padding. Column stdev represents the standard deviation of the accuracies from 5 runs. PT_official represents the corresponding official accuracies published on PyTorch website: https://pytorch.org/docs/stable/torchvision/models.html
@inproceedings{liu2018partialpadding, author = {Guilin Liu and Kevin J. Shih and Ting-Chun Wang and Fitsum A. Reda and Karan Sapra and Zhiding Yu and Andrew Tao and Bryan Catanzaro}, title = {Partial Convolution based Padding}, booktitle = {arXiv preprint arXiv:1811.11718}, year = {2018}, } @inproceedings{liu2018partialinpainting, author = {Guilin Liu and Fitsum A. Reda and Kevin J. Shih and Ting-Chun Wang and Andrew Tao and Bryan Catanzaro}, title = {Image Inpainting for Irregular Holes Using Partial Convolutions}, booktitle = {The European Conference on Computer Vision (ECCV)}, year = {2018}, }
Contact: Guilin Liu (guilinl@nvidia.com)
We thank Jinwei Gu, Matthieu Le, Andrzej Sulecki, Marek Kolodziej and Hongfu Liu for helpful discussions.
下一篇:grcuda
还没有评论,说两句吧!
热门资源
seetafaceJNI
项目介绍 基于中科院seetaface2进行封装的JAVA...
spark-corenlp
This package wraps Stanford CoreNLP annotators ...
Keras-ResNeXt
Keras ResNeXt Implementation of ResNeXt models...
capsnet-with-caps...
CapsNet with capsule-wise convolution Project ...
shih-styletransfer
shih-styletransfer Code from Style Transfer ...
智能在线
400-630-6780
聆听.建议反馈
E-mail: support@tusaishared.com