Caffe implementation of Deformable Convolutional Networks

• Thanks to offical code: https://github.com/msracver/Deformable-ConvNets

results:

faster rcnn(resnet50) result is implemented without OHEM and deformable roi pooling

train with pascal voc 2007 + 2012 test on 2007

Usage

Use modified caffe

The MNIST example is in caffe/defor/

Compile:

mkdir build cd build cmake ..  make  all

Train & test:

cd caffe/defor/ ./train_lenet.sh

and the model is in caffe/defor/model_protxt/

use faster rcnn

download voc07,12 dataset ResNet50.caffemodel and rename to ResNet50.v2.caffemodel

cp ResNet50.v2.caffemodel data/pretrained_model/

• OneDrive download: link

Train &test:

./experiments/scripts/faster_rcnn_end2end.sh  0 ResNet50  pascal_voc ./test.sh  0 ResNet50  pascal_voc

Use the codes in your caffe

All codes are in deformable_conv_cxx/

1. Add layer definition to caffe.proto:

optional DeformableConvolutionParameter deformable_convolution_param = 999;    message DeformableConvolutionParameter {   optional uint32 num_output = 1;    optional bool bias_term = 2 [default = true];    repeated uint32 pad = 3; // The padding size; defaults to 0   repeated uint32 kernel_size = 4; // The kernel size   repeated uint32 stride = 6; // The stride; defaults to 1   repeated uint32 dilation = 18; // The dilation; defaults to 1   optional uint32 pad_h = 9 [default = 0]; // The padding height (2D only)   optional uint32 pad_w = 10 [default = 0]; // The padding width (2D only)   optional uint32 kernel_h = 11; // The kernel height (2D only)   optional uint32 kernel_w = 12; // The kernel width (2D only)   optional uint32 stride_h = 13; // The stride height (2D only)   optional uint32 stride_w = 14; // The stride width (2D only)   optional uint32 group = 5 [default = 4];    optional uint32 deformable_group = 25 [default = 4];    optional FillerParameter weight_filler = 7; // The filler for the weight   optional FillerParameter bias_filler = 8; // The filler for the bias   enum Engine {     DEFAULT = 0;     CAFFE = 1;     CUDNN = 2;   }   optional Engine engine = 15 [default = DEFAULT];   optional int32 axis = 16 [default = 1];   optional bool force_nd_im2col = 17 [default = false]; }

you can read the template in deformable_conv_cxx/caffe.proto

2.Move codes to your caffe

move deformable_conv_layer.cpp and deformable_conv_layer.cu to yourcaffepath/srccaffelayers move deformable_conv_layer.hpp to yourcaffepath/includecaffelayers move deformable_conv_layer.hpp to yourcaffepath/includecaffelayers move deformable_im2col.cu to yourcaffepathsrccaffeutil move deformable_im2col.hpp to yourcaffepathincludecaffeutil

3.Compile in your caffe root path

mkdir build cd build  cmake ..   make  all

About the deformable conv layer

The params in DeformableConvolution:

bottom[0](data): (batch_size, channel, height, width) bottom[1] (offset): (batch_size, deformable_group * kernel[0] * kernel[1]*2, height, width)

Define:

f(x,k,p,s,d) = floor((x+2*p-d*(k-1)-1)/s)+1

the output of the DeformableConvolution layer:

out_height=f(height, kernel[0], pad[0], stride[0], dilate[0]) out_width=f(width, kernel[1], pad[1], stride[1], dilate[1])

Offset layer:

layer {   name: "offset"   type: "Convolution"   bottom: "pool1"   top: "offset"   param {     lr_mult: 1   }   param {     lr_mult: 2   }   convolution_param {     num_output: 72     kernel_size: 3     stride: 1     dilation: 2     pad: 2     weight_filler {       type: "xavier"     }     bias_filler {       type: "constant"     }   } }

DeformableConvolution layer:

layer {   name: "dec"   type: "DeformableConvolution"   bottom: "conv1"   bottom: "offset"   top: "dec"   param {     lr_mult: 1   }   param {     lr_mult: 2   }   deformable_convolution_param {     num_output: 512     kernel_size: 3     stride: 1     pad: 2     engine: 1     dilation: 2     deformable_group: 4     weight_filler {       type: "xavier"     }     bias_filler {       type: "constant"     }   } }

the prototxt model should like:

The following animation is generated by Felix Lau (with his tensorflow implementation):https://github.com/felixlaumon/deform-conv/

TODO List

•  all tests passed

•  evaluate performance on Regular MNIST

•  evaluate object detection performance on voc

Deformable Convolutional Networks

Dai, Jifeng, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. 2017. “Deformable Convolutional Networks.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1703.06211