PyTorch implementation of Deformable ConvNets v2

This repository contains code for Deformable ConvNets v2 (Modulated Deformable Convolution) based on Deformable ConvNets v2: More Deformable, Better Results implemented in PyTorch. This implementation of deformable convolution based on ChunhuanLin/deform_conv_pytorch, thanks to ChunhuanLin.

TODO

•  Initialize weight of modulated deformable convolution based on paper

•  Learning rates of offset and modulation are set to different values from other layers

•  Results of ScaledMNIST experiments

•  Support different stride

•  Support deformable group

•  DeepLab + DCNv2

•  Results of VOC segmentation experiments

Requirements

• Python 3.6

• PyTorch 1.0

Usage

Replace regular convolution (following model's conv2) with modulated deformable convolution:

class ConvNet(nn.Module):  def __init__(self):    self.relu = nn.ReLU(inplace=True)    self.pool = nn.MaxPool2d((2, 2))    self.avg_pool = nn.AdaptiveAvgPool2d(1)    self.conv1 = nn.Conv2d(3, 32, 3, padding=1)    self.bn1 = nn.BatchNorm2d(32)    self.conv2 = nn.DeformConv2d(32, 64, 3, padding=1, modulation=True)    self.bn2 = nn.BatchNorm2d(64)    self.fc = nn.Linear(64, 10)  def forward(self, x):
    x = self.relu(self.bn1(self.conv1(x)))
    x = self.pool(x)
    x = self.relu(self.bn2(self.conv2(x)))

    x = self.avg_pool(x)
    x = x.view(x.shape[0], -1)
    x = self.fc(x)    return x

Training

ScaledMNIST

ScaledMNIST is randomly scaled MNIST.

Use modulated deformable convolution at conv3~4:

python train.py --arch ScaledMNISTNet --deform True --modulation True --min-deform-layer 3

Use deformable convolution at conv3~4:

python train.py --arch ScaledMNISTNet --deform True --modulation False --min-deform-layer 3

Use only regular convolution:

python train.py --arch ScaledMNISTNet --deform False --modulation False

Results

ScaledMNIST