RGBD Based Dimensional Decomposition Residual Network for 3D SemanticScene Completion
Abstract
RGB images differentiate from depth as they carry more
details about the color and texture information, which can
be utilized as a vital complement to depth for boosting the
performance of 3D semantic scene completion (SSC). SSC
is composed of 3D shape completion (SC) and semantic
scene labeling while most of the existing approaches use
depth as the sole input which causes the performance bottleneck. Moreover, the state-of-the-art methods employ 3D
CNNs which have cumbersome networks and tremendous
parameters. We introduce a light-weight Dimensional Decomposition Residual network (DDR) for 3D dense prediction tasks. The novel factorized convolution layer is effective for reducing the network parameters, and the proposed multi-scale fusion mechanism for depth and color image can improve the completion and segmentation accuracy
simultaneously. Our method demonstrates excellent performance on two public datasets. Compared with the latest method SSCNet, we achieve 5.9% gains in SC-IoU and
5.7% gains in SSC-IOU, albeit with only 21% network parameters and 16.6% FLOPs employed compared with that
of SSCNet.