arxivst stuff from arxiv that you should probably bookmark

An Alternative to Fixed Convolution Architectures

Post · Mar 28, 2017 20:17 ·

CNN

Continuing the trend of more flexible convolutions, as seen in last week’s deformable convolution, the authors of this paper propose active convolution units (ACUs). During training an ACU learns the shape of the convolution, and can represent any fixed convolution. This technique shows promise because it allows you to skip hand-tuning some hyperparameters.

Arxiv Abstract

  • Yunho Jeon
  • Junmo Kim

In recent years, deep learning has achieved great success in many computer vision applications. Convolutional neural networks (CNNs) have lately emerged as a major approach to image classification. Most research on CNNs thus far has focused on developing architectures such as the Inception and residual networks. The convolution layer is the core of the CNN, but few studies have addressed the convolution unit itself. In this paper, we introduce a convolution unit called the active convolution unit (ACU). A new convolution has no fixed shape, because of which we can define any form of convolution. Its shape can be learned through backpropagation during training. Our proposed unit has a few advantages. First, the ACU is a generalization of convolution; it can define not only all conventional convolutions, but also convolutions with fractional pixel coordinates. We can freely change the shape of the convolution, which provides greater freedom to form CNN structures. Second, the shape of the convolution is learned while training and there is no need to tune it by hand. Third, the ACU can learn better than a conventional unit, where we obtained the improvement simply by changing the conventional convolution to an ACU. We tested our proposed method on plain and residual networks, and the results showed significant improvement using our method on various datasets and architectures in comparison with the baseline.

Read the paper (pdf) »