Elsevier

Neural Networks

Volume 142, October 2021, Pages 171-179
Neural Networks

Arguments for the unsuitability of convolutional neural networks for non-local tasks

https://doi.org/10.1016/j.neunet.2021.05.001Get rights and content
Under a Creative Commons license
open access

Abstract

Convolutional neural networks have established themselves over the past years as the state of the art method for image classification, and for many datasets, they even surpass humans in categorizing images. Unfortunately, the same architectures perform much worse when they have to compare parts of an image to each other to correctly classify this image.

Until now, no well-formed theoretical argument has been presented to explain this deficiency. In this paper, we will argue that convolutional layers are of little use for such problems, since comparison tasks are global by nature, but convolutional layers are local by design. We will use this insight to reformulate a comparison task into a sorting task and use findings on sorting networks to propose a lower bound for the number of parameters a neural network needs to solve comparison tasks in a generalizable way. We will use this lower bound to argue that attention, as well as iterative/recurrent processing, is needed to prevent a combinatorial explosion.

Keywords

Convolutional neural networks
Sorting networks
Relational reasoning
Attention
Locality

Cited by (0)