Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning

Abstract

Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as “catastrophic forgetting”, and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.

Introduction

In recent years Deep Convolutional Neural Networks (DCNNs) have emerged as the leading architecture for large scale image classification (Rawat & Wang, 2017). In 2012, AlexNet (Krizhevsky, Sutskever, & Hinton, 2012), an 8 layer Deep CNN, won the ImageNet Large Scale Visual Recognition Challenge (ISLVRC) and catapulted DCNNs into the spotlight. Since then, they have dominated ISLVRC and have performed extremely well on popular image datasets such as MNIST (LeCun et al., 1998, Wan et al., 2013), CIFAR-10/100 (Krizhevsky & Hinton, 2009), and ImageNet (Russakovsky et al., 2015).

Today, with increased access to large amounts of labeled data (e.g. ImageNet (Russakovsky et al., 2015) contains 1.2 million images with 1000 categories), supervised learning has become the leading paradigm in training DCNNs for image recognition. Traditionally, a DCNN is trained on a dataset containing a large number of labeled images. The network learns to extract relevant features and classify these images. This trained model is then used on real world unlabeled images to classify them. In such training, all the training data is presented to the network during the same training process. However, in real world, we hardly have all the information at once, and data is, instead, gathered incrementally over time. This creates the need for models that can learn new information as it becomes available. In this work, we try to address the challenge of learning on such incrementally available data in the domain of image recognition using deep networks.

A DCNN embeds feature extraction and classification in one coherent architecture within the same model. Modifying one part of the parameter space immediately affects the model globally (Xiao, Zhang, Yang, Peng, & Zhang, 2014). Another problem of incrementally training a DCNN is the issue of “catastrophic forgetting” (Goodfellow, Mirza, Xiao, Courville, & Bengio, 2013). When a trained DCNN is retrained exclusively over new data, it results in the destruction of existing features learned from earlier data. This mandates using previous data when retraining on new data.

To avoid catastrophic forgetting, and to leverage the features learned in previous task, this work proposes a network made of CNNs that grows hierarchically as new classes are introduced. The network adds the new classes like new leaves to the hierarchical structure. The branching is based on the similarity of features between new and old classes. The initial nodes of the Tree-CNN assign the input into coarse super-classes, and as we approach the leaves of the network, finer classification is done. Such a model allows us to leverage the convolution layers learned previously to be used in the new bigger network.

The remainder of the paper is organized as follows. The related work on incremental learning in deep neural networks is discussed in Section 2. In Section 3 we present our proposed network architecture and incremental learning method. In Section 4, the two experiments using CIFAR-10 and CIFAR-100 datasets are described. It is followed by a detailed analysis of the performance of the network and its comparison with transfer learning and fine tuning in Section 5. Finally, Section 6 discusses the merits and limitations of our network, our findings, and possible opportunities for future work.