OS20-2 Network with Sub-Networks

Title:	OS20-2 Network with Sub-Networks
Publication:	ICAROB2020
Volume:	25
Pages:	191-194
ISSN:	2188-7829
DOI:	10.5954/ICAROB.2020.OS20-2
Author(s):	Ninnart Fuengfusin, Hakaru Tamukoh
Publication Date:	January 13, 2020
Keywords:	Model Compression, Neural Networks, Multilayer Perceptron, Supervised Learning
Abstract:	We introduce network with sub-networks, a neural network which it's weight layers can be detached into subneural networks during inference phase. To develop trainable parameters which can be inserted into both base- and sub-models, firstly, the parameters of sub-models are duplicated to base-model. Each model is forward-propagated separately. All models are grouped into pairs. Gradients from selected pairs of networks are averaged and updated both networks. With MNIST dataset, our base-model achieves the identical test-accuracy to the regularly trained models. In other hand, the sub-models are suffered an extend of loss in test-accuracy, nevertheless the sub-models provide alternative approaches to be deployed with less parameters compare to the regular model.
PDF File:	https://alife-robotics.co.jp/members2020/icarob/data/html/data/OS/OS20/OS20-2.pdf
Copyright:	© The authors. This article is distributed under the terms of the Creative Commons Attribution License 4.0, which permits non-commercial use, distribution and reproduction in any medium, provided the original work is properly cited. See for details: https://creativecommons.org/licenses/by-nc/4.0/