Simulated Annealing Based Area Optimization of Multilayer Perceptron Hardware for IoT Edge Devices

Abstract

The deployment of highly parameterized Neural Network (NN) models on resource-constrained hardware platforms such as IoT edge devices is a challenging task due to their large size, expensive computational costs, and high memory requirements. To address this, we propose a Simulated Annealing (SA) algorithm-based NN optimization approach to generate area-optimized hardware for multilayer perceptrons on IoT edge devices. Our SA loop aims to change hidden layer weights to integer values and uses a two-step process to round new weights that are proximate to integers to reduce the hardware due to operation strength reduction, making it a perfect solution for IoT devices. Throughout the optimization process, we prioritize SA moves that do not compromise the model’s efficiency, ensuring optimal performance in a resource-constrained environment. We validate our proposed methodology on five MLP benchmarks implemented on FPGA, and we observe that the best-case savings are obtained when the amount of perturbation (p) is 10% and the number of perturbations at each temperature (N) is 10,000, keeping constant temperature reduction function ($\alpha$) at 0.95. For the best-case solution, the average savings in Lookup Tables (LUTs) and filpflops (FFs) are 24.83% and 25.76%, respectively, with an average model accuracy degradation of 1.64%. Our proposed SA-based NN optimization method can significantly improve the deployment of area-efficient NN models on resource-constrained IoT edge devices without compromising model accuracy, making it a promising approach for various IoT applications.