Three-dimensional hybrid circuits: the future of neuromorphic computing hardware

Recently there have been intensive research efforts to adopt emerging electronic devices for neuromorphic computing. However, the usage of these devices and arrays mainly was to implement parallel matrix multiplication in the two-dimensional (2D) space. This Perspective discusses the importance and implementation of three-dimensional (3D) hybrid circuits for neuromorphic computing, focusing on the integration density, data communication, and functional connectivity. We believe that 3D neuromorphic systems represent the future of artificial intelligence hardware with much-improved power efficiency and cognitive capabilities.


Benefits and applications of 3D neuromorphic circuits
The first large-scale application of 3D neuromorphic circuits could be the high-volume 3D synaptic arrays that can host all the weights of a modern deep neural network. The sizes of neural networks are growing rapidly to Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. accommodate more extensive input data and provide advanced functionalities such as multi-tasking and highlevel perceptions, which calls for significant expansion of on-chip storage. Although 3D FLASH has been mainly developed for high-capacity storage purposes, significant interest is rising to use 3D FLASH as a synaptic array for weight storage and compute-in-memory operations [4]. The state-of-the-art 3D FLASH product, fabricated with a 28 nm technology, can achieve an integration density exceeding 10 Gb mm −2 [5], compared to a 2D NAND FLASH with less than 1 Gb/mm2 made at the 14 nm node [6]. Emerging technologies such as memristors and phase change memories could achieve higher packing density due to their simple structure.
The second opportunity for 3D neuromorphic circuits lies in the capability to construct 3D connections between devices, which enables new computing paradigms for complex neural networks. In a standard 3D cell arrangement such as that used in 3D NAND and VRRAM [7], synaptic devices are connected through horizontal word lines and vertical bit lines. These 3D arrays implement a fully connected topology between input and outputs and are typically designed to serve as generic hardware for matrix computations. Neural networks with different topologies are divided into sub-networks that can be mapped to each array block. There is a compromise of speed and efficiency in most cases due to substantial control and communication overhead needed to route data between different physically separated sub-networks. Design and fabrication of custom arrays that can directly emulate the complex connections in neural networks become indispensable. As a proofof-concept demonstration, an eight-layer 3D array was designed with a new array topology that emulates the local connectivity of convolutional kernels [8]. The unique array design enables pixel-wise parallel convolutions for image classification and video processing applications with much-reduced processing overheads, otherwise impossible in conventional 2D and 3D array designs.
The third category of 3D neuromorphic hardware is those that integrate various functional layers, including sensors, memory, and logic blocks, into one 3D chip [9]. Each functional block can be connected through finegrained area interfaces, which provides high bandwidth for big data applications. Parallel processing could be highly beneficial for time-sensitive applications such as robotics and autonomous driving. In such a system, both the 'in-memory computing' and 'near-memory computing' concepts could be organically integrated, maximizing the parallelism in the computing while minimizing the data movement in the computing. It is projected that such an arrangement could bring 1000×improvements in the efficiency-speed product [10].

Potential issues and solutions
While 3D circuits have shown early promises in high density and design flexibility for neuromorphic computing, practical implementation of such architecture still requires overcoming challenges in devices, circuits, processes, and more. These topics have been discussed in detail in recent publications [11], [12]. As opposed to conventional 2D circuits, 3D circuits employ multiple device layers fabricated through a series of fabrication processes, some of which are unconventional. It requires a more delicately designed process to meet the thermal budget limit and maximize the fabrication yield. Since different layers are stacked on top of each other, thermal dissipation could be a bottleneck for 3D circuits.
Extensive research and development efforts are required to solve these issues fundamentally. We believe a short-term solution is to relax some non-critical requirements and focus on architecture and designs that maximize the benefits of 3D (such as high density and high parallelism). For example, a brain transmits signal orders of magnitude slower than a modern digital system but can still make instant decisions utilizing its vast parallel processing network. Similarly, we could use the high-density integration and parallelism while sticking to a lower operating frequency. An effective approach to reducing heat generation is to operate the circuits at a substantially lower speed when possible. Another method is to implement sparse neural networks in 3D, in which the overall activity of circuit components is kept relatively low.
Secondly, to maintain the claimed data throughput, the input and output of the 3D arrays should have a high enough bandwidth. This could be a potential issue since peripherals are usually placed on a 2D silicon surface. System-level co-design is needed to coordinate 2D/3D systems to avoid a potential speed bottleneck.
Finally, to use these 3D neuromorphic systems in real-world applications, a systematic design of the software is also required, ranging from the electronic design automation (EDA) tools to the operating systems, providing the compensation and management programs for the 3D circuits. The thermal issues could also be mitigated at the software level using thermal management programs. On the other hand, a significant advantage of the 3D neuromorphic circuit is its connectivity that brings in much-improved computing throughput. A future 3D neuromorphic chip may have a set of neural circuit modules to participate in a wide range of applications. A software abstraction of those 3D neuromorphic modules should be established for developers, and therefore dedicated simulators, compilers, and development environments for end-users should also be available.
In summary, ever since Carver Mead created the concept of neuromorphic engineering over thirty years ago, we have been aiming to build a brain-like computer. With the toolset in shape, it calls for more research efforts and more disruptive approaches. We project that the future of neuromorphic computing is in 3D form-a morphology that mostly resembles the brain. 3D circuits have symbolic similarities to the brain and offer significant benefits in functionality, performance, and scalability, all core needs to go beyond the current AI hardware.

Data availability statement
No new data were created or analysed in this study.