Physics > Optics
[Submitted on 19 Jun 2023 (v1), last revised 27 Jun 2023 (this version, v2)]
Title:Metasurface-based Spectral Convolutional Neural Network for Matter Meta-imaging
View PDFAbstract:Convolutional neural networks (CNNs) are representative models of artificial neural networks (ANNs), that form the backbone of modern computer vision. However, the considerable power consumption and limited computing speed of electrical computing platforms restrict further development of CNNs. Optical neural networks are considered the next-generation physical implementations of ANNs to break the bottleneck. This study proposes a spectral convolutional neural network (SCNN) with the function of matter meta-imaging, namely identifying the composition of matter and mapping its distribution in space. This SCNN includes an optical convolutional layer (OCL) and a reconfigurable electrical backend. The OCL is implemented by integrating very large-scale, pixel-aligned metasurfaces on a CMOS image sensor, which accepts 3D raw datacubes of natural images, containing two-spatial and one-spectral dimensions, at megapixels directly as input to realize the matter meta-imaging. This unique optoelectronic framework empowers in-sensor optical analog computing at extremely high energy efficiency eliminating the need for coherent light sources and greatly reducing the computing load of the electrical backend. We employed the SCNN framework on several real-world complex tasks. It achieved accuracies of 96.4% and 100% for pathological diagnosis and real-time face anti-spoofing at video rate, respectively. The SCNN framework, with an unprecedented new function of substance identification, provides a feasible optoelectronic and integrated optical CNN implementation for edge devices or cellphones with limited computing capabilities, facilitating diverse applications, such as intelligent robotics, industrial automation, medical diagnosis, and astronomy.
Submission history
From: Shijie Rao [view email][v1] Mon, 19 Jun 2023 05:01:07 UTC (2,003 KB)
[v2] Tue, 27 Jun 2023 12:16:44 UTC (2,006 KB)
Current browse context:
physics.optics
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.