Time-domain image processing using photonic reservoir computing

. Photonic computing has attracted much attention due to its great potential to accelerate artificial neural network operations. However, the processing of a large amount of data, such as image data, basically requires large-scale photonic circuits and is still challenging due to its low scalability of the photonic integration. Here, we propose a scalable image processing approach, which uses a temporal degree of freedom of photons. In the proposed approach, the spatial information of a target object is compressively transformed to a time-domain signal using a gigahertz-rate random pattern projection technique. The time-domain signal is optically acquired at a single-input channel and processed with a microcavity-based photonic reservoir computer. We experimentally demonstrate that this photonic approach is capable of image recognition at gigahertz rates.


Introduction
Recent studies on photonic computing have revealed the potential for overcoming major bottlenecks in electronic computing, suggesting that ultrahigh-speed computing with low energy consumption can be achieved.Photonic computing substrates have been predominantly used to process optical analog signals; thus, they are suitable for directly processing real-world optical information, such as image information.Up to now, many photonic or optoelectronic neural networks for image classification have been developed [1][2][3]; however, most of them rely on spatial parallelism to process image information.To optically process a large amount of image data, largescale photonic processors are generally required, but it is still challenging due to the low scalability of photonic integrations.
In this contribution, we propose a scalable, highspeed image processing approach based on the temporal degrees of freedom of photons [4].This is totally different from previous approaches, which rely on the spatial parallelism of photons using multipixel-based data acquisition.The proposed approach transforms the spatial domain information of a target object to time-domain information at tens of gigahertz rates and enables optical processing in a time-domain in a computationally resource-efficient manner.

System and its characteristics
The key techniques for the time-domain image processing are based on the combination of two different techniques, an optical random pattern projection and photonic reservoir computing (RC) (Fig. 1).The random pattern projection is used for a photonic domain transformation from the spatial-domain information of a physical object to a time-domain signal and enables compressive singlechannel image-data acquisition.The RC processor is used for directly processing the image-encoded time-domain signal.
The random pattern projector is based on a highspeed speckle generator developed in our previous study [5], which is composed mainly of a laser source, random number generator (RNG), phase modulator (PM), and multimode fiber (MMF).When coherent light is input into the MMF, the light is coupled into multiple propagation modes with different phase velocities, and their interference produces a speckle pattern at the end face of the MMF.Because speckles are sensitive to the phase change of the incident light, the speckle pattern can be switched at high-speed by modulating the phase of the incident light.This approach is different from conventional approaches used in single-pixel imaging or ghost imaging [6,7], where the switching rate of optical mask patterns have been limited to tens of MHz [7].By contrast, the switching rate in the proposed projector can exceed tens of gigahertz with the fast phase modulation, which is at least three orders of magnitude higher than that of conventional approaches.
Our photonic RC processor is based on an optical microcavity with the Bunimovich's stadium shape.The wave mixing due to the chaotic nature of the cavity forms a wave field inside the cavity corresponding to a spatially continuous optical random network with a small footprint that functions as a large-scale reservoir.Our previous study has shown that the stadium-shaped cavity-based RC has a higher computational performance for tasks requiring nonlinearity than nonchaotic cavity-based RC [8].In this study, we use the stadium-shaped RC processor coupled with input/output single-mode waveguides (see the inset of Fig. 1) to map the input information onto high-dimensional feature space and to facilitate inferences with a low training cost.

Results
To evaluate the image recognition performance of the proposed system, we used 28×28-pixel MNIST handwritten digit images from "0" to "3" as the target images.Random speckle patterns were generated and projected onto the target at a rate of 25 Gigasamples per second (GS/s).The reflected light was input into the photonic RC processor via an optical fiber (Fig. 1).In this setup, the image information was transformed to a timedomain signal at 25 GS/s [Fig.2(a)], and the time-domain signals were distributed using the RC processor and detected using fast-response photodetectors.The sampling time interval  ! was set as 20 ps.
Figure 2(b) shows the recognition accuracy as a function of the acquisition time  " = / !, where  is the number of data points of the acquired waveform.The compression sensing ratio  for the data acquisition was defined as /(28 × 28) and used to measure the compression capability of the image data.The results show that the classification accuracy exceeded 90% for  " ≥ 0.4 ns, which corresponds to the compressive sensing ratio C ≥ 1.28%, revealing the potential of the proposed approach for ultrafast image recognition at subnanoseconds with a substantial compression efficiency.For comparison, we also investigated the performance of the system without the RC processor.The recognition performance was worse than that of the system using the RC processor, suggesting the effectiveness of the RC processor.
See reference [4] for further investigations on the proposed approach, where the capabilities of dynamic image recognition, anomaly detection, and learning-based imaging have been experimentally shown.The accuracy for the system combined with the photonic RC was better than that without the photonic RC and comparable of that of a neural network model with a singlehidden layer.

Discussion
We proposed and demonstrated a gigahertz-rate image recognition approach based on the compressive transformation of spatial information to the time-domain and direct photonic processing with a photonic RC processor.The proposed approach is scalable, has a low training computational cost, and is suitable for deployment in edge-computing devices.The drawback of the time-domain processing is that there is the trade-off between the processing rate and image resolution.However, our approach can incorporate the other degrees of freedom of photons.This allows for parallel processing and can overcome the trade-off, combined with the spaceand wavelength-division multiplexing techniques [4].

Fig. 1 .
Fig. 1.Schematic of the proposed system based on the random pattern projector and photonic RC chip.RNG: random number generator, ISO: optical isolator, PM: phase modulator, MMF: multimode fiber.The RC processor was fabricated on a silicon chip (see the inset picture).

Fig. 2 .
Fig. 2. (a) Example of the image-encoded time-domain signal and RC outputs.(b) Recognition accuracy as a function of  ! and .900 images were used for training, and 100 images were used for testing.The accuracy for the system combined with the photonic RC was better than that without the photonic RC and comparable of that of a neural network model with a singlehidden layer.