Integration of a PQ:PMMA holographic memory device into the hybrid opto-electronic correlator for shift, scale, and rotation invariant target recognition

: The hybrid optoelectronic correlator (HOC) combines optical and electronic signal processing to achieve the same functionality as traditional optical correlators but without the need for dynamic materials. Here we propose and demonstrate the integration of a PQ:PMMA holographic memory device (HMD) into the HOC as a high-speed all-optical database for reference images. Using a PQ:PMMA HMD for one of the inputs eliminates one of the key speed limitations in the HOC. The observed correlation signal agrees with simulations but highlights the need for high quality holographic substrates in this application.


Introduction
In the age of big data, the need for high-speed image correlation systems has become increasingly apparent, as the need to scan large databases has provided a challenge to the computing world. Digital computation techniques are not ideal for these situations, as the processing time for each individual data set quickly adds up. In the past, Vander Lugt and joint transform correlators have been investigated as potential all-optical alternatives that could be used for high-speed correlation [1][2][3]. However, these have had steep drawbacks [4]: the former employs filters that have to be recorded for each particular correlation, and the latter utilizes delicate nonlinear dynamic materials [5] that are frequently damaged during operation [4]. We have been working towards developing a hybrid optoelectronic correlator (HOC) system [6,7] that replaces these nonlinear materials with detectors, thus overcoming some of these limitations. The advantages and full mathematical description of the HOC are discussed in more detail in Ref. [6], wherein we have demonstrated its functionality for position, rotation, and scale invariant correlation with use of the polar Mellin transform (PMT). We are still working towards achieving the HOC's ideal operational speed of ∼5µs per correlation, which would allow it to be used as a high-speed input filter for a more precise machine learning image recognition system. One improvement that will greatly increase the speed of the HOC is the incorporation of a holographic memory device (HMD) as a high capacity and high speed all-optical database.
Phenanthrenequinone (PQ) doped poly(methyl methacrylate) (PMMA) is a well-known polymer substrate that can yield highly stable holographic gratings [8,9]. Due to the exposure dynamics of the material, the dye is typically depleted after writing is done, making the polymer a 'write-once, read-many' (WORM) HMD [8]. As such, this medium is suitable for use in systems that require highly stable operation, such as correlators [10]. Furthermore, the substrate is first synthesized as a liquid and subsequently poured into a mold, allowing it to be made in arbitrary shapes [11].
For the HOC, a disk shape is ideal, as it can be rotated rapidly to access the database at the speeds required to achieve the optimum correlation times.
The rest of the paper is organized as follows. Section 2 describes PQ:PMMA based volume phase holograms and their use for optical database storage. An overview of the HOC is given in section 3, along with a description of how a PQ:PMMA HMD is incorporated into the architecture. A description of the PMT is summarized in section 4. Results are presented and examined in section 5, along with a comparison to the expected output from a simulation. Section 6 presents a discussion on the capacity of these holographic devices, and section 7 concludes with a summary and outlook.

Image phase holograms in PQ:PMMA
PQ:PMMA is a highly stable WORM substrate that uses a photosensitive dye in a polymer matrix to form phase gratings. To write these holograms, an expanded reference beam is interfered with an image beam within the substrate, generating a highly complex pattern of high and low intensity areas. The PQ dye is only activated by the light in the illuminated sections, attaching itself to MMA and PMMA, thus forming oligomers that have a different density than the host matrix [8,12]. This forms a modulation in the refractive index, i.e., a phase grating. After illumination is complete, the unreacted PQ diffuses, further strengthening the grating. Using this approach, a setup like the one shown in Fig. 1 can be used to store image databases in the optical domain. A laser at a wavelength that can activate the PQ molecules, in this case 532nm, is expanded and collimated prior to passing through a shutter, which allows us to precisely control the exposure time of the sample. A polarizing beamsplitter (PBS) separates the image arm, which has an amplitude modulated (AM) spatial light modulator (SLM), and the reference arm, which has a mirror (M1). The SLM allows us to change the spatial profile of the beam such that it matches a greyscale image that we wish to store on the PQ:PMMA HMD (i.e., white pixels have a high intensity and black pixels have no intensity). A half waveplate (HWP) placed before the PBS allows us to control the power of each arm, and another HWP in the reference arm corrects the polarization of one of the beams such that both arms have the same polarization. The image arm has a 4-f system that ensures that the stored image is a flat replica of the image generated by the SLM, without any phase curvature. The reference arm also has a 4-f system, but it is used for angle multiplexing, as explained below. Finally, after passing through the PQ:PMMA HMD, the writing beams are blocked via beam dumps to minimize unwanted exposure. In the case of PQ:PMMA, after writing, the samples are post-processed with incoherent UV light to deplete any remaining unreacted PQ, thus desensitizing the substrate and rendering the holograms permanent. For the purposes of this demonstration, a ∼2mm thick 50mm diameter disk of PQ:PMMA was used with the images measuring ∼5 mm x 7 mm. The PQ:PMMA was prepared using the method described in Ref. [8].
For simplicity, consider first the case where the image is uniform, and both writing beams can be approximated as plane waves. In this case, the grating formed within a holographic substrate will have a k-vector equal to that of the interference between the two writing beams. Furthermore, because of the thickness of the substrate, these gratings will fall into the Bragg regime. Consequently, in order for a read beam of the same wavelength as the writing beam to diffract efficiently, it will have to match the k-vector of one of the writing beams [13,14]. This allows us to change one or more of the parameters of the writing setup to create multiple superimposed gratings that will not generate cross-talk with one another in any significant way [15]. This is known as volumetric multiplexing. For the general case, the image would be non-uniform, and the reference beam would be Gaussian. As such, the k-vector for each of the two writing beams will have a finite spread around a mean value. Therefore, the grating corresponding to the recording of each image will have a finite spread. In order to suppress A polarizing beamsplitter (PBS) and two half waveplates (HWP) are used to control the power of the two arms. Finally, mirror M1 can rotate to vary the input angle of the reference beam on the PQ:PMMA HMD. The input location of this beam on the HMD will not change thanks to another 4-f system. cross-talk, it is necessary to ensure that this spread is less than the angular bandwidth of the Bragg selectivity.
For applications where the PQ:PMMA HMD is used as a database, the characteristics of the writing reference beam (i.e., wavelength, angle, etc.) are typically used for the reading beam. For practical reasons, the most common way to multiplex is by varying the angle of this reference beam [16,17]. In the setup shown in Fig. 1, mirror M1 is located at the input plane of a 4-f system, with the PQ:PMMA HMD at the output plane. When M1 rotates, it results in a corresponding 2x angle change for the reference beam at the HMD, without changing the location of said beam. By changing the reference beam angle, but maintaining a constant image beam angle, we can design a large multiplexed holographic database of images that is compatible with the HOC and can be rapidly accessed by changing the angle of the read beam. Furthermore, if the PQ:PMMA HMD is in the shape of a disk, images can be stored at distinct spatial locations along a constant radius, separated by distinct radial angles, allowing the disk to be rotated rapidly, thus creating a database with high reading speed and capacity. For a practical system, the angle at the same spatial location can be scanned very fast with an acousto-optic beam deflector, for example, while shifting from one spatial location to another at the same radius can be accomplished by using a servo-stabilized rotation stage. In order to allow for movement from a spatial location at a given radius to another one at a different radius, the whole reading assembly would be mounted on a servo-stabilized translation stage.

HOC architecture and functionality
Cross-correlation is an indication of the degree to which two signals are akin to each other. When two signals are said to be cross-correlated, it implies that they share similar properties. Perhaps the most practical way of performing this calculation, along with the closely related convolution, is by means of the Fourier Transform (FT), with the well-known relations: where ⊙ corresponds to the cross-correlation and ⊗ corresponds to the convolution. It should be noted that the only difference between these two expressions is that one of the FTs in the cross-correlation has been conjugated. In the optical domain, we are able to perform the two-dimensional spatial FT by means of a simple lens. This is achieved by placing a collimated image beam at the focal plane of the lens, whereby the FT will appear at the opposite focal plane.
Unfortunately, multiplication of arbitrary signals is much more difficult to achieve with all-optical components. The HOC solves this issue by performing this step with electronics. This, however, creates another problem: the FT of an image is a complex signal, and focal plane arrays (FPAs) can only detect intensity (i.e., the magnitude squared of the signal). The HOC's architecture, shown in Fig. 2, is built around this issue, and works by capturing six signals that together contain all of the magnitude and phase information of the Fourier Transform (FT) of the images that will be correlated. The signals are as follows: where M is the FT of an image, C is an auxiliary plane wave, and the subscripts 'r' and 'q' denote the reference and query, respectively. In the configuration shown in Fig. 2, the HOC only employs two FPAs to capture all six signals. Two shutters (S 1 , S 2 ) choose the signals that will be detected: For the HOC to reach its full speed, the architecture would have to be modified to have 6 total FPAs, one per signal, operating in parallel. These signals can be combined on an FPGA or a computer, as explained in Refs. [6,7,18], to obtain the following electronic signal: where α = C r C q , β = C r C * q .

Fig. 2.
Simplified HOC architecture with an SLM query arm and a PQ:PMMA HMD reference arm. The reference and query arms together form the image arm. The phase scanning and stabilization (PSS) arm employs a Mach-Zehnder interferometer with piezoelectric transducers (PZTs) to form a rudimentary optical phase locked loop, providing a method of controlling the relative phase between the C q and C r auxiliary plane waves. M is the FT of an image, C is an auxiliary plane wave, and the subscripts 'r' and 'q' denote the reference and query, respectively. Shutters S 1 and S 2 are used to select the signal to be measured.
The first two terms in this expression, containing α, have the multiplication of the FT of the two input images. Similarly, the last two terms, containing β, have the multiplication between a FT, and a conjugated FT. Comparing this to Eq. (1), it is clear that these terms can be used to find the convolution and cross-correlation, respectively. To do this, the signal is transferred back into the optical domain by means of an SLM, subsequently FT'd by a lens, and detected on a final FPA. Taking the auxiliary plane waves as constants, the final signal (Sf ) will thus yield two convolution terms associated with α, and two correlation terms associated with β, as shown in Eq. (4).
The HOC can employ three piezoelectric transducers (PZTs) in its phase scanning and stabilization (PSS) arm in order to eliminate the α term altogether, while maximizing the β term, effectively removing the convolution while performing the correlation [6]. The PSS section forms a rudimentary single-source optical phase locked loop (OPLL) by using a Mach-Zehnder interferometer (MZI) with PID control, as shown in Fig. 2. The source enters the MZI and is split into a static arm, employing a mirror, and a variable-length arm, employing a PZT (PZT-2). Cq is extracted through a beamsplitter (BS) prior to reaching PZT-2, and so its phase is independent of the position of the transducer. Cr, on the other hand, is extracted after PZT-2, and so its phase can be adjusted by changing the position of the transducer. The static and variable-length arms recombine, generating an interference pattern which is measured through a pair of matched detectors (MZI-PD). This is then fed into a PID system that controls the position of PZT-2. A 'bias' voltage is applied to vary the setpoint and thus control the set phase of Cr. This provides a method of directly controlling the phase difference between Cr and Cq, and thus, β. Two additional PZTs (PZT-1a and PZT-1b) are used to change the phase of Cr and Cq equally, thus controlling α.
PLLs are common components in electronic circuits, but their optical counterparts are notoriously difficult to construct and maintain [19]. Integrated versions of these circuits have proven to be more practical, but are still under development [20,21]. In the future, it may be beneficial to replace the laser source with a photonic integrated circuit altogether, incorporating a single laser with a Y-junction that guides the light down two paths: one section to directly output into the image arms, and another with an OPLL that produces the two reference beams, thus incorporating the PSS arm into the laser source. This would greatly benefit the HOC as this system would be almost completely insensitive to vibrations. In order to do this, however, the HOC would have to be operated at a distinct wavelength, e.g., 780nm, that is compatible with integrated photonics. Fortunately, PQ:PMMA substrates have been characterized for many wavelengths, and generally function identically in all cases, with only minor changes to the refractive index modulation and input/output angles [15,22].
The HOC consists of four main stages: The PSS arm, two image arms, and an output stage. In the conventional architecture, both image arms are identical, each consisting of an SLM, a biconvex lens, a beamsplitter, and an FPA. Using this for both image arms severely limits the operational speed of the HOC, as SLM switching speeds for greyscale images tend to be on the order of hundreds of microseconds. One solution to this issue is to replace the SLM in the reference image arm with an HMD. Figure 2 contains the simplified HOC architecture with a PQ:PMMA HMD reference arm, but without angle multiplexing. Because the PQ:PMMA HMD's holograms exist entirely in the optical domain, its response time is nearly instantaneous. This allows us to access the images at high speeds, limited only by our beam steering. With the query image arm using an SLM, and the reference image arm using a PQ:PMMA HMD, we can scan through the entire HMD database for every frame that is sent to the SLM.
While the advantages of incorporating a PQ:PMMA HMD into the HOC are clear, one must be careful to consider the effects of holographic lensing on the reference images. This is particularly true for polymer substrates, as the curing process often results in shrinking and distortions that may impart a strong lensing effect on the stored images. Because the HOC relies on the focal distance between the HMD, lens, and FPA, holographic lensing can significantly impede the system from functioning properly. Thus, it is necessary for the PQ:PMMA HMD to be either prepared in such a way as to eliminate the lensing altogether, or for the lensing to have a consistent and predictable value for all the images.

PMT for scale and rotation invariance in the HOC
The HOC is inherently shift invariant owing to its use of the FT. It is, however, insufficient for most applications, as rotation and scale invariance are an essential part of any robust image recognition system. This can be achieved by transforming the images themselves in such a way that they become shift, scale, and rotation invariant. The transformed images -also known as signatures-can then be input into the HOC, where the standard correlation operation can proceed [6,23].
Rotation invariance is achieved by first obtaining the magnitude of the FT of the image, which will always be centered at the 'DC' point. Taking this DC point to be the coordinate (0,0) of a polar plane, we can project the polar coordinates (r, θ) into a cartesian plane, where r is the horizontal axis and θ is the vertical axis. Through this process, any rotation in the original image results in a linear shift in the vertical axis of the signature. Scale invariance can be added on to the rotation component by taking the logarithm of the radial component, such that the new horizontal axis is given by ρ = log(r/r 0 ), where r 0 is a reference length that is equal for all images. With this transformation, if the FT of an image is scaled by a factor of γ, this will result in a rectilinear shift of log(γ) in the horizontal axis. Due to the logarithmic nature of this final transform, coordinates near r = 0 will yield ρ values that tend towards −∞; as such, we must block the DC component altogether to avoid this issue. The resulting signature is mathematically equivalent to the PMT and will be unique for every image, serving as a new input to the HOC.
When incorporating a PQ:PMMA HMD into the HOC, we may store PMT signatures instead of the raw images. This would still require the query arm to perform the PMT in real time, as the query images would typically be sourced from cameras or other unprocessed data sets. We have previously described a method to perform this [6].

Correlating SLM images with PQ:PMMA HMD images
For this experiment, an image of a square was written onto a PQ:PMMA HMD with the setup shown in Fig. 1. This shape was chosen for its simplicity and easily identifiable FT. The reference image retrieved from the HMD, as observed through a 4-f system, is shown in Fig. 3(A). The SLM query image, as observed through a 4-f system, is shown in Fig. 3(B). The PQ:PMMA HMD reference image shows spotting that is due to the writing setup, while the SLM query image shows a slight banding from imperfections in the SLM. Very similar imperfections are also found in the PQ:PMMA HMD reference image, as an SLM was used in the writing process. These images were used as inputs to the HOC setup shown in Fig. 2. The magnitudes of the detected FTs of these two images are shown in Figs. 3(C) and 3(D), respectively. Qualitatively, we observe that these two FTs are very similar, albeit with some distortions corresponding to the aforementioned imperfections. Figure 3(E) corroborates this assessment, as it shows a clear correlation peak at the center, along with two convolution peaks on the sides. The central peak is the sum of the T3 and T4 terms from Eq. (4) which, in the case of unshifted images, line up perfectly.
When the query input is shifted with respect to the reference image, as shown in Fig. 4(A) and 4(B), the T3 and T4 terms no longer line up at the center, as shown in Fig. 4(E). The result of this correlation was scaled to the maximum value of Fig. 3(E), and shows a peak of only 0.236, which is lower than the convolution terms of the unshifted experiment. This highlights the necessity for the scanning operation of the PSS arm, which will enable the HOC to eliminate the convolution terms altogether. We may compare these results with the expected output from a MatLab simulation using the same input images, as shown in Fig. 5. The unshifted simulation, shown in Fig. 5(A), presents a clear peak at the center of the output plane, implying a high degree of correlation between the two images. Simultaneously, we observe two side bands that result from the convolution terms. These bands have a maximum value of approximately 0.27, while the corresponding bands in Fig. 3(E) have peak of closer to 0.36. This shows that the experiment and simulation agree, but there is still room for improvement. The difference may be due to small imperfections in the optics and the holographic lensing effect that was previously described. In the future, great care must be taken when designing the PQ:PMMA HMD such that the holographic lensing can be avoided altogether. Similarly, the output of a shifted simulation, as shown in Fig. 5(B), agrees with the corresponding experiment shown in Fig. 4. The correlation terms appear in similar locations and have similar values: 0.267 for the simulation and 0.236 for the experiment.  Figure 6 shows a distinct experiment wherein the image is the PMT signature of an artificial FT that is shaped like the letter 'A'. The reference was stored in the same location on the PQ:PMMA HMD as in the previous experiments, albeit at a different multiplexed angle. The query was projected on a regular SLM as in the previous experiments and corresponded to the PMT of an identical letter 'A' scaled by a factor of γ = 1.56. This scaling factor in the artificial FT translates to a linear shift in the horizontal axis of the PMT. The experimental and simulation results are shown from a top view to emphasize the correlation peak separation that results from These results are in agreement with the simulations despite distortions in the stored reference images that stemmed from imperfections in the imaging setup. An additional source of potential noise is the scattering produced by the PQ:PMMA based HMD. We have taken special care to minimize such scattering. First, during the fabrication process, we ensure that no micro-bubbles are present in the HMD. Second, we encase the medium between two glass plates with optical quality surfaces, thereby minimizing any surface roughness. As a result, we see very little scattering during the image reconstruction process, as can be seen in Fig. 6(A). It should also be noted that the dark spots in Fig. 4(A) are due primarily to the imperfection in the optical imaging process, and not due to scattering from the PQ:PMMA based HMD. In the near future, a detailed analysis of the potential limitations due to residual scattering from the PQ:PMMA based HMD, as a function of parameters such as the density of image storage and the level of read-out laser power, will be carried out.

Discussion on the capacity of holographic devices
While the exact characteristics of any holographic substrate depends on many parameters [12,17,24], their high capacity and rapid reading speed make them ideal for the HOC. One of the most straightforward ways to characterize a substrate is by the M # , which roughly corresponds to the maximum number of 100% efficiency holograms that can be written at a single location [25]. Consider a PQ:PMMA HMD with an M # of 9.5 [24] used to store 1,000 images at one location: the efficiency of each individual grating is given by η d ≈ (M # /N) 2 = (9.5/1000) 2 ≈ 9 * 10 −5 , where N is the number of stacked gratings.
In order to understand how this diffraction efficiency affects the beam requirements of the HOC, we must consider the characteristics of the FPA that is reading the disk. Consider the case of an FPA that has a per-pixel well capacity −Q full − of ∼1,024e − , an exposure time −t e − of 5µs, a pixel size of 5µm x 5µm, a quantum efficiency −η q − of 70% at an operating wavelength −λ r − of 532nm, and a resolution of 1920 × 1200. This device would require N full = Q full /η q ≈ 1462.9 photons per pixel in order to perfectly fill the well. The equivalent photon flux would be: Φ = N full /(t e * (5um) 2 ) ≈ 1.17 * 10 19 photons s −1 m −2 , which corresponds to a power density of H = Φhc/λ r ≈ 4.37 W m −2 , where h and c are the Planck constant and the speed of light in vacuum, respectively. Taking into account the resolution of this particular device, the sensor area would be 6mm x 9.6mm, which may be used to find the total power of the beam at the sensor to be P sensor = H * Area ≈ 2.52 * 10 −4 W; this represents the scenario where the image is completely white and so each pixel in the FPA must be saturated. Finally, we may consider the PQ:PMMA HMD's diffraction efficiency to find the power of the readout beam to be P readout = P sensor /η d ≈ 2.8 W. The 5µs exposure time is required for optimum operation of the HOC. However, we may relax this requirement for evaluation purposes and allow for longer exposure times. If we instead consider an identical FPA with an exposure time of 30µs -a value available in current commercial devices-, then the required power of the readout beam can be reduced to P readout ≈ 0.467 W.
When designing or choosing the FPA for this operation, it is essential to also consider the analog-to-digital converter (ADC) resolution and noise characteristics. Consider the previously described FPA: if the device had an ADC resolution −R− of 8 bits, then there would be ∆Q = Q full /2 R ≈ 4 e − between each distinguishable digital output level. While there are various sources for noise in this system, the two most important will be read noise and shot noise, the root-mean-square values of which are denoted n r and n s , respectively. The read noise is a device parameter, and the shot noise depends directly on the number of incoming photons: The dark noise −n d − for this hypothetical FPA can be taken to behave as: n d = √ 5 · t e at room temperature, which is consistent with current commercially available devices [26] . At the required exposure time of 5µs for the HOC to operate at its ideal speed, this yields a value of n d = 0.005 e − , which is significantly lower than the shot noise for even one photon, and so can be ignored. If we instead consider the case with a 30µs exposure time, the dark noise would be n d ≈ 0.012 e − , which can also be ignored. The total noise can then be found to be n t ≈ √︁ n r 2 + n s 2 = √︁ n r 2 + Q. From here it is clear that, for low values of n r (e.g., 1e − ) the shot noise inevitably dominates most exposures. Under this condition, the darkest and brightest pixels will have a total noise of n t− min ≈ 1 e − and n t− max ≈ 32e − , respectively. Thus, taking into account the ∆Q of 4e − , the darkest pixels will be mostly unaffected by noise, and the brightest pixels will have a variability of ∼8 digital values, which corresponds to an error of ∼3% over the 256 possible readings. This much error should not have any significant effect on the spatial profile of the image being measured, and so is acceptable for the HOC.
If this PQ:PMMA HMD is in the form of a 2 mm thick disk with a radius of 10 cm, the disk would have a capacity of 1000 images per location, with 250 locations (each with a dimension of 9.6mm x 6mm) along four constant radii. This disk would have a maximum storage capacity of 250,000 images. If each image has a resolution of 1920 × 1200 pixels in 8-bit greyscale, the disk would have an equivalent storage capacity of approximately 576 GB. Furthermore, for a substrate of this thickness, we can estimate the angular selectivity of each grating to be ∼14.7 mdeg [13], which would then require an aperture of ∼14.7°for the full capacity of 1,000 images, a range that is achievable during the writing step. The proposed capacity is limited mostly by the diffraction efficiency of each image, which decreases by the square of N. The M # can be improved further by increasing the thickness or dye-concentration of the substrate. The capacity can otherwise be increased if locations at many different radii are used, by expanding the diameter of the disk, or by optically decreasing the size of the images.

Conclusions and outlook
Optical correlation has many advantages over its electronic or digital equivalent, but practical implementations have been difficult. We have recently made advances towards making the HOC fully functional, including demonstrating its use for shift, scale, and rotation invariant target recognition [6]. Here, we have proposed and demonstrated the feasibility of incorporating a PQ:PMMA HMD into the HOC architecture as a large-scale optical database, using it for shift, scale, and rotation invariant image correlation. The experimental results agree with the theory and provide a baseline for further development of high-capacity HMDs in this application.
The HOC's development is progressing in stages. In stage 1 we demonstrated the full functionality of the system for position, rotation, and scale invariant correlation with the help of computers to perform some electronic processing. The next stage in the HOC's development will be to fully automate its functionality through electronics, getting us closer to real-time operation and allowing for the full implementation of the scanning operation of the PSS arm to eliminate the convolution terms. We currently find ourselves between stages 1 and 2. In stage 3, fabrication of custom integrated circuits, both electronic and photonic, will aid in increasing the speed further as well as maintaining stability in the PSS arm. This final stage will finally permit the HOC to operate close to its ideal speed of ∼5µs per correlation. Disclosures. The authors declare no conflict of interest. Data availability. Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.