CURRENT SUMMARY OF THE PRACTICAL USING OF OPTICAL CORRELATORS

Optical Computing has long been a reportable as the solution of real time high resolution data processing. Optical processing uses the fastest medium (light) to same calculations as serial electronic systems so process time of these calculations is shorter. The best results in data processing can be obtained by using hybrid, digital – optical, processors as a Optical Correlators which use speed of light (optical part) in cooperation with the electronic components, (digital part). Optical Correlator is optical device to compare two images or two – dimensional data in real time. There are two main types of Optical Correlators, Match Filter and Joint Transform Correlator. In this paper are described both types of correlators and there are shown examples of realized Optical Correlators. Optical Correlators use Optical Fourier Transform to produce transformed image from input image and then second Fourier Transform to obtained correlations in correlation plane. Optical Fourier Transform vs. Fast Fourier Transform process time comparison will be discussed. Main components of Optical Correlators will be presented. Optical Correlators thanks its fast process time and recent advances in liquid crystal technology become more commercially available. Potential applications of these optical processors will be also presented and discussed.


INTRODUCTION
Optical Correlator (OC) is device to compare two or more images or two dimensional (2D) data sets in very high speed or in real time [1,5,12,18,24].Optical correlation is defined as the product of Fourier Transform of the two original functions, s(x,y) and r(x,y) which have themselves been Fourier transformed [1].There are two main types of OC, first based on match filtering called as Match Filter Correlator (MFC) and the other is based on Joint power spectrum, called Joint Transform Correlator (JTC).The Optical Fourier Transform (OFT) is one of the fundamental building blocks of frequency analysis of images [11,18].This transformation decompose twodimensional signal or image into its constituent frequency parts [1,3,11,12,15,18,23,25].
The method uses the OFT in OCs will be presented in Chapter 2. In this chapter also will be presented OFT vs. FFT (Fast Fourier Transform) process time comparison.
In Chapter 3 two types of OC will be presented and compared.All main components will be specifically described.
Chapter 4 will show realized OCs.In this chapter also applications of OC will be discussed.
Conclusions are covered in Chapter 5.

2D Fourier Transform
The Fourier transform (FT) of function g of two independent variables x and y will be represented here by F{g} and is defined by [1,2,20,21] { } ( , )exp[ 2 ( )] X Y F g g x y j f x f y dxdy On Fig. 1 is showed 2D function g(x,y) (square) and its FT, F(g).These examples were produced using MATLAB simulation software package.

Optical Fourier Transform Implementation
FT of 2D data (image) as is defined above can be performed using mathematical operations (FFT) or using optical lens (OFT) [10,19,21,22].The optical lens makes in focus of the lens the FT of the input image [1].Fig. 2 shows OFT using optical lens.Using second optical lens we get inverse FT and then get original image, as it is showed on Fig. 2.
Using masking filter in Transform plane we can make frequency filtering and remove some lines, shapes or contours from origin image.Using filters we obtain new modified image [1,2,3].Optical processing (Fourier transform and filtering) is a truly parallel method of calculation, using the fastest medium -the light.Because the process is parallel in nature, the process time does not scale with image resolution.This allows very large images to be FT at the speed of the light [1, 2, 4, 9, 19].In next diagram is compared process time between 2D FFT and OFT.The diagram shows process time for 2D FFT calculation using different processors (Xilinx Virtex -4 series -500MHz; TMS329C67 of DSP processor -1 GHz; Intel processor -3,74 GHz ) and process time for Optical transformation [4].

OPTICAL CORRELATORS
There are two main types of OC as was mentioned above.First type is most commonly used type called Matched Filter Correlator (MFC) based on Vanderlugt Filter.The second type is called Joint Transform Correlator (JTC) based on Joint Transform.It`s very sensitive to build, long optical train and owning to its strict alignment criteria, but still is commonly used type of OCs [2,4,9,19,20].
In Fig. 4, the principle of this correlator is described.Correlation consists of two independent correlations, correlation of input, s(x,y), and reference, r(x,y), image.Input scene is showed on Spatial Light Modulator (SLM) then is Fourier Transformed.Reference image is Fourier transformed off -line to make Matched Filter.This filter is used as a reference filter in FT plane [4].These transformed images are optically multiplied and using second lens are Fourier transformed again.
Correlation can be also written in form Operation sr means correlation between input and reference image and F is Fourier Transform.S(α,β) is Fourier Transformed input image and R*(α,β) is complex conjugate function of R(α,β) [1,2,3,9].
The most advantages of this method are high spacebandwidth product and extremely fast process time, but disadvantage is the need to create reference filter, MF from reference image, so also need reference image before correlation process [4,5].Output of this correlation is one correlation peak per match [2].

Joint Transform Correlator
The Joint Transform Correlator (JTC) was invented by Weaver and Goodman in 1966 and is traditionally the less commonly used type.The process differs significantly from the MF process, since no reference filter is required (Fig. 5) [4,5,9].JTC use Input and Reference image aligned and displayed on SLM.These before two images are now transformed together.FT of this aligned image produce Joint Power Spectrum.This spectrum is then threshold or binary processed to produce cross correlation.Output of this transform is correlation plane include correlations peaks per match.This found match is shown as highly localized intensities and the intensity of peaks provides a measure of similarity of the images being compared whilst position of spots denote how the images are relative aligned in the input scene [1,2,3,5,9].
Input plane of correlation process we can write as where s(x,y) and r(x,y) are input and reference images showed in input plane and create joint input scene v(x,y).
This input scene is then Fourier transformed, to produce V(α,β).
Using next equations is optical correlation produced by the square of absolute value of transformed input scene.
) ( , ) The square of absolute value of complex function is equal to multiplication of function with its complex conjugated function a then we can write [2,3] The result of equation ( 6) is showed in Fig. 6.

Fig. 6 Output of Joint Transform Correlator
The most advantages of this method of correlation are the simplified optical train (and therefore much cheaper to produce) and no strict alignment criteria.
JTC is actually correlator, which make autocorrelation from input image showed in input plane.As was mentioned above, position of correlation peaks is strongly addicted on position of images in input plane.Distance between these images in input plane is half of distance between correlation peaks in correlation plane.
That is, if distance between image and centre of input plane is x, then distance between correlation peak and centre of correlation plane is 2x (Fig. 7) [1,4,5,9].

PRACTICAL APPLICATIONS OF OPTICAL CORRELATORS
There are two types of optical correlators, MFC and JTC mentioned above, which can be used in many applications [4,5].Each of types has its advantages, which are important to specific applications.These correlators can be divided according type of object recognition.They can recognize faces or other objects, they can use Hilbert or Fourier Transform, they can be divided to two-dimensional or three-dimensional correlators or they can be used to Medical Analysis, Biometric Recognition, Motion Tracker, Traffic Sign Recognition, High Volume Data Searching and many other applications [4,5,7,8,9].Using of Optical Correlator is very helpful in applications, where fast process time is necessary [6,7,8].

Fast Face-Recognition Optical Correlator
Fast Face-Recognition Optical Correlator (FARCO) was developed at Japan Women`s University at Faculty of Science [6].This system has three different configurations depending on its recognition rate.There is shown improvement on one-to-one ID recognition system, which requires little calculation time.The FARCO system is a hybrid optical correlator that integrates an optical correlation technology and digital database.FARCO can be applied to several hundreds of images, with its operation speed of 1000 to 5000 faces per second.Due to greater demand to recognition, the S-FARCO (Super Fast Face-Recognition Optical Correlator) system is used [6].The S-FARCO is equipped with a holographic optical memory, which could store and process information optically.This enables optical correlation without decoding information in the database, greatly reduces processing time.In Fig. 8 block diagram is showed [6].
Fig. 8 Block diagram of FARCO [6] The facial image display SLM in the database is composed of an FLC-SLM (Ferroelectrics Liquid Crystal -SLM), with the capacity of high speed display (2500 frames per second).The FLC-SLM, featuring a reverse display, was constructed with a Laser diode of wavelength 635nm as a light source and driven by a pulse, flickering at positive values.NLC-SLM (Nematic LCl -SLM) is used as a spatial optical modulator for the display of the correlation filter.Moreover the MLZPA (Multi Level Zone Plate Array) is used for optical parallel processing.As receiving device, photo diode, divided into four parts was used.The diode can simultaneously receive four signals at the speed of 20 MHz.An optical isolator was installed as a reflection-proof instrument.Optical components are detailed described in Table 1 [6].

256x256 Binary Optical Correlator
The 256x256 Binary Optical Correlator utilizes two SLMs for inputting the data [7].It was constructed by Kipp A. Bauchert and Steven A. Serati, Boulder Nonlinear Systems, Inc., Boulder CO 80301.The output is a Dalsa 256x256 high-speed CCD camera with a maximum sustained throughput of 220 Hz.The 256x256 binary SLMs can modulate light in binary amplitude or binary phase.The type of modulation is selected by rotating an output analyzer.Current drive electronics refresh the SLM at sustained rate of over 18 kHz and support a useful frame rate of over 200 Hz.In Fig. 9 is shown block diagram and photo of 256x256 binary correlator [7].

Fig. 9 Block diagram and photo of 256x256 Binary Optical
Correlator [7] The theoretical full frame load time of the 256x256 VLSI chip is approximately 25 μs.Each SLM is driven from memory bank of 512 images.The Dalsa 256x256 feeds into grabber for capturing the correlation images.Some experiments are shown in Fig. 10 [7].

Hybrid Optoelectronic Joint Transform Correlator for the recognition of target in cluttered scenes
Hybrid Optoelectronic Joint Transform Correlator (HOJTC) exploiting the FT property of a lens implements target detection in real time [8].It was constructed at Changchun University of Science and Technology, Changchun, China.
Optical correlation methods are well established for target detection and recognition.The intensity of the cross-correlation between reference template and the target is decreased greatly when the target is in cluttered scenes [8].

Structure of the HOJTC
In order to detect and distinguish target in real time, the JTC was used, instead of 4f system.Comparing with MFC (Vanderlugt Correlator) has the characters of realtime recognition and auto -adaptation.The architecture of this HOJTC is shown in Fig. 11 [8].Light is launched from the Ar + laser and then its energy is adjusted through the attenuation filter.The laser power budgets through optical correlator are quite critical and should depend on the experiments.The pinhole in succession performs the function of special filtering.The laser beam must be expanded and collimated to uniformly a semi-reflection mirror, which illuminates the EALCD 1 and the EALCD 2 respectively images are recorded in real time and are inputted into the EALCD 1 and then FT by FTL 1 .
The EALCD 1 controlled by a microcomputer displays the signals of inputted target and template.The target The Joint Transform Power Spectrum is recorded by the CCD2 camera, inputted into computer and filtered by FT of Mexico-hat function.The filtered Joint Transform Power Spectrum is fed back to the EALCD 2 .The two EALCD are both placed close to the Fourier Transform (FT) lens [8].
After FT of the filtered Joint Transform Power Spectrum by FTL 2 , CCD3 recorded the correlation peaks of the joint image of the template and the target images.

Experiment with HOJTC
In experiment, an Ar + laser with a power 150 mW and a wavelength of 514 nm was used as the light source [8].
The laser beam was led through a variable neutral density filter, focused by a microscopic objective lens and spatially filtered.Through a collimator, the laser beam was expanded and transmitted through the EALCD1.The EALCD has the resolution of 800×600 pixels, in which the pixel transmission depended on the gray value that was written in.
In a Fig. 12 the reference image coupled with target is shown [8].These images are showed in one image and together enter to correlation process, (principle of Joint Transform Correlators).Fig. 12 Input Scene for HOJTC [8] In a Fig. 13 the results of correlation are showed [8].In Fig. 13a is showed unprocessed Joint Transform Power Spectrum (JTPS) of input image (Fig. 12), and in Fig. 13b is correlation obtained from unprocessed JTPS.We can see that the intensity of the correlation peaks is so weak that we cannot see it from the picture [8].
In Fig. 13c is JTCS showed processed by wavelet filtering.Fig. 13d present the cross correlation of Joint Transform Power Spectrum processed by wavelet filtering.This processing of JTPS allowed better detection of correlation peaks in correlations plane.Obviously, the intensity of the correlation peaks has been increased after wavelet filtering (red circles).

Cambridge Correlator
Cambridge Correlator (CC) belongs to family of Joint Transform Correaltors and was constructed by Cambridge Correlators Ltd.This device is coupled with software "Fourier Transform Experimenter", which allows students and researchers to practically explore a range of topics based around the OFT [4,5].

Optical system of Cambridge Correlator
Optical system of CC is based on Fourier Transform Engine© -a compact, yet hugely powerful processing system, built on the diffraction principles of the Optical Fourier Transform.Its new "W" design allows the electro -optical components to operate at their full potential.13 Correlation process [8] In Fig. 14 are shown main parts of CC [3].In the Table 2 are parameters of main components described [4,5].The SLM is driven from DVI port of PC via its own drive board.Images displayed on SLM are illuminated with a collimated low power laser beam, launched through an optical fibre.The light is modulated by images displayed in the SLM and then projected through the lens system [4,5].

Fourier Optics Experimenter
CC is coupled with software Fourier Optics Experimenter (FOE), which allows the user to run the FOE both Optical Fourier Transform (OFT) and Joint Transform Correlator (JTC) modes in easy to use graphical user interface.The OFT mode involves a single "pass" of the optical system, whereas the JTC mode requires two "passes".
Images and videos can be loaded into multiple tiles in input scene and furthermore, the user can add multiple files to each tile in input scene for sequential image correlation or FT analysis.
Correlation can be done between two or more images or videos.In Fig. 15 the correlation between three images is shown.In this experiment we can show that correlation process time do not increase with number of compared object.Correlation time is same as we compare two, three or more images.Number of compared object is limited just its size and resolution of SLM and Camera in FOE kit.In Fig. 15a we can see input plane of correlation process, where three same symbols (symbols of children) are shown.In Fig. 15b the Joint Transform Power Spectrum is shown.This spectrum is processed using Nearest Neighbours method and shown in Fig. 15c.
Correlation of three same symbols consist of three pairs (six) correlation peaks.In Fig. 15d we can see that peaks in green circles were produced from symbols in Fig. 15a localized horizontally.Position of correlation peaks is strongly depended upon position of compared symbols in input plane.
These advantages can be very helpful in many applications.Real-time recognition is really important aspiration in many systems.Cambridge Correlators Ltd report said wide use of their device in Medical Analysis, Biometric Recognition, High Volume Data Searching, Traffic Sign Recognition and many others [4,15].
In our department, Department of Electronic and Multimedia Communications, is available.Cambridge Correlator is used to design system of Traffic Sign Recognition System [10,13,14,16,18].

Experiments with Traffic Signs Recognition System
In experiment with OC based Traffic Sign Recognition System, four types of traffic signs are compared ("STOP", "CROSSWALK", "MAIN ROAD" and "YIELD") [13,14].
Hardware scheme of proposed system is showed in Fig. 16.As we can see, system consists of four main components.Traffic scene is captured by Color Camera and then is pre-processed in Computer (Region of Interest is detected).
Pre-processed image (detected ROI) and database of traffic sign are showed in input plane of correlation process of CC.Output of correlation (intensity and position of correlation peaks) are processed in Computer and recognized traffic sign is showed in Display.
For this study traffic scenes from road of Slovakia in daytime lighting conditions are taken, in urban traffic environment [10,16].Examples of Real Traffic Scenes used in experiments are showed in Fig. 17.
Input plane of correlation process is showed in Fig. 18a.The reference traffic signs are situated in top row and in a bottom row is tested traffic sign.Tested traffic signs were obtained from real traffic scenes (Fig. 17) and were pre-processed with colour filtering and detecting ROI.
Reference and Input traffic signs were pre-processed using Roberts -Cross operators to detect edge in these signs, to minimize DC noise in correlation plane [17].Intensity of obtained peaks (Fig. 18b) is represented in 8 bit depth greyscale, so every peak can take values from 0 to 255.Correlation peaks has exactly defined area of occurrence in correlation plane.Thanks these areas we can determine which traffic sign was recognized [10,16].Output of system is showed in Fig. 19.In Fig 19 we can see, that the highest intensity of correlation peak was detected in 4 th ROI, the ROI of Yield.All ROIs were exactly determined.The exactly positions of each ROIs were obtained by testing.In ROI 1, 2 and 3 were not detected any correlation peaks, or were detected only peak from DC noise.The intensities of peaks in these ROIs are much lower than intensity of peak in ROI 4, so we can definitely determine that the traffic sign in this position was recognized.
In these experiments were tested some items of each type of traffic signs (Table 3).From experiments we can see that the traffic sign of STOP and CROSSWALK has higher average intensity of correlation peaks because these types of signs contain more details than YIELD and MAIN ROAD.
Thanks this knowledge we can set decision level in correct peak position.The results of experiments are stored in Table 3.The average success of recognition was 93% for all traffic sign.

CONCLUSIONS
In this paper were presented principle of OFT and the application of this transform in optical processors as Optical Correlators.Both types of Optical Correlators, Matched Filter Correlators and Joint Transform Correlators, were detailed described.Practical applications of OC were discussed and some examples were detailed described, as FARCO, 256x256 Binary Optical Correlator, HOJTC and Cambridge Correlator.
Traffic Sign Recognition System with using Cambridge Correlator was presented.Some experiments have been done with this system.Traffic scenes were captured in road of Slovakia.These scenes were preprocessed to determine ROI and Cambridge Correlator were used to compare ROIs with database of traffic signs.Results of these experiments were stored in Table 3.
Usage of Optical Correlator has great significance in many applications, were fast processing of images or video is required or necessary.Advantages of these processors bring great potential to use Optical Correlator in many applications in all kinds of industry such as Traffic Sign Recognition System and many others.

Fig. 1
Fig. 1 Square and its Fourier Transform

Fig. 3
Fig. 3 FFT and OFT Process Time Comparison

Fig. 11
Fig.11Hybrid Optoelectronic Joint Transform Correlator[8] FTL 1 and FTL 2 are the FT lenses, while EALCD 1 and EALCD 2 are the SLMs.Light is launched from the Ar + laser and then its energy is adjusted through the attenuation filter.The laser power budgets through optical correlator are quite critical and should depend on the experiments.The pinhole in succession performs the function of special filtering.The laser beam must be expanded and collimated to uniformly a semi-reflection mirror, which illuminates the EALCD 1 and the EALCD 2 respectively images are recorded in real time and are inputted into the EALCD 1 and then FT by FTL 1 .The EALCD 1 controlled by a microcomputer displays the signals of inputted target and template.The target The Joint Transform Power Spectrum is recorded by the CCD2 camera, inputted into computer and filtered by FT of Mexico-hat function.The filtered Joint Transform Power Spectrum is fed back to the EALCD 2 .The two EALCD are both placed close to the Fourier Transform (FT) lens[8].

Fig. 14
Fig. 14 Main components of Cambridge Correlator

Fig. 15
Fig. 15 Comparing of three same symbols

Fig. 16
Fig.16 Hardware scheme of Traffic Signs Recognition System based on OC[16]

Table 2
Main components of Cambridge Correlator

Table 3
Traffic Signs classification rate