Fast reconfiguration algorithm of computer generated holograms for adaptive view direction change in holographic three-dimensional display

: Reconfiguration is a computational algorithm of adaptively updating computer generated holograms (CGHs) for the positional change of an observer’s viewing window with low computational load by efficiently using pre-calculated elementary CGHs. A fast reconfiguration algorithm of CGHs for three-dimensional mesh objects is proposed. Remarkable improvement is achieved in the computation speed of CGHs, which is at least 20-times faster than repetitive re-computation of CGHs. The image quality of reconfigured CGHs is analyzed.


Introduction
The most fundamental characteristic quantity of three-dimensional (3D) display is the information capacity, which is represented by the space-bandwidth product (SBP) [1,2].In general, it is well known that a fundamental trade-off relation exists between image resolution (space) and viewing angle (bandwidth) in 3D imaging [3].This is because the SBP of a 3D display system is a fundamental invariance of the system, but the image resolution and the viewing angle can be balanced within the limitation of the information capacity (SBP).This information capacity limitation makes the development of practical holographic 3D display with large-aperture and wide-viewing angle very challenging.In practice, the viewable space of a holographic 3D display is set to a narrow region to secure image size.Recently, attempts have been made to overcome the hindrance related to the SBP limitation by time-domain techniques.N. Leister et al. suggested a tracked viewing window (VW) technique [4,5].As an alternative way, Takaki and Okano proposed time-sequential binary holographic imaging methods independently [6,7].In the tracked VW technique, a specially designed eye tracking system senses the positional change of the VW of a freely movable observer and the VWlimited CGHs (referred to as sub-hologram) with parallax are adaptively re-computed in realtime to create parallax for the observer.The sub-hologram concept was devised to lighten the computation load by reducing the effective bandwidth of CGHs, but this real-time recomputation approach suffers from heavy computational burden still [8][9][10].
Various formats of CGHs and methods to enhance computation efficiency have been investigated.Linearity in CGH calculation can be exploited to parallel computation algorithm of CGH very naturally [8,9].In particular, the parallel computation using graphical processing units (GPUs) arises as the best solution to accelerate computation speed [10][11][12].The efforts to develop fast CGH algorithms are kept going with advances in computing hardware.For point cloud CGH model, various versions of look-up table (LUT) method and wavefront-recording plane method have been proposed [13,14].Implementation based on the fast Fourier transform (FFT) is also effective way to enhance computation efficiency [15].
However, for polygon or mesh CGH model, researches on fast algorithms have been relatively rare.In this paper, a novel approach to this problem called reconfiguration is proposed.The reconfiguration is a computational algorithm of adaptively updating CGHs of mesh 3D objects for the positional change of an observer's viewing window with low computational load by efficiently using pre-calculated elementary CGHs.Its feasibility is demonstrated numerically based on the recently developed semi-analytic CGH algorithm for 3D mesh objects [16].The fast calculation algorithm is derived from an analysis of the approximation of the mathematical formula of CGH, but the principle and algorithmic structure of the reconfiguration are accounted for with geometric interpretation.The image quality of the reconfigured CGHs and related optical effects are analyzed, and the computational advantages are discussed to show the validity and usefulness of the proposed algorithm.
In section 2, the model of mesh object CGH is briefly reviewed and the viewing characteristics of a holographic 3D image with limited SBP are addressed.In section 3, the reconfiguration algorithm is developed and its validity is proved with an analysis of image quality of the reconfigured CGHs.In section 4, concluding remarks are given.

Viewing characteristics of mesh object CGHs
Figure 1 shows the schematic of a holographic 3D display based on a Fourier transform optical system and the virtual object space created by the display.The finite size VW is opened at a specified position, where the human eye perceives a holographic image light field and forms corresponding 3D image on the retina via crystalline lens.We employ the numerical model of the mesh object CGH to synthesize and characterize holographic 3D images, where 3D objects in the image volume space are represented by the composite of triangle facets.The coherent superposition of the analytic complex angular spectrums of those triangle facet apertures in the image volume space are encoded in the form of a complex hologram in the CGH plane [11].Fig. 1.Holographic 3D display with a Fourier transform optical system.The complex angular spectrum of the image light field is displayed at the holographic display panel, and the observer can see floating holographic 3D images at a specific position where the crystalline lens is matched to the VW of the holographic display.
The delivery of the light field of a holographic 3D image to a VW without loss, i.e. vignetting, is crucial for an observer to correctly perceive the holographic 3D image.The basic building block of a CGH of a 3D mesh object is an elementary CGH that generates the light field of single triangle facet comprising the 3D mesh object.To deliver the light field of the kth single triangle facet to the VW, we have to use a directional carrier wave in the design of a CGH represented by the form of a local plane wave, where the direction of the wavevector, ( ) π α β γ , is determined by the direction connecting from the center of the VW ( ) x y z to the center of a triangle facet ( ) x y z , as indicated by the red arrow in Fig. 1.The first exponential term in Eq. ( 1) is the phase allocated to the triangle mesh facet, which is determined by the optical distance of the triangle facet from the center of the VW.Each triangle mesh comprising a 3D object is allocated with its own local plane carrier wave.This phase makes the wave fronts of the local carrier waves aligned spherically.The whole carrier wave profile for the 3D mesh object takes the form of a discrete bundle of local plane waves with a spherically aligned wavefront.
Let us assume that the kth triangle facet is placed on an infinite plane 0 where the exponential function of the carrier wave in Eq. ( 1) is immersed in the angular spectrum representation.In Eq. (2a), the localization operator ( ) which is used to restrict the angular spectrum to a range around the spatial frequency of the carrier wave, ( ) α β , with the circular bandwidth of a radius ρ .In practical computation, this localization function is employed to enhance the computational efficiency by reducing the computational size of the angular spectrum CGH (ASCGH), the angular spectrum of the triangle facet, ( ) A α β .The radius of the bandwidth is tuned into a value that does not induce considerable degradation in observed image quality.
The ASCGH of a holographic image is formed in the CGH plane (α-β plane).In this paper, the ASCGH of triangle facet is considered as the fundamental data format, and its description follows a previously developed theoretical framework [11].The ASCGH , ( , ) A α β is proportional to the analytic Fourier transform of the triangle and the auxiliary phase term related to spatial translation of light field as where ( ) is the analytic Fourier transform of the triangle facet, and the parameters in A , are expressed for α and β , respectively, as [11]: In Fig. 2, the vignetting effect of an example holographic 3D image with narrow viewing angle is simulated.In Figs.2(a) and 2(d), the light field profiles of non-vignetted and vignetted cases measured in the plane of a crystalline lens (see Fig. 1) is presented, respectively.The light field distribution shown in Fig. 2(a) is well concentrated within the 2mm-diameter pupil of a human eye.In this case, the holographic 3D image is reconstructed without distortion in the retina plane since the image information of a 3D scene is correctly transferred to the eye.In Figs.2(b) and 2(c), the holographic 3D images observed at two different foci are presented, and the results clearly show the accommodation (focus-defocus) effect, one of the 3D image cues.In the case of Fig. 2(d), the convergence of the carrier wave is not enough to concentrate the light field inside the pupil.As presented in Figs.2(e) and 2(f), the rejection of a part of incoming light field results in significant deterioration in observed images.These proof simulations manifest the physical meanings of the vignetting effect and the role of the converging carrier wave for displaying a narrow-bandwidth holographic 3D image.

Reconfiguration algorithm of CGH
In a holographic 3D display with narrow viewing angle, if the observer's position varies, serious deterioration of the image would occur without appropriate management.In Fig. 3, the observer's positional change and subsequent variations in imaging condition are illustrated.The VW at the initial position ( ) x y z is moved to a new position ( ) Let us consider what should be taken into account in updating an ASCGH so that the observer can see correct holographic 3D images.proper light field focused to the VW is generated by the display in real-time.In addition, the holographic 3D image of a triangle facet must be placed at the specified position ( ) x y z regardless of the change of carrier wave.This management is key to creating the correct motion parallax with-viewing angle of holographic 3D scenes.
For computation of the updated ASCGH, ( ) A α β , the simplest approach is to totally re-compute the ASCGH for the new VW and display the recomputed ASCGH to the observer simultaneously.The ASCGH for the new VW position takes the form of Here, it is argued that the re-computation of the Fourier spectrum L A is a heavy computational burden.
In this paper, we propose a novel algorithm of computing the updated CGH, ( ) efficiently referred to as reconfiguration algorithm.The mathematical derivation of the reconfiguration algorithm is elucidated and its physical meaning is interpreted based on the geometrical interpretation of the light wave propagation through the 3D display system.The first parameter of L A in Eq. ( 4) is analyzed as follows: ( where the paraxial approximation ( ) From the above analysis, it is proven that the first part of the CGH is a shift-variant function for the positional change of VW because of the nonlinear term of ( ) (5a).The second parameter is analyzed in shift-invariant form for ( ) The motivation of the reconfiguration algorithm is the enhancement of calculation speed of mesh object CGH, ( ) , , G k A α β , for the positional change of an observer.For efficient computation, we invoke an approximation in the calculation of , ( , )   G k A α β to use prepared

CGH data of , ( , )
G k A α β , which ignores the nonlinear term ( ) ).Then, the first parameter of L A in Eq. ( 4) is given by Consequently, with the approximation to the shift-invariant form, the view direction change ( ) Meanwhile, the third exponential term of Eq. ( 4) is manipulated with an approximation as follows; ( ) where , whose geometrical meaning is explained in later.The updated ASCGH is obtained as where c is a constant term, x y z .From this notion, x Δ and y Δ in Eq. ( 8) are equivalently defined, respectively, by Thus, the spatial shift by ( ) A α β , with multiplication, duplication and stamping operations of the pre-calculated elementary CGHs.The reconfiguration with elementary operations is expected to be more efficient than totally re-computing the CGH.There is similarity between the proposed reconfiguration and the LUT method for point cloud CGHs in an aspect that they use pre-calculated elementary patterns, but the reconfiguration is differentiated in that it manages mesh CGHs in the angular spectrum domain.In Fig. 4, the numerically reconstructed holographic 3D images with the reconfiguration are compared with those obtained by recomputation.The simulation results confirm that the holographic images obtained by the reconfiguration present parallax in 3D images clearly.The phase irregularity is revealed in the surface texture of the 3D object, but the overall shape of the 3D object and the accommodation effect are expressed well with the reconfiguration.The most advantageous point of the reconfiguration is the computation time.The CGH computation was performed in MATLAB with 2.4GHz Intel Xeon E5620 and 48GB of memory.An average of 0.58s is taken to calculate 1000 1000 × ASCGH of one triangle facet.In the reconfiguration method, only 0.029s is taken for reconfiguring a pre-calculated ASCGH of the same size.The dinosaur model shown in Fig. 5 has 2189 visible triangles.It takes about 21 minutes to be recomputed while the reconfiguration takes only 1 minute.An average 13 minutes (0.36s per a triangle) is needed for a common routine to stamp the elementary CGHs on the total 4000 4000 × ASCGH, but the code optimization will be able to reduce the time for stamping operation.In Fig. 6, it is schematically illustrated that a human eye observes the holographic image of a tilted triangle located in front of the reference plane.The distance of the VW from the origin is set to 300mm.For the specific horizontal position of the VW, the root-mean square errors (RMSE) of two holographic images calculated by the reconfiguration and recomputation are estimated, respectively.
We can see that the reconfiguration error is dependent on the tilting geometry of a triangle facet.The error estimation plots in Figs.6(a) and 6(b) present the cases of a triangle parallel to the x-y plane and a triangle tilted 45 degrees relative to the x-y plane, respectively.The RMSEs for the horizontal shift (x-axis distance) of the VW and the axial shift of the triangle object are plotted.The reference ASCGHs for reconfiguration are computed at the reference VW position that is positioned on the optical axis with 300 . From the data of the plots in Figs.6(a) and 6(b), we can see that the RMSE is basically proportional to the xdirectional horizontal shift of VW and the error due to the reconfiguration is inversely proportional to the effective projection area of the triangle to the observer at a specified horizontal shift.In the case of the non-tilted triangle, the effective projection area of the triangle for the VW position decreases as the triangle approaches the observer, and as a result, increase of the RMSE is observed, while for the 45 deg.tilted triangle, the effective projection area of the triangle increases as the triangle approaches the observer, so decreasing behavior of the RMSE clearly appears in Fig. 6(b).In this analysis, the overall RMSE is less than 20%.

Conclusion
We have proposed the reconfiguration algorithm of ASCGH for fast application to eyetracking-based holographic 3D display with large aperture and small VW.With numerical simulations, we have verified the feasibility and at least 20-fold enhancement of the computational efficiency without considerable image quality degradation.It is expected that the proposed reconfiguration algorithm can greatly enhance the efficiency of CGH content generation for real-time eye-tracking based holographic 3D displays.

φ
where the vector components of the normal vector of the triangle k are the longitudinal angle and azimuth angle of k n in the global Cartesian coordinate system.The light field of the kth single triangle facet is expressed by the angular spectrum form: #176061 -$15.00USD Received 12 Sep 2012; revised 15 Nov 2012; accepted 16 Nov 2012; published 6 Dec 2012 (C) 2012 OSA

Fig. 2 .
Fig. 2. Vignetting effect on holographic 3D image seen in observation simulation.(a) light field distribution at the eye pupil plane in the case of light convergence (b) observed 3D image with a focus on the far object (c) observed 3D image with a focus on the near object (d) light field distribution at the human eye pupil plane in the case that light field is vignetted by the finite human pupil (e) observed vignetted 3D image with a focus on the far object (f) observed vignetted 3D image with a focus on the near object .

Fig. 3 .
Fig. 3. Geometric interpretation of reconfiguration algorithm; (i) redirection of carrier wave and (ii) spatial translation of complex light field at the reference focal plane The direction vector of the carrier wave carrying the light field to the VW indicated by the red-lined vector ( ) , , k k k α β γ . The reconfiguration algorithm is defined by the two-step process based on Eq. (8): (i) the spatial translation of light field distribution at the reference plane and (ii) the redirection of the light field.The former part is performed by The obtained reconfiguration algorithm can be figured out intuitively with geometrical interpretation.The light field at the reference focal plane may be a forward or backward diffraction field of the tilted triangle facet according to the z-axial position of the tilted triangle ( 0k z < or 0 k z > , respectively).We should note that the diffraction light field of the triangle facet in the reference focal plane is the optical Fourier transform of the ASCGH in the angular spectrum CGH plane as shown in Fig.3.Hence the directional control of the light field of the triangle facet can be performed by translating the elementary ASCGH in the angular spectrum domain.The shift operation of the ASCGH, leads to a change in the directional vector of the light field from , respectively.As mentioned above, the important point to create correct motion parallax of the holographic image of a triangle facet is that the holographic triangle facet must be floating at its original position () .In a geometrical sense, to fix the position of the triangle facet regardless of the positional change of VW, the diffraction light field distribution of the triangle facet at the reference focal plane (x-y plane, 0 z = ) should be spatially shifted from its original position by x Δ and y Δ as indicated in Fig.3.The spatial shift ( ), y x Δ Δ is the point symmetric translation of the diffraction field in the reference focal plane to the positional change of the VW with respect to the symmetry point ( ) the reference focal plane is performed by the multiplication of the phase function .The aim of the #176061 -$15.00USD Received 12 Sep 2012; revised 15 Nov 2012; accepted 16 Nov 2012; published 6 Dec 2012 (C) 2012 OSA reconfiguration algorithm is to create motion parallax by elementary mathematical operations on the elementary ASCGHs without recomputation.In practice, we can replace the computation of

Fig. 4 .
Fig. 4. Comparison of the quality of the holographic images generated by reconfiguration and recomputation.(a) and (d) are the numerically reconstructed holographic 3D images observed at two different VW positions with the left part focused.For (a) and (d), the corresponding ASCGH patterns were totally recomputed.(b) and (e) are the numerically reconstructed holographic 3D images generated by the ASCGHs obtained by the reconfiguration of the original ASCGH that is designed for the VW located on the optical axis.(c) Phase profile of reconfigured complex ASCGH in the angular spectrum domain for the first VW position and (f) that for the second VW position.The distance of the center of mass of the object from the eye lens is set to 300mm.

Fig. 5 .
Fig. 5. Comparison of the quality and computation time of the holographic images generated by reconfiguration and recomputation.The height of the object is 8mm and the distance of the object from the crystalline lens plane is set to 300mm.(a) and (d) are the numerically reconstructed holographic 3D images observed at two different VW positions with the left part focused.The corresponding ASCGH patterns were totally recomputed.(b) and (e) are the numerically reconstructed holographic 3D images generated by the ASCGHs obtained by the reconfiguration of the original ASCGH that is designed for the VW located on the optical axis.(c) Phase profile of reconfigured complex ASCGH in the angular spectrum domain for the first VW position and (f) that for the second VW position.Next, the accuracy of the proposed reconfiguration algorithm is estimated.The accuracy degradation is mainly ascribed to the approximation of eliminating the nonlinear term ( ) 0 sin k θ γ γ − in Eq. (5).Considering the second nonlinear term in Eq. (5a), [ ] [ ] ( ) sin k

Fig. 6 .
Fig. 6.(a) Setup for analyzing the accuracy of reconfiguration algorithm and estimation of accuracy of the reconfiguration algorithm (b) RMSE (%) of a triangle facet on the x-y plane, and (c) RMSE(%) of triangle 45(deg.)-tiltedrelative to the normal vector of the x-y plane