Semi-analytic texturing algorithm for polygon computer-generated holograms

A texturing method for the semi-analytic polygon computergenerated hologram synthesis algorithm is studied. Through this, the fullpotential and development direction of the semi-analytic polygon computergenerated holograms are discussed and compared to that of the conventional numerical algorithm of polygon computer-generated hologram generation based on the fast Fourier transform and bilinear interpolation. The theoretical hurdle of the semi-analytic texturing algorithm is manifested and an approach to resolve this problen. A key mathematical approximation in the angular spectrum computer-generated hologram computation, as well as the trade-offs between texturing effects and computational efficiencies are analyzed through numerical simulation. In this fundamental study, theoretical potential of the semi-analytic polygon computer-generated hologram algorithm is revealed and the ultimate goal of research into the algorithm clarified. ©2014 Optical Society of America OCIS codes: (090.1995) Digital holography; (090.2870) Holographic display. References and links 1. J. Hong, Y. Kim, H.-J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display technologies of recent interest: principles, status, and issues [Invited],” Appl. Opt. 50(34), H87–H115 (2011). 2. R. Häussler, A. Schwerdtner, and N. Leister, “Large holographic displays as an alternative to stereoscopic displays,” Proc. SPIE 6803, 68030M, 68030M-9 (2008). 3. N. Leister, A. Schwerdtner, G. Füutterer, S. Buschbeck, J.-C. Olaya, and S. Flon, “Full-color interactive holographic projection system for large 3D scene reconstruction,” Proc. SPIE 6911, 69110V (2008). 4. http://www.israel21c.org/health/revolutionary-hologram-guided-heart-surgery-is-a-heartbeat-away/ 5. M. Janda, I. Hanák, and L. Onural, “Hologram synthesis for photorealistic reconstruction,” J. Opt. Soc. Am. A 25(12), 3083–3096 (2008). 6. T. Ito, N. Masuda, K. Yoshimura, A. Shiraki, T. Shimobaba, and T. Sugie, “Special-purpose computer HORN-5 for a real-time electroholography,” Opt. Express 13(6), 1923–1932 (2005). 7. E. Zschau, R. Missbach, A. Schwerdtner, and H. Stolle, “Generation, encoding, and presentation of content on holographic displays in real time,” Proc. SPIE 7690, 76900E (2010). 8. L. Ahrenberg, P. Benzie, M. Magnor, and J. Watson, “Computer generated holography using parallel commodity graphics hardware,” Opt. Express 14(17), 7636–7641 (2006). 9. T. Shimobaba, T. Ito, N. Masuda, Y. Ichihashi, and N. Takada, “Fast calculation of computer-generatedhologram on AMD HD5000 series GPU and OpenCL,” Opt. Express 18(10), 9955–9960 (2010). 10. T. Shimobaba, H. Nakayama, N. Masuda, and T. Ito, “Rapid calculation algorithm of Fresnel computergenerated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display,” Opt. Express 18(19), 19504–19509 (2010). 11. H. Nakayama, N. Takada, Y. Ichihashi, S. Awazu, T. Shimobaba, N. Masuda, and T. Ito, “Real-time color electroholography using multiple graphics processing units and multiple high-definition liquid-crystal display panels,” Appl. Opt. 49(31), 5993–5996 (2010). 12. Y.-Z. Liu, J.-W. Dong, Y.-Y. Pu, B.-C. Chen, H.-X. He, and H.-Z. Wang, “High-speed full analytical holographic computations for true-life scenes,” Opt. Express 18(4), 3345–3351 (2010). #224568 $15.00 USD Received 9 Oct 2014; revised 26 Nov 2014; accepted 26 Nov 2014; published 9 Dec 2014 (C) 2014 OSA 15 Dec 2014 | Vol. 22, No. 25 | DOI:10.1364/OE.22.031180 | OPTICS EXPRESS 31180 13. Y. Pan, Y. Wang, J. Liu, X. Li, and J. Jia, “Fast polygon-based method for calculating computer-generated holograms in three-dimensional display,” Appl. Opt. 52(1), A290–A299 (2013). 14. M. Lucente, “Interactive Computation of holograms using a Look-up Table,” J. Electron. Imaging 2(1), 28–34 (1993). 15. J. Cho, J. Hahn, and H. Kim, “Fast reconfiguration algorithm of computer generated holograms for adaptive view direction change in holographic three-dimensional display,” Opt. Express 20(27), 28282–28291 (2012). 16. H. Nishi, K. Matsushima, and S. Nakahara, “Rendering of specular surfaces in polygon-based computergenerated holograms,” Appl. Opt. 50(34), H245–H252 (2011). 17. T. Ichikawa and Y. Sakamoto, “A rendering method of background reflections on a specular surface for CGH,” J. Phys. Conf. Ser. 415, 012044 (2013). 18. K. Matsushima, “Wave-field rendering in computational holography,” 9th IEEE/ACIS International Conference on Computer and Information Science, 846–851 (2010). 19. T. Ichikawa, K. Yamaguchi, and Y. Sakamoto, “Realistic expression for full-parallax computer-generated holograms with the ray-tracing method,” Appl. Opt. 52(1), A201–A209 (2013). 20. H.-G. Lim, N.-Y. Jo, and J.-H. Park, “Hologram synthesis with fast texture update of triangular meshes,” Digital Holography and 3D Imaging Technical Digest, DW2A.8 (2013). 21. T. Kurihara and Y. Takaki, “Shading of a computer-generated hologram by zone plate modulation,” Opt. Express 20(4), 3529–3540 (2012). 22. K. Matsushima, S. Nakahara, Y. Arima, H. Nishi, H. Yamashita, Y. Yoshizaki, and K. Ogawa, “Computer holography: 3D digital art based on high-definition CGH,” 9th International Symposium on Display Holography (ISDH2012), 012053 (2013). 23. M. A. Neifeld, “Information, resolution, and space-bandwidth product,” Opt. Lett. 23(18), 1477–1479 (1998). 24. Y. Takaki and Y. Hayashi, “Increased horizontal viewing zone angle of a hologram by resolution redistribution of a spatial light modulator,” Appl. Opt. 47(19), D6–D11 (2008). 25. T. Mishina, M. Okui, and F. Okano, “Viewing-zone enlargement method for sampled hologram that uses highorder diffraction,” Appl. Opt. 41(8), 1489–1499 (2002). 26. H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008). 27. D. Im, E. Moon, Y. Park, D. Lee, J. Hahn, and H. Kim, “Phase-regularized polygon computer-generated holograms,” Opt. Lett. 39(12), 3642–3645 (2014). 28. L. Ahrenberg, P. Benzie, M. Magnor, and J. Watson, “Computer generated holograms from three dimensional meshes using an analytic light transport model,” Appl. Opt. 47(10), 1567–1574 (2008).


Introduction
Recently, the computational algorithms of computer-generated holograms (CGHs) have been actively researched.Promising technological applications such as holographic threedimensional (3D) displays [1][2][3] and holographic medical imaging technologies [4] have motivated world-wide research interest and formed a competitive CGH development environment.The technological goal of this research is the real-time generation of photorealistic holographic scenes [5][6][7][8].Therefore, in the field of CGH algorithms, most research effort is directed towards the development of fast computation algorithms using parallel computing devices such as GPUs [9][10][11][12][13][14][15] and representation theories such as those about texturing [16][17][18][19][20], shading [21,22], and the creation of hybrid scenes from 2D and 3D images [16,17].However, the real-time computation of large-scale wide-viewing angle CGHs [23][24][25] are still challenging.In practice, the numerical representation of wide-viewing angle CGHs with a wavelength scale pixel pitch leads to a large-scale complex matrix.The real-time management of such a large-scale data requires huge computing resources and advanced computing devices.Also, more advanced efficient mathematical models of CGHs need to be developed.
According to the mathematical models of 3D target objects, CGH synthesis algorithms can be classified into (i) point cloud CGH synthesis algorithms [5,6] and (ii) polygon CGH synthesis algorithms [18,22,[26][27][28].Within these two basic frameworks, various algorithmic variants have been derived to enhance computational efficiency and holographic image quality.In the point cloud CGH model, the phase and amplitude profiles of the point cloud distribution representing target 3D objects are individually controlled.In fact, a massive point cloud set has the maximal degree-of-freedom to represent arbitrary textures on the surface of holographic 3D objects.The phase and amplitude values of each point in the point cloud 3D model can be easily modulated to represent intended surface texture effects.However, computation times can become a serious issue.On the other hand, polygon CGH algorithms, a parallel research direction, have been intensively researched.In this model, 3D objects are represented by a collective continuum of light fields of unit triangular facets.From a technical point of view, polygon CGH synthesis algorithms can be sub-divided into the full-numerical method based on the fast Fourier transform (FFT) [16,18,22] and the semi-analytic method based on the mathematical formula of the angular spectrum of unit polygon facet [12,[26][27][28].The full numerical implementation [16,18,19] based on the FFT is considered effective in terms of not only computational efficiency but also expressiveness.The key parts of the FFT based method are (i) the regular grid FFT calculation of the angular spectrum of a unit triangle facet in its own local coordinate and (ii) the bilinear interpolation based coordinate transformation of the angular spectrum from the local coordinate system to the global coordinate systems.This total process can be referred to as non-uniform FFT based CGH synthesis.The non-uniform grid inside the non-uniform FFT should be adaptively changed according to the tilting geometry of unit triangular facet.Thus, the algorithm is composed of two separate steps.The potential for theoretical improvement of the FFT-based method seems limited.That is, the lower limits of the FFT-based method have already been established and stabilized, and research into further enhancing the method's computational efficiency is now oriented towards parallel processing based on advanced computing hardware such as substantial improvements in computation efficiency.Actually, this research line does not fundamentally modify the mathematical basis of the algorithm or the CGH information, but simply makes more efficient use of advanced parallel machines using the same mathematical framework [8,9].
Meanwhile, the motivation for the development of the semi-analytic polygon CGH algorithm [26] was the efficient computation of the required non-uniform Fourier transform using the de facto standard mathematical formula.From this respect, the research contrasts that recently carried out on the conventional fully numerical FFT-based method.The semianalytic polygon CGH algorithm seems to have been successful for 3D objects that are composed of bare triangle facets and has increased the possibility of achieving substantial improvements in computational efficiency and accuracy.
Regarding expressiveness such as texturing that is a fundamental element for the realistic rendering of reconstructed holographic 3D objects, the FFT-based method has an advantage of that its computational complexity is constant regardless of whether the 3D objects are textured or bare.In this method, texturing is performed directly using an approthe FFT of the textured polygon including the texture pattern with a non-uniform interpolation technique.In the case of the semi-analytic polygon CGH mode, however, the texturing problem is unexplored and remains a theoretical hurdle.
In this paper, this texturing problem is precisely defined and the difficulty is clearly manifested.An approach to tackle this problem is described and evaluated with numerical simulations.The basis of the devised approach is the mathematical approximation used in the calculation of the angular spectrum of unit polygon facet.The trade-off between texturing effects and computational efficiency is analyzed.This fundamental study on semi-analytic CGH paves the way to novel semi-analytic fast textured CGH algorithm, allows an understanding of their theoretical potential, and clarifies the research direction necessary to achieve the ultimate goal of real-time photorealistic CGH generation.

Holographic light field representation of textured polygon CGH
In this section, the mathematical expression of a holographic light field distribution of textured polygon objects is described and the related computational problem isaddressed.In the derivation process, we follow the same theoretical framework as in the previous papers [15,26].A target 3D polygon object composed of 81 triangular facets is used as an example for the analysis.The bare 3D polygon objects without and with texture are compared in Fig. 1(a) and 1(b), respectively.
Let us consider the light field distribution and angular spectrum of the kth triangular facet with the center of mass, ( ) , in the global coordinate system as shown in Fig.
Here, the angular spectrum of the holographic light field, ( ) A α β , is formed in the ASCGH plane.Let us refer to it as the angular spectrum CGH (ASCGH) and formed in the ASCGH plane (α-β plane).
The local coordinate system ( ) , , x y z ′ ′ ′ of the titled triangular facet is constructed as shown in Fig. 1(c) such that the z′ -axis is matched to k n .A point ( ) , , x y z ′ ′ ′ in the local coordinate system corresponds to the point ( ) , , x y z in the global coordinate system by the rotational matrix transform in Eq. ( 2), cos cos cos sin sin sin cos 0 .sin cos sin sin cos Similarly, the vector in the local coordinate system, ( ) ′ ′ ′ , is also related to the spatial- frequency vector in the global coordinate system, ( ) α β γ , by the same rotational matrix transform.Let ( ) ′ ′ ′ represent the carrier wave vector in the global and local coordinate systems, respectively [15].Using the coordinate transform of Eq. ( 2), the light field of Eq. ( 1) can be expressed in the local coordinate system as follows: where γ ′ is defined by and λ is the free space wavelength of the light field.The ASCGH, ( ) [15,26,28].The ASCGH in the global coordinate system is solved for ( ) where the parameters in Eq. (4a), and k β , respectively, as ′ ′= [26].Next, let us consider the texturing process of the bare triangular facet.The texture function is given by a complex function ( ) , T x y ′ ′ in the local coordinate system.It is assumed that the texture pattern of ( ) The complex light field distribution of the textured triangle facet, , .
By substituting Eq. (3) and Eq. ( 5) into Eq.( 6), we can obtain the angular spectrum representation of ( ) , , ,0 where the angular spectrum of the textured triangular facet is identified as which is equivalent to the discrete convolution of the shifted angular spectrums of the triangle facet and the Fourier series coefficients of the texture pattern.
In the global coordinate system, the ASCGH of the textured triangular facet, ( ) The partial light field distribution of the textured triangular facet in the global coordinate system is represented by The computation of Eq. ( 9) becomes an important issue and practical computation strategies for Eq. ( 9) are described in the following section.

Computational textured CGH synthesis algorithms
The computation of the ASCGH of the textured triangular facet of Eq. ( 9), ( ) be approached from two different directions: the FFT-based texturing algorithm or the semianalytic functional method for the synthesis of textured polygon CGHs.

FFT-based textured CGH synthesis algorithm
The FFT-based texturing algorithm takes two sequential steps: (i) the computation of the sampling matrix of Let us employ an abstract operator description of the CGH synthesis and texturing processes.The aforementioned process can also be abstractly modeled as ( ) where L Λ represents the angular spectrum of the textured triangle facet in the local coordinate system and L G I → represents the overall coordinate transformation which may contain a nonlinear interpolation process.The subscript L G → means that the coordinate transformation from the local coordinate to the global coordinate systems is conducted.Likewise, the ASCGH of a bare triangular facet without texture can be expressed by ( ) ∆ where L ∆ represents the angular spectrum of the bare triangular facet.
It is noted that in the FFT-based method, the computation times of A are approximately the same since the FFT computation is applied in the same way regardless of whether a triangular facet is textured or non-textured.Particularly, the time-consuming resampling process L G I → is required in the FFT-based method whether the triangular facet is textured or not.One of the major advantages of the semi-analytic CGH algorithm, however, is that the resampling process from the local to global ASCGH can be omitted.are shown in Fig. 3.The analytic result G ∆ shows faster computation since the coordinate transformation process L G I → can be carried out directly in the analytic formula G ∆ , so the ASCGH is a mathematically rigorous form which does not include any distortion unlike the FFT-based method ( ) L G L I → ∆ .Although the semi-analytic method is faster and more accurate for non-textured polygons, no analytic model for textured polygons has been reported until now.In the following section, the details of our trial to develop a semi-analytic approach to the texturing problem are described.

Semi-analytic textured CGH synthesis algorithm
A texturing method retaining the analytic calculation of G ∆ is investigated.The main problem is how to efficiently treat Eq. ( 9) using the analytic formula of G ∆ .As seen in Eq. ( 9), when the process for synthesizing textural patterns is included, the analytic method loses the advantage of being a perfect analytic method.If the convolution sum of Eq. ( 9) is used directly, we need to repeatedly calculate ( ) variables α ′ and β ′ .In order to resolve this problem, we use a mathematical approximation and change the order of operation for the calculation of Eq. ( 9).We take the approximation in Eq. ( 9) for a uniformly sampled discrete spatial frequency point ( ) , , cos cos cos sin sin , where the subscripts p and q represent the ( ) , p q th sampling point in the global spatial frequency coordinate α and β axes.Here, let us assume that we take a plausible approximate function, Γ , for p m α − and q n β − , designed to effectively approximate , , p q m n γ γ − , as which can make Eq.(11a) a shift-invariant form of Eq. (12b) for p m α − and q n β − represented as ( , , cos cos cos sin , sin cos cos cos sin , sin , .
As a result, the angular spectrum , , , Let us also denote the ASCGH obtained by the semi-analytic method by G ∆  .By substituting Eq. (11b) and Eq.(12b) into Eq.( 9), we can simplify the angular spectrum in the global coordinate as where In practical implementation, a rough approximation of removing ( ) ) can be expressed using the convolution of the texture angular spectrum and the triangular facet in the global angular spectrum domain as , .
As mentioned in Eq. ( 5), the texture pattern ( )  This texturing process is illustrated in Fig. 4. The calculated load of texture pattern generation and bilinear interpolation can be significantly reduced using this method, in contrast to the FFT-based method, wherein the bilinear interpolation is applied to a large matrix with the same dimensions as the ASCGH of the triangular facet.G T  is independently calculated in the global coordinate system.Thus, the matrix size of the texture pattern, which determines the quality of resolution, can be controlled according to user demand.It would be interesting to control the resolution of texture separately and independently in the semianalytic texturing process.By controlling the size of each texture pattern matrix, we can reduce computation time while maintaining the resolution of the triangular facet outline.The simulation results supporting those features are shown in Fig. 5 The approximate ASCGH of a triangular facet, G ∆  , does not lose accuracy substantially.
It is seen that although the resolution of the texture pattern changes with the size of G T  , the details of the triangular facet representation are retained.This is because the details in the triangular facet and the texture pattern are independently affected by G ∆  and G T  , respectively.The simulation results show that the texture representation resembles the original, without obvious distortion nor loss of accuracy when the size of G T  is greater than 161 × 161.It is expected that an observer's eye could not easily recognize the difference between 161 × 161 and 801 × 801 textures when the CGH is displayed by a practical holographic 3D display system.
In addition, the quantitative comment on the effect of the approximation of ( )

Trade-off in the semi-analytic texturing algorithm
In terms of computational efficiency, the flexible control of the size of G T  allows a considerable improvement in the computational efficiency of the semi-analytic texturing T ∆   .This approach manifests a trade-off between computational efficiency and texture representation detail.In Fig. 6(a), the computation times of the FFT-based and semianalytic methods measured are plotted against the size of G T  and compared.In Fig. 6(b), observed computed CGH scenes from this experiment at selected texture sizes are presented.
The example object is composed of 81 triangular facets and the texture size of each case is denoted in the figures.The total holographic light field is obtained the coherent superposition of all partial light fields, wherein the occlusion effect is implemented by visibility filtering from the observer's viewpoint [2,3].The first case is a bare polygon object followed by those with 101 × 101, 141 × 141, and 201 × 201, respectively.The black dashed line represents the computation time of the FFT-based method and the red line represents that of the semi-analytic method.As seen in the graph, the computation time of the FFT-based method is constant regardless of texture representation detail.In contrast, the computation time of the semi-analytic method decreases sharply with a reduction in the size of G T  .It should be noted that the computation time of the semi-analytic method is faster than the FFTbased method under a threshold size of G T  .In this experiment, the threshold value is measured to be 680 × 680 for an 801 × 801 size of G ∆  .It is ensured that the semi-analytic method is superior to the FFT-based method in computational efficiency.The semi-analytic method has a great potential for further computational efficiency enhancement after further investigation of fast computational algorithms for G ∆  and the associated optimal fast convolution operation.Meanwhile the partial computation times of the key algorithmic steps implemented in the FFT-based and semi-analytic methods are analyzed and compared in Figs.6(c) and 6(d), respectively.It has been known that the bilinear interpolation process is a common computational load for both method.For the semi-analytic method, the computation time in the bilinear interpolation process is quadratically proportional to the size of G T  , whilst it is constant for the FFT-based method.An additional computational burden in the semi-analytic method is put on the convolution process, which is linearly proportional to the size of G T  .From the analysis, we can see that, to enhance the computational efficiency, the key computational functions of bilinear interpolation and convolution should be optimized for the semi-analytic method.One of our current research topics is the development of the optimal low-level fast routine of those function for the semi-analytic CGH algorithm.Also, it is noteworthy that the ASCGH of a non-textured triangular facet is very sparse and the sparsity can be smartly exploited in this context.

Conclusion
We have investigated the fast texturing algorithm for the semi-analytic polygon CGH synthesis method to overcome the theoretical hurdle of the conventional FFT-based CGH synthesis method.The possibility of achieving a great improvement in computational efficiency has been shown by numerical analysis.A series of our to develop the mathematical algorithms contrast to recent research direction on parallel acceleration of the CGH algorithms using high-performance parallel-machines with multi-core CPUs or multiple GPUs.We believe that this study shows future research direction of the semi-analytic polygon CGH synthesis algorithm that have not yet been fully revealed.

Fig. 1 .
Fig. 1.Observation of 3D target objects (a) without texture and (b) with texture patterns.(c) The relationship between the local coordinate system of the kth triangular facet and the global coordinate system.The holographic light field distribution

Fig. 2 .
Fig. 2. FFT-based texturing process: (a) texturing of a triangular facet in the local coordinate system by direct multiplication of a texture pattern with a triangular facet function and (b) the local-to-global coordinate transformation of the holographic light field of the textured triangular facet.

Fig. 3 .
Fig. 3. Comparison of the computation times and reconstructed images of (a)


given by the non-uniform discrete Fourier series for the local spatial frequency ( ) the relation Eq. (13c).In practical calculation, the non-uniform Fourier coefficients , sampling grid of the local spatial frequency domain.Thus, be denoted by the texture angular spectrum matrix G T  .Then, the method of Eq. (13d) can be modeled by the operator form, * G G T ∆   .The most interesting feature of this method is that it is semi-analytic in fashion.The whole process is divided into the analytic calculation G ∆  , the numerical bilinear interpolation , and their numerical convolution.In this method, the matrix sizes of G ∆  and G T  are independent and the size of G T  is a free parameter.

Fig. 4 .
Fig. 4. Semi-analytic texturing process: (a) texturing of a triangular facet in the global coordinate system by direct multiplication of a texture pattern to a triangular facet function.(b)The equivalent convolution process of the angular spectrums of the texture and the triangular facet in the global coordinate system.Here the bilinear interpolation process is eliminated.

Fig. 6 .
Fig. 6.(a) A comparison of computation times of the conventional FFT-based CGH method and the semi-analytic CGH method.(b) Numerically observed holographic images of 3D objects without a texture pattern and with texture patterns synthesized using the semi-analytic method.The texture pattern sizes shown are 1 × 1 (non-texture), 101 × 101, 141 × 141 and 201 × 201.The partial computational times of the key algorithmic steps of (c) the FFT-based method and (d) the semi-analytic method.