Continuous shading and its fast update in fully analytic triangular-mesh-based computer generated hologram

Fully analytic mesh-based computer generated hologram enables efficient and precise representation of three-dimensional scene. Conventional method assigns uniform amplitude inside individual mesh, resulting in reconstruction of the three-dimensional scene of flat shading. In this paper, we report an extension of the conventional method to achieve the continuous shading where the amplitude in each mesh is continuously varying. The proposed method enables the continuous shading, while maintaining fully analytic framework of the conventional method without any sacrifice in the precision. The proposed method can also be extended to enable fast update of the shading for different illumination directions and the ambient-diffuse reflection ratio based on Phong reflection model. The feasibility of the proposed method is confirmed by the numerical and optical reconstruction of the generated hologram. ©2015 Optical Society of America OCIS codes: (090.0090) Holography; (090.1760) Computer holography; (100.0100) Image processing. References and links 1. M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging 2(1), 28–34 (1993). 2. K. Wakunami, H. Yamashita, and M. Yamaguchi, “Occlusion culling for computer generated hologram based on ray-wavefront conversion,” Opt. Express 21(19), 21811–21822 (2013). 3. K. Matsushima, H. Nishi, and S. Nakahara, “Simple wave-field rendering for photorealistic reconstruction in polygon-based high-definition computer holography,” J. Electron. Imaging 21(2), 023002 (2012). 4. K. Matsushima, “Computer-generated holograms for three-dimensional surface objects with shade and texture,” Appl. Opt. 44(22), 4607–4614 (2005). 5. K. Matsushima and S. Nakahara, “Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method,” Appl. Opt. 48(34), H54–H63 (2009). 6. Y. Pan, Y. Wang, J. Liu, X. Li, and J. Jia, “Fast polygon-based method for calculating computer-generated holograms in three-dimensional display,” Appl. Opt. 52(1), A290–A299 (2013). 7. H. Nishi, K. Matsushima, and S. Nakahara, “Rendering of specular surfaces in polygon-based computergenerated holograms,” Appl. Opt. 50(34), H245–H252 (2011). 8. Y. Pan, Y. Wang, J. Liu, X. Li, J. Jia, and Z. Zhang, “Analytical brightness compensation algorithm for traditional polygon-based method in computer-generated holography,” Appl. Opt. 52(18), 4391–4399 (2013). 9. H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008). 10. L. Ahrenberg, P. Benzie, M. Magnor, and J. Watson, “Computer generated holograms from three dimensional meshes using an analytic light transport model,” Appl. Opt. 47(10), 1567–1574 (2008). 11. Y.-Z. Liu, J.-W. Dong, Y.-Y. Pu, B.-C. Chen, H.-X. He, and H.-Z. Wang, “High-speed full analytical holographic computations for true-life scenes,” Opt. Express 18(4), 3345–3351 (2010). 12. D. Im, E. Moon, Y. Park, D. Lee, J. Hahn, and H. Kim, “Phase-regularized polygon computer-generated holograms,” Opt. Lett. 39(12), 3642–3645 (2014). 13. D. Im, J. Cho, J. Hahn, B. Lee, and H. Kim, “Accelerated synthesis algorithm of polygon computer-generated holograms,” Opt. Express 23(3), 2863–2871 (2015). #253869 Received 12 Nov 2015; revised 9 Dec 2015; accepted 16 Dec 2015; published 23 Dec 2015 © 2015 OSA 28 Dec 2015 | Vol. 23, No. 26 | DOI:10.1364/OE.23.033893 | OPTICS EXPRESS 33893 14. J.-H. Park, H.-J. Yeom, H.-J. Kim, H. Zhang, B. Li, Y.-M. Ji, and S.-H. Kim, “Removal of line artifacts on mesh boundary in computer generated hologram by mesh phase matching,” Opt. Express 23(6), 8006–8013 (2015). 15. B. T. Phong, “Illumination for computer generated pictures,” Commun. ACM 18(6), 311–317 (1975). 16. S. Oh and I. K. Jeong, “Cylindrical angular spectrum using Fourier coefficients of point light source and its application to fast hologram calculation,” Opt. Express 23(23), 29555–29564 (2015). 17. K. Matsushima, “Exact hidden-surface removal in digitally synthetic full-parallax holograms,” Proc. SPIE 5742, 25–32 (2005). 18. H. Kim and D. Im, “Representation and synthesis of computer-generated holograms,” in 15th International Meeting on Information Display (IMID 2015), paper 38–2.


Introduction
Hologram is a useful tool in recording and reconstructing three-dimensional (3D) scene.Computer generated hologram (CGH) calculates the optical field of the 3D scene, enabling its numerical synthesis and reconstruction.In CGH, the 3D scene is considered as a collection of many primitives.The optical field of each primitive is calculated and accumulated to obtain the optical field of entire 3D scene.The primitive of the 3D scene for CGH includes a point light source [1], a light ray [2], and a triangular mesh [3][4][5][6][7][8][9][10][11][12][13][14].Among these, the mesh based CGH has advantage in the point that it is compatible with usual computer graphics techniques and efficient in representing large 3D scene.Among various methods for mesh-based CGH, the fully-analytic mesh-based CGH finds the angular spectrum of an arbitrary triangular mesh in space from the analytic formula of the angular spectrum of a reference triangle [8][9][10][11][12][13][14].As the angular spectrum is found by mapping the uniform spatial frequency grid in the hologram plane to the transformed spatial frequency grid in the local plane of the triangle, any artifact from the resampling of the spatial frequency can be avoided.This advantage of the fullyanalytic mesh-based CGH encourages increasing attention to this technique.
In the fully-analytic mesh-based CGH, the amplitude of the reference triangle is set to a unit value inside the triangle [9,10].The amplitude of the arbitrary triangle in space is assigned by considering the illumination direction and the normal vector of the triangle.Therefore in the reconstruction, the amplitude is uniform inside each triangle, resulting in the flat shading of the 3D scene.This flat shading makes the mesh boundary visible, degrading the reality of the reconstruction.One obvious solution would be to increase the number of the triangles with smaller size, but it would increase the computational load.
In this paper, we propose an extension of the fully-analytic mesh-based CGH.The proposed method uses a reference triangle whose amplitude is not uniform but spatially varying inside its area.In the proposed method, the amplitude of each vertex is assigned by considering its normal vector and the illumination direction, and the amplitude distribution within the triangle is given by interpolation according to the distances from three vertices.Therefore the 3D scene is reconstructed with continuous shading, enhancing the reality at the same number of the triangular meshes.In the proposed method, we derive an analytic formula of the angular spectrum of the reference triangle with the spatially varying amplitude.Hence the proposed method maintains the fully-analytic framework of the conventional flat shading method without sacrificing the precision of the calculation.
The proposed method is further extended in this paper to realize the fast update of the shading for different illumination direction and the ambient-diffuse reflection ratio.In conventional CGH, if the illumination direction or the ambient-diffuse reflection ratio is changed, the whole CGH calculation should be repeated to represent the correct shading of the 3D scene.In the proposed method, the ambient and directional diffuse reflection components of the angular spectrum of the 3D scene can be calculated and stored separately.The hologram for the arbitrary illumination direction or the ambient-diffuse reflection ratio can simply be synthesized by adding these pre-calculated angular spectrums with proper weights.Therefore the update of the CGH for different shading can be performed very fast in the proposed method.In the followings, we present the principle of the proposed method and experimental verification.

Review of conventional fully-analytic mesh-based CGH
In this section, the procedure of the fully-analytic triangular-mesh-based CGH synthesis [9,10,14] is briefly reviewed.The conceptual diagram is shown in Fig. 1.The angular spectrum in the hologram plane G(f x ,f y ) which corresponds to a single triangle in 3D space is first related to the angular spectrum of the triangle in its local plane G l (f xl , f yl ) by [9,10] ( ) ( )exp 2 , where Then the G l (f xl , f yl ) is related to the angular spectrum of a reference triangle G r (f xr , f yr ) by [9,10] where A and b are 2 × 2 affine matrix and 2 × 1 shift vector between the given triangle in the local plane and the reference triangle.The shifted local spatial frequency f′ xl,yl in Eq. ( 2) is given by where u xl and u yl are unit vectors of x l and y l axes represented in the global coordinates and u c is the unit vector of the carrier wave.The last term C in Eq. ( 2) matches the phase on the boundary between the neighboring triangles and is given by [14] , where r v,xlyl is the 2D position vector of any of three vertices of the triangle represented in the (x l , y l ) local coordinates, and d v,vf is the distance between the vertex and the carrier wavefront that passes through a reference point in the hologram plane as shown in Fig. 1.Using Eqs. ( 1) and ( 2), the angular spectrum of a single triangle can be calculated in the hologram plane.The angular spectrum of the entire 3D scene can be obtained by accumulating the angular spectrums of each triangle by where a m is the amplitude of the m-th triangle and determined in the simple Phong reflection model by [15] ( ) where a o,m and n m are the reflectance and the normal vector of the triangle and s is the illumination vector.Two coefficients k a and k d determine the ratio between the ambient and diffuse reflections in the Phong model.
In the conventional method, the reference triangle is assumed to have a uniform unit amplitude and a single amplitude value a m is assigned to each triangle.Therefore in the reconstruction, the 3D scene is represented with the flat shading, degrading the reality.

Proposed continuous shading
Figure 2 shows the concept of the proposed continuous shading method.In the proposed method, the amplitude inside the reference triangle is spatially varying by the interpolation of the amplitudes of three vertices.The shape of the reference triangle used in the proposed method is an isosceles right triangle with three vertices at V 1 (0, 0), V 2 (1, 0), and V 3 (1,1) as shown in Fig. 2. Given the amplitudes of three vertices a v1 , a v2 , and a v3 , respectively, the amplitude of the point (x r , y r ) inside the triangle, g r (x r , y r ) is linearly interpolated by ,1 , inside triangle ( , ) , 0, outside triangle which is based on the distances of the point (x r , y r ) from three vertices.With this interpolation model, the angular spectrum of the reference triangle G r (f xr , f yr ) can be found by taking Fourier transform of Eq. ( 7).After some mathematical manipulations, we have ( ) , where ( ) Equations ( 8)-( 11) are analytic formulas, and thus the fully-analytic framework of the conventional flat-shading method is maintained in the proposed method without any need for additional resampling of the spatial frequency grid.The amplitudes of three vertices a v,1 , a v,2 , and a v,3 are determined in the proposed method by ( ) where a v,i,o and n v,i are the reflectance and the normal vector of i-th vertex.
The procedure of the proposed method for the continuous shading is as follows.First the mesh-modeled 3D scene data is prepared.The normal vectors for the vertices are usually available in 3D modeling software or they can be calculated by averaging the normal vectors of the associated triangular mesh planes.Then the angular spectrum for each triangular mesh G m (f x , f y ) is calculated by Eqs. ( 1), (2), and (8) in a common spatial frequency grid (f x , f y ) in the hologram plane.Finally they are accumulated to yield the angular spectrum for the entire 3D scene by Note that the amplitude factor is now considered in the angular spectrum of the reference triangle G r (f xr , f yr ) and thus it does not appear in Eq. ( 13) unlike the conventional method given in Eq. ( 5).

Experimental verification of the continuous shading
An optical reconstruction was performed to verify the effect of the proposed continuous shading.Two holograms for a teapot object were generated in the conventional flat shading and the proposed continuous shading method, respectively.The teapot object consists of 756 triangular meshes and its size is 6.5 × 3.5 × 2.9 mm 3 .The center of mass of the teapot was located at 2cm in front of the hologram plane in the hologram generation.The reflectance of all meshes and vertices, i.e. a o,m , and a vio in Eqs. ( 6) and ( 12) were set to 1 and the ambient and diffuse coefficients k a and k d were set to 0.5 and 1.0, respectively.The number of the sampling points in the hologram was 1019(H) × 1019(V) and the sampling pitch was 8um.
The carrier wave used in the calculation was a plane wave slanted horizontally by 1° from the optical axis of the hologram so that the DC term and the reconstructed image are separated in the observation.The wavelength was 532nm.The complex optical field calculated by taking Fourier transform of G entire (f x,y ) in Eqs. ( 5) and ( 13) was encoded to an amplitude hologram by interfering the complex optical field with a plane wave numerically.In the optical reconstruction, a reflective spatial light modulator (SLM) of 8um pixel pitch was used.The polarization of the laser was adjusted so that the SLM operates as an amplitude modulator.The encoded amplitude hologram was loaded to the SLM and its reconstruction was captured using a digital camera.Figure 3 shows the numerical and optical reconstruction results.In the reconstruction of the hologram generated by the conventional flat shading method, i.e. Figure 3(a), it is seen that each mesh has uniform amplitude given by Eq. ( 6), revealing individual mesh.This flatshading effect is more obvious in the main body of the teapot as indicated by arrows in Fig.

#253869
3(a) where the surface is smooth and consists of small number of the meshes of large size.On the contrary, in the proposed method shown in Fig. 3(b), it can be observed that each mesh is reconstructed with spatially varying amplitude, achieving the continuous shading effect successfully.The mesh model of the teapot object used in the conventional and the proposed method is the same.Therefore the experimental result shown in Fig. 3 confirms that the use of the reference triangle with spatially varying amplitude in the proposed method effectively realizes the continuous shading as expected.
Note that the proposed continuous shading method requires calculation of Eqs. ( 9) and ( 10) in addition to Eq. ( 11) of conventional flat shading method for each mesh, which results in the increase of the computation time.In our current implementation using Python language without any hardware acceleration, the computation time for three teapot hologram resolutions, i.e. 255(H) × 255(V), 512(H) × 512(V), and 1019(H) × 1019(V) was measured to be 47.0 sec, 207.3 sec, and 859.0 sec, respectively, in the proposed continuous shading method and 28.8 sec, 124.2 sec, and 523.6 sec in the conventional flat shading method, indicating that the proposed method requires approximately 65% increase of the computation time.However, it is believed that the quality enhancement by the proposed method is more significant than the 65% increase of the computation time and there are large room for the computation time reduction by better implementation.Note also that a few bright artifacts due to non-proper treatment of the occlusion of the teapot object are observed in the reconstruction results shown in Fig. 3. Several methods have been reported to realize proper occlusion in the hologram generation [2,[16][17][18] and especially the polygon based methods [17,18] can be applied to the proposed continuous shading method for better reconstruction quality.

Fast update of the shading
The proposed method for the continuous shading can be easily extended to achieve the fast update of the illumination direction s and the ambient-diffuse reflection coefficients k a and k d .Using Eqs. ( 1), ( 2), (8), and (12), Eq. ( 13) can be rearranged into where s x , s y , and s z are the x, y, and z components of the illumination vector s.G ambient (f x,y ), G diffuse,x (f x,y ), G diffse,y (f x,y ), and G diffuse,z (f x,y ) are the angular spectrums of the entire 3D scene which are calculated by setting the amplitudes of the vertices a v,i in Eq. ( 12) as a v,i,o , a v,i,o n v,i,x , a v,i,o n v,i,y , and a v,i,o n v,i,z , respectively, where n v,i,x , n v,i,y , and n v,i,z are directional components of the normal vector of the corresponding vertex.Because the four angular spectrums, i.e.G ambient (f x,y ), G diffuse,x (f x,y ), G diffse,y (f x,y ), and G diffuse,z (f x,y ) are not dependent on the illumination vector s, the ambient and diffuse reflection coefficients k a and k d , they can be pre-calculated regardless of them.Then the final angular spectrum G entire (f x,y ) can be easily obtained by simple weighted addition of these four angular spectrums following Eq.( 14) for any illumination directions and ambient-diffuse ratios, enabling fast update of these parameters in the CGH and their reconstruction.Note that this proposed method does not require any additional calculation, thus it does not sacrifice the computation time.Although the storage requirement increases four times, it is manageable considering increasing storage capacity of recent devices.The fast update of the illumination directions was verified experimentally.The object and the specifications of the experimental setup are the same as described in the previous section.The only difference is that the ambient and diffuse coefficients k a , k d , and the illumination vector s are not fixed in this experiment.Figure 4 shows the pre-calculated four angular spectrums, G ambient (f x,y ), G diffuse,x (f x,y ), G diffse,y (f x,y ), and G diffuse,z (f x,y ) in Eq. ( 14) each of which has 1019(H) × 1019(V) sampling points.By adding these four angular spectrums with proper weight following Eq.( 14), the angular spectrum of the teapot object for the given ambient and diffuse reflection coefficients k a , k d and the illumination vector s can be obtained.Note that all holograms used in Figs. 5 and 6 were generated by simply adding four angular spectrums shown in Fig. 4 with different weights following Eq.( 14).Therefore it can be confirmed that the hologram synthesis for given k a , k d , and s can be performed very fast, once the four angular spectrums are calculated for given 3D scene.

Conclusion
In this paper, we proposed an extension of the conventional fully-analytic triangular-meshbased CGH method.While the conventional method assigns a uniform amplitude inside the reference triangle, the proposed method allows the amplitude inside the reference triangle to be spatially varying by the linear interpolation of the amplitudes of three vertices.Therefore in the reconstruction, the 3D scene can be represented with continuous shading effect, enhancing the reconstruction quality over the conventional method which shows the flat shading effect.It is also shown that the angular spectrums of the 3D scene can be decomposed into four components which correspond to ambient and diffuse reflections of the object in the Phong reflection model.Therefore the holograms for different illumination directions, ambient-diffuse reflection ratios can be quickly synthesized by simple weighted addition of four pre-calculated angular spectrum components, instead of repeating the whole hologram calculation.The feasibility of the proposed method was confirmed by the numerical and optical reconstruction results.

Fig. 2 .
Fig. 2. Concept of the proposed continuous shading method: Reference triangle has (a) uniform amplitude in the conventional method and (b) spatially varying amplitude in the proposed method.

Fig. 3 .
Fig. 3. Numerical and optical reconstruction of the hologram generated by (a) conventional flat-shading method, (b) proposed continuous shading method.

Figure 5
Figure 5 shows the reconstruction results of the hologram with different ambient and diffuse reflection coefficients.When only the ambient reflection exists without the diffuse reflection, i.e. k a = 1.0, k d = 0.0, it can be observed from Fig. 5(a) that the reconstructed teapot has uniform amplitude without shading.As the ratio of the diffuse reflection increases as shown in Figs.5(b) and 5(c), the shading of the reconstructed teapot becomes more significant as expected.

Figure 6
Figure 6 shows the reconstruction results of the hologram with different illumination vectors s.In 3 × 3 reconstructions shown in Fig. 6, the x and y components of the illumination vector s were selected from −0.4, 0.0, and + 0.4 and the z component was calculated to maintain the length of s to be 1.0 without change.The ambient and the diffuse coefficients were set to k a = 0.0, k d = 1.0 to clearly show the effect of the illumination direction change.The numerical and optical reconstruction results shown in Fig. 6 reveal that the holograms for different illumination directions are successfully synthesized, showing proper shading effects in the reconstruction.