Depth-of-focus Resolution and Viewing Angle Improvement of Integral Imaging System Using Multi-directional Projections and Non-uniform Lenslets Array with Intermediate View Reconstruction Method

This study presents a three-dimensional (3-D) integral imaging system to improve concurrently the viewing angle, Depth-Of-Focus (DOF) and resolution of the reconstructed object images by using multi-directional projections and non-uniform lenslets array with Intermediate-View Reconstruction Method (IVRM). In this method, each elemental lens array collects multi-directional illuminations of multiple Elemental Image (EI) sets and generates multiple Point Light Sources (PLSs) at the different positions in the two focal planes. The viewing zone and depth-of-focus is larger than the conventional method because of multi-directional projections of multiple EI sets and non-uniform lenslets; whereas a conventional method produces a viewing zone using only a single set of EI projection and uniform lenslets. For the non-uniform lenslets, DOF-improved integral images of 3-D objects can be picked-up and then by applying the IVRM to these picked-up integral images. Hence, the viewing angle, DOF as well as resolution enhanced object images can be reconstructed. To find out the possibility of the proposed system, experimental results are discussed with mathematical expressions.


INTRODUCTION
Integral imaging has been regarded one of the most contemporary three-dimensional display technique for the next generation that was firstly introduced by Lipmann (1908) named as integral photography.A basic integral imaging technique has two parts, pick-up and reconstruction (Okano et al., 1997;Song et al., 2005;Okano et al., 2006;Arai et al., 1998;Martinez-Cuenca et al., 2006;Islam et al., 2015) as shown in Fig. 1.In the process for pickup of integral imaging direction and intensity information of the rays coming from a three-dimensional (3-D) object are spatially sampled by using of a lens array (or pinhole array) and two-dimensional (2-D) image sensor.The lens array are made of many convex elemental lenses is positioned directly in front of the photographic film as a capture device.The capture image is consisted of many small elemental images that are imaged by lens array with their number corresponding to that of the elemental lenses.As a result, a size of elemental image is equal to a size of elemental lens.The striking features of integral imaging technique are full colour, full parallaxes both horizontal and vertical, natural depth perception with relatively low eye fatigue, real 3-D image with continuous view point, incoherent illumination in the pick-up process and specially does not require any extra viewing aid.
Even though the integral imaging has some advantages but it is still suffering from some problems, namely: low viewing angle, low image depth and low image resolution.Many researchers have been tried to improve the resolution, image depth and viewing angle of integral imaging three-dimensional display system.They were successful but not able to improve all the three at a time.In this study, we attempt to improve the viewing angle, depth-of-focus and resolution of integral imaging system using multi-directional projections and non-uniform lenslets array with Intermediate-View Reconstruction Method (IVRM).

MATERIALS AND METHODS
Depth of focus of the non-uniform lenslets array: By using the non-uniform lenslets array it is possible to increase the depth-of-focus of 3-D image (Seung-Cheil et al., 2011;Ju-Seog and Baharam, 2004).In this system, we used a lenslets array with two types of focal lengths and aperture sizes.Figure 2 illustrates the integral imaging with a non-uniform lenslets array.To improve the depth-of-focus, every lenslets set to produce the same spot size and different depth of focuses are ∂ I-1 , ∂ I , ∂ I+1 and so on with different Lenslet Imaging Planes (LIPs) L i1 , L i2 and so on as shown in Fig. 2 and 3.
Here, we consider the gap between the display panel and the lenslets array, the aperture size, focal length of the lenslets and illumination wavelength to be g, P L , f and λ respectively for uniform lenslets array.Optical ray and P L will vary for non-uniform lenslets array as shown in Fig. 4. From Gauss lens law, the LIP is located at: where L = L i1 , L i2 and so on.So, the spot size: where, P L is the aperture size.The spot size ∂ will vary with 'z'.From the Fraunhofer diffraction theory, ∂ can be approximately represented by the following Eq.( 3) and (4): The resolution of the reconstructed image within depth of focus is maximum with the minimum spot size and depth of focus is minimum with the maximum spot size.The ∂ i1 , ∂ i2 based on Rayleigh limit and the diverging ray angles θ 1 , θ 2 from the Fraunhofer diffraction theory are given by Eq. ( 5) and (6): The reconstructed image within the depth of focus of the lenslets array is highly focused.On the other hand, the reconstructed image out of the depth of focus is blurred owing to off-focusing effect of the lenslets array.
Overall proposed system: In our proposed system, the effective DOF of the pick-up lenslets array can be increased by using the lenslets with non-uniform focal lengths and aperture sizes and as a result, depth enhanced integral images can be obtained.At the same time, by applying the IVRM to these integral images, DOF as well as resolution improved of the object image can be reconstructed.Figure 5 illustrates the proposed method for reconstruction of the resolution and DOF improved object image from the integral images picked-up using a non-uniform lenslets array.This system is considered three parts: pick-up of the Elemental Image Array (EIA) by using a non-uniform lenslets array with multi-directional projections, generation of Intermediate Elemental Images (IEIs) with IVRM and reconstruction of object image from the synthesized EIA.At last, from the newly synthesized EIA 3-D object image is reconstructed along the output plane by using the Computational Integral Imaging Reconstruction (CIIR) technique.Pick-up system of elemental images: Here, we consider a non-uniform lenslets array is composed of 2×1 lenslets having two kinds of focal lengths and aperture sizes.Figure 6 illustrates an operational principle of the non-uniform lenslets array composed of lenslets with two kinds of focal lengths and aperture sizes (Seung-Cheil et al., 2011).In this system, the DOF range of the non-uniform lenslets array can be increased noticeably because of lenslets having different focal lengths and aperture sizes that are responsible forming their own Lenslets Image Planes (LIPs) at different distances from the lenslets, namely: LIP (1) , LIP 2 .The overall DOF range of the non-uniform lenslets array is made by adding these two LIPs together.Consequently, the effective DOF of the non-uniform lenslets array should be extended in the pickup process as shown in Fig. 6.On the contrary, the uniform lenslets array is composed of lenslets with one kind of focal length and aperture size, so that only one LIP of the lenslets array will be formed in front of lenslets array.Although, the effective DOF of the reconstructed image in the non-uniform lenslets array system can be increased but resolution of the reconstructed object image decreased due to sampling rate of rays emitting from the object at the specific depth plane might be decreased.To compensate this problem, a method is proposed for the resolution enhanced of integral imaging 3-D displays by using time-multiplexed multi-directional elemental image sets (Alam et al., 2012).For practical applications, a forceful 3-D integral imaging system having not only large DOF but also has high image resolution as well as wider viewing angle.As a result, IVRM is required to improve the resolution and DOF of the reconstructed images.

Resolution and viewing angle improvement of integral imaging:
In this study, a method has been proposed for the resolution and viewing angle enhancement of integral imaging 3-D displays by using time-multiplexed multi-directional elemental image sets with non-uniform lenslets array.Figure 7 illustrates the principle of the resolution enhancement of multidirectional elemental image projection compare with a conventional method.It is noticeable that one PLS produces in the case of single plane illumination whose angle with the optical axis of elemental lens is equal to zero.If this angle is not zero, the light is collected at a different distance D PLS from the lens axis in the focal plane as shown in Fig. 7.There are three PLSs are generated on PLS plane for multi-directional elemental image projections.The distance D PLS is given by: tan( ) where, AI is the angle of illumination and f is the focal length of the lens array.On the basis of Eq. ( 7), it can be write that the lateral position of a PLS depends on the angle of illumination (AI).As a result, it is possible to create a number of PLSs for one elemental lens if illumination projected from the multi-direction projections, namely; directional elemental image, upper (DEI upper ), directional elemental image, centre (DEI lower ) and directional elemental image, lower (DEI lower ).Therefore, one elemental lens creates three PLSs.For uniform spatial resolution, a distance between the PLS is: 3 where, P L is the size of elemental lens.So, we can write the angle of illumination: Figure 8 and 9 show the principle of the resolution enhancement as well as the viewing angle of the integral imaging three-dimensional (3-D) display system using a uniform and a non-uniform lenslets array.In the case of multi-directional elemental image projections using non-uniform lenslets array, resolution, viewing angle as well as depth of the constructed image is enhanced due to the two point line source planes.On the other hand, multi-directional elemental image projections using uniform lenslets array, resolution as well as viewing angle is enhanced.In this system, the viewing angle can be determined by the following equation (Alam et al., 2014;Islam et al., 2015): where, P L = The pitch (size) of a elemental lens of lenslets array f = The focal length of a elemental lens of lenslets array

Method of generating intermediate elemental images:
As aforementioned discussion, it can be prove that DOF is enhanced using non-uniform lenslets array system.It is also noticeable that non-uniform lenslets array system performance can be extended depending on the number of non-linear lenslets.If we increase the ).However, if we use multi-directional projections then resolution will be increase compare than single illumination.In the case of multi-directional projection through non-uniform lenslets array the resolution of the 3-D reconstructed object image will be not uniform everywhere due to different PLSs produced in the different PLS planes.
For getting the uniform resolution of the reconstructed 3-D object image, a method, named IVRM is used; hence, the object image resolution as well as image quality is enhanced.For integral imaging system, the reconstructed image resolution is highly dependent on the number of picked-up elemental images of a 3-D object.So, IVRM is responsible for doing this.Because of the number of picked-up elemental images might be limited owing to physical limitation of the lenslets array.By applying the IVRM to the picked up elemental images it is possible to increase the number of the elemental images computationally as much as we want.So, the resolution of the reconstructed object image from these newly synthesized elemental images can be improved.Figure 10 illustrates the operational principle of IVRM.In this case, we consider two elemental images (left and right elemental images) and then (left upper, left lower, right upper, right lower elemental images) disparity data among them are given from the picked up EIA, after it can be synthesize as many intermediate elemental images as we want.
Now, an EIA is picked up through a N×N nonuniform lenslets array where, one sub-lenslets array is composed by 2×1 lenslets with two types of focal lengths and aperture sizes.It is very important to enhanced integral imaging of the reconstructed 3-D object image depth of focus by using non-uniform lenslets array.So, different types of focal lengths and aperture sizes of elemental lenses are desirable but will be decreased resolution drastically.In this study, we are trying to enhance all three weak points of integral imaging three-dimensional techniques.For these reasons, we used here two types of focal lengths and aperture sizes of elemental lens of the lenslets array.Through Fig. 11 has been shown the process of IVRMbased synthesis of IEIs from the elemental images picked-up by using the non-uniform lenslets array both two positions, namely: horizontally and vertically.
As mentioned earlier that Fig. 11 shows an ingredient of the EIA picked-up through N×N lenslets array from there a 2×1 sets of the elemental image array picked-up by sub-lenslets array.Each subset is composed of 2×1 elemental images picked-up by 2 types of lenslets.Then IVRM is applied on these picked-up elemental images as shown in Fig. 11a.It is also shown in Fig. 11b and c synthesized IEIs both horizontally and vertically.After that, the picked-up 2×1 sets of elemental images are possible to increase computationally into 2×3 sets of elemental images by using the IVRM as shown in Fig. 11d.App.Sci. Eng. Technol., 12(3): 294-303, 2016 301

RESULTS AND DISCUSSION
Figure 12 illustrates the experimental setup for D objects using a nonbased synthesis of tion of the DOF and resolution enhanced object images.In this experiment, we used a D objects of 'Hat' and 'Car' that is bearing with resolution of 155×155 and 121×194, that is located at z = 40 mm and 70 mm, respectively.The non-uniform lenslets array (100 ×100) with two kinds of focal lengths and aperture sizes and Lens 2 are shown in Table 1.
We used here a 30×30 non-uniform lenslets array that consisted of two types of focal lengths, and f 2 = 3.3 mm of Lens 1 , Lens 2 respectively, whereas; a 30×30 uniform lenslets array consisted of only one focal length f = 3 mm of each elemental lens of the  lenslets array.The gap 'g' is taken 5.06 mm measured from display panel to lenslets array and DOF and DOF 1 of LIP are calculated to be the sa denoted by ψ.Fig. 13 and 14 reveal that elemental images of the trial objects picked-up by using non uniform and uniform lenslets array.
In this stage, the resolution of each elemental image picked-up by Lens 1 and Lens 2 is supposed to be 25×25 and 30×30 pixels, respectively.These resolution values are analyzed with calculation from the aperture size of lenslets while the pixel size of the worked Liquid Crystal Display (LCD) is given by 0.251 mm.The object image can be computationally reconstru through the uniform lenslets array in which the are calculated to be the same value is 13 and 14 reveal that elemental up by using non-In this stage, the resolution of each elemental is supposed to be and 30×30 pixels, respectively.These resolution values are analyzed with calculation from the aperture size of lenslets while the pixel size of the worked Liquid Crystal Display (LCD) is given by 0.251 mm.The object image can be computationally reconstructed through the uniform lenslets array in which the Lens 1 has a LIP 1 of 40 mm and a DOF 1 LIP 1 .The DOF of Lens 1 is approximately (40 (70-ψ) mm.As mentioned before, the car is clearly reconstructed at z = 40 mm due to this originally located in this point.In this point, it can be found that the object is clearly reconstructed through the uniform lenslets array.On the contrary, the red hat is reconstructed at z = 70 mm but noticed that the reconstructed object image is unclear owing to off focusing effect in which out of the lenslets array.On the other hand, for non lenslets array the two objects, namely; white car and red hat is clearly reconstructed at z = 40 mm with the DOF 1 = (40+ψ) mm and DOF around the LIP 1 and LIP 2 .In comparison, for uniform and non-uniform lenslets arrays the DOF can be increased from (40-ψ) mm to (70+ aforementioned discussion it is mathematically proved that for the non-uniform lenslets array the Fig. 15: Reconstructed 3D images for the multi-directional projections with large viewing angle of ψmm around the is approximately (40-ψ) mm to ) mm.As mentioned before, the car is clearly due to this object is originally located in this point.In this point, it can be found that the object is clearly reconstructed through the uniform lenslets array.On the contrary, the red hat but noticed that the ge is unclear owing to offfocusing effect in which out of the DOF range of the lenslets array.On the other hand, for non-uniform lenslets array the two objects, namely; white car and red = 40 mm and z = 70 mm DOF 2 = (70+ψ) mm .In comparison, for uniform uniform lenslets arrays the DOF can be mm to (70+ψ) mm.In the aforementioned discussion it is mathematically proved uniform lenslets array the DOF of reconstructed object image is enhanced remarkably but image resolution of the reconstructed object image is degraded by one half (for two types of focal lengths and aperture sizes) compare than uniform lenslets array.If we used multi-directional projections (upper, center and lower sides) for the reconstruction of the object image through non-uniform lenslets array the resolution of the reconstructed object image will be increased because of every elemental lens will produce three PLS in two different PLS planes.Since, non-uniform lenslets array, the resolution of the reconstructed image will be not uniforms everywhere.To get the uniform resolution image through the non-uniform lenslets array, a method named IVRM is used.So, through the process, it is possible to get newly synthesized high resolution reconstructed 3-D object image.For the limitation of the laboratory instruments, not possible to show the original experimental 3-D picture through this study.A three-dimensional image picked-up by using the integral imaging system is shown in Fig. 14.
In the case of multi-directional elemental image projections using a non-uniform lenslets array the viewing angle (according to Eq. 10) of the reconstructed 3-D object image will be enhanced two times compare than conventional one as shown in Fig. 15.In traditional system, the viewing angle is measured by only 'θ' whereas; an existing system (Alam et al., 2014) the viewing angle is measured by '3θ'.Another method proposed (Islam et al., 2015) for increasing viewing angle of the reconstructed 3D images and measured by '4θ' but this method is not suitable in this system due to enhance all limitations of integral imaging three-dimensional display system at a time.
Figure 15 reveals that the reconstructed 3D images are captured from different viewing angles using conventional system shown in Fig. 15a and b with multi-directional projections shown in Fig. 15c.By the figure, we observed that the viewing angle enhanced remarkably of the reconstructed 3D image using multidirectional projections through non-uniform lenslets array.

CONCLUSION
Integral imaging three-dimensional display technology system has been suffering from the three limitations, namely: low viewing angle, image resolution is low and limited depth of images.All the limitations are significantly overcome through the paper.In this study, a three-dimensional (3-D) integral imaging system to improve simultaneously the viewing angle, Depth-Of-Focus (DOF) and resolution of the reconstructed object Images by using multi-directional projections and non-uniform lenslets array with intermediate-View Reconstruction Method (IVRM) is proposed.To find out the possibility of the proposed system, experimental results are discussed with mathematical expressions.

Fig. 3 :
Fig. 3: Ray integration to produce 3D images with two lenslets image plane

Fig. 6 :
Fig. 6: Pick-up and reconstruction processes for non-uniform lenslets array based integral imaging

Fig. 11 :
Fig. 11: Pictorial representation of the IVRM-based synthesis of IEIs; (a): elemental image sampled picked up by 2 ×1 sublenslets array; (b): vertically synthesized EIA; (c): horizontally synthesized EIA and; (d): overall synthesized EIA number of non-linear lenslets in a non-uniform lenslets array the reconstructed 3-D object of integral imaging DOF will be gradually improved but inversely decrease the resolution of the 3-D object (Seung-Cheil et al., 2011;Ju-Seog and Baharam, 2004).However, if we use multi-directional projections then resolution will be increase compare than single illumination.In the case of multi-directional projection through non-uniform lenslets array the resolution of the 3-D reconstructed object image will be not uniform everywhere due to different PLSs produced in the different PLS planes.For getting the uniform resolution of the reconstructed 3-D object image, a method, named IVRM is used; hence, the object image resolution as well as image quality is enhanced.For integral imaging system, the reconstructed image resolution is highly dependent on the number of picked-up elemental images of a 3-D object.So, IVRM is responsible for doing this.Because of the number of picked-up elemental images might be limited owing to physical limitation of the lenslets array.By applying the IVRM to the picked up elemental images it is possible to increase the number of the elemental images computationally as much as we want.So, the resolution of the reconstructed object image from these newly synthesized elemental images can be improved.Figure10illustrates the operational principle of IVRM.In this case, we consider two elemental images (left and right elemental images) and then (left upper, left lower, right upper, right lower elemental images) disparity data among them are given from the picked up

Figure 12
Figure 12 illustrates the experimental setup for pick-up of the EIA of the test 3-D objects using a non uniform lenslets array with IVRM-based synthesis of IEIs and reconstruction of the DOF and resolution enhanced object images.In this experiment, we used a test 3-D object formed from two 2-D objects of 'Hat' and 'Car' that is bearing with resolution of 155×155 Experimental setup for pickup of EIA, synthesis of IEIs and reconstruction of object image using non directional projection (a) (b) Elemental image arrays of 155×155 pixels picked up uniform lenslets array and (b): uniform Fig. 14: Reconstructed three-dimensional (Markman et al., 2014) and 121×194, that is located at z = 40 mm and 70 mm, uniform lenslets array (100 ×100) with two kinds of focal lengths and aperture sizes Lens 1 uniform lenslets array that consisted of two types of focal lengths, f 1 = 3 mm respectively, whereas; a 30×30 uniform lenslets array consisted of only one of each elemental lens of the ge using non-uniform lenslets dimensional images

Table 1 :
Specifications of the non-uniform lenslets