Depth-tunable three-dimensional display with interactive light field control
Introduction
With the development of 3D display, glasses-free 3D displays have attracted much more attentions. The 3D display with large size, high resolution and dense viewpoints were demonstrated [1], [2], [3], [4]. However, it is difficult to present a perceptually pleasing 3D experience by directly displaying the multi-view content on the 3D display, because the acquired depth of the 3D scene does not fit well with the display capacity of display devices and the 3D salient regions in the reconstructed light field differ from one individual to another.
As the main depth cue for 3D scene perception is binocular parallax, the post-process to process the disparity information of multi-view contents is indispensable. However, the disparity compatibility problem is not fully considered in the 3D display and it is still underdeveloped for adjusting disparity structure for multi-view contents. Lei et al. extended the disparity handling method for two-view or stereoscopic display [5], [6] to deal with the multi-view situation, which controlled the disparity of corresponding points in each view [7]. However, it is unpractical for the dense multi-view 3D display which contains tens of viewpoints. The disparity of each view is computed separately and it is hard to maintain a smooth motion parallax after the disparity control due to the error of disparity acquisition. Recently, a depth map assisted disparity control method for the multi-view content was presented, and the projection relationship between views was taken into consideration to decide the disparity between views [8]. Selected region was projected to other views to find its corresponding regions, and the disparity information of the region was obtained. The accurate disparity adjustment was done by shifting the image of each view respectively. However, it suffers from the same problem with [7] because the shift value for each view is also separately acquired and the reference region is only adjusted to the zero disparity plane (ZDP) in both works. Masia et al. proposed a 3D depth remapping method which took the depth reconstruction capability of display device into consideration [9]. The depth map was remapped to adapt the display device, but a side-effect of depth remapping was the virtual viewpoint distortion since the depth map was altered before DIBR [10]. Moreover, the most serious weakness in all the above previous works is that the knowledge of per pixel depth is indispensable when adjusting the disparity, and the knowledge is difficult and inconvenient to obtain, especially for real scene. The characteristics of the inherently structure of multi-view contents with the depth information are not fully considered in previous works.
Inspired by the analysis of light field [11], [12], the depth-tunable 3D display is presented by controlling the dense light field. Benefiting from a wealth of scene information of the dense light field, the 3D display disparity control is possible without priori depth information of the scene, which eases the priori demand of multi-view content for disparity adjustment [8], [9]. Different from previous works which shift the multi-view images according to the priori depth information, the proposed method handles the disparity by shifting multi-view images according to light field structure: EPI-strip introduced in the next section. With different image shift values, light field structures are different, which correspond to different disparities. Therefore, the proposed disparity adjustment method is not limited to adjust the reference region to ZDP. As the light field structure is formed by all the multi-view images, the depth information is estimated by statistically analyzing the linear pattern of the light field structure, which considers all the multi-view images as a whole and leads to a more accurate disparity estimation. The previous point pattern based methods not only estimate the disparity information separately: stereo-matching [5], [6], [7] or depth map [8], [9], but also deal each image separately with the disparity adjustment. To break this independence, a linear regression disparity line is further proposed to represent the global depth information of the light field, and the disparity control process becomes a global operation, which improves the accuracy of disparity control and maintains robustness in the presence of disparity noise. Moreover, a smooth motion parallax is maintained with the disparity adjustment if the original multi-view content exhibits a smooth motion parallax. The rest of the paper is organized as follows, in Section 2 the concept of light field is briefly introduced, and then the depth expression in the light field structure and the basic light field structure control idea are explained. In Section 3 the detailed processes of the depth-tunable system are shown. In the experiment and discussion section, the effectiveness of the proposed method is verified and a depth-tunable 3D exhibition system is carried out to show the freedom of the disparity adjustment with the dense light field.
Section snippets
Depth expression in the three-dimensional light field
The light field concept was firstly proposed with a 4D function called the “Lumigraph” [11] and considered as the content form of free viewpoint TV (FTV) [13]. Here, instead of processing 4D light field, the 3D light field is processed since the content for the 3D display satisfies the horizontal parallax only stereo constraint.
Fig. 1 shows the representation of 3D light field, where Fig. 1(b) is a 3D light field, created from a set of multi-view images in Fig. 1(a). Each light ray in the light
Depth-tunable 3D display system
The depth tuning process has three detailed processes: the reference region selection, slope acquisition and ZDP refocusing. The reference region selection and the slope acquisition processes are the preliminary work for disparity adjustment, which obtain the original slope value of the reference region. Then zero disparity plane of the 3D content can be refocused and the reference region is commonly adjusted to the zero disparity. Here, the disparity adjustment is called the zero disparity
Experiment and discussion
To demonstrate the performances of our depth-tunable 3D display, three experiments are carried out. There are 32 viewpoints in our experimental 3D display with a resolution of 3840×2160. The display is a lenticular LCD autostereoscopic display and with the pixel arrangement method in our previous work [15], a dense view 3D display with 32 views is realized. The main device parameters of the display are given in Table 1. The multi-view contents used are shown in Fig. 5, which are called the
Conclusion
In summary, a depth-tunable 3D display system by operating the light field structure is presented. The proposed method takes the advantage of the inherent linear structure in the light field for depth information acquisition, which eases the priori demand of multi-view contents for disparity adjustment. The disparity line is proposed to represent the global depth information of the light field, which is more accurate and robust depth estimation with the existence of capture error. Experimental
Acknowledgment
This work is partly supported by the “863” Program (2012AA011902 and 2015AA015902), the National Natural Science Foundation of China (61575025) and the Program of Beijing Science and Technology Plan (D121100004812001).
References (18)
- et al.
Natural three-dimensional display with smooth motion parallax using active partially pixelated masks
Opt. Commun.
(2014) - et al.
Display adaptive 3D content remapping
Comput. Gr.
(2013) - et al.
Depth map misalignment correction and dilation for DIBR view synthesis
Signal Process.: Image Commun.
(2013) - et al.
Extracting layers and analyzing their specularproperties using epipolar-plane-image analysis
Comput. Vis. Image Underst.
(2005) - et al.
Three-dimensional displays: a review and applications analysis
IEEE Trans. Broadcast.
(2011) - et al.
Demonstration of a large-size real-time full-color three-dimensional display
Opt. Lett.
(2009) - et al.
Aberration analyses for improving the frontal projection three-dimensional display
Opt. Express
(2014) - et al.
Real-time vergence control for binocular robots
Int. J. Comput. Vis.
(1991) - et al.
Vergence control of binocular stereoscopic camera using disparity information
J. Opt. Soc. Korea
(2009)
Cited by (9)
Crosstalk reduction in large-scale autostereoscopic 3D-LED display based on black-stripe occupation ratio
2017, Optics CommunicationsCitation Excerpt :For autostereoscopic displays, two types of optical methods, parallax barriers and lenticular sheets, are used to create a 3D perception for real visual performance. Owing to the absence of additional viewing auxiliary aids, these types of autostereoscopic displays have been rapidly developed and commercialized in markets [1–6]. 3D display devices with parallax-barrier technology have some inherent advantages, such as glassless configuration, simple preparation and low cost, to form a horizontal sequence of 3D images [7–9].
Depth-reversal-free three-dimensional display by adjusting the light field alignment
2017, Optics CommunicationsIntended 3D Content Expressions on Light-field Displays using Adaptive Depth Compression
2022, ITE Transactions on Media Technology and ApplicationsDynamic zero-parallax-setting techniques for multi-view autostereoscopic display
2020, IS and T International Symposium on Electronic Imaging Science and TechnologyEffect of the vertical/slanted staggered barrier on autostereoscopic displays
2019, Chinese Journal of Liquid Crystals and DisplaysEstimating Depth Range Required for 3-D Displays to Show Depth-Compressed Scenes Without Inducing Sense of Unnaturalness
2018, IEEE Transactions on Broadcasting