Elsevier

Optics Communications

Volume 371, 15 July 2016, Pages 166-172
Optics Communications

Depth-tunable three-dimensional display with interactive light field control

https://doi.org/10.1016/j.optcom.2016.03.066Get rights and content

Highlights

  • This is the first time that the disparity adjustment for autostereoscopic display is conducted in light field, and benefit from the depth information inherently contains in the light field, the disparity adjustment can be conducted without the assistance of priori depth information.

  • A statistical analyzed disparity line is proposed, which links the disparities of all the multi-view images. The disparity adjustment is done by changing its slope, with this global operation, the accuracy of disparity control is improved, robustness is maintained in the presence of noise and a smooth motion parallax is always maintained after disparity adjustment.

Abstract

A software-defined depth-tunable three-dimensional (3D) display with interactive 3D depth control is presented. With the proposed post-processing system, the disparity of the multi-view media can be freely adjusted. Benefiting from a wealth of information inherently contains in dense multi-view images captured with parallel arrangement camera array, the 3D light field is built and the light field structure is controlled to adjust the disparity without additional acquired depth information since the light field structure itself contains depth information. A statistical analysis based on the least square is carried out to extract the depth information inherently exists in the light field structure and the accurate depth information can be used to re-parameterize light fields for the autostereoscopic display, and a smooth motion parallax can be guaranteed. Experimental results show that the system is convenient and effective to adjust the 3D scene performance in the 3D display.

Introduction

With the development of 3D display, glasses-free 3D displays have attracted much more attentions. The 3D display with large size, high resolution and dense viewpoints were demonstrated [1], [2], [3], [4]. However, it is difficult to present a perceptually pleasing 3D experience by directly displaying the multi-view content on the 3D display, because the acquired depth of the 3D scene does not fit well with the display capacity of display devices and the 3D salient regions in the reconstructed light field differ from one individual to another.

As the main depth cue for 3D scene perception is binocular parallax, the post-process to process the disparity information of multi-view contents is indispensable. However, the disparity compatibility problem is not fully considered in the 3D display and it is still underdeveloped for adjusting disparity structure for multi-view contents. Lei et al. extended the disparity handling method for two-view or stereoscopic display [5], [6] to deal with the multi-view situation, which controlled the disparity of corresponding points in each view [7]. However, it is unpractical for the dense multi-view 3D display which contains tens of viewpoints. The disparity of each view is computed separately and it is hard to maintain a smooth motion parallax after the disparity control due to the error of disparity acquisition. Recently, a depth map assisted disparity control method for the multi-view content was presented, and the projection relationship between views was taken into consideration to decide the disparity between views [8]. Selected region was projected to other views to find its corresponding regions, and the disparity information of the region was obtained. The accurate disparity adjustment was done by shifting the image of each view respectively. However, it suffers from the same problem with [7] because the shift value for each view is also separately acquired and the reference region is only adjusted to the zero disparity plane (ZDP) in both works. Masia et al. proposed a 3D depth remapping method which took the depth reconstruction capability of display device into consideration [9]. The depth map was remapped to adapt the display device, but a side-effect of depth remapping was the virtual viewpoint distortion since the depth map was altered before DIBR [10]. Moreover, the most serious weakness in all the above previous works is that the knowledge of per pixel depth is indispensable when adjusting the disparity, and the knowledge is difficult and inconvenient to obtain, especially for real scene. The characteristics of the inherently structure of multi-view contents with the depth information are not fully considered in previous works.

Inspired by the analysis of light field [11], [12], the depth-tunable 3D display is presented by controlling the dense light field. Benefiting from a wealth of scene information of the dense light field, the 3D display disparity control is possible without priori depth information of the scene, which eases the priori demand of multi-view content for disparity adjustment [8], [9]. Different from previous works which shift the multi-view images according to the priori depth information, the proposed method handles the disparity by shifting multi-view images according to light field structure: EPI-strip introduced in the next section. With different image shift values, light field structures are different, which correspond to different disparities. Therefore, the proposed disparity adjustment method is not limited to adjust the reference region to ZDP. As the light field structure is formed by all the multi-view images, the depth information is estimated by statistically analyzing the linear pattern of the light field structure, which considers all the multi-view images as a whole and leads to a more accurate disparity estimation. The previous point pattern based methods not only estimate the disparity information separately: stereo-matching [5], [6], [7] or depth map [8], [9], but also deal each image separately with the disparity adjustment. To break this independence, a linear regression disparity line is further proposed to represent the global depth information of the light field, and the disparity control process becomes a global operation, which improves the accuracy of disparity control and maintains robustness in the presence of disparity noise. Moreover, a smooth motion parallax is maintained with the disparity adjustment if the original multi-view content exhibits a smooth motion parallax. The rest of the paper is organized as follows, in Section 2 the concept of light field is briefly introduced, and then the depth expression in the light field structure and the basic light field structure control idea are explained. In Section 3 the detailed processes of the depth-tunable system are shown. In the experiment and discussion section, the effectiveness of the proposed method is verified and a depth-tunable 3D exhibition system is carried out to show the freedom of the disparity adjustment with the dense light field.

Section snippets

Depth expression in the three-dimensional light field

The light field concept was firstly proposed with a 4D function called the “Lumigraph” [11] and considered as the content form of free viewpoint TV (FTV) [13]. Here, instead of processing 4D light field, the 3D light field is processed since the content for the 3D display satisfies the horizontal parallax only stereo constraint.

Fig. 1 shows the representation of 3D light field, where Fig. 1(b) is a 3D light field, created from a set of multi-view images in Fig. 1(a). Each light ray in the light

Depth-tunable 3D display system

The depth tuning process has three detailed processes: the reference region selection, slope acquisition and ZDP refocusing. The reference region selection and the slope acquisition processes are the preliminary work for disparity adjustment, which obtain the original slope value of the reference region. Then zero disparity plane of the 3D content can be refocused and the reference region is commonly adjusted to the zero disparity. Here, the disparity adjustment is called the zero disparity

Experiment and discussion

To demonstrate the performances of our depth-tunable 3D display, three experiments are carried out. There are 32 viewpoints in our experimental 3D display with a resolution of 3840×2160. The display is a lenticular LCD autostereoscopic display and with the pixel arrangement method in our previous work [15], a dense view 3D display with 32 views is realized. The main device parameters of the display are given in Table 1. The multi-view contents used are shown in Fig. 5, which are called the

Conclusion

In summary, a depth-tunable 3D display system by operating the light field structure is presented. The proposed method takes the advantage of the inherent linear structure in the light field for depth information acquisition, which eases the priori demand of multi-view contents for disparity adjustment. The disparity line is proposed to represent the global depth information of the light field, which is more accurate and robust depth estimation with the existence of capture error. Experimental

Acknowledgment

This work is partly supported by the “863” Program (2012AA011902 and 2015AA015902), the National Natural Science Foundation of China (61575025) and the Program of Beijing Science and Technology Plan (D121100004812001).

References (18)

There are more references available in the full text version of this article.

Cited by (9)

  • Crosstalk reduction in large-scale autostereoscopic 3D-LED display based on black-stripe occupation ratio

    2017, Optics Communications
    Citation Excerpt :

    For autostereoscopic displays, two types of optical methods, parallax barriers and lenticular sheets, are used to create a 3D perception for real visual performance. Owing to the absence of additional viewing auxiliary aids, these types of autostereoscopic displays have been rapidly developed and commercialized in markets [1–6]. 3D display devices with parallax-barrier technology have some inherent advantages, such as glassless configuration, simple preparation and low cost, to form a horizontal sequence of 3D images [7–9].

  • Intended 3D Content Expressions on Light-field Displays using Adaptive Depth Compression

    2022, ITE Transactions on Media Technology and Applications
  • Dynamic zero-parallax-setting techniques for multi-view autostereoscopic display

    2020, IS and T International Symposium on Electronic Imaging Science and Technology
  • Effect of the vertical/slanted staggered barrier on autostereoscopic displays

    2019, Chinese Journal of Liquid Crystals and Displays
View all citing articles on Scopus
View full text