SHAPE RECONSTRUCTION OF HERITAGE ASSETS BY POLARIZATION INFORMATION

: Three-dimensional modeling of heritage assets poses a challenge when high reflectance and featureless surfaces are involved. Because of the complex reflection characteristics, traditional methods, either passive or active, tend to generate noisy and incomplete reconstruction results. To address this, we introduce in this paper a shape-from-polarization system and modeling strategy for the 3-D reconstruction of heritage assets. As demonstrated, by analyzing polarization properties of reflected light, we can generate by a single image pixel-level-resolution normal and depth maps. We consider two system prototypes involving a point light source but different in the polarizing camera configurations. They provide solutions for different reconstruction requirements and acquisition conditions. Our reconstruction model is common however to both involving a closed set of simple computation steps. Our evaluations demonstrate how the proposed method significantly improves the 3-D surface model completeness and level of detail, showing its suitability and value to reach the goal of a high-resolution survey of heritage assets, where traditional approaches tend to fail.


INTRODUCTION
Three-dimensional active and passive sensing technologies provide a non-invasive solution for the documentation, conservation, restoration, and valorization of cultural heritage assets (Soler et al., 2017;Felicísimo and Polo, 2022). By analyzing features such as color and symmetry of 3-D entities, incomplete assets can be restored in virtual space to enrich their interpretation, understanding, and research (Montusiewicz et al., 2019). Such digital forms can also be combined with technologies including computer graphics (CG), virtual reality (VR), and augmented reality (AR) to set up attractive and interactive exhibitions (Comes et al., 2022).
Due to the object characteristics, 3-D modeling of heritage assets places high requirements on reconstruction methods and sensing technologies. These requirements become more pronounced considering their shape complexity, completeness, and material characteristics, e.g., diffused or specular, or transmissive. Therefore, the modeling technologies should be flexible to tackle data acquisition of objects and different surface reflection properties (Apollonio et al., 2021). In addition, depending on the number of objects to be reconstructed and the available resources, modeling efficiency, accessibility, and cost must be taken into consideration when tackling the acquisition and processing phases (Farella et al., 2022a). Despite these requirements, it is customary to apply multiview stereo-(MVS) based methods, which require designated adaptations and settings in textureless or specular regions, or active-sensor based, which are costly and time-consuming. As a result, they face limitations in handling surface complexity and provide incomplete, and sometimes noisy results (Wu et al., 2018;Adrian and Pham, 2019;Li et al., 2021). To address these limitations, we introduce in this paper polarization-based setups, techniques, and algorithms aiming to generate high-fidelity surface-shape

RELATED WORK
It is common to apply mainstream 3-D reconstruction techniques to cultural heritage 3-D modeling. Typical passive methods, structure-from-motion (SfM) and MVS are widely used for small and medium-sized objects (Gallo et al., 2014;Farella et al., 2022b;Vuković et al., 2022). Since passive reconstruction depends on the image quality, elaborate camera adjustment and image post-processing are required to reduce the image blur caused by camera depth-of-field or motion (Farella et al., 2022b). Nonetheless, the existence of featureless or glossy areas on the object surface may still leave voids in the reconstruction result (Cui et al., 2017). To address them, active 3-D reconstructions like laser triangulation scanners and phase shift laser scanners are also used for small and medium-sized objects, such as porcelain, metal, or stone sculptures, fragments of wooden shipwrecks, and various metal products, etc (Hess et al., 2015;Montusiewicz et al., 2019). These scanners offer good geometric accuracy but they are costly and are limited in color fidelity due to the use of laser light sources (Felicísimo and Polo, 2022). Therefore, an additional camera for photometric measurement under controlled illumination is required to render the color-accurate texture map for the 3-D model (Apollonio et al., 2021). By using precisely calibrated cameras and multiple customized flashlight sequences, shape from shading (SfS) has also been used in the reconstruction of ancient coins and medals with bas-relief structures (e.g., MacDonald et al., 2017;Betti, 2022). However, being an inherently underconstrained problem, to apply the SfS with a sufficient number of photometric constraints, multiple calibrated cameras, and customized flashlight sequences are required (Betti, 2022).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy

METHODOLOGY
In this paper, our focus is on the reconstruction of objects such as ones made of marble, ceramics, or glazed pottery materials, usually featuring complex reflective properties. Their textureless form lacking sharp geometries makes them featureless, while smooth areas produce specular highlights. As a result, their reconstruction using traditional methods is difficult, and as the review has noted the application of traditional approaches is likely to generate incomplete or noisy reconstruction. To address these limitations, we study the application of shape-frompolarization (SfP) methods for the reconstruction. SfP has been showing promise in reconstructing challenging surfaces, where dense polarimetric surface shape cues can be obtained from featureless and glossy surfaces (Zhu and Smith, 2019;Fukao et al., 2021;Lei et al., 2021). To facilitate the estimation of shape information from surfaces with different reflection properties, we utilize polarization reflection models that incorporate both diffuse and specular reflections. We show that a polarizationdriven shape reconstruction pipeline can be roughly partitioned into two main steps. The first calculates all the possible normal solutions of the object surface and then eliminates the inherent normal ambiguity from polarimetric cues. The second solves the relation between the surface gradient and depth to estimate the object depth map from the disambiguated normal map. The outcome is a pixel-level dense monocular 3-D reconstruction model, independent of surface features, demonstrating stability in structurally complex regions, featureless regions, and regions characterized by specular reflection. In that respect, SfP produces an accurate 3-D reconstruction and facilitates the computations of the surface normal and depth for heritage assets with featureless and glossy surfaces.

Polarization imaging background
SfP is a physically-based method that relies on Fresnel equations to derive surface information by light polarization properties. According to the Fresnel equations, an unpolarized light becomes partially linearly polarized when reflected from a surface (Goldstein, 2011). The properties of the reflected polarized light, obtained by polarization cameras, can inform the object surface shape at each pixel. To compute these surface-related parameters, a sequence of intensity images is acquired in which a linear polarizer positioned in front of the camera lens is rotated to yield n ≥ 3 images, each with polarizer angle ϕi, i ∈ {1, ..., n}. The intensity, Ii, at a single image pixel, varies sinusoidally as a function of ϕi according to: where Imax and Imin are the respective maximum and minimum intensities obtained by rotating the polarizer; I = (Imax + Imin)/2 is the average intensity; ρ = (Imax − Imin)/(Imax + Imin) ∈ [0, 1] is the degree of polarization (DoLP); and ϕaop ∈ [0, π) is the angle of polarization (AoP), where I(ϕaop) = Imax (Goldstein, 2011). The three parameters,Ī, ρ, and ϕaop, completely describe the polarization state of the incoming light and can be estimated by Eq. (1). The SfP task is to parameterize the object surface shape as a function of these parameters.
Polarization reflection process for textureless objects The reflection process is generally divided into diffused and specular reflections. A relatively smooth non-metallic object surface can be regarded being made of as a Lambertian substrate (generating total diffused reflection) coated with a uniform dielectric layer that contributes all the specular reflection (Fig. 1). The Lambertian substrate contributes diffused reflection, which is being refracted by the dielectric layer. Following Fresnel equations it can be shown that when unpolarized light incident on the surface, the reflection from the dielectric layer becomes partially linearly polarized ( Fig. 1-a). The reflected light by the Lambertian substrate is also unpolarized, but the refracted light at the dielectric layer becomes partially linearly polarized as well ( Fig. 1-b). Therefore, the recorded light at the camera image plane is partially linearly polarized and is a mixture of the diffused and specular reflections.

Acquisition system setup
To facilitate the reconstruction, our image acquisition system consists of a polarizing camera and position-known distant point light source to realize active illumination (Fig. 2). In addition to illuminating the object surface, the point light source is also used to disambiguate the surface normal direction. Normal ambiguity is an inherent polarization-based feature and its address is detailed in Sec. (3.3).
We use the distant point light source as a means to disambiguate AoP-introduced normal/convexity ambiguity and DoLPintroduced normal ambiguity. Our point light source in the present setup is a Wiz A67 RGBW LED bulb, which can achieve a maximum luminous flux of 1521 lm, has 16 million adjustable colors (CRI90), and adjustable white light color temperature from 2200-6500 K, which can meet the related requirements on illumination and color restoration. Though any point light source can be used here, the value of this specific illumination lies in the ability to control the color temperature, a useful property to obtain accurate color content.
The polarization imaging system can be implemented in two ways, i) division-of-time (DoT), and ii) division-of-focal-plane (DoFP). Our DoT polarizing camera (Fig. 2a)

uses a Nikon
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy D7200 DSLR camera (sensor size 23.5×15.6 mm, image dimension 6000×4000 pixels) with an AFS Nikkor 18-70mm f/3.5-5.6G VR lens, and a B+W KSM POL linear polarizer mounted on the lens to achieve polarization filtering. During image acquisition, we capture the intensity images by rotating the polarizing filter to at least three known angles (usually 0 • , 45 • , 90 • , and 135 • ). The polarizer rotation angle is then calibrated by least squares adjustment and the scene polarization information can then be computed from these intensity images according to Eq. (1). Since the DSLR camera has high resolution, a large numerical aperture of the lens, and as the rotation angle of the polarizer can be calibrated, the DoT system is suitable for objects that require high precision and spatial resolution. It exerts, though, a lengthy image acquisition process. After removing the polarizer, the DSLR is also used to accurately acquire object surface color information.
Our DoFP polarizing camera (Fig. 2b) uses a LUCID Vision Labs TRI050S1-P monochrome camera with a 2048×2448 pixels resolution, an 11.1 mm sensor size, and an AFS Nikkor 18-70mm f/3.5-5.6G VR lens (with an F to C-mount adapter). The sensor incorporates a layer of polarizers above the photodiodes. Four different angled polarizers (0 • , 45 • , 90 • , and 135 • ) are placed above the sensor's pixels with every block of four pixels making up a calculation unit. From the relationship between the different directional polarizers and the resulting pixel output, standard algorithms can calculate both the DoLP and AoP. The computed polarization image resolution is 1024×1224 pixels. Compared with the DoT configuration, the DoFP camera has a lower resolution, but the system is compact, lightweight, and does not require rotating and calibrating the polarizer, which can realize real-time imaging, suitable for scenes with limited acquisition space and time.

Polarimetric surface normal estimation
Fresnel equations associate the surface normal parameters with the observed polarization state, specifically t he D oLP a nd the AoP.
Estimating the zenith angle from DoLP Following the common convention, we define t he z enith a ngle, θ , a s t he one formed between the camera optical axis and surface normal (implying that the reconstruction is performed in reference to the camera frame). The zenith angle is encoded in Fresnel equations (Goldstein, 2011), and when substituted into the DoLP definition, we can express the latter as a function of θ and the refractive index η. The DoLP for specular and diffuse dominant surface facets is different. Thus, we assume that reflection from a point can be classified as diffuse or specular dominant, and designate saturated pixels in the average intensity image as specular and the rest as diffused. As demonstrated by Atkinson and Hancock (2006), the expression for the DoLP of a diffuse dominated surface facet, ρ d , reads: and has only a single zenith angle solution for θi, given ρ d . Therefore, the incident angle of a diffused surface can be uniquely solved. The DoLP for specular surface reflection, ρs has the form (Tozza et al., 2017): ρs = 2sin 2 θi cos θi η 2 − sin 2 θi η 2 − sin 2 θi − η 2 sin 2 θi + 2sin 4 θi .
( 3) and note that two possible solutions exist for θi given an observed ρs. Therefore, the incident angle of a specular surface is ambiguous when solved from the DoLP.
Estimating azimuth angle from AoP Fresnel equations show that the AoP at each point corresponds to the azimuth angle of the plane of incidence (also the azimuth angle of surface normal). However, as ϕaop ∈ [0, π) while the azimuth angle is defined in the range of ϕn ∈ [0, 2π), there is a π ambiguity in the surface azimuth, yielding for diffused dominated surface facet normals, the two following values: We also note that there is a π/2 difference in the normal azimuth angle between diffuse dominant surface facets and specular ones (Goldstein, 2011). Therefore, where the s and d superscripts relate to the diffuse and specular domination of the surface facets, respectively.
To obtain surface normal information for the zenith and azimuth angles, under the orthographic projection of the camera coordinate, we have: Surface normal disambiguation Considering the DoLPand AoP-related ambiguities, there are two possible surface normals for a diffuse dominant facet, while there are four possible normals for a specular dominant one. To resolve these ambiguities a position-known distant point light source is introduced. The point light source provides an additional photometric constraint to disambiguate the polarization-derived normals.
For a diffuse dominant facet, we follow Lambert's reflection law which describes I as a function of the surface normal n and the normalized point light source direction, s = [sx, sy, sz]: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy where a is the albedo of the diffuse facet. The estimation of a can be carried out using two or more diffuse-related pixels from which the normal ambiguity can be resolved, and a can be computed. To disambiguate the polarization-derived diffusedsurface-related normal direction, one can select the candidate that best agrees with Eq. (7). We also assume that a specular dominant facet is ideally mirror-reflective. In such a case, the surface normal can be expressed as the halfway vector between the viewing and light source directions: To disambiguate the polarization-derived specular normal, of the four options, one can select the one agreeing the most with Eq. (8).

Depth estimation from surface normals
Using the estimated normal map, we can now compute surface depth. Under the orthogonal projection assumption, the surface gradient p(x, y), q(x, y) on a point (x, y) is defined as (Durou et al., 2020) p(x, y) = ∂z(x, y) ∂x q(x, y) = ∂z(x, y) ∂y (9) where z(x, y) is surface depth at (x, y). According to the relation between gradient and normal, the surface depth can be expressed as a function of normal: where z0 is the known depth of a starting point, and nx, ny, and nz are the components of normal n in the x, y, and z directions, respectively. The surface depth can theoretically be reconstructed by locally integrating Eq. (10). However, due to the image noise and surface discontinuity, the surface normal obtained usually does not satisfy the integrability constraint, resulting in the integration result being related to the selection of the integration path. The depth reconstruction by the direct integration is also very sensitive to noise. Therefore, we use here the Poisson solver (Agrawal et al., 2006) to robustly solve the integral equation for surface height reconstruction (Fig. 3).

RESULTS
Our proposed algorithm is verified on both synthetic and realworld data. All are characterized by smooth and glossy surfaces. In Fig. (4) we demonstrate a synthetic example, applied on the carnelian intaglio with a glossy surface and spatialvarying reflectance map. We specify its pBRDF and the refractive index η. For physically based rendering, we used the Mitsuba 2 software (Cui et al., 2017), which calculates the object polarized reflection according to the reflectance map and surface roughness and synthesizes high-fidelity polarization images via dense ray tracing. Visualization results (Fig. 4) show the estimated normal map and reconstructed 3-D model with fine surface reliefs. The simulated polarization imaging system consists of a DoFP polarizing camera and a point light source. The SfP completely reconstructed the surface shape without distortions, and the surface normals are also smooth. Quantitative evaluations of the normal estimation show mean angular errors of 3.15 • ; and mean relative depth error of 0.38%, indicating the successful application of the SfP. The angular error map also shows small and smooth forms, largely unaffected by specular highlights.
For the real-world validations, we also compared the reconstruction results of the photogrammetric SfM-MVS application to our proposed polarimetric method on a white porcelain vinegar pot featuring a smooth, featureless surface. For the SfM-MVS evaluation, 850 RGB images were acquired using the DSLR camera and introduced to the Meshroom software (Griwodz et al., 2021) for reconstruction. For our SfP modeling, the object was imaged by the LUCID TRI050S1-PC DoFP camera. Figs. 5 & 6 present the reconstruction results using the SfM-MVS and the SfP methods. Despite a large number of images used, the SfM-MVS reconstruction is still relatively sparse, making the reconstructed meshes incomplete and seriously distorted (Fig. 5). In contrast, our SfP method yields a Figure 4. SfP reconstruction on the high-fidelity digital replica of a pre-Islamic carnelian intaglio engraved with Roman portrait, 1st Century BCE.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy dense and smooth reconstruction of both the pot body, lid, and handle (Fig. 6). One can also clearly observe the shape variation of the pot surface from the mesh, allowing the reconstruction of minute concave/convex, usually unobservable. Fig. (7) shows the comparison of the contour of the reconstructed pot and the ideal arcs based on manually measured radii. The polarimetric result has millimeter-level depth errors and negligible distortions, demonstrating its high accuracy. Both synthetic and real-world examples demonstrate the value of the polarizationbased approach and the promise it offers in modeling, particularly with the realization that only a single image is needed for their application.

CONCLUSIONS
Shape reconstruction of objects featuring textureless and specular surfaces is a challenge, becoming relevant with heritage assets, where object surfaces may be made of marble, ceramics, or glazed pottery materials, which usually present complex reflective p roperties. To address this challenge, this paper proposed a shape-from-polarization modeling strategy. As demonstrated, only simple means in the form of a single point light source and a polarizing camera are needed for the reconstruction. The proposed model considered surface reflection properties, including specular and diffused ones. Using a relatively direct means to disambiguate the derived zenith and azimuth angles, surface normal information has been derived. By locally integrating the surface gradient, depth was estimated from the generate normal map regardless of the presence of specular and featureless regions. The outcome is a pixel-level dense monocular 3-D reconstruction model, demonstrating stability and completeness in featureless regions and one characterized by specular reflection. The visualization of 3-D reconstruction results also showed rich details and minute structures/edges, which are valuable in heritage documentation. Our proposed system is also compact and easy to operate, which is ideal for a non-intrusive modeling of artifacts where the requirements on acquisition time and space are high. Additionally, the insights gained by polarimetric analysis of textureless and specular surfaces offer value in developing methods and strategies to protect these historic assets.