When a tree model meets texture baking: an approach for quality-preserving lightweight visualization in virtual 3D scene construction

ABSTRACT Three-dimensional (3D) high-fidelity surface models play an important role in urban scene construction. However, the data quantity of such models is large and places a tremendous burden on rendering. Many applications must balance the visual quality of the models with the rendering efficiency. The study provides a practical texture baking processing pipeline for generating 3D models to reduce the model complexity and preserve the visually pleasing details. Concretely, we apply a mesh simplification to the original model and use texture baking to create three types of baked textures, namely, a diffuse map, normal map and displacement map. The simplified model with the baked textures has a pleasing visualization effect in a rendering engine. Furthermore, we discuss the influence of various factors in the process on the results, as well as the functional principles and characteristics of the baking textures. The proposed approach is very useful for real-time rendering with limited rendering hardware as no additional memory or computing capacity is required for properly preserving the relief details of the model. Each step in the pipeline is described in detail to facilitate the realization.


Introduction
The evolution of three-dimensional (3D) modeling techniques has enabled us to produce digital replicas of spatial objects with high fidelity.In urban scene construction, typical objects relevant to 3D depictions are buildings, streets and trees (Cura, Perret, and Paparoditis 2018;Musialski et al. 2013;Wu et al. 2018).Urban vegetation reconstruction is crucial because vegetation occupies a large part of the land surface and greatly influences urban environments.To restore the structure and appearance of urban vegetation, especially old and valuable trees, detailed geometries and highresolution textures are created to represent trees and guarantee photorealistic visualization in a virtual scene.However, scenes containing urban vegetation are large-scale scenes that contain many tree models (Niese et al. 2022).The complexity and quantity of high-fidelity models place a large burden on rendering.Therefore, to improve the rendering efficiency, the geometries and textures of 3D tree models require appropriate processing.
Numerous approaches have been proposed in the field of 3D model processing.A common processing method for reducing the data size of geometries is mesh simplification (Burghardt and Cecconi 2007;Chen, Li, and Li 2015;Li and Nan 2021;Tariq et al. 2021).Mesh simplification algorithms transform a given polygonal mesh into another mesh with fewer faces by removing vertices or resampling.Although mesh simplification can accelerate vision algorithms by generating more compact geometry models, an oversimplified mesh may change the structure of a model such that the model cannot properly maintain texture mappings (Cohen, Manocha, and Olano 2003;She et al. 2022).Another limitation associated with mesh simplification lies in the ability of the model to preserve details.Mesh simplification maintains the main structures of a mesh but may obscure the underlying morphological details (Merlo et al. 2013).For old and valuable trees, it is difficult to obtain the desired models with pure mesh simplification.In texture processing, reducing the texture resolution is an effective method for optimizing model storage and transmission (Wolski et al. 2019).Several application instances, such Unreal Engine 4 and Unity, build mipmaps of a model by converting high-resolution textures into low-resolution textures, which support texture streaming for fast model loading (Beers, Agrawala, and Chaddha 1996).Nevertheless, low-resolution textures will obviously lead to a blurred surface where pixels cannot convey the precise color of the original model.
Additionally, tree modeling is different from the building façade modeling.First, the geometry of trees is highly complex and varied, while most building façades are generally composed of regular structures represented by vertical and horizontal lines.Hence, the simplification algorithms that are usually implemented for modeling building do not meet with the requirements for displaying photorealistic tree models.Second, the textural appearance of the trunk in a tree model is exactly critical because it is the most visible part and directly reflects the quality of the tree model.For example, low-resolution or repeated bark textures greatly influence visual realism.Third, compared with buildings, planted trees are much more numerous in urban environment; thus, tree models need to be used repeatedly in scenes.Consequently, the challenge of tree modeling is to strike a balance between the level of the details and the visualization efficiency.
Our objective is to reduce the geometric complexity of tree models while preserving visually pleasing texture details.In this paper, we introduce a process pipeline called texture baking for generating morphologically detailed tree models for quality-preserving and lightweight visualization.Texture baking overcomes deformations caused by pure mesh simplification by converting the decimated geometry information into particular textures (Merlo et al. 2013;Verhoeven 2017).The textures preserving the details of the corresponding models are well suited for parallel calculations to restore this geometric information.As a result, the method can significantly simplify the geometry of the models and provide a highly realistic and efficient visual experience without texture resolution reduction, which supports the ability to use 3D models intensively for scene construction if the rendering hardware is limited.Notably, this study highlights two critical techniques for implementing the method.First, the concept of a 'cage' for baking a model is adopted.The cage plays an important role in recording detailed information of the original model.Second, a displacement map is used to offset the information lost during the simplification process.The displacement map is also a significant reflection of the transformation from geometry to image.
The main contributions of this paper are as follows.
(1) We propose a systematic technological workflow for creating target models.Both the geometric and textural aspects of the models are considered in the workflow.Although the workflow is not fully automated, operational details are further explained to facilitate the reuse of the method.
(2) We attempt to explore the influencing factors of texture baking.In light of the unique parameters in these steps, by categorizing and comparing, we analyse the influencing mechanisms of the parameters and make suggestions for obtaining better baking textures in practice.
The article is organized into six sections.The second section provides an overview of related work.The third section describes the methodology.The fourth and fifth sections present the experimental results and further discuss several aspects of the results, respectively.The last section presents the conclusions of the study and discusses our future work.

Related work
To describe the baking procedures and results more clearly, we discuss the related works on data representation for a 3D models and applications in the field of texture baking.

Model representation
The structure and appearance of the model can be represented by geometries and textures.A geometry refers to a body defined by a collection of points in 3D space connected by various geometric entities, such as vertices, lines, triangles and polygons (Li and Nan 2021).The spatial organization is irregularly structured, and a specific vertex index file is required to record the ordering (Schilling, Bolling, and Nagel 2016).Apart from the basic geometric elements used to shape the model structure, each vertex is usually associated with texture coordinates, a normal, a tangent and other attribute for using textures or calculating illumination information (Miao, Song, and Zhu 2017;Schuster et al. 2020).Therefore, in many cases, the geometry files of a 3D model store not only thousands of vertices but also the corresponding attributes.Recent mesh modeling algorithms are closely associated with point clouds, and these dense point clouds have immense geometric data sizes (Fabio 2003;Mei et al. 2017).The main goal of mesh simplification is to generate a low-point-count approximation that maintains the basic morphology of the original mesh.Many mesh simplification algorithms remove sizable numbers of homogeneous faces from a detailed model full of redundant triangles (Bahirat et al. 2018;Fuhrmann, Umlauf, and Mantler 2005;Lee and Kuo 2010;Xie et al. 2012).These algorithms can effectively reduce the number of vertices, but they tend to create poor texture mappings that considerably change the original appearance.
The other important aspect of a 3D model is texture mapping, which is a method for defining normal or color information by attaching textures along the geometric surfaces of 3D models (Tsai and Lin 2007;Yoo 2013).Texture mapping originally referred to only diffuse maps, which are images obtained by simply mapping color pixels onto a model surface (Heckbert 1986).In recent decades, the advent of physically based rendering (PBR) systems has made it possible to achieve near-photorealism in real time via simulation by vastly reducing the number of polygons and the amount of lighting calculations needed (Greenberg et al. 1997;Pharr, Jakob, and Humphreys 2016).In a PBR system, more textures are created to express and record various physical properties, such as normal maps, displacement maps, specular maps, reflection maps, occlusion maps, roughness maps and glossiness maps.These textures replicate the physical ranges of values to accurately simulate a model of the real world.
Additionally, for urban objects such as trees, many other methods are used to represent 3D models, such as point clouds and voxels.In photogrammetry, they are usually represented by a set of triangles constructed from point clouds and diffuse textures converted from photography data.Therefore, we process tree models mainly from the perspective of mesh and textures in this study.

Texture baking
To eliminate the problem of sacrificing the appearance fidelity for simplification, researchers have employed texture baking on a high-poly and a low-poly mesh pair to transform information to compensate for the geometric deformations and to store valuable attributes (Kumar 2020;Merlo et al. 2013).Very little research has been conducted on texture baking in 3D real scene cases.The prior task solved by texture baking is accelerated modeling of lighting interactions in realtime rendering.Schäfer et al. (2012) proposed a memory-efficient and high-performing hybrid approach that combines texture-based and vertex-based lighting baking.The approach significantly reduces the memory requirements without negatively impacting on the rendering quality.Spini et al. (2016) implemented texture baking to create lightmaps to support synthetic 3D indoor environments of quasiphotorealistic quality.These lightmaps focus on preserving special lighting interactions such as accurate shadows and light reflections, which may impose a large computational burden in real-time rendering.Finally, a nearly photorealistic indoor scene can be directly obtained by a remote web service.
Apart from lighting baking, other types of textures are also exploited to increase the model quality (Schuster et al. 2020).In a study on 3D modeling in the cultural heritage domain, Verhoeven (2017) explored the capacity of integration of computer graphics (CG) and image fusion and developed a processing method for generating a detailed and morphologically relevant diffuse texture of a 3D model.The paper presented a new method for conveying the geometrical features and the visible spectral reflectance of 3D models, and the method is highly suitable for rendering spatial objects with high fidelity.Following their lead, Tousant and Fai (2019) employed texture baking to generate five textures, namely, a diffuse texture, a normal map, an ambient occlusion map, a roughness map and a metallic map, which are the essential components of PBR materials.Compared with (Verhoeven 2017), this paper introduces more complex textures to transfer surface details from a high-poly mesh.The improvement demonstrates that texture baking is capable of producing more sophisticated textures.However, the above works do not explain any principles of texture baking or the compromise in converting geometries to images.Therefore, our work proposes a specific texture baking pipeline after obtaining complex 3D models, as well as a special baking texture that plays an important role in geometric detail transformation.

Processing pipeline overview
In general, the proposed method principally follows the flowchart in Figure 1, which contains four parts: a baking group, baking procedures, baked maps and a baked model.The target of texture baking is to preserve the high polygon (high-poly) mesh, i.e. the original mesh, details in the low polygon (low-poly) mesh.Thus, the processing objects in the baking procedure are the high-poly mesh and the low-poly mesh simplified from the high-poly mesh.Since mesh simplification necessitates extra steps and has many specialized algorithms in CG, the generation of a low-poly mesh will not be described in detail in this work.Finally, it is important to mention that the processing workflow is conceived with functional principles and operation availability.Although each step can be executed within various software programs, we describe each step by mainly following the common terms and approaches in CG rather than by using terms and approaches specific to one software.The software used in each procedure of the texture baking process is included in Appendix.
We create three baked maps: a diffuse map, normal map and displacement map.Although other textures, such as specular and occlusion maps, have been used to generate photorealistic effects, the three maps can guarantee the preservation of base shapes and colors.The proposed method can be applied to different texture types.

Mesh repair
When a mesh is created, geometrical and topological requirements are imposed.The mesh should be watertight (or closed) to guarantee that the geometry precisely represents the outer surface of a real object (Dey and Goswami 2003;Zhang, Gain, and Norato 2020).In many cases, however, a mesh generated using 3D data creation software may suffer from various geometrical and topological errors.For instance, gaps, holes and nonmanifold edges typically lead to triangular surfaces that cannot be used to obtain continuous UV coordinates in a normal way and, thus, affect the texture baking results in subsequent steps.Therefore, it is of the utmost importance to repair the highpoly mesh to a watertight and manifold mesh before mesh simplification.
As shown in Figure 2, before specifically addressing geometrical and topological errors, we extensively remove duplicate vertices and degenerated triangles to guarantee that the vertices and indices of the mesh are strictly correct.We mainly address three error types that occur when repairing a mesh.A gap is defined as an empty region between two sets of triangulated surface pieces.The effect of gaps in a mesh is that the boundary of the resulting model contains the edges that are the boundaries of the gaps.The mistake makes the surface of the model incomplete.Thus, we close the gaps in a mesh to guarantee that the boundary can embrace all triangles.A hole is an  empty area within a mesh, and the boundary of a hole usually consists of one or several triangle edges loops.Holes conform to neither the actual construction of the object nor the aesthetics and, more importantly, make the subsequent operations difficult, such as mesh simplification.We repair these defects by a hole-filling method to generate a continuous surface.Edges of meshes at which more than two faces are joined are nonmanifold edges.However, in practice, for a mesh containing nonmanifold edges, it is difficult to find spatial objects to physically represent these edges.In many software programs, models with nonmanifold edges may return an error when attempting UV unwrapping.Therefore, we identify all nonmanifold edges and eliminate the nonmanifold surface.

UV unwrapping
The textures of a 3D model are generally 2D and cannot be overlaid directly on a mesh, so the surface of a 3D mesh needs to be transformed into a 2D image, which is known as UV unwrapping or surface parameterization (Li et al. 2018;Poranne et al. 2017;Sheffer and Hart 2002).To accurately assign color or normal attributes to each vertex in the original mesh, the vertices have their associated coordinates on the 2D image, which are called texture coordinates.The expression of each vertex is also changed.Given that (x, y, z) is used to indicate a vertex of a 3D mesh in model space, the corresponding coordinate on the 2D image are represented by (u, v).The values of u and v are defined on a scale of 0.0-1.0.In the subsequent processing, the (u, v) coordinates are stored in the vertices of a simplified mesh and used to generate baked textures.
The principle of UV unwrapping is shown in Figure 3.Given that the original mesh is a cylinder, there are two main steps for illustration.First, we expand the cylinder into a rectangle along one vertical line called a seam, and then the points on the sides of the cylinder are projected onto the rectangle.Second, we can determine the directions of the UV coordinate axes based on the usage statement and the specific texture coordinates calculated according to the texture resolution.In this process, the choice of seams influences the texture layouts.The optimal texture layouts are crucial for minimizing any stretching, incorrect scaling or overlap effects that might reduce the final visual quality of the textured model.Therefore, many methods are provided for creating UV layouts, such as obtaining a UV map with minimal area deformation or a minimal seam.In baking procedures, the relation between UV unwrapping and texture baking is mainly reflected in the recorded pixel properties.The pixel properties of baking textures are eventually visualized in a UV map.Regardless of the type of texture, baking should follow the common UV unwrapping method, which guarantees that the resulting texture layout is uniform and easy to compare.Additionally, although many software programs support automatic UV unwrapping, several manual operations, such as seam adjustment, tend to be required for generating clean and intuitive UV maps.

Surface subdivision
Surface subdivision is a recursive algorithm that is used to add triangles or polygons to a mesh by dividing the faces into smaller faces without changing the overall shape.Figure 4 shows the principle of the Catmull-Clark algorithm (Halstead, Kass, and DeRose 1993;Ni et al. 2009).For a mesh composed of triangles, the algorithm subdivides its surface into a quadrilateral mesh.First, the mesh generates new points, called face points, on each triangle by the following formula: where v a , v b and v c are the three vertices of the triangle on the original mesh.Then, for the mesh with face points, the algorithm uses the following formula to generate new points called edge points: where f m and f n are the face points of two adjacent triangles.Furthermore, the algorithm updates vertex points with face points and edge points by the following formula: where q is the average of the face points for all faces adjacent to v and r is the average of the midpoints of the edges associated with v. Finally, the algorithm connects the newly generated points in each step, namely, (v ′ , e) and (e, f ), to create a new mesh, which has many more faces than the original mesh.In the baking procedures, the mesh generated by surface subdivision is the foundation of the cage used in the next step.Denser surface subdivisions enable the use of more pixel records, which preserves more surface details.In addition, surface subdivision plays an important role when rendering a mesh with the corresponding baking textures.The input low-poly mesh is converted into a high-poly mesh by surface subdivision, and then the textures baked by the subdivided surface are used as a cage source for rendering, which significantly enhances the realism of the rendering effects.

Cage creation
A cage is created to record more appearance details of the high-poly mesh by ray casting.Ray casting is one of the fundamental elements of baking; it penetrates both high-poly and lowpoly meshes simultaneously and then records the differences between them.The concept of a 'smooth group' is conducive to understanding the pertinence and purpose of a cage.A smooth group involves a pair of concepts 'hard edge' and 'soft edge'.A hard edge is also regarded as a seam or a sharp edge, which refers to an edge where each vertex has more than one normal.Conversely, a soft edge is an edge where each vertex has only one normal.Then, a smooth group is a group of triangles in a mesh surrounded by a series of hard edges, where all triangles consist of soft edges.Supposing that there are more than two smooth groups in a mesh, as shown in Figure 5(A), the cast rays derived from the low-poly mesh in the same direction as the normal will intersect with the high-poly mesh at a point.However, a gap will appear, causing small areas on the high-poly mesh to be ignored since the common points in two smooth groups have different normal directions.Given that all triangles are in a smooth group, common points share the same normal direction to avoid the emergence of gaps.The principle for this case is presented in Figure 5(B).
Nevertheless, it is rare for all triangles to be in a smooth group for a 3D model because real spatial objects have various shapes, such as cubes and cones, which have obvious hard edges.Therefore, as shown in Figure 5(C), we use the new mesh obtained by surface subdivision from the lowpoly mesh as the cage and expand it to fully wrap the high-poly mesh.In other words, the cage is a duplicate of the subdivided surface, which is scaled to completely encase both meshes.Then, we use the normals emitted by the vertices on the cage as cast rays, which are capable of intersecting both high-poly and low-poly meshes to record the difference.Compared with the case illustrated in Figure 5(A), the method produces more cast rays and, thus, covers more mesh surfaces.Significantly, two factors can affect the baking results.First, a cage should be guaranteed to expand to fully cover all surfaces of the high-poly and low-poly meshes; otherwise, baked textures may contain holes with no pixel properties.Second, the ray casting directions should be set towards the side where the high-poly and low-poly meshes are.
As the source layer for ray casting, a cage solves the problem of generating a gap area by using the normals of the vertices of the low-poly mesh.Although the baking time will increase accordingly, this time is better spent than the time consumed in real-time rendering.

Texture baking
In essence, texture baking transforms geometric complexity into textural complexity.All the fine details from the high-poly mesh are stored in multiple high-resolution textures, which are applied in a mesh featuring far fewer polygons.Their color, light and shade information instruct the rendering engine on how to properly render all the fine details and create a visual illusion of the real object.
Texture baking supports multiple texture types, and the steps in our work contain three types: diffuse maps, displacement maps and normal maps.The textures record different information from real objects, i.e. diffuse maps for color, displacement maps for concave-convex attributes and normal maps for illumination.The information effectively restores the surface appearance of real objects and is also the most important type of information for improving realism and stereognosis displays.A diffuse map is the most fundamental and prerequisite type of map.It reflects the color of the model surface, which is also the visual information most directly perceived by the human eye.A normal map is a classic texture used to compute illumination.Specifically, a normal map adds surface details by mimicking the lighting of bumps and dents, which makes the surface appear to be rough.For a 3D model, a normal map is also generally a prerequisite.A displacement map is a special map for 3D modeling.Different from a normal map, a displacement map truly changes the positions of the vertices of a model to add surface details.Additionally, a displacement map has a more complex usage mode than the other two types of maps.The introduction of a displacement map produces surface details that are different from the traditional textures used in highefficiency and high-quality rendering.Apart from the above textures, other textures can also be obtained through texture baking if necessary, such as specular maps, roughness maps and ambient occlusion maps.These textures are used to represent various properties of a PBR material, and the means of creation and usage are similar to those of diffuse maps and normal maps.
With the previous preparation steps, the objects in texture baking consist of four elements: highpoly mesh, low-poly mesh, cage and ray casting elements.The process is illustrated in Figure 6.When beginning texture baking, each point on the cage emits a group of rays along its normal direction that intersects the high-poly mesh and low-poly mesh simultaneously.Here, we discuss three cases: (1) If baking a diffuse map, cast ray will transfer the pixel color of the high-poly mesh to the lowpoly mesh and fill the color at the corresponding UV coordinates.(2) If baking a displacement map, cast ray will record the distance difference from the high-poly mesh to the low-poly mesh and normalize it to a stored value range of 0-255, so the displacement map is generally greyscale (Doggett and Hirche 2000).
(3) If baking a normal map, ray casting will transfer the normals of the high-poly mesh to the lowpoly mesh and convert the coordinates of the object space to tangent space coordinates for storage, which will make the normal map in tangent space generally appear blue (Peercy, Airey, and Cabral 1997).
The resulting textures from texture baking should use the immutable UV coordinates created in UV unwrapping.Additionally, the size of the baked textures depends on the resolution at the time when the parameters are set.A higher texture resolution will produce a better rendering effect but lead to a longer baking time and a larger texture size.

Texture baking results
The section presents the results of each module in the baking process separately and the final baked textures.The original high-poly mesh and the simplified low-poly mesh of a tree model are compared in Figure 7.The number of triangles is substantially reduced, but the low-poly mesh still preserves the basic trunk shape.Figure 8 is a partial detail comparison between a high-poly mesh and low-poly mesh.The results reveal that simple simplification cannot produce the expected tree model because all surface details of the tree trunk are lost.Many parts turn into plain outlines rather than detailed sculptures.
Figure 9 shows the results of UV unwrapping by two approaches: (A) is the UV map obtained by unwrapping with fewer faces; this approach preserves the continuity of the surface to the greatest extent.In other words, this approach uses fewer seams in the process.(B) is the UV map obtained by unwrapping with slight distortion.The distortion occurs in the projection from a curved surface to a planar surface.The approach increases the seam of this curved surface, which can effectively reduce the deformation but creates more fragmented plane surfaces.The criteria for choosing the approach for UV unwrapping generally depends on the properties of the 3D model on which developers faces.Concretely, unwrapping with fewer faces will finally create clearer and more connected textures, while unwrapping with slighter distortion can guarantee the correct pixel color and locations but produce a relatively fragmented texture.In the subsequent procedures, we select (A) as the final texture coordinates.There are three reasons for the decision.First, according to the  appearance of the simplified model, the size and distribution of triangles in the low-poly mesh are roughly well-proportioned, which does not lead to severe distortion in the UV map.Second, a smaller number of faces makes it easier to manually adjust the texture coordinates.Third, especially for a tree bark model, we prefer to use several complete single surfaces rather than more divided surfaces to represent a large area of bark because more continuous pixels will be clearer in the final baked results.
The subdivided surface obtained by the Catmull-Clark algorithm is shown in Figure 10.The mesh is much smoother than the low-poly mesh after a surface subdivision because the essence of surface subdivision is vertex interpolation.The operations inevitably result in an increase in the number of triangles.A comparison of the number of triangles among the low-poly mesh, high-poly mesh and subdivided surface is shown in Table 1.Although the number of triangles in the subdivided surface approximates or, in most cases, exceeds that in the high-poly mesh, the subdivided surface is still not as detailed as the high-poly model.For this reason, the subsequent employment of the baked textures is required and plays a critical role in reflecting model appearance.
Figure 11 describes the appearance and use of a cage.The cage is duplicated from the subdivided surface in Figure 10(B), so the number of triangles is much greater than that of the high-poly mesh.The dense triangles enable more rays to be cast and to pass through the high-poly mesh.To fully enfold the high-poly mesh and low-poly mesh, we scale the entire cage to create a suitable gap existing between the cage and the surface of the high-poly mesh.For the uneven mesh, as shown in Figure 11(C), the cage still covers the rough surface to guarantee that no information is omitted.
The final UV map and baked textures are shown in Figure 12.The three baked textures have a resolution of 4 K. Figure 12(B) is the diffuse map.Compared with the original tree model, the pixels that are originally connected to the model can be separated by the diffuse map since they belong to different smooth groups of the UV map.We initially observe that the diffuse map is used to create a tree model from the colorful pixels.Figure 12(C) shows the normal map.The color of a pixel in the normal map indicates the normal vector of the matching point.The normal encoded in a tangent space compensates for the behavior of the low-poly mesh normal, so the normal texture is blue.Figure 12(D) shows the displacement map, which is a greyscale map.There are two major findings for the baked texture group.First, baked textures and UV maps have the same pixel distribution regardless of the texture type.Second, different types of baked textures have different pixel properties.The diffuse map and normal map record color and normal values as RGBA values (which are RGB values with an alpha channel that specifies the opacity), while the displacement map records the distance value as an integer from 0 to 255.

Model comparison
We compare the model attributes and visualization effect between the high-poly mesh and the lowpoly mesh with the baked textures.The model attributes encompass the following metrics of geometry and texture.
. Vertices: The number of vertices.Each triangle has three vertices, and each vertex can be shared by multiple triangles.A lower value is better for the loading mesh. .Indices: The number of indices for constructing triangles.Each index corresponds to a vertex in space, and stores a texture coordinate, a normal and other information.A lower value is better for loading mesh. .Triangles: The number of triangles.The number of indices is triple the number of triangles.
. Mesh size: The data size of the geometry loaded in real time.A lower value is better for the loading mesh. .Texture size: The data size of textures loaded in real-time.In our work, the high-poly mesh only uses the diffuse map, and the low-poly mesh uses three baked textures.
The attribute comparison is shown in Table 2. On the premise of maintaining the basic shape, the numbers of vertices and triangles of the low-poly mesh are reduced by 97% and 95%, Table 1.Numbers of triangles in the low-poly mesh, high-poly mesh and subdivided surface.respectively, compared with those of the high-poly mesh.Additionally, the data sizes of the baked textures are 12.21 M (diffuse map), 6.23 M (normal map) and 2.64 M (displacement map).The total data size for the geometry and textures is reduced by 54% in storage.
Figure 13 shows the model comparison.Figure 13 (A), (B) and (C) represent the mesh, the white model and the mesh with its diffuse map of the high-poly model.Figure 13 (D), (E) and (F) represent the mesh, the mesh with the displacement map and the mesh with all baked textures of the low-poly model.As shown in Figure 13(D) shows, although it can be identified as a trunk, the simplified mesh loses many details compared with the high-poly mesh shown in Figure 13 Figure 12.UV map and the corresponding baked textures.(A).When we perform surface subdivision and set the displacement map for the low-poly mesh, the visualization effect, shown in Figure 13 (E), is very close to that of the white model shown in Figure 13 (B).Especially after setting the diffuse map and normal map, the color, light reflection and surface details of the synthetic tree model are consistent with those of the original model, as shown in Figure 13 (F) and 13 (C).The results demonstrate that a low-poly mesh with the blended baked textures can provide a precise visualization of the original tree model.Moreover, we compare the treatment results of different methods on the tree model.Two classic simplification algorithms are implemented: quadric edge collapse decimation (Garland and Heckbert, 1997) and a quadric edge collapse decimation with textures (Garland and Heckbert, 1998).In Figure 14, (A) shows a detailed partial image of the tree model.(B) is the result from a pure simplification using quadric edge collapse decimation.Obviously, the base color in (B) is completely incorrect, and the geometry is fragmented in the detailed partial image.(C) is the result of quadric edge collapse decimation with textures, in which the simplification ratio is the same as that in texture baking.Compared with (B), (C) retains the generic pixel color, making the result look integrally like a tree.However, the result is not as vivid as (A) because the geometry is incondite and lacks abundant surface details. (D) is the result of quadric edge collapse decimation with textures, but with fewer triangles preserved than in (C).Black pixels appear on the surface, and many pixels are separate.The geometry is also jagged due to the reduction in the number of triangles.

Visualization in unreal engine 4
We use Unreal Engine 4 (UE4), a current mainstream gaming engine, to render a simplified tree model.The geometry of the model is imported directly and converted into the corresponding format in UE4.(B) is a transparent mode of (A), which reveals a clearer difference in the geometry.As indicated by the red arrow, a relatively distinct gap exists in the right bark of the model.The red area in (C) indicates the pixel difference between the two images.The red area contains 3404 pixels, which is 11.8% of the whole tree model.Correspondingly, 88.2% of the pixels are the same in both images.
In terms of the improvement of the rendering efficiency, the number of frames per second (FPS) is an important indicator for evaluating the effectiveness of texture baking.A single tree model does not have an active impact on the FPS of the scene, so we generate 400 duplicates of the high-poly model and baked model, which leads to a remarkable drop in the FPS.We measure the FPS of the high-poly model and the FPS of the baked model in the UE 4 environment.The FPS of the highpoly model is 17.09, and the FPS of the baked model is 24.30.The results demonstrate that apart from reducing the storage volume, texture baking is also conducive to improving the rendering efficiency, which facilitates the visualization of a large-scale vegetation scenes.
Finally, a complete presentation of a scene is included in the Appendix by way of the video.The scene is a real campus and contains many trees.We reconstruct the entire scene and use the proposed method to generate photorealistic tree models.

Discussion
In this section, we discuss the effect of the simplification ratio, cage and the iterations of surface subdivision.They have a critical impact on the baking result.Additionally, we introduce the concept of geometry images and discuss the methods of rendering optimization from the perspective of converting geometry to textures.

Simplification ratio
We simplify the mesh with different ratios and record the attributes and the final effects.Applying the simplification ratio means reducing the number of triangles until the current number equals the original number multiplied by the ratio.The attributes are compared in Table 3.The results indicate that the simplification ratio has no impact on the texture size.Furthermore, as shown in Figure 17, we compare the texture baking results of models that use different simplification ratios.The results suggest that there is a significant difference among these models.As the simplification ratio decreases, the restoration effect of the baking model worsens because many details are lost, as shown in Figure 17(D) and (E).
From the perspective of cast rays, a lower simplification ratio means that more geometric information is lost and that more rays need to be cast to record the differences between a high-poly mesh and low-poly mesh.The cast rays are derived from the cage, and the cage is transformed from the subdivided surface.However, we use the same parameters when performing surface subdivision in this case, which leads to similar numbers of cast rays and, thus, similar texture sizes.Therefore, it is necessary to augment the iterations of surface subdivision when using a lower simplification ratio.

Cage
We perform a texture baking experiment in which the cage does not fully wrap the high-poly mesh.
In Figure 18, (A) is the incorrect cage, and (B) is the corresponding baking result.In this cage, many vertices are less than zero units away from the high-poly mesh, i.e. inside the high-poly mesh.From the perspective of the functional mechanism of the abovementioned ray casting approach, the rays cast from these vertices are invalid because they cannot cross both the high-poly mesh and low-poly mesh at the same time.Compared with the original model, the surface of the baking result is abnormal.The vertex positions have changed dramatically, and many pixels are incorrect or even black.The result suggests that there is not enough recorded information to restore the original model.Therefore, the cage should be ensured to cover the entire surface of the high-poly mesh to avoid this phenomenon.

Iterations of surface subdivision
Surface subdivision is a recursive algorithm that is used to add triangles or polygons to a mesh by dividing the faces into smaller faces.In addition to creating a cage by surface subdivision, it is the operation that combines the displacement map to produce an actual change of the low-poly during rendering.Notably, the operation is resource-consuming for real-time rendering.Controlling the iterations of the surface subdivision is critical for the final effects, i.e. a high iteration leads to high resource consumption, and a low iteration cannot achieve the expected visualization result.As shown in Figure 19, we conduct a comparison of the visual effect which contains the surface subdivision with different iterations and the high-poly mesh.First, whether or not the surface subdivision is performed has an impact on the result, as shown in (A) and (B).From the perspective of the mechanism of baked textures creation, the absence of surface subdivision means the invalidation of those textures.Second, more iterations will perform better.As shown in (B) and (C), when the iteration is set to 10, the partial details in the red circle become clearer.Third, increasing the iteration does not further improve the model quality.We consider that it is a significant finding because continuous iteration will bring a burden on graphics processing unit (GPU).The finding also demonstrates that the final visualization depends only on the quality of the baked texture rather than the geometry, which reflects the advantage of transformation from geometry to image.

From geometry to image
Until now, triangular mesh representations of geometries and image representations of textures have been distinct, and an image has been attached to a triangular mesh.We tend to create levels of detail or even reduce the quality of the model in the face of a complex triangular mesh to decrease the pressure on graphics rendering.However, it is difficult to guarantee photorealistic visualization with this approach.For 3D model rendering, GPU are remarkably proficient at processing images,  in contrast to geometries, because of efficient parallel computing.Thus, due to this feature, an alternative method has emerged for optimizing graphics rendering by converting geometric calculations to image calculations.
In (Gu, Gortler, and Hoppe 2002), a basic entity, called a geometry image, that remeshes an arbitrary surface onto a completely regular structure was first proposed.The entity captures the geometry as a simple 2D array of quantized points and provides an approach that converts geometric complexity into image complexity.Texture baking is also a process of saving geometric information as image information.We compare these approaches and draw the following two conclusions to illustrate their differences.
(1) Converting different objects.In texture baking, we use images to record the differential information between high-poly and low-poly meshes while geometry images convert a whole mesh by creating a completely regular structure.Compared with texture baking, geometry images are more direct and efficient because they preserve all geometric information without simplification.
(2) Using different coordinates.Before texture baking, we perform a UV unwrapping of the lowpoly mesh and obtain the corresponding texture coordinates.We use the constant coordinates to carry the valuable information in the original model, so the pixel distribution of the baked textures is consistent with the distribution in the UV map.The pixel distribution of the geometry image is generated by a regular surface parametrization, and the parametrization is implicit and highly customized.
In summary, as GPU hardware evolves, converting geometric calculations to image calculations will become the focus in graphics rendering, and texture baking and geometry images are two available methodologies.Although there are some differences, both are extremely important and used in different scenarios for rendering optimization.

Conclusion
In our continued work, we construct virtual urban scenes, which are characterized by their subsistent spatial information representation and photorealistic visualization.For the high-efficiency rendering of a virtual scene containing numbers of tree models, this paper presented an approach for producing a quality-preserved and lightweight tree model.We proposed a processing pipeline of texture baking and described the specific technical principle of each step in the pipeline.The approach was employed to generate special textures to compensate for the missing geometric information caused by pure simplification.Our experimental results showed that the final high-fidelitypreserved tree model comprises a simplified mesh and three baked textures.The total data size of the model is reduced in storage, and the baked result is more suitable for rendering a virtual 3D scene with limited rendering hardware.Finally, we discussed the impacts of the simplification ratio, cage and the iterations of surface subdivision on the baked result and compared the similarities and differences between texture baking and geometry images.The method is conducive to facilitating the visualization of large-scale and multi-object scenes in 3D real scene domain.
This approach is not perfect, and improvements are always possible.First, the approach relies strongly on mesh simplification.The current simplification algorithm uses traditional edge collapse operations.However, in addition to tree trunks, however, many other urban objects, such as building facades, have characteristic geometric features.The handling of such objects involves a trade-off between hard and soft edges.General simplification algorithms are not appropriate for these objects, which require slimming down the meshes while preserving their structures (Li & Nan, 2021).Thus, developing specialized simplification algorithms for different urban objects is necessary.Additionally, fusing texture baking and geometry images to store complex 3D models is an approach worth pursuing.

Figure 4 .
Figure 4. Four steps in a surface subdivision process by the Catmull-Clark algorithm.Apart from the Catmull-Clark algorithm, a rich variety of subdivision schemes have been developed.However, the Catmull-Clark algorithm creates a smooth surface by the recursive application of simple rules and is increasingly prevalent in real-time applications, which are well supported by most software and rendering engines.

Figure 5 .
Figure 5. Creating cage to avoid the ray casting gap.

Figure 6 .
Figure 6.Principles of texture baking for a diffuse map, displacement map and normal map.

Figure 7 .
Figure7.High-poly mesh and low-poly mesh of a tree model.

Figure 8 .
Figure 8. Partial detail images of a high-poly mesh and low-poly mesh.

Figure 9 .
Figure 9. Different texture coordinates of the low-poly mesh obtained by UV unwrapping.

Figure 10 .
Figure10.Surface subdivision of the low-poly mesh.

LowFigure 11 .
Figure 11.Cage generation from a subdivided surface.The figure shows that the high-poly mesh is completely wrapped in a cage.
Figure 15 shows several cases of the tree model in UE4.(A) is the low-poly mesh.The material is implemented in the low-poly mesh.(B) is the low-poly mesh with only a diffuse map.(C) is the low-poly mesh with a diffuse map and normal map.Compared with (B), (C) shows a relatively dedicated surface because of the normal map.The normal map records the normal of each vertex of the tree model, which is used to compute reflection, refraction and scattershot with natural lights.In other words, the normal map plays an important role in reflecting lights and shadows in the illumination system.This feature is also a typical feature of PBR materials.(D) is the low-poly mesh with three baked textures.The color and surface details of the tree model are represented clearly.(E) is the original high-poly model.Furthermore, as shown in Figure16, we compare the image-based difference between a highpoly model and a low-poly mesh with the baked textures.(A) is a combination of two images.

Figure 13 .
Figure13.Model comparison between a high-poly mesh with the original diffuse texture and a low-poly mesh with baked textures.

Figure 14 .
Figure 14.Comparison of different methods on the tree model.

Figure 16 .
Figure 16.An image-based difference between a high-poly model and a low-poly mesh with the baked textures.

Figure 17 .
Figure 17.Baking results of models that use different simplification ratios.

Figure 18 .
Figure18.Baking result of the model when using a cage that does not wrap the original model.

Table 2 .
Attribute comparison between the high-poly mesh and the low-poly mesh.

Table 3 .
Attribute comparison among the low-poly meshes obtained by different simplification ratios.