Aesthetic style transferring method based on deep neural network between Chinese landscape painting and classical private garden’s virtual scenario

ABSTRACT Most of the existing virtual scenarios built for the digital protection of Chinese classical private gardens are too modern in expression style to show the aesthetic significance of their historical period. Considering the aesthetic commonality between traditional Chinese landscape paintings and classical private gardens and referring to image style transfer, here, a deep neural network was proposed to transfer the aesthetic style from landscape paintings to the virtual scenario of classical private gardens. The network consisted of two parts: style prediction and style transfer. The style prediction network was used to obtain style representation from style paintings, and the style transfer network was used to transfer style representation to the content scenario. The pre-trained network was then embedded into the scenario rendering pipeline and combined with the screen post-processing method to realise the stylised expression of the virtual scenario. To verify the feasibility of this methodology, a virtual scenario of the Humble Administrator’s Garden was used as the content scenario and five garden landscape paintings from different time periods and painting styles were selected for the case study. The results demonstrated that this methodology could effectively achieve the aesthetic style transfer of a virtual scenario.


Introduction
Urban cultural heritage is an important symbol of human culture, which condenses the most valuable achievements in the history of human civilisation development and is a bridge between history and modernity (Zhao 2008;Yang and Wu 2008).However, with the continuous acceleration of urban modernisation and the frequent occurrence of natural disasters, many traditional urban cultural heritage buildings, including tangible gardens, have suffered unprecedented impacts and damage.Therefore, properly protecting and visually restoring urban cultural heritage, including Chinese classical gardens, through three-dimensional (3D) digitisation and other technical means to realise the sustainable development of the urban context is an important issue being faced.(Skrede and Berg 2019).
To date, many studies have been conducted on the digital 3D virtual restoration of urban cultural heritage.The visualisation method can not only realistically restore the real appearance of urban cultural heritage buildings (Marques et al. 2017;Boutsi, Ioannidis, and Soile 2019;Liang et al. 2018;Pavel and Cajthaml 2020;Nguyen et al. 2021;Smith, Walford, and Jimenez-Bescos 2019) but can also reproduce the topography, landform, and other natural environment elements where the heritage buildings are located (Boutsi, Ioannidis, and Soile 2019;Liang et al. 2018;Pavel and Cajthaml 2020;Smith, Walford, and Jimenez-Bescos 2019).However, existing urban cultural heritage restoration methods lack expression of the rich artistic aesthetic features, resulting in the construction of digital virtual scenarios that are too modern in style (Marques et al. 2017;Boutsi, Ioannidis, and Soile 2019;Liang et al. 2018;Pavel and Cajthaml 2020;Nguyen et al. 2021;Smith, Walford, and Jimenez-Bescos 2019) and are unable to show users the aesthetic significance of urban cultural heritage within its historical period.This study was motivated by the following considerations.It is necessary to study the aesthetic style expression of urban cultural heritage virtual scenarios.As a representative of China's urban cultural heritage, Chinese classical gardens are materialised history, personified carriers, and famous for their natural landscape style.They are known as a pair of "Twin Art" with Chinese traditional landscape paintings (Chen 2011) and a form of expression of literati expressing their feelings for landscapes.Traditional landscape paintings and classical gardens have similar cultural and ideological origins.They permeate and blend in the process of historical development; hence, they have the same aesthetic style (Zhang and Sha 2018;Zhu 2019).This study intended to realise the aesthetic style expression of the digital virtual scenario of a classical garden through the transfer of aesthetic information from traditional Chinese landscape paintings.
Chinese classical gardens are mainly divided into three categories according to their affiliations: royal, private, and temple gardens.Among these, private gardens are models of Chinese classical gardens.Their gardening art is deeply influenced by painting theory and creation techniques of traditional Chinese landscape painting (Chen 2011;Liu and Huang 2007;Jiang 2010).In this study, the Humble Administrator's Garden, which is the most representative of Chinese classical private gardens, was selected as the research subject.Taking the aesthetic information of Chinese traditional landscape painting as the entry point and combining it with the theory of image style transfer (Gatys, Ecker, and Bethge 2015), we conducted the research on the transfer of Chinese traditional landscape painting's aesthetic styles to a classical private garden's virtual scenario based on a deep neural network.In this process, the following two crucial issues were addressed: (1) Identification of the types of aesthetic information in traditional landscape paintings that are suitable for transfer and represent the aesthetic style of the classical private garden's virtual scenario.
(2) Extraction of aesthetic information from traditional landscape paintings and transferring it to a classical private garden's 3D virtual scenario.
The remainder of this paper is organised as follows: Section 2 provides a literature analysis of aesthetic commonality (Chinese traditional landscape paintings and classical private gardens), a review of style transfer research, and a discussion of the differences between the present study and previous studies.Section 3 summarises the geographical location, historical development, and other information on the research subject and describes the experimental data sources used in this study.Section 4 describes in detail the methodology workflow of the virtual scenario aesthetic style transfer proposed in this study.First, the deep neural network architecture is explained in subsection 4.1, then the general steps of aesthetic style transfer in a 3D virtual scenario are described in subsection 4.2.Section 5 discusses and compares the research methods and results of this study.Section 6 presents the conclusions and future work.

Analysis of aesthetic commonality
Hegel, a famous philosopher, believed that Chinese classical private gardens were a type of painted art (Xu 2019).These gardens reproduce the beauty of nature with natural environmental elements such as water, vegetation, rockery, and pavilions (Zhu 2019).They combine with other cultures and art, such as traditional landscape paintings, to form the main characteristics of Chinese classical private gardens.As two different dimensional art forms, the expression contents of Chinese classical private gardens and traditional Chinese landscape paintings are basically the same.Both attach great importance to the creation of artistic conception and have commonalities in aesthetic style, which is mainly shown in the following three aspects: (1) Common creative sources.Both Chinese classical private gardens and traditional Chinese landscape paintings originate from the landscape culture.Landscape culture includes not only the famous mountains and rivers as sites of human activities but also philosophical thoughts relative to temple culture, that is, the so-called "Seclusion Culture" (Lu 2018).This kind of thought focuses on the relationship between humans and nature, guides people to re-understand the natural landscape, and furthers the understanding of nature from an aesthetic perspective.
The idea of leisure and seclusion was gradually integrated into gardening theory, which was reflected in the gardens created in the Ming and Qing Dynasties.(2) Same creation purpose.Traditional landscape paintings and classical private gardens express the masters' feelings.Chinese literati are good at injecting subjective thoughts and feelings into objective things.From their point of view, the lotus leaves the mud and does not stain, the plum blossom is proud and independent, and every plant, stone, and water body in poetry, calligraphy, painting, and the construction of a garden provides a medium through which they can express their emotions (Zhu 2019).In garden creation activities to imitate natural landscapes, garden designers express their pursuit of such virtues as indifference to fame and wealth, purity, and nobility through the image of the garden, so as to express their fresh, elegant, and extraordinary interests.Therefore, the creation purpose of classical private gardens and landscape paintings is the same; both are the sustenance of the creators' emotions and aspirations.
(3) Similar creative ideas.Both gardening techniques and painting theories appreciate nature and the freedom to achieve harmony and compatibility between humanity and nature.This is the case with existing Chinese classical private gardens and traditional landscape paintings; all elements are "inspired by nature from the outside and derived from the heart" (Ye 2005), are surrounded by mountains and rivers, have roads that are quiet and tortuous, and have buildings that are scattered among the mountains and forests.Nature inspires creators to strive to make works free and natural through clever ideas.It can be seen that the creators' pursuits of natural beauty are consistent in classical private gardens and traditional landscape paintings.
Based on the above analysis, this study considered transferring the fresh, elegant, free, and natural aesthetic style of Chinese traditional landscape paintings to the virtual scenario of Chinese classical private gardens, which mainly corresponds to the presentation of aesthetic information, such as the colours and brushwork of paintings (Chen 2011;Jiang 2010).Therefore, from the perspective of common aesthetic styles, this study mainly focused on realising the aesthetic style expression of virtual scenarios of Chinese classical private gardens through the transfer of the above aesthetic information from traditional landscape paintings.

Applied research on style transfer
Traditional image style transfer is mainly based on model rendering and texture synthesis.Efros and Freeman (2001) proposed a simple texture algorithm that combined and recombined the texture of a model to synthesise a new texture.Hertzmann et al. (2001) introduced a method based on the analogy that synthesises images with new textures through image feature mapping.Although the stylisation outcomes achieved by these methods are good, they only extract the low-level features of the image and not the high-level abstract features.When processing an image with complex colours and textures, the final synthesised image outcome is rough, and it is difficult to meet actual needs.
With the rise of deep learning research (Yin, Wang, and Wang 2015;Guo and Ding 2015;Mao et al. 2016;Gatys, Ecker, and Bethge 2015), an image style transfer method has been creatively proposed based on a convolutional neural network that separates the content abstract feature representation and style abstract feature representation of an image through a convolution network, and effectively accomplishes image style transfer by independently processing these high-level abstract feature representations.Previous studies (Gatys, Ecker, and Bethge 2015;Gatys et al. 2017) attracted extensive attention.Subsequent studies on image style transfer mainly included image-based iterations (Li et al. 2017;Li and Wand 2016;Liao et al. 2017) and model-based iterations (Johnson, Alahi, and Li 2016;Wang et al. 2017;Li et al. 2017;Ghiasi et al. 2017).
In addition to the style transfer of images, style transfer research has been conducted on texture, materials (Nguyen et al. 2012;Goudé et al. 2021), and the shape of 3D models (Ma et al. 2014;Lun et al. 2016;Friedrich and Menzel 2019) in recent years.However, these studies were unable to realise the direct and fast transfer of image style information to a virtual scenario.

Differences between the present study and previous studies
The differences between the method of the present study and that of previous studies are as follows: (1) This study was guided by the aesthetic commonality theory of style (traditional Chinese landscape painting) and content (virtual scenario of the Chinese classical private garden) to fulfil the content stylised expression.In contrast, previous studies have mainly selected arbitrary images for style transfer, and have had no theoretical connection between style (arbitrary images) and content.( 2) Previous studies of style transfer have mainly focused on images, single models, and local 3D scenarios with few models, which cannot directly and efficiently transfer aesthetic information to a large virtual scenario with many models.In contrast, the present study initially involved the design of a deep neural network that could predict the image style at runtime and then realises the style transfer of a large virtual scenario by means of GPU pipeline rendering and screen post-processing method.

Study subject
In this study, the Humble Administrator's Garden was chosen as the research subject (Figure 1).The garden is located on Northeast Street of the Gusu District, Suzhou, China.This is one of the most representative classical private gardens in China.It was built in the fourth year of Zhengde during the Ming Dynasty (A.D. 1509).In the following five centuries, it was rebuilt many times and has had many owners.In the 10th year of Shunzhi in the Qing Dynasty (A.D. 1653), Chen Zhilin reorganised and repaired the Humble Administrator's Garden.At this time, the garden reached its most celebrated period in history, as the overall natural landscape of the garden was maintained in the best condition (Wang and He 2021).From 1662 to 1949, the Humble Administrator's Garden experienced complicated changes, dilapidation damage, and lost the natural artistic style that it had at the beginning of its design.In 1951, the garden became a historical and cultural relic protection object of the government.In 1997, it was selected for the World Cultural Heritage List by the United Nations Educational, Scientific, and Cultural Organisation (UNESCO). 1

Data
The experimental data used in this study mainly include the digital virtual scenario of the Humble Administrator's Garden and the content image and style image data-sets used for deep neural network model training.A detailed description is provided in the following sections.

Virtual scenario of the humble administrator's garden
In this study, a virtual scenario of the Humble Administrator's Garden was constructed by integrating the 3D modelling of the garden elements and the virtual reality platform.1 3D modelling of garden elements.
To represent the scenario of the Humble Administrator's Garden as accurately as possible, plane CAD data of the garden were used as a reference, as shown in Figure 2 (a). 2 The data were imported into SketchUp software and 3D modelling of garden elements was conducted through extrusion, chamfering, smoothing, cutting, and connecting operations.The results of the modelling are shown in Figure 2 (b).(2) Virtual scenario construction and the typical shots.
The results of the 3D model of the garden elements were used as the model basis for the interactive virtual scenario, as shown in Figure 2 (b).Then, the C# script, Unity3D visualisation engine, and HTC Vive VR hardware equipment were utilised to construct a virtual scenario of the garden.The constructed virtual scenario was employed as the test material for scenario style transfer (Figure 3).
The gardening elements of classical private gardens mainly include rockery, water, vegetation and pavilions (Zhang 2010;Zhu 2019).Through the mutual configuration relationship of these four elements (Yu 2009;Hu 2010), the garden designers created a poetic and picturesque artistic conception.To show readers the content of the virtual scenario of Humble Administrator's Garden, we selected four typical shots from the virtual scenario by combining the typical elements of classical private gardens and referring to their mutual configuration relationship (Yu 2009;Hu 2010), as shown in Figure 3.

Data-sets of content and style images
The content and style image data used in the deep neural network training in this study were collected from the internet.Content images were obtained from the Baidu Picture search engines.The style images were mainly obtained from the Palace Museum, Nanjing Museum, National Palace Museum Taipei, and Harvard University Art Museum, and traditional landscape paintings from the early Northern Song Dynasty to the late Qing Dynasty were selected.
To reduce the noise interference of non-garden elements in the image data as much as possible, a brief denoising process was conducted on the image obtained from the internet by means of manual visual interpretation: (1) Content Image.Cutting off modern architectural elements in the image; (2) Style Image.Only retaining elements such as rockery, water, vegetation, and pavilions.After the image data de-noising process was completed, the TF-slim method was used to construct style and content image data-sets for neural network training with reference to the ImageNet data-set format.

Methodology
Under the guidance of the aesthetic commonality between Chinese classical private gardens and Chinese traditional landscape paintings, a deep neural network was designed that could quickly transfer styles, which was mainly used to predict and transfer the aesthetic styles of traditional landscape paintings.Then, it was combined with GPU pipeline rendering, screen post-processing, and other methods to achieve aesthetic style expression of the virtual scenario.The methodology workflow is illustrated in Figure 4.

Neural network architecture
Combined with the aesthetic commonality analysis of subsection 2.1, this study mainly focused on the transfer of aesthetic style information, such as colour and brushwork, in traditional Chinese landscape paintings.In combination with the principle of image style transfer (Gatys, Ecker, and Bethge 2015), a deep neural network (Figure 5) capable of fast transfer of aesthetic style was designed.The network architecture was similar to that of Ghiasi et al. (2017), and mainly consisted of the following two parts: (1) Style Prediction Network.The pre-trained Inception-v3 network architecture was employed (Szegedy et al. 2016) and the mean value across each activation channel of the Mixed6 layer was computed, which can take any style image as the input and predict the embedding vector S of normalisation constants, as shown in Figure 5.This network can infer a compact representation of the style from unknown style images.( 2) Style Transfer Network.This part of the network differed from that of Ghiasi et al. (2017).It mainly consisted of two downsampling layers and symmetrical upsampling layers, and the inbetween layer included five residual blocks.Considering the limited computational performance of the device, the size of the neural network was reduced, as shown in Figure 6.To stylise a 3D virtual scenario, the compact representation inferred in (1) was injected into this network.
The style loss L s (x, s) and content loss L c (x, c) of the neural network model were calculated using the spatial distance of the VGG-19 image classification network, and were defined as follows:  where f l (x) is the activation function of the l-layer network, n l is the total number of l-layer network elements, and g[ f l (x)] is the Gram matrix related to the activation of the l-layer network.The Gram matrix is a square symmetric matrix that is primarily used to measure the spatial average correlation of filters in the network activation layer.In addition, the total loss L of the style transfer network is the weighted sum of the content loss and style loss, as shown in formula (3): where λ s and λ c are scalable hyperparameters.λ s = 1.0 was set, and λ c was retained as a free hyperparameter.

Implementation of aesthetic style expression in the virtual scenario
The Unity3D visualisation engine was used to transfer aesthetic information, such as colours and strokes of traditional Chinese landscape paintings, to the virtual scenario of Chinese classical private gardens, and the HTC Vive VR device was combined to realise the visual expression of the aesthetic style of the virtual scenario.This process is illustrated in Figure 7.

Scenario rendering stage
The scenario rendering stage involved using Unity's built-in rendering pipeline to realise scenario object occlusion elimination, sequential rendering, circular rendering, and output to the frame buffer after the 3D scenario was built (subsection 3.2.1),as shown in Figure 8.In this study, the scenario rendering image of the research subject calculated in the scenario rendering stage was the focus (Figure 7), which was used as the content image for style transfer.The specific rendering process of the 3D scenario was not within the scope of this study and has not been discussed here.

Scenario post-process stage
In the post-processing stage of virtual scenario style transfer, the Unity light-weight cross-platform neural network inference library Barracuda was used to embed the deep neural network designed in this study into the frame buffering step of GPU pipeline rendering to create the stylised expression of a virtual scenario.The process was as follows: (1) Considering that Unity Barracuda recommends that the neural network model be imported into unity through ONNX (which is an open format that can be converted by most deep learning libraries), the pre-trained style transfer deep neural network model was first converted into the .onnxformat.( 2) The deep neural network model converted in step (1) was then imported to the Unity platform, in which the input, output, and network layer information of the neural network were displayed on Unity's asset inspector.(3) Finally, a custom post-processing script was created in Unity to load the neural network model with barracuda, and the scenario rendering image of the VR camera in each frame was acquired, the network style according to the input was inferred, and the stylised image was copied to the VR camera screen, thus, realising the aesthetic style expression of the virtual scenario.

Training the deep neural network
The aesthetic style expression method of the virtual scenario proposed in this study was implemented in the running time of the application programme.Therefore, the entire deep neural network must be pre-trained offline.After the training of the neural network was completed, it was used during the running time.To this end, custom image datasets obtained from the internet were used for neural network training.Among them, there were approximately 1,100 content images and 1,800 style images.The neural network was implemented using Keras and TensorFlow was used as the backend.The training of the neural network used the Adam (Kingma and Ba 2015) optimiser to carry out random gradient descent, and all of the hyperparameters and detailed architecture of the model are shown in Appendix Table A1 and A2.To ensure that the output stylised virtual scenario had style fidelity and content identifiability, the weights of the neural network were optimised in the training process.The hardware configurations used for the neural network training are listed in Table 1.Compared with the neural network of Ghiasi et al. (2017), the network model in this study could reduce the training time under the same parameters and epochs (Table 2) and better retain the content characteristics of the initial virtual scenario, as shown in Figure 9.

Stylised expression of the virtual scenario
To verify the feasibility of the virtual scenario aesthetic style expression methodology proposed here, the virtual scenario of the Humble Administrator's Garden was selected to achieve the aesthetic style transfer of the traditional landscape painting.In consideration of the post-processing stage, the style rendering of the virtual scenario, especially the inference of the neural network, required extensive graphics calculation, thus, NVIDIA's high-end graphics card with high video memory was chosen.Specific hardware information is presented in Table 1.
There are different types of Chinese traditional landscape paintings, among which the content of garden landscape paintings is closest to the elements of classical private gardens (both contain elements such as rockery, water, vegetation, and pavilions).Therefore, in this study, five Chinese garden landscape paintings with different aesthetic styles were selected to transform the style of the virtual scenario of the Humble Administrator's Garden (Figure 10).Background information and aesthetic characteristics of the five paintings are provided in Table 3.
The garden landscape paintings' content expression effect and artistic conception vary with colour, brushwork, painter, and time period.Therefore, the aesthetic style of the virtual scenario obtained by style transfer was also different (Figure 10).The differences in the virtual scenario style transfer results from the two aspects of colours and textures were analysed as follows: (1) Main colours of the virtual scenario.As shown in Figure 10, the colours of the virtual scenario transfer results corresponding to the five different landscape paintings were different.The neural network effectively transferred the background colours of different paintings to the skybox of the virtual scenario, and the element colours of the paintings were also transferred to the virtual scenario, such as rockery, water, vegetation, and architecture.However, the colour application of the five garden landscape paintings differed because of the influence of each painter's identity, which resulted in different colours in the virtual scenario.The painter in Figure 10(a) was a Buddhist, and his work incorporates Zen-like features and the artistic conception expressed is more tranquil and natural, thus, the colour style adopted was mainly light and the corresponding virtual scenario style was also relatively light.The painters of 10(b) and (c) were both literati, and their paintings are poetic and picturesque.The artistic conception that they express is quiet and elegant, with ochre as the main colour, and the painting style is dark.The corresponding scenario style was also dominated by dark ochre.The painters in 10(d) and (e) were both court painters, and their paintings are elegant and mainly use turquoise.The corresponding virtual scenario had a rich turquoise background.(2) Textures of the virtual scenario.The texture of the virtual scenario corresponded to the brushwork of the painting techniques.As shown in Figure 10, the brushwork information of the landscape paintings was transferred to the textures of the virtual scenario.However, owing to the influence of painting techniques, the fineness of the texture content of the virtual scenario was different.Figure 10 It can be seen from Figure 10 that the transferred virtual scenario not only had the aesthetic style information of garden landscape paintings, but also retained the content characteristics of the initial scenario, which realised the aesthetic style expression of the virtual scenario.
The virtual scenario aesthetic style transfer realised in this study was a dimensional expansion of the traditional two-dimensional image style transfer (Gatys, Ecker, and Bethge 2015;Gatys et al. 2017) and an extension of the expression medium, which was different from the existing style transfer research of video (Huang et al. 2017;Li et al. 2019;Xu et al. 2021), texture and material of a 3D model (Ghiasi et al. 2017;Nguyen et al. 2012), and surface shape of a 3D model (Goudé et al. 2021;Ma et al. 2014;Lun et al. 2016).The corresponding analysis undertaken was as follows: (1) The existing video style transfer and the style transfer of the virtual scenario realised in this study stylise each frame of the scenario and then output it to the user screen to realise the stylised expression of the entire scenario.The difference is that the virtual scenario style transfer  implemented here could change the input style image through the intervention of user interaction when the application was running to achieve the effect of dynamically changing the scenario aesthetic style and improving the user experience of the virtual scenario of the research subject, as shown in Figure 11.(2) Studies on the style transfer of texture and material of a 3D scenario model can realise the fine transfer expression of style, and the detailed resolution of the stylised scenario is better than that of the scenario style transfer method in this study.However, these procedures are relatively complex and the number of calculations required is large.It is generally used for the style transfer of a single 3D model or a local scenario with a small number of models (Liao et al. 2017;Johnson, Alahi, and Li 2016).The methodology employed in the present study can perform fast style transfers for a large scenario or a large number of scenario models.In addition, the stylised scenario resolution can reach 1,080 p (determined by the parameters of the graphics card), which satisfies the practical application requirements.(3) Traditional Chinese landscape painting artists use exaggeration and other painting techniques (Jiang 2010) to express different subjects in a personalised manner under the influence of their subjective wills in the process of creation rather than fully showing the true shape of objects.Hence, to avoid readers' misunderstanding of the appearance of cultural heritage, we didn't conduct research on the style transfer of a 3D model shape.This study was conducted based on historical reality, without changing the real surface shape of the urban cultural heritage and only changing the expression style of the virtual scenario to show the aesthetic significance of the urban cultural heritage in the historical period to the user.
The methodology of this study is also applicable to the aesthetic style expression of other types of urban cultural heritage digital virtual scenarios, as shown in Figure 12.Through the transfer of aesthetic information from traditional paintings and other objects, realising the expression of the aesthetic style of 3D digital scenarios of urban cultural heritage can provide support for the digital protection and restoration of urban cultural heritage.

Conclusion
The digital 3D virtual restoration of urban cultural heritage can not only realistically restore the real appearance of heritage buildings but also reproduce the terrain, landforms, and other natural environment elements where the heritage buildings are located (Figure 3).However, it fails to express the historical and aesthetic characteristics of urban cultural heritage, and the construction of a 3D scenario does not have the aesthetic significance of urban cultural heritage in its historical period.If a virtual scenario is manually stylised, it is very time consuming.Therefore, in this study, we proposed a methodology to express the aesthetic style of a virtual scenario of urban cultural heritage automatically and quickly.By analysing the aesthetic commonality of Chinese classical private gardens and traditional Chinese landscape paintings, aesthetic characteristic information that could be transferred to a virtual scenario was obtained.Combined with this aesthetic characteristic information and the principle of image style transfer, a neural network design was carried out for aesthetic style transfer.The network was embedded in the frame buffering step of the Unity pipeline rendering to realise the stylisation of each frame image of the VR camera to present a stylised virtual scenario of cultural heritage to the user.In this study, the Humble Administrator's Garden in Suzhou, China was used as a case study to verify the feasibility of the methodology.
This study was a preliminary attempt to transfer the style of a 3D scenario.In future research, exploration of the potential of the deep neural network style transfer method in the aesthetic style expression of the digital virtual scenario of urban cultural heritage will be continued.(1) The types of neural network training samples can be increased to improve the generalisation ability of the deep neural network.This is mainly because different countries and regions have different types and styles of urban cultural heritage sites.Users can select any cultural heritage scenario and corresponding style images to see the aesthetic style effect of the selected scenario.(2) The authors of the present study plan to introduce a self-attention mechanism to provide shading information of the style image into the network.When stylising the virtual scenario in this study, it was found that the sky and water in the garden scenario often had different degrees of artefacts because in many landscape images, the colours of these parts are bright and the brightness distribution is uneven.Therefore, introducing a controllable self-attention threshold to reduce the attention paid to the brighter part of the style image will help optimise the aesthetic style expression of the virtual scenario.

Figure 2 .
Figure 2. The 3D modelling process of the Humble Administrator's Garden.(a) CAD data of the garden; (b) 3D model of the garden.

Figure 3 .
Figure 3. 3D virtual scenario shots of the Humble Administrator's Garden.(a)-(d) Respectively take rockery, water, vegetation and pavilions as the main content of the shot, showing the mutual relationship with the other three elements.

Figure 4 .
Figure 4.The methodology workflow of this study.

Figure 5 .
Figure 5. Diagram of the neural network architecture.

Figure 7 .
Figure 7. Style rendering process of the virtual scenario.
(a) adopts freehand brushwork with rough strokes, whereas 10(b)-(e) adopts a fine brushwork technique with delicate and soft strokes.From the style transfer results of the virtual scenario, it can be seen that the textures of the scenario elements transferred through 10(a) are rough and the boundary is fuzzy, and the scenario elements transferred through 10(b)-(e) have fine and precise textures and very clear boundaries.

Figure 10 .
Figure 10.Virtual scenario style transfer results (partial shots of the results) with different aesthetic styles.

Figure 11 .
Figure 11.Demonstration of the aesthetic stylised virtual scenario.The red box indicates (a) the initial virtual scenario of the Humble Administrator's Garden; (b) the aesthetic stylised virtual scenario of the Humble Administrator's Garden.

Figure 12 .
Figure 12.Zhongshan Gate (Nanjing City Wall) virtual scenario.(a) Original scenario; (b)-(d) The style transfer results of the scenario corresponding to images with different aesthetic styles.

Table 1 .
Hardware information of network training.
Figure 9.Comparison of the aesthetic style expression of the three-dimensional virtual scenario of Ghiasi et al. (2017) with that of the present study.