EMBEDDING KNOWLEDGE IN 3 D DATA FRAMEWORKS IN CULTURAL HERITAGE

At present, where 3D modeling and visualisation in cultural heritage are concerned, an object’s documentation lacks its interconnected memory provided by multidisciplinary examination and linked data. As the layers of paint, wood, and brick recount a structure’s physical properties, the intangible, such as the forms of worship through song, dance, burning incense, and oral traditions, contributes to the greater story of its cultural heritage import. Furthermore, as an object or structure evolves through time, external political, religious, or environmental forces can affect it as well. As tangible and intangible entities associated with the structure transform, its narrative becomes dynamic and difficult to easily record. The Initial Training Network for Digital Cultural Heritage (ITN-DCH), a Marie Curie Actions project under the EU 7th Framework Programme, seeks to challenge this complexity by developing a novel methodology capable of offering such a holistic framework. With the integration of digitisation, conservation, linked data, and retrieval systems for DCH, the nature of investigation and dissemination will be augmented significantly. Examples of utilisating and evaluating this framework will range from a UNESCO World Heritage site, the Byzantine church of Panagia Forviotissa Asinou in the Troodos Mountains of Cyprus, to various religious icons and a monument located at the Monastery of Saint Neophytos. The application of this effort to the Asinou church, representing the first case study of the ITN-DCH project, is used as a template example in order to assess the technical challenges involved in the creation of such a framework.


INTRODUCTION
Cultural heritage is changing rapidly with the influx of new technologies that are influencing the way that heritage is managed, studied, curated, shared, and published.This fast growth has produced a wide range of new tools and methods, yet these new tools and methodologies are rarely integrated, and we have yet to see a framework that demonstrates best practices from start to finish; from acquisition to dissemination and publication.The Initial Training Network for Digital Cultural Heritage is a Marie Curie Actions programme funded by the European Commission's Seventh Framework Programme that proposes to deal with cultural heritage in a holistic fashion, filling in the gaps and unifying the pieces of heritage managed through all the steps.The ITN-DCH envisions heritage as bringing together all the sub-disciplines that work with heritage, unifying the data and practises so that CH professionals and the public alike and access these data for generations to come.This paper explores one of the ITN-DCH's case studies in Cyprus, the church of Asinou as well as another famous monument recently documented, the Monastery of Saint Neophytos.Both of these demonstrate the challenges and the reality of the proposed framework, and show how the fundamental focus of cultural heritage management must be able to deal with much more than two dimensional media and documents, but being able to embed and retrieve knowledge in 3D and 4D research environments.
Inscriptions from the founding of the church in the 12th century, written records spread throughout numerous archives, contemporary video recordings, thermal imaging, and 3D models of the structure as well as the objects that previously or currently occupy it are only some of examples which comprise the documentation available.Such a design of data retrieval and future accessibility will allow researchers to better study and illustrate the site as one complex unit.Once this can be achieved, DCH can facilitate public outreach through interactive education and virtual museum exhibitions.
Although the Asinou church is the first test case of this project, other sites and objects have also begun to have this framework applied.One of these is the Monastery of Agios Neophytos near Paphos, Cyprus where Saint Neophytos created his enkleistra, a natural cave modified into a chapel, living chamber, and later, his own tomb.The lower enkleistra chambers are currently being modelled using photogrammetry as well as surviving regions in the upper chambers which were constructed in 1197.The oldest remaining icons currently on display at the on-site museum were also modelled as well.
Following the holistic framework prototype of the Asinou church, these models represent the first steps in collecting and linking the metadata of this significant site into the cultural history of Cyprus.Future work is under consideration to expand the ITN-DCH framework to the entire monastery and assembling a CH management system, of the tangible and intangible, in order to assemble all available data, found locally as well from various national and international archives and museums.
We will first briefly present the sites of Asinou and Neophytos, giving a brief background of their history and current state.Following that, the acquisition campaigns at both sites will be outlined, with particular emphasis on not only the acquisition and processing of cultural heritage data, but also the importance of the gathering of metadata.

CASE STUDIES
This paper looks at two different projects, one of the official case studies of the ITN-DCH, that of Asinou, and an additional supporting case study, Agios Neophytos, a monastery near Paphos, Cyprus.The following sections will provide brief introductions to the sites and their importance as case studies demonstrating the integration of cultural heritage data into 3D platforms.

Asinou
Officially known as the Panagia Forviotissa Asinou church located in the Troodos Mountains of Cyprus.It is a UNESCO World Heritage site as well as a case study of the ITN-DCH project.Constructed in the 12th century, this church has been previously documented by a number of researchers (Winfield and Hawkins, 1967) (Carr and Nicolaïdès, 2012).This Byzantine-era church is comprised of a heterogeneous set of cultural heritage information ranging from historical inscriptions and frescoes to contemporary television broadcasts.In addition, geometric documentation has previously been acquired through the use of both terrestrial laser scanning and photogrammetry (Sofocleous et al., 2006).The current focus of this monument by the ITN-DCH project is to design a single holistic framework in order to best document and disseminate its heterogeneous data for both future academic research and the public.

Agios Neophytos
The Monastery of Saint Neophytos, located outside of Paphos on the island of Cyprus, was founded by its namesake in the 12th century.According to Galatarioutou (Galatariotou, 2004), Neophytos first found the cave (known as the enkleistra, meaning a place of seclusion) on the feast-day of Saint John the Baptist (24 June) in 1159.From that time, he dwelled in the enkleistra, making it his home for over a decade before the monastic community grew up around his solitary home by request of Bishop Basil Kinnamos.
Today, the monastery continues, attracting tourists and pilgrims from around the world to see the wall paintings inside the enkleistra, which were painted sometime towards the end of the 12th century and restored at least once after 1215 and also in 1503 (Papageorgiou, 1998).The seismic activity has already destroyed some of the enkleistra as can be seen when one visits the site today.This seismic activity is a major danger to the heritage of the site, and preventative measures must be taken in the preservation of the enkleistra, ideally high-resolution laser scans in addition to photogrammetric acquisitions.

Asinou
Close range photogrammetric field acquisition was carried out at two locations for two religious icons from the Asinou church with coded targets in order to properly record scale and assist with alignment.The oldest surviving icon from the Asinou church is currently on display at the Byzantine Museum in Nicosia.Due to time constraints and lacking optimal acquisition conditions, it was decided that documentation had to be accomplished in the museum exhibition without any control of lighting conditions.Although these settings were not optimum, the icon was placed on a table and 246 images were taken in RAW format with a calibrated DSLR camera of each of its sides as well as the along its edges.
Next, field acquisition was accomplished at the Asinou church where the current Asinou icon dating from the 19th century was on display.The conditions of this acquisition were more challenging due to unsuitable lighting conditions and mobility within the church.The geometric shape of this icon was more complex as well, which required that images needed to be taken with the icon from three positions, face up, face down, and finally balanced by two individuals on its side.The complete image set totalled 900 RAW-format images.
Figure 1: Earliest Icon from Asinou: Saint John the Baptist

Agios Neophytos
Using a combination of a calibrated Canon DSLR camera as well as a calibrated Apple iPhone 5 camera, photogrammetric acquisition was accomplished of religious icons currently on display at the onsite museum as well as the enkleistra, which is today a complex of three rooms interconnected by small doorways.We placed visible coded markers between the rooms to easily join them later during processing stage.The total size of the dataset was 1222 images from both cameras.Furthermore, the second room is made up of entirely irregular features, low ceilings, and varying light conditions, making it very difficult for standard photogrammetric acquisition.For example, parts of the room required the acquisition to be done while laying on the floor in order to capture the entire area.Photogrammetric acquisition was also conducted at the monastery's museum of three of the oldest religious icons, Agios Neophytos, Christ Filanthropos, and Virgin Eleousa.The Agios Neophytos icon could not be removed from its display frame so only the front side of the icon could be documented (see Figure 3).The conditions of this acquisition were similarly challenging due to lighting conditions within the museum.As completed for the John the Baptist icon from Asinou, the Christ Filanthropos and Virgin Eleousa icons were also placed on a table with images taken of both sides of the two icons.The image set for the icons of Agios Neophytos, Christ Filanthropos, and Virgin Eleousa were 118, 196, and 142, accordingly.GmbH2 (release 1.0.9.38) was used in order to generate an ideal geometric record of each religious icon as well as the enkleistra (Rothermel et al., 2012).Both high and low resolution point clouds and textured meshes were required for diverse documentation and visualisation purposes.Although more processing still required for the enkleistra dataset due to its size, all of the religious icons have been processed at different resolution levels.

Development of the 3D Viewer
Our first decision was how one might disseminate the data, and it seems that the best option available to us was through a webbased platform.However, this platform required specific features: • Ability to visualise and navigate a 3D model online with desktop, tablet, or phone.
• Ability to link to other resources (JSON-LD, RDF, HTTP, FTP, etc.) • Capacity to view very large point clouds

• Service management
We tested out different tools such as 3DHOP (Potenziani et al., 2014), Potree3 , WebGL4 , and made our choice on making our own real time viewer with the Unity3D engine for the 3D Viewer.This choice gave us more flexibility in implementation and integration.For the large point cloud viewing, we opted to use Potree.

Integration of resources
For testing purposes, we tried to integrate different types of media through the 3D viewer: 3D photogrammetric model, 3D geometrical model, texts, pictures, point cloud, etc.

Photogrammetric model
The main point here that we recognised is how difficult it is to display very large and complex meshes online which have millions of faces.The computational cost is too high.Therefore, we had to determine how best to simplify the 3D data without losing much visual information.We then reduced the topography, reduced the noise, and reduced the texture size and the entire polygon structure.
Figure 5: Geometrical model carrying semantic historical information

Geometrical Model
The main idea was to visualise a different architectural state of the church through time.We created 5 different CAD models.We decided to allow the user to switch between the photogrammetric model which was displayed by default and the geometrical models.The user can also change between the different CAD models with a slider at the a bottom of the viewer.
Figure 6: Asinou with Integrated Digital Photographs

Photos
We also implemented the possibility to display photos taken on site, overlaying the camera's position around the virtual church.

Motion Capture Data
In order to not only focus on case study's architecture, we also integrated motion capture data, under the form of an animated avatar going around the virtual church to test the viewer's capabilities on complex motion data.
Furthermore, if we want to visualise some specific character such as religious Pope, we tested the workflow of facial modelling and texturing.Special programs was used to make a pre-defined ideal representation of a humanoid character head and match it to imported images.We could also produce so called "morph targets" to be integrated to a lip sync engine for the animation of the character to test if we can do it from a painting.

Graphical User Interface & interactivity
The graphical user interface consists of on-screen elements and virtual world space elements: • Button that controls specific views (fixed one and the main view where you can rotate around) • Toggle allowing the user to switch between 3D models • Measuring tool allowing to measure distance in meters • Sliders to switch between the different architectural periods.
• Links with external resources Since the large amount of points to render for such high resolution data made the display and interaction of these meshes online impossible, we needed a strategy to visualise them online.Finally, making links to an external viewer was the best idea, which also allows us to point to other external resources.
We decided to make some part of the UI of the church to link to different 3D point cloud and mesh viewers (Potree, 3DHOP, BlendWeb), giving the user the option to decide which best meets their needs for the research they might want to do.One example of how easily a user can interact with a large dataset is the potree viewer of the enkleistra point cloud.The point cloud totals over 252 million points from the filtered cloud generated by SURE, where the unfiltered cloud totalled more than 3.7 billion points.Most researchers do not have the computational access required to open and manipulate such large datasets however we found the potree viewer was easily capable of providing a responsible interactive tool that only requires a web browser (Figure 8).
As Figure 9 highlights, the platform could include multiple sites throughout the island, with all the appropriate 3D models, metadata, and information a researcher might like to use.While this is a prototype demonstrating the theoretical possibility, it is important to note that this was developed in an extremely short amount of time using readily available tools.Not all of these tools are robust enough for the types of research applications being suggested by this paper, nor are they mature enough to contain all the features necessary for creating a full-fledged research web-portal.Hardesty, 2014, Koh, 2014, Gil et al., 2015).All data entered into Apsida is eventually harvested by Europeana and made publicly available there (Figure 10).These data can then be linked to the virtualisation platforms to connect the metadata of the digitised models to the viewer, as well as related data provided by Europeana.More specifically, the development of this proposed framework integrates previous documentation strategies in cultural heritage while integrating novel approaches from disciplines such as remote sensing and semantics.The management of digital cultural heritage metadata is something that is not static but should be evolving over the full life-cycle of a project.Metadata must be properly recorded and organized during the preliminary planning stages, throughout the acquisition and processing steps, and chronicle the final stages of visualisation and dissemination.

CONCLUSIONS
The speed of digitisation is ever decreasing, and the quantity of 3D information is increasing rapidly.Large repositories make Figure 10: Apsida Metadata Platform it possible to share and link data in ways that were not possible not that long ago, and tools exist that allow us to publish all of these data online and more easily accessible to other researchers.However, all these are pieces that do not necessarily make a whole, and what we require now are workflows and frameworks that bring together all these disparate parts into a single unit that make it possible to explore all forms of data in a single place.Researchers studying the history of a monument should easily be able to access the history of archaeological interventions, restorations, material characterisations, or geophysical surveys.These platforms need to allow researchers the ease of visualising 3D data while also working in the traditional 2D realm.
This paper has demonstrated that we can embed the traditional knowledge into 3D platforms, knowledge usually presented in the flat space of the web.At the same time, we've shown that the tools exist to make this dream possible, it only requires assembling the parts into a cohesive whole.While much of what was done requires supervised or manual creation of the models, it is likely that with continued research and development, these processes can become automatic, allowing for researchers with little or no development experience to publish their data online and therefore make a contribution to the rest of the academic community.
Future work will continue to integrate data from different disciplines into the framework to demonstrate how this might be done, building on existing platforms and services to bring 3D cultural heritage informatics onto the web in a way that is accessible by all.
consortium as a whole, nor a certain participant of the ITN-DCH consortium, warrant that the information contained in this document is capable of use, nor that use of the information is free from risk, and accepts no liability for loss or damage suffered by any person using this information.

Figure 3 :
Figure 3: Close up of the textured model of the Icon of Saint Neophytos

Figure 4 :
Figure 4: Implementation of the Photogrammetric Model of the Asinou Church in a Virtual Environment

Figure 7 :
Figure 7: Measuring tools in 3D virtual environment