VPL FOR CULTURAL HERITAGE MONITORING: SAMPLING AND COMPARISON OF PHOTOGRAPHIC IMAGES

: In the last decades, ICT strongly support cultural heritage conservation, management, valorisation and communication activities. Digital systems record series of data in real time, facilitating the automation of comparison, control and preventive evaluation activities aimed at the conscious management of the phenomenal reality based on the interrelationship between different information. According to these premises, this contribution presents first results of a research activity aimed at testing VPL parametric tools aimed at mapping environmental data and phenomena involving architectural surfaces and explicitating the comparison between data by graphical alert system. VPL tools are designed for a thematic image colour sampling. The main goal is monitoring the state of conservation of decorated architectural surfaces or artworks placed in unfavourable environmental conditions to facilitate preventive and intervention activities. The case study chosen to test this approach and digital tools is the fresco preserved in the Filangieri Hall of the State Archives in Naples.


INTRODUCTION
This paper shows the preliminary results of a research work focused on the development of algorithms aimed at monitoring the state of conservation of decorated architectural surfaces and artworks. The main goal is to support prevention and intervention activities through a specific semi-automatic alert system designed as a graphic warning of potentially damage events. This study proposes a methodological workflow for acquiring and processing big data using low-cost and user friendly VPL tools The VPL algorithm allows reading, discretizing, and comparing multiple thematic images and/or photographs produced from monitoring and diagnostic activities. Specifically, the research dealt with capturing in time frescoes decay phenomena. The decision to work with photographic images depends on the fact that most decay pathologies involve a surface alteration or material addition manifested as different colour to the original state. Furthermore, many diagnostic activities use thematic images and colour maps to describe the influence of environmental conditions on the artefact. The correspondence between these data and evident colour alterations can provide more complete and useful information to correctly define interventions, avoiding an excessive artworks damage. Therefore, this research work uses parametric digital tools to support decision-making processes. Monitoring data allow to identify and express ongoing phenomena through graphic manipulations. According to these premises, this work shows preliminary results about semi-automatic signaling of decay phenomena and environmental data variations acquired on architectural surfaces by of RGB images sampling.

Theoretical background
In recent decades, research and protection activities of Cultural Heritage have benefited from ICT solutions for the conservation, management, valorisation and communication of artworks. The preservation of artworks is important to witness human culture and artistic expression. The artworks are naturally deteriorated over time or may be damaged due to artificial processes.
Art conservators strive to protect artwork through nondestructive preservation and restoration procedures (Ligang et al. 2022, Sileo et al., 2022, Ergün et al., 2021, Borg et al., 2020. Quantity and typology of the modern information from acquisition and analysis require an adaptation of the taxonomic ordering structures to understand the phenomenal reality and to consciously dealing it, searching for a solution to big data management/visualization problem (Buratti et al., 2021). Technology facilitates the preservation of artworks otherwise compromised and not accessible. The delocalized synchronous and asynchronous digital information are continuously detected in real time by appropriate electronic devices. Many and various information require optimized management systems favouring the definition of alternative tools for re-elaboration and the comparison of the acquired data. Implementation of predictive models of risk, damage and structural stability and optimization of materials management processes are some of the possible fields of application of artificial intelligence (Kim et al., 2020). The specialized literature offers specific examples of analysis techniques to verify frescoed structures damage (Xiang et al., 2021). Change detection is the process of finding and evaluating access points in multi-spectra images that have undergone spatial or spectral changes and it is often defined as the comparison of two co-registered views of the same geographic area captured at successive periods (Goswam et al., 2022). Furthermore, analysing satellite images and remote sensing (RS) data using artificial intelligence (AI) tools and data fusion strategies has recently opened new perspectives for environmental monitoring and assessment (Himeur et al., 2022). Recently, interoperability between VPL and other digital products/tools is often used for monitoring architectures. An example is the VPL-CAD or VPL-BIM interoperability to enrich the potential of the digital twin thanks to the AIM_Algorithmic Information Modeling approach aimed at building a model to merge the object-oriented BIM hierarchical structure and the flexibility of VPL (Lo Turco et al., 2022, Lo Turco et al., 2021. This work proposes to use the potentialities of VPL algorithmic tools for the graphical comparison between geo-localised finite numerical datasets characterising graphic units of images (RGB codes -pixels) acquired over the time for preventive monitoring and thematic mapping applied to Cultural Heritage.

RESEARCH GOALS AND ACTIVITES
The proposed methodological approach is based on phenomenological reading of decay pathologies facing architectural surfaces over time. The regulations and guidelines consolidated at national and international level (UNI 11182) provide a series of descriptions and timely examples to distinguish decay pathologies according to the variations of the affected surface compared to its original state. In these cases, these are macroscopic variations in progress, which experts can easily recognize with the naked eye: surface alteration generally involves an immediately detectable colour variation (as well as the texture). Thus, an expert could recognize a pathology by comparing photographic images acquired at different times, before and after the appearance of the symptomatology associated with specific decay. Monitoring systems may not always prevent some complex diseases. However, the protection of valuable surfaces of historical and artistic interest requires designing and installing timely monitoring systems. According to these premises, the research groups in Survey and Representation of Architecture of the Universities Federico II and Suor Orsola Benincasa of Naples are defining a methodological approach and a digital toolkit to face this scenario. The acquisition activity involves the installation of simple photographic sensors in monitored architectural environments, set to capture many frames of a fixed scene at predefined time intervals. Through a wi-fi control unit, each photo shot is automatically sent to an archive (the data can be sent to a cloud or a physical archive placed in the structure), each frame is semiautomatically imported into the special defined VPL algorithm and automatically divided, pixel by pixel, into smaller analysed quads. The same scene is captured at different times according to constant lighting conditions. Then, the algorithm compares pairs of frames and reports any anomalies in the colour data between corresponding pixels. The main goal of this process is a constant monitoring system through low-cost resources, requiring expert human intervention only for data reporting. In addition, such data can be correlated with monitored environmental conditions (temperature, humidity), providing a useful information framework to understand both causes and effects to support decision-making processes in any interventions (Fig. 1).

Artworks presentation
Wall paintings are extremely complex works as integral part of monuments and sites. Their state of conservation must be related to the condition of the entire building and environment. The wall paintings degradation can be caused by changes in environmental parameters and/or serious external causes. Therefore, decay diagnosis needs environmental information to find all possible causes of deterioration. The case study chosen to test the procedure is an important fresco preserved in the State Archives of Naples. In this place once stood an imposing Benedictine monastery and it preserves many artworks and documents of the history of Naples. In particular, the largest room surrounding the Atrio dei Marmi is now known as the Sala del Filangieri. Once this room housed the ancient Refectory of the Benedictine monastery. The dimensions of about 36.00 x 11.00 m and the height of about 12.50 m make it one of the most majestic rooms of the complex. The back wall of the room is characterized by the presence of the fresco chosen as a case study (Fig. 2). Made by Belisario Corenzio in the 30s of the seventeenth century, it represents the affirmation of the Benedictine order depicting St. Benedict in the act of distributing bread to the friars of the order. The artist compares one of the symbolic Christian stories, that is the story of the multiplication of bread and fish by Jesus (Mazzoleni, 1964). The fresco is considered one of the most important composed by Belisario Corenzio for the Benedictine monastery and it was made in just forty days inspired by the painting by Giacomo da Ponte known as Bassano for the Monastery of Montecassino. In total, one hundred and seventeen human figures are depicted in bright colors in contrast with an austere background. The technique of realization involves an indissoluble relationship between the work and the wall and, therefore, with the host architecture. It is only a part of a much larger work considered not only for its historical value but above all from a technical point of view in terms of conservation (humidity of the walls, ambient humidity, temperature, lighting, etc.). The preservation of the former Benedictine monastery also favors the preservation of the thousands of relevant paper documents kept inside after its transformation into the State Archives. Over the years, the fresco has suffered several damages strictly caused by the vicissitudes of the architectural structure. An important restoration took place between the second and third decade of the '900 when the weight of the structures above (before) and the earthquake (after) generated serious injuries to the room and the wall that houses the fresco (Rosanova et al., 1999). Thus, effectively preserving of the entire architectural structure is essential for the optimal preservation of the works. However, the value of the artwork justifies timely operations for its preservation and protection, implementing diagnostic interventions with monitoring of the state of preservation and changes caused by time or unfavorable environmental conditions. Very often, decay phenomena on the frescoes depends on excessive levels of humidity, inadequate temperatures or states of ventilation and disadvantageous lighting conditions and its preservation cannot consist in a single and definitive intervention (Mora et al., 1999). As for the artworks preserved in museums, the frescoes of monumental complexes need a constant monitoring of the degenerative states and The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy periodic interventions to preserve the conditions reached with previous restorations and verified the reactions of the works to the new changes affecting the surrounding environment.

Design of the infrastructural acquisition system
A laser scan of the Filangieri's hall and adjacent spaces, also acquired by the research team for other purposes, is among the data available for experimentation. The point cloud, acquired with a Faro focus 3D s120 laser scanner, with a resolution of approximately 6 mm at 10 m, makes it possible to understand the position of the fresco, to digitally verify the relationship with the artificial and natural lighting of the environment, and to design, with preventive simulations, the arrangement of the cameras necessary to acquire the frames useful for monitoring the fresco and its variations over time, to acquire its evolutionary history, and to immediately verify anomalous states by identifying any potentially damaging alterations. The analyzed portion of the point cloud has been lightened and uploaded to the Rhinoceros modeling software. Here, virtual cameras have been created to simulate shots from different points of view, varying the focal distance and field of view according to the possible optics and the distance from the fresco. This made it possible to understand the geometric characteristics to be set in the actual chambers so that the result is appropriate for the investigation (Fig. 3).  According to the digital simulation, a single high-resolution camera in front of the fresco is sufficient; however, the artistic value of the entire physic architectural environment influences the position and anchorage of the fixed and visible instruments. Alternatively, two cameras were placed on the shelves of the side walls allowing two opposite and convergent angular framing on the fresco. The on-site infrastructure also provides a constant data connection for the streaming of photos to the dedicated software system (Fig. 4). A VPL algorithm (Grasshopper, Rhino) has been designed to compare pairs of photos captured in streaming. The algorithm allows images upload and comparison according to the RGB code of each pixel, identifying anomalies and providing a signal to the user for the intervention. The whole process is based on the comparison of the colors recorded in the photographic images; therefore, the process foresees constant light conditions and the digital simulation has considered the artificial lights of the hall, by setting the photo acquisition at night. In this way, color variations do not depend on light altered by the solar phases. The algorithm compares the first frame with a predefined number of frames previously captured from the same photographic location under constant conditions, records comparison and displays anomalies to the user.
The time distance between frames allows the comparison between minimal variations. The alert recognizes clusters of pixels with RGB code no longer matching. This property of the system, extensible and applicable to many types of investigation, in this case is aimed at identifying degradation phenomena of specific areas of the artifact.

METODOLOGICAL APPROACH
The methodological approach is based on the use of semiautomatic and user -friendly VPL digital tools to graphically explicit and compare numerical data lists corresponding to different properties of the analysed system or environmental parameters interacting with artwork, such as surface alterations and decay, thermographic maps, reflectance values, deviation maps, localised acquisitions (humidity, temperature, illuminance etc.). The algorithm specifically allows a constant low-cost monitoring system of the artifact through manipulation of data from images acquired by photographic sensors in different time and under the same controlled lighting conditions. The photographic series records the evolution of the state of conservation. The algorithm manipulates numerical data sets characterizing graphic units (RGB codes -pixels) by graphically translating information or implicit properties expressed in the form of numerical data. Therefore, images (complex system) and numerical data lists represent the input elements. The process consists in the graphic expression and subsequent comparison of sampled implicit data characterizing the images datasets of the same subject, acquired from the same position at different times (geometric -semantic decomposition). The output is a graphic alert system coinciding with the pixels clusters resulting from the automatic comparison between the numeric values characterizing pairs of pixels with corresponding index belonging to the same images acquired in different time. Therefore, the algorithm explicits the evolution of the phenomenon according to the variation of the pixels colour set as the check specific parameter. The post-acquisition workflow, developed and managed in the VPL environment, consists of the following steps: -real-time and/or post-acquisition monitoring (datasets acquisition); -decomposition of high-resolution image in smaller quadrants (if necessary to fix computational overload, fig. 5); -semi-automatic upload of images within the algorithm (automatic upload is work in progress); -image sampling (integral image or smaller quads); -extraction of RGB list for each pixel (each pixel/cell corresponds to a specific identification index); -construction of the neutral geometric grid as a graphical support (number of cells = number of pixels of the compared images = same indexes) to explicit comparison of the data lists resulting from separate image sampling; -comparison of RGB code lists extracted from images by equality criterion (equal values = True; different values = False); -comparison between average values extracted from monitoring activities and threshold environmental values (in the case of numerical lists not resulting from images); -definition of alert system to graphically explicit/modelling data comparison; -VPL/CAD/BIM tools interoperability aimed at recording and mapping the comparison of information on appropriate graphic support. Next paragraphs deepen each step of the process: the first step consists in identification, detecting and interpretation of phenomena and corresponding data/parameter (e.g. Image Sampling to explicit relation between decay/photographs/RGB channel code); the second step is to compare/overlap acquired datasets (Data Sets Management); the third step consists into recording and geometrically translate differences to convey pixels clustering (Alert system).

DIGITAL TOOLS
The algorithm is structured to compare acquired/extracted data lists or to compare the average/medium value from a single list with the optimal maximum and minimum value for a given parameter. In the first case, the comparison is expressed for each pixel of a neutral geometric grid by attribution of Boolean values True and False according to different colours (first type of alert: True = empty cells -False = red cells); in the second case, the change in value is expressed by the comparison between acquired values and threshold values: the difference represents the value of the input parameters aimed to control a geo-localized geometric signaler (e.g. the radius value of a sphere, second type of alert). Data sets come from a previous manipulation of the acquired digital supports (e.g. codes from Image Sampling), or it can be directly acquired by special devices in situ and compared (e.g. temperatures values, humidity values, etc.). Then, each data is subsequently geolocated on a digital media (neutral grid) for graphicization or information display. For each punctual variation resulting from comparison of data with same geolocation it will be possible to have a single alert. More alerts (clustering of coloured pixels) make up a colour region that represent the phenomenon in progress. The proposed algorithmic tools adapt the approach and process for manipulating different data lists regardless of the specific survey performed. Moreover, a point corresponds to the vertex/centre of a pixel or of a mesh face whose chromatic data, or also other type of data, can always be detected as a numeric code. Each face/cell will always correspond to an index that identifies its position/geometric coordinate on the element (planar grid, complex mesh). Therefore, regardless of the shape, it will always be possible to geolocate data and any changes starting from a discrete system, whether it is a planar image or a complex mesh. According to these premises, VPL definition was declined according to the information/data analysed, in this case pairs of photographs (planar grid) and comparison of lists of numerical data (RGB codes) resulting from different images sampling to create a geometric signaler/alarm system as pixels colour clustering (Figg. 6, 7).

Image Sampling: pixels indexing (input data definition)
This paper proposes the use of consolidated digital procedures for image manipulation, such as Image Sampling and Clustering, aimed at the search for semi-automatic and automatic solutions to approach images decomposition and semantic-geometric interpretation, according to the relation parameter : phenomenon = numeric value : pixels/cell. The representation of any implicit information requires the identification of a geometric support. The acquired information characterizes the geometric original place and describes characteristics of point or surface in relation to the specific minimum unit acquired. Considering the most common decay manifestations, the algorithm is defined to compare the photos according to the color of the pixels as the main parameter. A digital image consists in a set of converted continuous sensed data into digital form (Logeshwaran et al. 2022). The photographic sensors produce an image in form of analogue signal, which is then digitized to produce a digital image. A digital photograph is a bitmap image composed of coloured pixels. A bitmap image is an array of bits specifying the colour of each pixel.
A pixel (px_picture element), in computer graphics is the minimum conventional unit of the surface of a digital image. Pixels consist of a matrix (rows by columns) of square cells expressed by coordinates fixed by a graphic processing program. The number of pixels depends on the image resolution. Sampling allows the digitisation of a continuous image f(x,y) as function in both co-ordinates and amplitude. Therefore, this process includes two main steps: sampling (co-ordinate value digitisation) and quantization (amplitude value digitisation). The pixel of a binary image at a specific location (x,y) assumes value 0 for black or 1 for white. Grayscale images have intensity values ranging from 0 (black) to 255 (white). RGB image contains three channels: red, green, and blue: each channel has intensity values ranging from 0-255. Therefore, Sampling and quantization result in a matrix of rows and columns consisting of real numbers: according to RGB image, each cell or pixel corresponds to a set of values that compose the final colour. In this specific case, the Image Sampler component (Grasshopper, Rhino) uses the RGB code as a parameter for discretization/segmentation of pixels with corresponding indices belonging from images data set acquired in different periods. Pixeling/sampling an image (input) means to create a vector grid of cells equal to the number of pixels per height and width (output), as semi-automatic solutions to facilitate decomposition and geometric interpretation -semantics of images, expressed as a relationship between phenomenon/parameter -pixels/cell center -numerical value/colour. The Image Sampler VPL component allows to work with different colour channels. The algorithm provides two Image Samplers to separately explicit photographs as lists of data/RGB codes and compare the acquired value for each index corresponding to each cell/pixel (Fig. 8). This system allows the comparison between pairs of images of a same subject as data sets related to specific characteristics acquired in two successive periods, corresponding to the same indices. Process speed depends on different factors, such as size and resolution of the image, the level of detail. A high-resolution image, characterized by a high number of pixels, can be decomposed into quads to lower the number of compared pixels, speeding up the system but retaining a high level of detail. Pixels can be clustered according to the common RGB code by manually color detecting allowed by a Color Picker (semiautomatic selection), or by automatic extraction of the RGB code output list from the Image Sampler (automatic selection).
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-M-2-2023 29th CIPA Symposium "Documenting, Understanding, Preserving Cultural Heritage: Humanities and Digital Technologies for Shaping the Future", 25-30 June 2023, Florence, Italy The process adopted for this research work exploits the automatism allowed by the second approach (Figg. 9, 10). Next step describes comparison and selection automatic processes between sampled datasets, i.e. the lists of RGB values for pairs of corresponding pixels (same index), aimed to structure the alert system as pixel clustering as containers of unmatched values. Figure 9. Application of the algorithm on a fresco quad. Figure 10. The comparison between the same quad belonging to the same photo of the fresco acquired in different time shows a decay area corresponding to a pixels cluster with colour variation.

Data Sets Management: pixels comparison (data manipulation and selection)
Images sampling and comparison (cell/pixel with same index), according to different time acquisition and specific parameters/values (numerical variation) monitor and translate phenomenological manifestations, in this specific case architectural surface decay and/or crack pattern (image sampling and numerical values comparison, e.g. decay areas and crack amplitude). Overlapping between pixels allows contextual reading and comparison of different information (Lanzara et al., 2022a, Lanzara et al., 2022b. The main goal of data set management is data selection to activate an alert system. The designed alert system works with boolean operators. Booleans can assume two values: True -False (1 -0). Boolean identifiers and Boolean operations are widely used to provide instructions. The algorithm, according to comparison between images, determines whether an equality condition is true or false and it will manifest a specific result, otherwise invalid. VPL specific components for Data Sets management (items selection, list management, Data Sets creation/comparison/manipulation, Data Tree management), combined with Math Operators (in this case, Equality, Larger Than and Smaller than), allow to compare numeric geo-localized data by dispatch of Boolean pattern true/false, according to specific indices to localize the specific container cell for each Boolean value. By imposing equality condition between the lists of RGB codes extrapolated from different images, the algorithm returns a True -False list. The pairs of RGB lists (output from Image Sampler) are the input elements in the mathematical operator that compares the numeric values index by index: if equality is satisfied, the result is True; if equality is not satisfied, the result is False (Figg. 11,12). False indicates that a value change has been recorded at specific pixels or cells. Therefore, at this step, the main goal of the algorithm is to list only the indices corresponding to the False. The clustering of cells corresponding with False values reproduces the areal extent of the decay phenomenon.  This solution allows two-dimensional variation visualization and can be extended to three-dimensional data, adding the thickness parameter. By extending this approach to the management of generic numerical data, the Smaller than and Larger than mathematical operators allow the comparison between numerical data and threshold or optimal values. The difference between the recorded mean value and the reference value can be represented according to the parametric properties of the geometric primitives (formal and dimensional variation, e.g. radius-sphere relationship or radius-circumference). The next paragraph introduces colour attribution to pixels clusters according to the comparison result.

Alert system (type A): pixels overlapping (output)
The first type of signal consists of colored pixels corresponding to False value, i.e. RGB code variation, then decay manifestation. Therefore, this solution represents the ongoing evolution of the monitored phenomenon as surface development/extension (pixels clustering of mismatched values = semantically smart polygon), according to the physical manifestation of the phenomenon. Data Sets operators divide True -False list in two different lists, according to the specific boolean in input (Fig. 13).
True and False values are distributed in a neutral geometric grid composed of a cells matrix equal to pixels matrix (same number of rows and columns and same index for each pixel/cell). The Custom preview and Colour swatch components allow color attribution to grid cells: True = empty; False = red (Fig. 14).

Alert system (type B): parametric primitives (output)
Sensors can be directly installed on the artwork. This condition extends the algorithm utility to display point variations recorded on flat works and complex shapes. The variation of surface or environmental numerical data can be visually translated through primitives, e.g. spheres, geo-located on the digital model on the same position of the sensor; the variation of the difference between the acquired values and the optimal value represents the input value of the numerical slider associated with the parameter, e.g. radius of the sphere. The primitive changes size and/or color according to the variation of the parameter. Different size and/or color of the sphere represent a different type of signaler for the alert system.

CONCLUSION AND FUTURE WORKS
The main goal of this user-friendly and low-cost digital tool is expeditious and semi-automatic visualization of a decay phenomenon afflicting artworks comparing localized data acquired over time. Furthermore, the intersection of this information with microclimate monitoring datasets can generate a first link between cause and effect. This condition demonstrates the applicability of the system to any geolocated data sets to define different thematic mapping systems aimed at explicit data representation and comparison. The algorithm is being tested and improved. For the phenomena report system, the design of a cloud space aimed to record the date and time of the shots and the anomaly on the frame is in progress. However, high computational load generates problems in importing high-definition images in the algorithm, due to the weight of the image and the large number of values to be compared. The showed experiments are performed on limited quad of the whole images, also manipulated and transformed into low resolution images, therefore containing fewer pixels for the same image size. This manipulation does not affect the process, approach and outcome. The comparison between data lists is always possible regardless of the input image size and resolution. The component can be optimized to allow the import of the images file as a link to automatically update the captured photos time by time through two separate links. Work in progress and future works also concern the testing of the system in interoperability with BIM software completed by the consequent attribution of the properties of degradation to areas identified by pixel clustering.