1 Introduction

Heritage Buildings (HBs) are structures that have been around for a long time and are of cultural or architectural significance. Therefore, safeguarding and conservation of cultural heritage is an important problem whose solution requires both theory (Feilden 2007; Mehr 2019) and good practices (Mekonnen et al. 2022).

The protection of the world’s cultural and natural heritage has received a great impetus since 1972 when the UNESCO Paris Convention established its goals. However, HBs around the world are still confronted with the unending problem of unsatisfactory maintenance management (Akasah et al. 2011; Adegoriola et al. 2021). It has been recently observed that, although the HBs are lucrative assets able to generate millions of jobs and billions of euros in revenue yearly, there is still a need to devote more attention to their preservation (Otero 2021). What appears to be often overlooked is the issue of conservation (Woodward and Heesom 2019), a problem that has many facets.

An important research area related to the issue of conservation is the condition monitoring of HBs by means of sensor networks (Ceriotti et al. 2009; Shen et al. 2022). On the one hand, there is a growing interest in structural health monitoring as a knowledge-based assessment tool to quantify and reduce uncertainties regarding their structural performance (Lorenzoni et al. 2016). Sensors placed at critical points in the structure of HBs enable the detection of anomalies due to various causes such as earthquakes, deterioration of stone or cement, structural damages, pollution, and anthropic activities. On the other hand, preserving works of art usually displayed or maintained in HBs requires appropriate technological solutions (D’Amato et al. 2012; Landi et al. 2022) to support the roles responsible for their state of preservation. This paper aims to contribute to HBs management by proposing a situational awareness (SA) model that integrates human and automatic decision processes. This model is used as a reference for the design of Human-in-the-Loop Cyber-Physical Systems and offers new opportunities to improve monitoring, control, maintenance, and other management processes related to HBs. The case study discussed in the paper describes the cyber-physical system implemented during the realization of the RE-SEMIRTO project at the Royal Palace of Carditello, a major cultural attraction located in Campania, Italy.

The paper is organized as follows. The next section reviews the theoretical background, first discussing the foundations of Intelligent Building (IB) and Heritage Building Maintenance Management (HBMM) and then evaluating the state of the art of edge computing and cyber-physical systems. Section 3 first presents the proposed model of SA that will guide the design and implementation of a management system for HBs; then, a discussion about the goal-directed task analysis (GDTA) used to identify SA requirements and develop solutions to meet user needs follows. Section 4 describes the edge-cloud architecture of the Cyber-Physical System (CPS) for the monitoring and control of HBs. The case study in Sect. 5 presents the system implemented at the Royal Palace of Carditello (San Tammaro, Caserta–Italy), showing the technological solutions used to improve the knowledge base necessary for the conservation of the HBs and the exposed artworks. Section 6 discusses the benefits of our approach to the management of HBs highlighting the contribution concerning the state of the art. The conclusions resume the findings and discuss some limitations that require additional research.

2 Literature review

The literature related to the monitoring of historic buildings is mainly rooted in the research streamlining of IB and HBMM. Many concepts, models, and technologies introduced for IB apply to the maintenance management of HBs buildings (Purwantiasning and Bahri 2017). However, the characteristics of the monumental heritage, and the new technologies, outline a scenario in which additional lines of research and next-generation systems make it possible to improve the efficiency and effectiveness of processes aimed at the monitoring and control of HBs. The theoretical background and the role of edge computing and cyber-physical technologies are the foundation for developing our approach.

2.1 Theoretical background

In the past few decades, the research area of IB has developed from a debate concerning its foundations. There is a multiplicity of definitions of an IB, and each reflects particular aspects that research communities, institutions, or professionals wish to explore (Clements-Croome 2004; Wong et al. 2005). As science and technology advance, breakthroughs aimed at improving IB performance are possible, particularly by leveraging state of art technologies applied to smart building management. Recent works focus on the following aspects: improvement of automation control systems (Yu 2021); artificial intelligence applied to the preventive conservation of HBs (Prieto et al. 2019) and to intelligent decision-making systems in IB (Zhao 2022).

Even if many research results and best practices adopted to manage IB can be applied to different types of buildings, the management of HBs requires special attention due to the long-term goals of heritage conservation and promotion. Indeed, specific practices in heritage conservation and promotion strategies, such as the adaptive reuse of HBs and the conservation of works of art are necessary. In addition, the role of tourism is an important part of HBs management and can influence the decision-making process regarding their conservation and valorization (Biderman and Bosak 1997; Jamieson 2019). Finally, there is a need for a rethinking of consolidated practices and concepts to consider the new scenario of Sustainable Development Goals (Lerario 2022).

The review presented by Adegoriola et al. (2021). highlights the role of HBMM in ensuring sustainable maintenance of HBs aiming at reducing cases of dilapidation, demolition, and loss of social and economic benefits. The authors propose a classification scheme where five broad categories concerning research in HBMM are identified: (1) Decision-making frameworks in HBMM, (2) Integration of digital technologies and HBMM, (3) Condition assessment and maintenance practice, (4) Sustainability and HBMM, and (5) Emerging research trends in HBMM.

In the literature, it is possible to find only a few works that refer to SA theory applied in the IB or HB research. In the paper by Giallonardo et al. (2019), an ontology-based approach to provide runtime models of physical entities in context-aware reactive systems is proposed and validated using a case study of IB for cultural heritage preservation. Context awareness also helps to improve recommendation systems. The work presented in Casillo et al. (2023) describes a general overview of context analysis techniques in recommender systems and discusses some challenging applications aiming at the improvement of the interaction between human beings and artistic-cultural heritage.

An application of GDTA (Endsley and Jones 2004) to obtain the key goals and SA requirements of tasks is discussed in (Irizarry and Gheisari 2014) to solve safety and facility management problems in the construction sector. However, what is less clear in the literature is how SA models and related GDTA provide orientation to developers for the implementation of CPSs to improve the management of HBs. To fill this gap, we propose an SA model that combines decisional processes made by humans and automated systems and show how this model can be used as a guideline for the realization of CPSs to support the management of HBs.

2.2 Technologies for heritage building management

The last decade has seen the rise of smart objects as key components in the implementation of sensor networks. A smart object (SO) is essentially a small micro-electronic device that consists of a communication device, typically a low-power radio, a small microprocessor, and a sensor or actuator (Vasseur and Dunkels 2010). In Kortuem et al. (2010) a SO is defined as an autonomous physical/digital entity augmented with sensing, processing, and network capabilities; furthermore, it carries application logic. A smart object can sense, actuate, log, and interpret what is occurring within itself or in the world in which it is immersed, and can interact with people and other smart objects (Liu and Baiocchi 2016). Figure 1 shows the evolution path from sensors to smart objects.

Fig. 1
figure 1

Adding memory, processing, and communication capabilities to things: from sensors to smart objects

Full size image

The presence of a processor as a component of a SO affects the distributed computing model. In the model known as edge computing, the “edge” is defined as any computing and network resources along the path between data sources and cloud data centers (Shi et al. 2016). In this model, data processing occurs as close as possible to where the data are generated, improving response times and saving on bandwidth.

The survey on edge computing for the Internet of Things (Yu et al. 2018) compares the characteristics of IoT, edge computing, and cloud computing proposing a three-layer architecture (IoT devices, intelligent gateway edge computing, and cloud servers) of edge computing-based IoT. Focusing on the transmission, storage, and computation characteristics, the authors highlight how the integration of IoT, and edge computing improves network performance.

An important aspect concerning the acquisition of data of different types and formats in a wireless sensor network (WSN) is data ingestion. Utilizing the proper tools, it is possible to capture, import, clean, filter, manipulate, merge, store, and export data of various formats from many sources having different capabilities of producing data and different transmission rates. A comparative study of data ingestion tools is presented in Sharma et al. (2021).

Research on systems for HBs management takes advantage of the opportunities made available by the advancement of technology. One of the first works that used a WSN to monitor the internal and the external state of an HB is presented in Ceriotti et al. (2009). The authors describe how custom hardware and sensors efficiently capture a high volume of vibration data, using them to detect the potential deformation of a medieval tower. Open-source hardware can also be used for monitoring purposes. The study presented by Mesas-Carrascosa et al. (2016) shows how a network of 15 microclimatic stations was developed using open-source hardware at the Mosque-Cathedral of Córdoba.

Recent research focuses on designing and implementing cyber-physical systems specialized in monitoring and controlling buildings. Essentially, a cyber-physical system (CPS) consists of two parts, a “physical” one that is a selected aspect of the real world and a “cyber” one that is the corresponding virtual representation (Wu et al. 2011). These two parts communicate and influence each other to reach a goal such as the execution of an automated production process or monitoring a building. Typically, communication from the physical system to the virtual system reports real-world status messages (captured by sensors) while reverse communication carries command messages that are executed by actuators. Examples of applications of this technology for building efficiency include Smart Buildings based on IoT and CPS technologies (Bakakeu et al. 2017), and a kind of Cyber-Physical-Social Eco-Society oriented to the current generation of museum visitors, who are influenced by their exposure to modern technology (Nisiotis et al. 2020).

Fig. 2
figure 2

Situational awareness decision process for HBs management

Full size image

Smart sensor systems have employed classic and modern Machine Learning (ML) algorithms to develop sophisticated models for sensing applications. These models can draw from multiple sensing modalities to gain a comprehensive understanding of the system being monitored. Ha et al. (2020) describes how the sensor technologies are coupled with ML “smart” models and how these systems achieve practical benefits. Afshan and Rout (2021) highlights how ML has the power to handle different challenges associated with IoT data and can be deployed effectively as they require minimal human intervention.

3 The role of situational awareness

Since its introduction, SA has been applied to various application domains. As outlined by Endsley (Endsley 1995; Endsley and Jones 2004), SA is defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”. SA implies that when we need to accomplish a task or reach a goal, we must be aware of our surroundings. The model of SA and dynamic decision-making consists of three distinct levels:

Level 1:

Perception of the elements in the environment.

Level 2:

Comprehension of the current situation.

Level 3:

Projection of future status.

The SA theory provides an effective approach to monitoring, controlling, and HB management. Inspired by the Endsley model (Endsley 2017), the model shown in Fig. 2 is our proposal for SA and the related decision-making processes oriented toward the management of HBs. The model shows how this application domain can take advantage of decisional processes that involve both humans and automated systems.

Table 1 Capabilities and strengths of human beings and automatic systems
Full size table

First, it separates the behavior of automatic systems and humans involved in a decision problem to achieve a goal or perform a job. Although the perception-comprehension-projection scheme is the same in both cases, it is necessary to highlight the peculiarities of the two decision-making processes for each of them. Second, the influence of several stakeholders concerning decisional processes must be taken into account. Third, the role of technology in building automatic systems for monitoring and control purposes must be clearly stated. For what concerns the goals and objectives related to HB management, the main stakeholders are people who have governance roles and people who work in the socio-technical system charged with day-by-day HB management. These people have expectations that must be considered and strongly influence the decisional process oriented to the achievement of organizational purposes. Other stakeholders can influence the decisional processes as well. For example, tourists have various expectations when visiting an HB such as convenient visiting hours, comfortable air conditioning, valorization of the building and the surrounding environment, and easy access to the works of art.

The human decisional process can receive an important contribution from an automatic support system. For what concerns data collection and elaboration, from one side the model shows how heterogeneous data acquired from a sensor network is transformed and organized through a data ingestion phase. The availability of tools to visualize data provides the decision-maker with information on the current situation to make an informed decision downstream of the “human perception, human understanding, human projection” path. On the other side, data output from the data ingestion phase can follow the path “automatic perception, automatic comprehension” feeding a decision support system (DSS) that can trigger a new SA cycle aiming at improving human decision-making. The DSS also receives information stored in the data lake of cloud servers, i.e., building architecture drawings, photos, videos, and other kinds of structured, semi-structured, and unstructured data. In our model, automatic perception has two specific purposes. The first is to execute some transformation and merging operations not strictly necessary for data visualization thus making simpler the data ingestion phase. The second is to organize data in structures that facilitate the subsequent phase of automatic comprehension.

Indeed, there are decisions for HB management that can be taken by an automatic system; typically, they are simple routinary decisions that can be taken autonomously by the system without the intervention of humans. Therefore, the model is enriched by the automatic decision process that follows the path “automatic perception, automatic comprehension, automatic projection”. The choice of smart objects as state-of-the-art technology for the realization of automatic decision-making systems is fundamental to guiding the design of support CPSs which will be discussed in the following section. Smart objects are chosen to implement the necessary sensor network from which environmental variables are collected; they are the fundamental components that allow the feeding of the data ingestion phase. Furthermore, they contribute to the realization of the perception-comprehension-projection modules and influence the quality of the automatic decisional process.

Fig. 3
figure 3

Categorization of goals, decisions, and requirements in GDTA

Full size image

The proposed model suggests that an integrated decision-making system comprises the decision-making processes of human beings together with decision-making processes taken from automatic systems. Table 1 shows the capabilities and strengths of human and automatic systems. Essentially, humans can solve problems and make decisions in complex and changing scenarios where uncertainty makes it difficult to pursue decisions based on incomplete information. On the other hand, there are simpler decisions that can automatically be made based on the analysis of the situation starting from measurements of environmental variables captured by sensors. In these circumstances, the strengths of automatic systems refer to the continuous assessment of the situation and the ability to respond in real-time in an emergency or dangerous situation. It is worthwhile to highlight that there are strengths of automatic decision systems such as “real-time reaction to known environmental stimuli” or “monitoring of variables undetectable by humans” difficult or unreachable by humans. For example, a crack not visible to the human eye in a load-bearing wall of an HB or the physical-chemical characteristics of a painting is perceivable and measurable by specialized sensors and not by humans.

By following the methodology of GDTA (Endsley and Jones 20042004), a hierarchy of goals, decisions, and related information requirements can be developed starting from goals that a human operator must achieve.

Because our study considers the combination of decision-making processes taken by both humans and automated systems, the GDTA hierarchy has been divided into two basic categories concerning the goals of humans and those of automated systems. This approach is driven by the consideration that combining the capabilities and strengths of humans and automated systems recalled in Table 1 can lead to more complete and better-performing DSSs. Furthermore, explicit categorization facilitates the execution of GDTA to identify SA requirements (Nasser-dine et al. 2021). Figure 3 shows the generic hierarchy that is taken as a reference for the development of the case study.

4 The role of cyber-physical systems

The role that CPSs can play in the realization of HB management systems is based on the features that emerging technologies make available to create advanced monitoring, control, information retrieval, and digital services devoted to various kinds of stakeholders. The SOs, their memorization, processing, and interconnection capabilities, coupled with the use of distributed algorithms are the building blocks to build a CPS that supports the SA decision process. They are key components for the implementation of a smart sensor network and the implementation of the automatic perception, comprehension, and projection subsystems described in Fig. 2.

4.1 Smart objects

From the analysis of the literature about SOs emerges that object intelligence comprises several features: self-awareness, interaction with the surrounding environment, data processing, machine learning, and connectivity. These features can be further specified as follows.

  1. 1.

    Self-awareness. The capability of an SO to recognize itself as an individual entity and its current state. It covers the aspects of sensor/actuator identification to establish an access path to the object, status diagnosis, localization, etc.

  2. 2.

    Interaction with the surrounding environment. This functionality is manifested through the relationships that the SO establishes with elements of the surrounding environment using sensors or actuators.

  3. 3.

    Data processing. Processing activities refer to basic processing, i.e., processing the collected primitive data by filtering, algebraic aggregation, conversion, and encryption, and advanced processing by algorithms for statistical analysis, inference, and prediction.

  4. 4.

    Machine Learning. The presence of embedded microprocessors that have memories and sufficient computing capabilities contributes to realize multi-agent applications for ML.

  5. 5.

    Connectivity. The different communication possibilities are manifested through the various available technologies (Wi-Fi, Ble, LoRa, 4 G/5 G). Proper communication management is ensured in a multi-threaded scenario, supported by the GPIO subsystem connected to the microcomputer.

SOs today can be equipped with a dual-core processor that can perform code for:

  1. 1.

    Dialogue with routers via a WiFi connection and with smart IoT in the WSN via LoRa connections.

  2. 2.

    Dialogue via a BLE connection, for example in the case of oscillometric sensors to create a network of NFC microelectromechanical systems (MEMS) or nanoelectromechanical systems (NEMS).

  3. 3.

    Computing for several functionalities such as first processing of data captured from sensors connected directly via some control IO ports (analog, digital, serial, parallel), direct alarm functionality (actuator), and advanced computations executed by ML algorithms.

A final consideration about SOs refers to the fact that they can be both physical and digital objects. This characteristic makes them ideal for implementing cyber-physical systems in which the cyber part constitutes the virtual representation of a physical system that is to be controlled to achieve one or more purposes.

4.2 The edge-cloud approach to the implementation of cyber-physical systems

In the edge-cloud approach, data computing is done closer to the device or equipment that collects it, rather than sending all the raw data to a centralized data center or cloud. This can significantly reduce latency and improve overall performance. However, the integrated edge-cloud approach can coexist with previous architectural solutions, e.g., cloud computing, to cover the different application scenarios potentially realizable. In this section, we explore the edge-cloud approach to the implementation of a cyber-physical system that can be used to sustain the decision processes of Fig. 2.

Fig. 4
figure 4

IoT-edge-cloud architecture

Full size image

Edge where? Compute nodes and gateway nodes are more complex devices than IoT sensors because of the large number of functionalities they must provide. Compute and gateway nodes are necessary when the network is primarily made of conventional IOT or Smart IoT nodes with sensing and simple control logic capabilities. This is a typical scenario for smart cities and smart homes where a high number of low-cost nodes suggest the prevalent development of WiFi and BLE telecommunication systems. This scenario is shown in Fig 4 where:

  1. a)

    The architectural segment “Smart IoT-gateway/router-Cloud” is required when smart IoT components are used as end devices.

  2. b)

    The architectural segment “IoT-Edge server-Cloud” can be chosen when conventional IoT is used.

When a timely response to changes detected by the sensor is required, IoT smart sensors show limitations due to the absence of an immediate and direct control system. In this case, the SO appears more appropriate and can be used as shown in the architectural segment “Smart object-Edge server-Cloud” of Fig. 4. Examples of applications are:

- Outdoor areas (e.g., climate monitoring, valuable forest areas, marine protected tidal areas, highways, bridges, etc. ).

-Indoor areas where the active or passive anthropogenic risk is considerable (e.g., museums, places of worship, hospitals, convention halls).

Cloud where? Cloud storage is the place where data, information, and knowledge generated by both automated devices and humans are maintained for knowledge management and decision-making purposes. There is no single vision of the cloud in edge-cloud systems as its location may depend on the application scenario. For example, in the case study discussed in Section 5, the edge server should be understood as the bridge that connects the network of SOs and smart IoTs to the cloud with both data ingestion and temporary datastore functions.

The enhancement of the smart function in distributed IoT systems allows the technology associated with the cloud process to be separated into two parts that we could call edge-server and remote cloud. This separation is critical because the edge-server environment allows for completion, with relatively fewer resources than the remote cloud, of:

  • the ML process initiated in the SO nodes

  • the entire ML process in the case of intelligent IoT nodes.

The duality of edge-server and remote-cloud best characterizes the latter in that it leaves the former with the responsibilities of data ingestion and network manager on the edge side of the architecture. The remote cloud, freed from these responsibilities, can exercise the primary role of long-term storage implementing digital services for:

  • data-ingestion for DSS,

  • datalake for other services.

Indeed, given its nature, it can encompass other databases from specialized stations on audio-video, GIS, and other sources.

4.3 Data ingestion

The variety of sensors that can potentially be used in an edge-cloud architecture and their different characteristics require, from a data acquisition perspective, a data ingestion process to acquire data from various sources and load it into a central data repository. This data can then be used for transmission, analysis, ML, and other purposes. In this study, the data ingestion process is critical to arrive at both automated and human-made decisions based on:

  1. 1.

    real-time analysis (done on the edge branch of the architecture) to recognize anomalies and generate notifications, warnings, and alarms;

  2. 2.

    the feeding of a DSS system (done on the cloud side of the architecture) to make available to humans the knowledge they need to make informed decisions.

Automatic data ingestion is developed into two phases. During the first phase, the data detected from various types of sensors connected to a SO are collected and transformed. The SO performs basic functions (capture, import, clean, filter, manipulate, merge, and store data) to increase the quality of collected data. Then, the SO provides functions to export data in various formats for transmission to other SOs or the edge server for further processing. During the second phase, the data ingestion is executed at the edge server or the cloud side of the IoT-Edge-Cloud architecture.

The edge server manages:

  • The data received from an SO in various formats, for example, JSON and CSV, normalizing them.

  • The fusion of data received by several SOs.

  • The generation and visualization of sensor timelines (CFR. Fig. 9).

  • The analysis of incoming anomalies detected and received by the connected SOs, taking care of the delivery of messages sent by SOs.

The cloud manages:

  • The acquisition of big quantities of data either from smart IoT or from edge servers.

  • The feeding of the knowledge base that supports the DSS. For this process, data come in many different shapes and sizes, and their structure depends entirely on the sources from which they were generated and the downstream use cases.

Captured data can be stored in a variety of formats: structured, semi-structured, and unstructured. For example, numeric data for monitoring are presented in the Cloud in Row Key form converted to JSON mode, while diagnostic data are typically unstructured. For what concerns the transmission modality we have two possible choices, streaming and batch.

  1. 1.

    Streaming. An asynchronous message stream containing data from all sensors connected to a SO is triggered periodically (e.g., hourly). As an example of streaming data, we recall those used for telemetry, collected by an SO or IoT-Edge systems, and stored in CSV-type files, JSON, etc. Streaming data can be used to trigger alarms on events, perform timeline studies, or be used as input for specific ML tasks.

  2. 2.

    Batch. Big Data are stored in a set of files that are transferred in bulk and consist of large data sets whose transfer requires high bandwidth. The source files could also be on local servers or other cloud platforms.

Data ingestion initiated by humans complements the automatic data ingestion process. It is necessary because many instruments for observing and measuring phenomena require human intervention to be used. Typically, these instruments produce files of relatively large graphical models, photos, sounds, and movies. The transmission mode of these files to the cloud storage is batch.

Fig. 5
figure 5

The Royal Palace of Carditello

Full size image

5 Case study

The Royal Palace of Carditello, shown in Fig. 5, was designed by the architect Francesco Collecini in 1787 at the behest of Carlo di Borbone and developed under the reign of Ferdinando IV.

After years of neglect and decay, several actions of recovery, protection, preservation, and enhancement have affected this monument. The case study described in this section shows the cyber-physical system implemented during the realization of the RE-SEMIRTO project for the enjoyment, preservation, and enhancement of the HB with the intention of introducing some degree of automatic intelligence to support the management processes.

Fig. 6
figure 6

The GDTA hierarchy for the safeguarding and conservation of the Royal Palace of Carditello

Full size image

The first step in designing a CPS implementing the SA model of Fig. 2, is the application of the GDTA to determine the objectives of the operators and the objectives of the automatic systems from which to derive decisions and requirements. For brevity, we report in Fig. 6 only some system goals by dividing them into operator goals (OG) and automated goals (AG) and developing the corresponding sub-goals, decisions, and requirements. For example, goal AG2 (indoor monitoring) is developed into 5 subgoals of which AG2.4 and AG2.5 are about the smooth operation of the processing and communication system. The goal AG2.2 (vibration/oscillation monitoring of artworks) corresponds to decision AD2.2.1 (generate an alarm in the case of sudden movement of the sensor connected to the work of art). Finally, the automatic system SA requirements detail the software functionalities necessary to implement decisions and reach goals.

The second step of the CPS design process is to map the SA model by distributing the automatic intelligence to the nodes in the edge-cloud architecture. To implement the model in Fig. 2, this distribution may depend on the problem to be solved. Among the scenarios we encountered during the development of the case study, two are worth highlighting:

1) Problems that require real-time response where fully automated solutions prevail.

2) Problems that require the skills of humans in which the automated system acts as a support to human decision-makers. In this case, automated behavior and human behavior are combined to improve the decision-making process; the human work is typically long-term analytics done in a much wider time interval than in the previous scenario.

Fig. 7
figure 7

Mapping the SA model in the edge-cloud architecture

Full size image

An architectural segment of Fig. 4, describing how the stages of perception-comprehension-projection can be mapped according to the two previous scenarios, is shown in Fig. 7. When real-time problems can be solved with the limited computational resources available in an IoT smart sensor (CFR. Figure 2.c), the entire decision-making process takes place in the IoT smart node. An example is the evaluation of parameters that must remain within predefined thresholds. In this case, the sensor perceives the environmental data and the computational capability of the IoT enables the implementation logic for the understanding phase (e.g. when the data goes beyond the predetermined limits) and projection phase (sending alert messages or triggering feedback actions).

The second scenario is more complex. Examples are the analysis of structures of a historic building or the analysis of a painting. In such cases, the decision-making process combines, as shown in Fig. 2, the following steps: (1) automatic perception, (2) automatic comprehension, (3) human perception, (4) human comprehension, and (5) human projection. For the analysis of paintings described in Sect.  5.2, the automatic process starts sensing environmental variables using various kinds of tools for the capture of photos, videos, and numeric data and continues sending the perceived data to the cloud server. The automatic comprehension is based on data lake technology and follows the server-side ingestion of data from any system at any speed organizing them in a centralized repository of large amounts of structured, semistructured, and unstructured data in native format. Then, software tools and digital services interpret the data (automatic comprehension) and set the scenario for the initiation of the perception-comprehension-projection cycle carried out by the human being. The repository is enriched with data collected from sensors but also with information from other sources that allow the decision-maker to have an adequate knowledge base for understanding the phenomenon he or she is analyzing. The projection phase concerns the identification of a future scenario that may emerge from the data analysis, which, in the case of paintings, could be the need to obtain a restored copy of a painting.

The decision on which nodes to deploy the intelligence (smart IoT, gateway/router, edge server, or cloud server) can be more effective if multi-core processors are used where convenient. For the SOs in our case study, we used dual-core processors of which one dedicated to I/O that can be managed by multi-threaded processes, and another to the execution of software agents running concurrently to implement applicative functions. This approach to the division of computational tasks improves the performances of SOs and makes them critical for the implementation of IoT-edge-cloud architectures.

5.1 Monitoring and control subsystems

Outdoor monitoring is used to assess the effects of air pollution and global climate change (from traffic and urban and industrial settlement) or vibrations/oscillations due to traffic, human activities, or natural phenomena on Cultural Heritage. In Fig. 8 the SO for monitoring is shown. It can be used for outdoor as well as indoor contexts and is equipped with sensors mounted on the board. Moreover, it is connected to many external MEMS-BLE sensors for oscillometric monitoring. This SO consists of 5 sensors and 3 actuators (fan, micro camera, Wi-Fi). Visible is the dual-core processor, 3 families of IoT-class environmental sensors, an IoT sensor control system, the micro camera, and BLE and WiFi antennas for connecting to MEMS and networking. As an example of an actuator, we highlight the fan start/stop mechanism using an NPN-type transistor connected to the GPIO.

Fig. 8
figure 8

SO for indoor/outdoor monitoring

Full size image

The transmission modality managed by the SO is streaming and has the following characteristics:

  • MEMS to SO is on BLE always active transmission channel.

  • SO to edge server is start-stop type driven by ESP32 over WiFi channel.

  • by start-stop type micro camera (triggered periodically or by motion sensors) driven by ESP32 over WiFi channel to the edge server.

The trend of data detected by MEMS sensors is shown in Fig. 9. The dashed frame mark anomalies due to abrupt accelerations of the MEMS sensor in the x, y, and z-axes due to minor displacements of the sensor in three-dimensional space.

Fig. 9
figure 9

A trend of data detected by MEMS sensors with alarm points marked on the curves

Full size image

Indoor monitoring and control of environmental and thermohygrometric parameters is necessary for the evaluation of the conditions of conservation and enjoyment of works in museums or other contexts of historical and archaeological interest.

5.2 Periodic activities

Periodic activities, carried out by humans with the help of supporting tools, are aimed at diagnosing and assessing the condition of the building and the works of art in it. Photographs, data streaming, and video footage are acquired and transmitted to the cloud repository. The RE-SEMIRTO project includes three kinds of periodic activities:

  1. a)

    Materials diagnostics for understanding the effects of climatic and atmospheric agents.

  2. b)

    Analysis of the state of preservation and deterioration of the historic building.

  3. c)

    Analysis of the state of artworks.

In this way, it is possible to compare the condition of the building and artworks and assess their health or deterioration process with the passage of time.

Fig. 10
figure 10

Analyses of a painting

Full size image

Figure 10 shows activity (c) for indoor analysis of paintings that involve roles with different professional profiles in a typically multidisciplinary context. The paintings displayed in the “camera oscura” room were analyzed by IR Reflectography, X-Ray Fluorescence, and Reflectance Spectroscopy for the identification and mapping of the different constituting materials (original and restoration pigments) on the pictorial surface, to detect the execution technique and to evaluate persistent degradations or effects due to previous interventions (Spagnuolo et al. 2022).

5.3 Digital services

An important part of the RE-SEMIRTO concerns the creation of Apps aimed at researchers, technicians, guides, and tourists to take advantage of digital services of

  1. a)

    Dissemination. Implementation of a web-based presentation platform of the results of measurements on cultural heritage, differentiated for the monitoring part, the control part, and the archaeometric and diagnostic surveys. Figure 11 shows an excerpt of detected data available as open data on the platform.

  2. b)

    Conflict of interest. Definition of Scientific, Technical, and Cultural Pathways of the Real Site of Carditello with the support of appropriate Apps driven by BLE beacons. Figure 12.a) shows the App being used by researchers and technical staff to monitor the values returned by environmental sensors.

  3. c)

    Participation. Virtual tours to enhance the immersive experience. Virtual tours make visitors aware of ongoing diagnostic investigations and allow visitors to “visit” sites during excavation and artifact analysis/restoration work.

Fig. 11
figure 11

Data dissemination (https://www.resemirto.it/)

Full size image
Fig. 12
figure 12

a mobile application for microclimatic monitoring; b virtual tour

Full size image

6 Discussion

One of the first papers that applied SA to the construction industry was published by Adamu et al. (2015). The authors present a framework, based on shared SA to enrich co-creation processes and virtual collaboration using cloud-based servers. Their focus is on the design of new constructions rather than safeguarding and conserving HBs, which is the focus of our work. In addition, we use an edge-cloud architecture instead of a traditional cloud architecture allowing us to solve real-time monitoring and control problems. The application of the SA to HBs to evaluate their health through real-time monitoring of key parameters is recent (Xiao 2022). However, this solution does not deepen the decisional process that characterizes SA models as we do in Sect. 3 and does not address the issue of control. Indeed, most of the literature concerning the preservation and conservation of HBs focuses on monitoring several aspects such as:

  • the assessment of their structural integrity using vibration and deformation sensors (Ceriotti et al. 2009; Shen et al. 2022),

  • the interior environments and its artworks using microclimate stations (Mesas-Carrascosa et al. 2016),

This makes sense because monitoring systems are critical to support humans in the decisions they must make to safeguard and conserve cultural heritage and to activate consequential actions.

However, the control component adds new possibilities for managing the HB (Purwantiasning and Bahri 2017).

SA provides a novel way to address the issue of HBs monitoring and control and to frame their decision-making processes in a sound theoretical framework. Our model combines decision-making processes involving humans and automated systems, exploiting their strengths and managing monitoring and control processes. The approach of this study is that of action research (Willis and Edwards 2014), which seeks to solve the current practical problems and simultaneously expand scientific knowledge. Therefore, we chose to identify an SA model that is representative of a family of applications where the asset to be protected is subject to conservation programs. As a result, the application of our model can be replicated in scenarios similar to or even more complex than the case study described in Section 5. For example, the model has been the reference for the design and implementation of the Human-in-the-Loop CPS implemented in the TISMA project (www.tisma.eu) that supports monitoring and control processes of the complex ecosystem of Sant-Agata de- Goti, a historic village in southern Italy (Nota and Petraglia 2023).

From an implementation perspective, the monitoring and control functions are made possible by the implementation of a CPS in which key components are the SOs on which a multiagent system responsible for the various application functions has been implemented. This approach is, in some respects, like that proposed by Fortino et al. (2017) in which agent-based cooperating smart objects are used to develop smart and dynamic IoT systems. For the implementation of automatic decision processes related to HBs management, a CPS has been realized by integrating both the traditional Cloud architecture and the IOT-Edge-Cloud architecture. This solution has several advantages. First, automated decision processes that require real-time or near real-time responses can be implemented by edge devices. Typical functionalities achieved are monitoring and control of the environmental elements with which sensors and actuators interact. Second, the cloud side of the architecture can be used when a greater amount of computational and storage resources is required by applications and there are no urgent real or near real-time constraints. This is the case of DSSs whose knowledge base can be fed either by activities initiated by humans or by automatic mechanisms of data collection. Third, new digital services make it possible to reach dissemination, disclosure, and participation goals.

7 Conclusions

The preservation and conservation of cultural heritage involve the implementation of decision-making processes, taken both by humans and by automated systems. Starting from the proposed SA model for HB conservation, we illustrate how a CPS can be designed and implemented to make automatic decisions and to support different human roles, using the SA model and the GTDA method as design guidelines. The paper highlights the role that SA and CPS can play in the improvement of HB preservation and conservation processes. According to the action research approach, the case has involved collaboration between the researchers from academia and the managers working in the Royal Palace of Carditello so that the implemented CPS and the supported decisional processes have been validated by the managers, especially for the aspects concerning: (a) the benefits of combining the capabilities of humans and the automated components of the CPS that have been summarized in Table 1; (b) the effectiveness of the three-tier edge-cloud architecture to implement the SA model and real-time capabilities for the protection and preservation of historic buildings. Although the case study carried out from the research results presented in the paper met stakeholders’ expectations, there are several limitations that require additional research efforts to address many unresolved aspects. First, a more in-depth study of how risk management (e.g., fire, hydrogeological, pollution, anthropogenic) can be improved through CPS systems is necessary. From a methodological perspective, this requires revisiting the current risk management approaches and the study of a risk ontology in HB management. Another promising research area concerns the development of Cyber-Physical-Social Systems. This is an aspect that is included in our SA model because socio-technical systems, governance, and other stakeholders exercise an important influence on the decisional processes supported by CPSs. However, further study is required to deep how this class of CPSs implements the human-in-the-loop paradigm (Nunes et al. 2015) and how new forms of relationships between humans and ML algorithms, generically called human-in-the-loop ML (Mosqueira-Rey et al. 2022), improve the collaboration between AI models and humans. This is considered increasingly important in the future research due to the knowledge learned by machine learning cannot win human domain knowledge in the near future (Wu et al. 2022). Finally, human-computer interaction is an area of research that is usually considered to improve communication between people and machines so that the performance of related systems and processes can, in turn, be improved. In the scenario of HBs management, the design of human-computer interfaces will draw on studies from other research sectors that are based on context awareness (Bisio et al. 2018), usability (Di Gregorio et al. 2020), and visual pattern design (Fenza et al. 2021).