Integrating expert feedback on the spot in a time-efficient explorative CT scanning workflow for cultural heritage objects

Computed Tomography (CT) has proven itself as a powerful technique for analysing the internal structure of cultural heritage objects. The process followed by conservators and technical art historians for investigating an object is explorative: each time a new question is asked based on the outcome of the previous investigation. This workflow however conflicts with the static nature of CT imaging, where the planning, execution and image analysis for a single CT scan can take days, or even weeks. A new question often requires conducting a new experiment, repeating the process of planning, execution and image analysis. This means that the time that is needed to complete the investigation is often longer than originally anticipated. In addition, it brings up more practical challenges such as the transportation of the object, facility availability and dependence on the imaging operator, as well as the cost of running additional experiments. A much needed interactive imaging process, where the user can adapt the CT scanning process based on the insights discovered on the spot, is hard to accomplish. Therefore, in this paper we show how a time-efficient explorative workflow can be created for CT investigation of art objects, where the object can be inspected in 3D while still in the scanner, and based on the observations and the resulting new questions, the scanning procedure can be iteratively refined. We identify the technical requirements for a CT scanner that can address the diversity in cultural heritage objects (size, shape, material composition), and the need for adaptive steering of the scanning process required for an explorative workflow. Our approach has been developed through the interdisciplinary research projects The See-Through Museum and Impact4Art. We demonstrate the key concepts by showing results of art objects scanned at the FleX-ray Laboratory at CWI, Amsterdam. © 2021 The Author(s). Publi? par Elsevier Masson SAS. Cet article est publi? en Open Access sous a w a R c


Introduction
Imaging science and computational methods have increasingly been applied to cultural heritage objects over the past decades [1][2][3][4][5], including optical coherence tomography [6], non-destructive X-ray imaging modalities such as radiography [7], phase contrast [8] and macroscopic X-ray fluorescence [9] and investigations combining multiple techniques [10,11]. The focus in this article is on In CT-based research in the fields of technical art history and conservation, the research questions are linked to internal structures and features that are difficult to access such as toolmarks or fingerprints inside the object, or separation lines between different materials. The governing questions can be related to manufacturing process and structure, requiring information on the material composition of the object and the artist's techniques. For example, imaging the tree rings in wooden objects can assist in dating the object [30] and searching for lines in a terracotta statue can shed light on whether the clay was press molded or freely worked by hand [31]. Investigation by CT scanning can also be used to assess the current condition of the object for restoration and conservation purposes, revealing information on internal damage such as cracks, gaps and filler material [32,33]. This will reveal any structural modifications that have an impact on the original state of the object.
The exploration process for studying art objects differs from medical applications, for which CT was originally developed [19,34,35]. For medical CT scans the position and general shape of the interior is known beforehand; the densities of the subjects are similar [36,37]. The state-of-the-art medical scanners have been optimized pertaining to these characteristics. In the case of cultural heritage studies, it is unknown beforehand which features are present in the object and what scan specifications are needed to image them. Currently, the investigation process for each project involving CT scans, starts by selecting a scanner. Which scanner is appropriate for a given purpose is closely linked to the size of the object and the required resolution of the reconstructed 3D image, as these lead to specific hardware requirements [18,38,39]. Published research often concerns a single object or multiple with similar characteristics, as these can be scanned at the same facility. Depending on the requirements of the CT scan, one selects either a medical scanner (resolution in the range of 1 mm [24,28,30,40,41]), industrial scanner (resolution in the range of 100 m [13,30,[42][43][44]), a lab-based scanner (resolution below 100 m [22,[45][46][47][48]), a synchrotron facility (resolution in the range of 10 m or smaller [39,49]) or specialised small-scale CT scanners (sub-micron to nanoscale resolution) [50].
In case there are no suitable in-house CT facilities, the process of CT scanning a cultural heritage object requires extensive planning with respect to transport of the object and the setup of the experiment, often taking a few weeks or even months for synchrotron facilities [39] to arrange. Once at the facility, the focus is to acquire as much data as the allocated time allows. The reconstructions are examined by the art expert at a later time. CT scans offer a wealth of information about the interior, which often stimulates new investigations. This exploration aspect clashes with the static nature of CT imaging, in which we collect data, reconstruct images and analyse results in a sequential order. Explorative investigation implies the repetition of these steps, greatly increasing the time needed to complete the research and raising challenges such as logistics, additional experiments and increased cost.
The long-term goal of the CT scanning approach presented in this article is an efficient implementation of an exploratory CT imaging process within museum research facilities. The key objectives of this paper are i) detailing the requirements for a single scanner to perform explorative imaging; ii) describing the link between the scanner design, algorithm development and expert (technical and heritage) involvement in the process, to carry out the exploration efficiently; iii) demonstrating that the timespan of the investigation is essential: carrying out the investigation, with the object and experts jointly present, in a single scanning session enables to expose details that might otherwise go unseen.
To our knowledge, we describe for the first time how the combination of a flexible scanning setup and direct expert feedback enables immediate follow-up actions, influencing the CT scanning process by insights gathered on the spot. This leads to an interactive workflow for explorative CT scanning, which potentially reduces the work that can span over a couple of weeks at different scanners (or even facilities) down to a single day, increasing both time-and cost-efficiency and research throughput.

Research aim
We present the insights and experiences gained from two interdisciplinary research projects: the See-Through Museum [51] and Impact4Art [52]. It presents the collaborative work of the CT imaging scientists from the Centrum Wiskunde & Informatica (CWI), and conservators, curators and researchers from the Rijksmuseum, Amsterdam.
We aim to establish explorative CT imaging as a way to enable efficient collaborative CT-based research of art objects, driven by developments and expertise in both these fields. As we want the exposition to be accessible to a broad audience (possibly with less detailed knowledge in CT imaging), we include detailed explanations of technical concepts using diagrams and case studies.
We describe the key characteristics of cultural heritage objects that are significant for CT scanning (object shape, feature resolution, object materials) and outline which degrees of freedom are required to accommodate a broad range of investigations in a single scanner design (zooming, tiling and object orientation). For each individual degree of flexibility, the implementation is illustrated on a small example. As a proof-of-concept, the complete explorative workflow is then illustrated with a case study of scanning a wooden cornett, in which the experts' feedback was essential to steer the scanning process in response to their observations, answering a chain of questions within a one day timespan.

Material and methods
In this section, we first address the key characteristics of 3D cultural heritage objects that may be investigated by CT scanning. We then introduce the basics of CT imaging as well as the technical requirements to carry out the explorative workflow.

Key characteristics and technical requirements
The study of cultural heritage objects by CT scanning is substantially different from other CT-based investigations, such as in the fields of (bio)medical imaging and industrial quality control. In particular, the following key characteristics apply to the broad set of cultural heritage objects we encountered in the See-Through Museum and Impact4Art projects: The internal features vary in scale independently of the size of the object. Toolmarks, for example, range from coarse to finely detailed. In order to image them at the required resolution one needs to be able to zoom into the object.
2 Sizes and shapes of objects Cultural heritage objects have hugely varying sizes, from small ivory beads to large wooden cabinets. The shapes can also differ from one another: from a simple, spherical object to more complex shapes, such as the elongated shape of a statue with outstretched arms. To accommodate the range of sizes and shapes, at the desired resolution, tiled CT scans are required and have, for example, been applied to large musical instruments [45]. 3 Multi-material objects The objects in cultural heritage research can consist of multiple materials. The density of each material may vary. Cultural heritage objects often contain metal parts [13,53] or might be entirely made of metal [54], causing artefacts in the reconstructed image such as streaks or cupping effects that are due to the beam hardening, photon starvation or scattering of the X-rays [48,[55][56][57][58]. The barrier imposed by dense materials should therefore be avoided as much as possible, creating a need for flexibility in object orientation and positioning.

Principles of CT imaging
A CT setup consists of the following components: an X-ray source, a rotation stage and a detector. The diagram in Fig. 1 illustrates a point source that emits X-rays in a conical shape onto a flat panel detector. These X-rays travel through the object, which is mounted on the rotation stage. The absorption is material dependent [35,37,38]. The energy of the X-rays and the exposure time influence the quality of the data and determine the effective radiation dose the object receives. The potential effect of the radiation exposure depends on the settings of the scan and the characteristics of the object [59,60].
A single 2D detector image is called a radiograph or a projection. For a CT scan, the object is rotated and projections are taken from different angles. Typically, a full 360 • rotation is performed, with a constant rotation step size between the projection images in the order of 0.1 • . The inner features are thus captured from many viewpoints. Mounting the object in a fixed and stable manner is particularly important in obtaining accurate results and ensuring the safety of the object.

Flexible setup
The concepts discussed in the following subsection, namely zooming, tiling and object orientation, are imposed by the variety of characteristics of the objects. Although there exist systems that facilitate one or more of these concepts [61,62], it is rare to encounter a single scanner that has the degrees of freedom in each of the system components (source, detector, rotation stage) needed to facilitate them in a single apparatus. The FleX-ray Laboratory at the Centrum Wiskunde & Informatica (CWI) [63] is an example of such a system. The setup consists of a cone-beam X-ray source with spot size 17 m and energy range 20 kV-90 kV and a flatpanel detector of 1536 × 1944 pixels with pixel size 0.0748 mm 2 . The freedom of movement of the components is outlined in Fig. 2. The laboratory combines a highly flexible CT scanner with newly developed algorithms and software for on-the-fly image analysis, enabling the prototyping of a wide range of X-ray based scanning methodologies.

Tailored reconstruction algorithms
The data from a CT scan consists of a large number of projections. Next to the hardware requirements that these large datasets (in the order of several GB) pose, the computing infrastructure needs to be available on the spot to process the data for inspection. A fast network connection that directly interfaces the data collection by the CT scanner with the accompanying reconstruction software is therefore a key requirement of an explorative workflow. At the FleX-ray Laboratory, the acquired data can be processed using the provided software, Acquila (TESCAN-XRE), within minutes of a scan being completed. In order to take the flexible movement of components into account for a full object reconstruction, tailored algorithms have been developed that are immediately available while operating the CT scanner. For example, if there is no single orientation that gives accurate results, it is possible to scan in multiple orientations and combine the data to improve the image quality [64]. For the reconstructions in the next sections we use the FleXbox software that was developed at CWI for this purpose [65]. The resulting 3D reconstruction, which is saved as a stack of slices through the object, can be visualised for inspection in any direction, using for example the freely available Fiji/ImageJ software [66].

Degrees of freedom for adapting the scanning process
We now illustrate how the various degrees of freedom in the settings and positioning of source, stage, and detector are linked to the characteristics of cultural heritage objects outlined in Section 3.1.
Zooming can be achieved by increasing the magnification of the object, for example by moving it closer to the source. This is shown in Fig. 3.
In Fig. 4 an example of zooming is shown on a CT scan of a fragment of fabric [67], a mock object resembling a type of object we may encounter in cultural heritage studies.
Tiled scans can be performed by moving the source and detector around between CT scans, a full rotation of the object is recorded for each location. Examples of tiling modes are given in Fig. 5a,b.
In Fig. 6 we present an example of a vertically tiled CT scan [68,69] of an oak sculpture of a Holy woman with lantern, 35.8 cm high, c. 1500-1525 (Rijksmuseum, Amsterdam) [70]. It was scanned for the purpose of dating by tree ring measurement [23].
Object orientation can be adjusted either by manually changing the positioning of the object on the stage (possibly changing the mount of the object) or by adjusting the trajectory of the source and detector to achieve a similar change of viewpoint. In Fig. 7 the Fig. 3. Schematic of the zooming in on an object in a CT setup. a) Imaging a pyramid in a box: at this magnification, the entire pyramid is visible on the detector. b) Zooming in to achieve higher resolution, by moving the rotation stage with the object closer to the source. More details can now be seen, such as the tiny void within the pyramid. effect of object orientation on the radiograph of an object with a metal support is shown.
As an example object, we CT scanned an ivory bead with a metal bar through the middle to show the effect of object orientation on the reconstructed image, shown in Fig. 8.

Results: case study of a wooden cornett
The examples in the previous section were chosen to illustrate each of the three technical requirements. In this section, we present a proof-of-concept study that brings together the flexible scanner design and the software to rapidly inspect the object after the CT scan using both radiographs and 3D reconstructions. New questions, based on the observations of the art expert, guide the settings of the next CT scan to perform. The demonstration was carried out in the FleX-ray Laboratory at CWI.
The object under investigation is a cornett (see Fig. 10a), a curved woodwind musical instrument made in Italy between 1600-1650 (Rijksmuseum, Amsterdam) [71]. The cornett is made of wood, entirely lined with leather and is 56 cm long. A recent study [72], performed during the conservation treatment on a fracture in the upper part of the cornett, has shown that it consists of different wood species: the upper section containing the mouthpiece, identified as cherry wood, being severely damaged by insect infestation, while the lower part containing the finger holes, probably made of boxwood, was left almost untouched. The initial question concerned the manufacturing method and conservation status of the cornett (Question 1 in Fig. 9). As the leather lining does not allow a visual inspection of the wooden parts of the cornett, CT scanning was employed to visualise the interior.
To ensure that the object was securely mounted on the rotation stage, a mount was specifically designed for the cornett (see Fig. 10b). It was made from a rectangular piece of Ethafoam ® [73], attached with polyethylene hotmelt adhesive within a groove in an octagonal foam base to create a vertical stand. One end of the object rested in a cut-out in the foam base, and the object was secured upright with a cotton tape through a hole in the vertical stand. This made it possible to safely modify the object position and to flip it vertically on the stand. Ethafoam ® is often used for mounting as it is cheap and widely available, easy to mould into an appropriate shape, and lightly absorbs the travelling X-rays.

Tiled inspection with radiographs (Question 1: "How was this object made an what is its condition?")
The conservators questioned the current condition of the object and the manufacturing process, e.g. the damage by the insect infestation and how the curved object was hollowed out. An inspection with radiographs of the entire object was carried out in a dynamic process in which the scanner components were moved around at the request of the conservator and the resulting radiographs were shown directly on the screen next to the scanner. Due to the height constraint of the scanner, first the lower half of the object was inspected, and then the object was flipped within the mount to inspect the remaining half. The vertical and horizontal range of motion needed to provide a full view of the object while rotating it over 360 • is illustrated by the radiographs in Fig. 10c. Both object orientation and tiling were thus necessary to obtain a full view of the object. The values for the power and voltage were investigated and chosen to be 70 kV and 42 mA, respectively, and were kept the same throughout the scanning process. The radiographs confirmed that there were two different species of wood as the densities of the two sections were different. Based on the inspection, the conservators raised Question 2 in Fig. 9: How are the pieces connected precisely? This required a higher resolution CT scan of the region containing the joint (ROI1). The total time needed for the preparation and inspection was approximately 1.15 h (mounting: 15 min, parameter investigation: 15 min, inspection and discussion: 45 min).

CT scanning the joint (Question 2: "How are the pieces of wood connected precisely?")
Following the inspection by the conservators (Feedback 1), we performed a CT scan of the region containing the joint (ROI1) at image resolution 50 m. Each of the ROI scans consisted of 1200 projections. The data was used for a reconstruction on the spot and shown on the screen next to the scanner for analysis by the conservators. They asked if it was possible to perform a CT scan of the region in higher resolution. The total time for this investigation was 40 min (preparation: 5 min, scan: 15 min, reconstruction: 10 min, discussion: 10 min).
We zoomed in by moving the object closer to the source to focus on the region of interest with image resolution 25 m (ROI 1, see Fig. 10e,i). Analysis of the reconstruction of this CT scan confirmed that the pieces of wood were connected with a lap-joint with different layers of material (leather and wood) clearly identified. It also became apparent that both the lower and upper sections consist of two longitudinal pieces of wood, as a thin glue joint was visible on the horizontal slices (Fig. 10e). The conservators then wanted to investigate the difference in woodboring damage between the two species (Question 3). Total time needed was 35 min (preparation: 5 min, scan: 15 min, reconstruction: 10 min, discussion: 5 min).

4.3.
CT scanning the infested section (Question 3: "What are the differences in woodboring damage between the two species?") In the CT scan of the joint, it became apparent that both parts had been damaged by the insects. The conservators expected only the upper part to be damaged, as there were no holes on the outside on the lower part. The ROI1 reconstructions showed that the lower wood had been infested to some degree close to the joint but was otherwise nearly untouched. The conservators were interested in visualising the damage in both parts further from the joint (Feedback 2). Keeping the same object position and moving the source and detector up and down to image different sections and slightly changing the object orientation to make sure the region of interest stayed in the field of view, two more ROI CT scans at image resolution 25 m were performed to investigate these questions (ROI 3,4). ROI3 revealed the devastating effect of the insect infestation in the upper half of the cornett and the tunnelling structure. The conservators remarked how porous the wood had become, and how little wood was left to support the instrument (see Fig. 10g,k). This new information clearly documents the condition, illustrating how fragile the substrate has become and why the damage occurred precisely in this area. The ROI4 reconstruction revealed that the lower part was indeed almost untouched by the insects (see Fig. 10h,l). The thin glue joint of the two longitudinal pieces of wood was again visible on the horizontal slices (Fig. 10g,h). The time needed for each ROI CT scan was 40 min (preparation: 5 min, scan: 15 min, reconstruction: 10 min, discussion: 10 min).

CT scanning a section containing finger holes (Question 4:
"What is the shape of the finger holes?") The location for the ROI4 CT scan was chosen to include a finger hole, which could provide a possible insight into the manufacturing process: the shape of the finger and thumb holes (Feedback 4). Based on the analysis, the conservators identified the finger hole to be undercut to account for intonation corrections by the maker.
In addition, the CT scan showed in more detail the longitudinal joint. From this the conservators concluded that to construct   the hollow interior of the cornett, the makers used a single piece of wood, split and carved out each half to create the curved bore before putting the halves back together. The tree ring patterns were visible on the reconstruction, and indicated that the halves were indeed from the same piece of wood. A new question arose based on this CT scan, namely whether the size of the curvature and thus the diameter of the tree section could be estimated from these images. Fitting the curvature lines to a circle, we estimated that the section used to make the instrument can be placed at approximately 10−20 cm from the pith in the transversal section of the tree trunk. The curvature excludes the possibility that the cornett was made from a branch or a section including the pith.
4.5. CT scanning the restoration (Question 5: "What is the structure and condition of the restoration?") Following the inspection by radiographs of the full object, the conservators were also interested in the location where the cornett had been broken and later restored in 2018 [72]. In order to investigate and monitor the structure and long term effect of the restoration, ROI2 was scanned at 25 m image resolution. According to the conservation report, small custom-shaped sticks were inserted to replace missing wood. The inserts were covered with Japanese paper and retouched so that the intervention is practically invisible from the outside. The region of interest was larger than the previous ones, meaning a 2-tile CT scan was necessary to capture the details at the same resolution. The reconstruction showed the conservation method clearly (Fig. 10f,j), providing an excellent documentation for future reference and for the monitoring of long term effects. The total time needed for this investigation was 1 h (preparation: 10 min, scan: 30 min, reconstruction: 10 min, discussion: 10 min). 4.6. Further research (Question 6: "Is the manufacturing method original?") As it would not be practical to bring the object to the scanner again, we decided to use one more day to CT the entire object to facilitate further research and have a complete digital representation of the object for future reference. We performed a 30-tile CT scan at the resolution of 50 m to image the entire object. As during the inspection, it was necessary to first scan the lower half in 15 tiles, then flip the object in the mount and scan the other half. For each tile 1200 projections were taken. The tiling was automated by writing a script with exact locations for source and detector positions during the CT scan (locations corresponding to the radiographs in Fig. 10c). The scanning took in total 7.5 h (preparation: 1 h, first tiled scan: 3 h, repositioning: 30 min, second tiled scan: 3 h).
The investigation of this cornett using CT imaging illustrates how an explorative workflow facilitated by one scanner increases both time-efficiency and research throughput. The inspection and consequent refinement of scans by the experts was performed on a single day (taking in total approximately 5 h), and led to more questions and analysis. It stimulated further research outside the CT facility mainly regarding the originality of the manufacturing method with two wood species. Investigations are currently carried out to determine whether other cornetts have comparable structures with multiple wood species, and to further analyse the CT scans for art historical and conservation purposes [74].

Discussion and conclusions
The wide range of objects investigated by CT in cultural heritage is imposing challenges for applying an explorative workflow, where new questions are continuously asked based on the outcome of imaging observations. In this paper we have discussed the key requirements for enabling a time-efficient workflow for CT scanning of cultural heritage objects and presented our implementation of such a workflow in the FleX-ray Laboratory at CWI, where a CT scanner with several degrees of freedom is combined with a fast computational imaging solution that allows to inspect results while the object is in the scanner, asking new questions and planning new scans that can be carried out immediately.
A flexible imaging workflow with support for a wide range of magnification factors and detector tiling can effectively address the key characteristics of CT scanning for cultural heritage research: multi-scale features, sizes and shapes of objects and multi-material objects. Through fast user feedback, this allows for the adjustment of scanning parameters on the spot, giving a wide range of possibilities to investigate features in detail that are discovered during the scanning process. The case study shows how the time-efficient explorative workflow can benefit research, as for example Question 5 was inspired by the initial inspection with radiographs of the entire object. The scans of ROI2, ROI3, and ROI4 are direct consequences of expert feedback, in which the experts enquired about specific sections of the object interior based on the analysis of the radiographic inspection. This was only made possible via the explorative workflow, where we modified the scanning parameters to investigate on the indication of the experts.
Our approach requires a highly flexible CT scanner and the presence of both the CT imaging scientist and the curator or conservator. The FleX-ray scanner at CWI (Amsterdam) provides the required scanning flexibility for relatively small objects. Many cultural heritage objects do not conform to this size constraint, or are difficult to move outside the museum. The technical requirements for implementing such a workflow on-site should therefore be taken into account in the design of X-ray facilities in museums. Although our approach does not replace the investigations with different X-ray imaging modalities such as phase contrast, or very high resolution scanning at synchrotrons, it offers a broad range of applications where absorption imaging is the main investigation. Follow-up research, such as the development of a more automated scanning and feature extraction process, is required to turn it into a methodology that can be applied at large. With the approach described in this paper, we hope to contribute to the establishment of a more time-and cost-efficient workflow to optimise the knowledge gain from CT scanning cultural heritage objects.

Funding
The See-Through Museum project is supported by Netherlands Organisation for Scientific Research (NWO), project number 341-60-001. The Impact4Art project is supported by the Netherlands Institute for Conservation, Art and Science (NICAS) and NWO, project number 628.007.033. The FleX-ray Laboratory is supported by the NWO, project number 639.073.506. The sponsors were not involved in the research and writing process.

Data availability
The datasets associated with two of the examples in Section 3 are available on Zenodo; the fabric example (DOI: 10.5281/zenodo.3741311) and the sculpture Holy woman with lantern example (DOI: 10.5281/zenodo.3747192 and 10.5281/zenodo.3747327).