Advanced Video-Based Processing for Low-Cost Damage Assessment of Buildings under Seismic Loading in Shaking Table Tests

This paper explores the potential of a low-cost, advanced video-based technique for the assessment of structural damage to buildings caused by seismic loading. A low-cost, high-speed video camera was utilized for the motion magnification processing of footage of a two-story reinforced-concrete frame building subjected to shaking table tests. The damage after seismic loading was estimated by analyzing the dynamic behavior (i.e., modal parameters) and the structural deformations of the building in magnified videos. The results using the motion magnification procedure were compared for validation of the method of the damage assessment obtained through analyses of conventional accelerometric sensors and high-precision optical markers tracked using a passive 3D motion capture system. In addition, 3D laser scanning to obtain an accurate survey of the building geometry before and after the seismic tests was carried out. In particular, accelerometric recordings were also processed and analyzed using several stationary and nonstationary signal processing techniques with the aim of analyzing the linear behavior of the undamaged structure and the nonlinear structural behavior during damaging shaking table tests. The proposed procedure based on the analysis of magnified videos provided an accurate estimate of the main modal frequency and the damage location through the analysis of the modal shapes, which were confirmed using advanced analyses of the accelerometric data. Consequently, the main novelty of the study was the highlighting of a simple procedure with high potential for the extraction and analysis of modal parameters, with a special focus on the analysis of the modal shape’s curvature, which provides accurate information on the location of the damage in a structure, while using a noncontact and low-cost method.


Introduction
Several techniques for seismic monitoring are available, and multiple methods and sensors are used. They are generally intended to sense the vibrations that occur in a structure or the possible changes in the physical parameters related to the structural health as a consequence of a seismic event. In fact, several types of earthquakes may occur depending on the site where the structure is located. In particular, sudden and unexpected high amplitude earthquakes, or so-called high activity earthquake swarm events, can occur, potentially causing damage to civil engineering structures [1].
In recent years, the application of image-based techniques to structural monitoring has gained increasing interest [2][3][4]. One the one hand, digital image and video processing

Proposed MM-Based Procedure
The proposed procedure comprises different processing steps. In Figure 1, the essential flowchart of the proposed procedure is illustrated. A video camera records footage of only a few minutes of the object. In the second step, the footage is processed using a software tool provided by the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT-CSAIL). The phase-based MM version (PBMM) of the algorithm is implemented with this software is used [31]. The main parameters to be set are the frequency range and the amplification factor.
The frequency range indicates the group of frequencies whose magnification will be enhanced with respect to the others. However, the precise specifications of their tuning are lacking, as warned by the MIT researchers themselves.
Therefore, it is advisable to collect as much information as possible on the studied object, especially in terms of the frequency range of interest, using a frequency range and required level of amplification within that range. An excessive amplification or too large frequency range produces disturbing image distortion and noise. Therefore, a trade-off between noise and amplification is always necessary. The duration (T) of the video is inversely correlated to the frequency resolution in the frequency domain: the larger the T, the better the frequency resolution. Unfortunately, a large T means a much longer elaboration time. Moreover, taking into account the Shannon-Nyquist theorem, the camera frame rate (fps) becomes an important parameter. It is suggested, however, to adopt a significant data oversampling to compensate for possible aliasing phenomena. A video camera records footage of only a few minutes of the object. In the second step, the footage is processed using a software tool provided by the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT-CSAIL). The phase-based MM version (PBMM) of the algorithm is implemented with this software is used [31]. The main parameters to be set are the frequency range and the amplification factor.
The frequency range indicates the group of frequencies whose magnification will be enhanced with respect to the others. However, the precise specifications of their tuning are lacking, as warned by the MIT researchers themselves.
Therefore, it is advisable to collect as much information as possible on the studied object, especially in terms of the frequency range of interest, using a frequency range and required level of amplification within that range. An excessive amplification or too large frequency range produces disturbing image distortion and noise. Therefore, a trade-off between noise and amplification is always necessary. The duration (T) of the video is inversely correlated to the frequency resolution in the frequency domain: the larger the T, the better the frequency resolution. Unfortunately, a large T means a much longer elaboration time. Moreover, taking into account the Shannon-Nyquist theorem, the camera frame rate (fps) becomes an important parameter. It is suggested, however, to adopt a significant data oversampling to compensate for possible aliasing phenomena.
Then, an appropriate region-of-interest (ROI) is selected for analysis in the frequency domain. In fact, the whole full-resolution image could be theoretically processed, but it would be computationally uselessly heavy, as not all pixels in the image provide equally significant information. So, the choice of an efficient ROI is crucial for an efficient identifica-Sensors 2023, 23, 5303 5 of 27 tion of the modal frequencies. On the one hand, the ROI should contain a portion of the studied structure with the highest image contrast with respect to the background so that structural motion is captured with the highest accuracy. To this purpose, an image entropy criterion can provide a useful indication [32]. On the other hand, the best results can be obtained by points of the structure, with the highest expected displacements associated to the considered modal shapes. Without going into details about the several existing optimal sensor placement approaches [12], the ROI selection can be treated following somewhat similar approaches. However, contrarily to conventional sensor placement, the ROI size, shape, and position can be easily changed at any time with great flexibility so that an optimal ROI can be found manually in a few attempts. Nonetheless, it is highly recommended to preliminarily calculate the expected mode shapes of the structure using finite element analysis (FEA), which provides useful indications to guide the ROI selection. Moreover, the ROI size should consider the available computational resources, as the larger the ROI, the higher the computational time. Finally, the ROI shape is a rectangle that should be as flat as possible in the direction orthogonal to the main direction of motion.
The values of the pixels extracted from the ROI are processed in the frequency domain, typically using power spectral density (PSD) to identify the main frequencies. Once the pixel time-series are extracted from the magnified video, they are averaged and processed just as the usual standard signals. The somewhat smooth appearance of the PSD depends on the short time span of the video that decreases the PSD frequency resolution. Instances of this procedure can be easily found in the literature, see for example [33].
After the identification of f r , a new MM video is produced with the passband filter centered on f r and with a sufficiently large amplification factor (α) so that the MM video actually visualizes the structure moving according to the modal shape.
This latter MM video is then processed by searching for the frame with maximum displacement (FMD) and frame with zero displacement (FZD) of the studied structure. The FMD is assumed as the best frame to identify the modal shape, while the FZD represents the undeformed structure and is used as a reference. The FZD might be identified as a frame with minimum displacements, as no frame will possibly have zero displacements, given the unavoidable noise in the image.
At this point, the FMD and FZD undergo further processing. Preparatory image processing steps, which essentially make the image's structural lines more evident, might include color adjustment and contrast maximization. Then, an inverted edge detection algorithm [34] is applied in order to enhance the main structural lines in the image (e.g., lines of columns and floor slabs). Such lines are extracted from both the FMD and FZD. The relative displacements between corresponding structural lines in the FMD and FZD are calculated to obtain a magnified modal shape (MMS), which can be demagnified by dividing by α to obtain an estimate of the real modal shape.
In order to verify the consistency of the results, the proposed method can be analogously applied to more couples of the frames corresponding to the relative maximum and minimum displacements in any period of the oscillation of the studied structure in the magnified video at a given modal frequency. For example, in a magnified video with a duration of 20 s at a modal frequency of 3 Hz (e.g., three oscillations per second), we have 60 oscillations; therefore, we can identify 60 couples of relative FMD and FZD frames. Finally, an averaged modal shape can be calculated and analyzed. However, this would imply a much greater computational time. Consequently, we limited the extraction of the modal shape to the three amplest oscillations just to demonstrate that the result obtained with only the absolute maximum FMD and minimum FZD frames is not an outliner.

Lab Experimentation
The experiment was carried out at the seismic hall of the ENEA Casaccia Research Center, near Rome, Italy, using a 4 m × 4 m, 6 degrees-of-freedom (DOFs) shaking table. The shaking table was used for seismic testing of a two-thirds scale, two-story mockup of a framed weakly RC building. It was designed in accordance with the Italian codes and standards of the 1960s and 1970s, when a large part of the Italian building stock was designed during the real estate "boom" of the post-war reconstruction, so that it nowadays represents one of the most widely common construction typologies on the Italian territory [35].
The building mockup was designed to resist only to vertical loads according to the Italian codes of that time.
As for the material properties, the utilized concrete had a characteristic compressive strength of 20 MPa, and the steel bars had a tensile characteristic yield strength of 450 MPa. Extracts of the design drawings and the dimensions of the tested mockup are illustrated in Figure 2. In addition, the slabs were typical of common Italian building stock, and they were built as one-way ribbed floor slabs (drawing of the design shown in Figure 3) and lightened in weight through the use of hollow clay bricks, and the floor slab weight was approximately 21 kN. The overall mass of the building was approximately 150 kN. An additional load of 12 kN was added to each floor by steel plates placed at nine positions in a 3 × 3 matrix. The configuration of the additional load is shown in Figure 4a, with slight eccentricity with respect to the building center of mass.

Lab Experimentation
The experiment was carried out at the seismic hall of the ENEA Casaccia Research Center, near Rome, Italy, using a 4 m × 4 m, 6 degrees-of-freedom (DOFs) shaking table. The  shaking table was used for seismic testing of a two-thirds scale, two-story mockup of a  framed weakly RC building. It was designed in accordance with the Italian codes and standards of the 1960s and 1970s, when a large part of the Italian building stock was designed during the real estate "boom" of the post-war reconstruction, so that it nowadays represents one of the most widely common construction typologies on the Italian territory [35].
The building mockup was designed to resist only to vertical loads according to the Italian codes of that time.
As for the material properties, the utilized concrete had a characteristic compressive strength of 20 MPa, and the steel bars had a tensile characteristic yield strength of 450 MPa. Extracts of the design drawings and the dimensions of the tested mockup are illustrated in Figure 2. In addition, the slabs were typical of common Italian building stock, and they were built as one-way ribbed floor slabs (drawing of the design shown in Figure 3) and lightened in weight through the use of hollow clay bricks, and the floor slab weight was approximately 21 kN. The overall mass of the building was approximately 150 kN. An additional load of 12 kN was added to each floor by steel plates placed at nine positions in a 3 × 3 matrix. The configuration of the additional load is shown in Figure 4a, with slight eccentricity with respect to the building center of mass.  After positioning on the shaking table, the building's base was fixed to the table and a 3D geometrical survey using a laser scanner was carried out. Four stations placed near each building column were acquired (Figure 4b). This acquisition was repeated after the end of the shaking table tests.
Several instruments were utilized to the acquire motion data of the structure positioned on the shaking table (Figure 4c). In particular, the positions of the MM camera, as well as the accelerometers and markers of an optical 3D MCS, are illustrated in Figure 5. For a nomenclature of the measurement points, the building columns were named from 1 to   After positioning on the shaking table, the building's base was fixed to the table and a 3D geometrical survey using a laser scanner was carried out. Four stations placed near each building column were acquired (Figure 4b). This acquisition was repeated after the end of the shaking table tests.
Several instruments were utilized to the acquire motion data of the structure positioned on the shaking table (Figure 4c). In particular, the positions of the MM camera, as well as the accelerometers and markers of an optical 3D MCS, are illustrated in Figure 5. For a nomenclature of the measurement points, the building columns were named from 1 to 4 counterclockwise, and the building facades were named S, E, N, and W according to cardinal points.
The camera used for the MM acquisitions had mounted a CMOS sensor Onsemi PY-THON 1300 with a full resolution of 1280 × 1024 pixels (1264 × 1016 pixels in the color mode) and a maximum frame rate of 210 fps with reduced resolution (Figure 6a).
The optical 3D MCS is based on Vicon technology with a constellation of 13 near-infrared (NIR) cameras positioned at safe distances around the shaking table (see Figure 6b). The 3D motion capture NIR cameras with 5 Mpixel at 200 fps, acquired the position of 68 markers located on the building mockup. The used markers were simple 25 mm diameter plastic balls covered with a retro-reflecting coating. They were placed at the base of each column,    After positioning on the shaking table, the building's base was fixed to the table and a 3D geometrical survey using a laser scanner was carried out. Four stations placed near each building column were acquired (Figure 4b). This acquisition was repeated after the end of the shaking table tests.
Several instruments were utilized to the acquire motion data of the structure positioned on the shaking table (Figure 4c). In particular, the positions of the MM camera, as well as the accelerometers and markers of an optical 3D MCS, are illustrated in Figure 5. For a nomenclature of the measurement points, the building columns were named from 1 to 4 counterclockwise, and the building facades were named S, E, N, and W according to cardinal points.
The camera used for the MM acquisitions had mounted a CMOS sensor Onsemi PY-THON 1300 with a full resolution of 1280 × 1024 pixels (1264 × 1016 pixels in the color mode) and a maximum frame rate of 210 fps with reduced resolution (Figure 6a).
The optical 3D MCS is based on Vicon technology with a constellation of 13 near-infrared (NIR) cameras positioned at safe distances around the shaking table (see Figure 6b). The 3D motion capture NIR cameras with 5 Mpixel at 200 fps, acquired the position of 68 markers located on the building mockup. The used markers were simple 25 mm diameter plastic balls covered with a retro-reflecting coating. They were placed at the base of each column, The camera used for the MM acquisitions had mounted a CMOS sensor Onsemi PYTHON 1300 with a full resolution of 1280 × 1024 pixels (1264 × 1016 pixels in the color mode) and a maximum frame rate of 210 fps with reduced resolution (Figure 6a).
The optical 3D MCS is based on Vicon technology with a constellation of 13 nearinfrared (NIR) cameras positioned at safe distances around the shaking table (see Figure 6b). The 3D motion capture NIR cameras with 5 Mpixel at 200 fps, acquired the position of 68 markers located on the building mockup. The used markers were simple 25 mm diameter plastic balls covered with a retro-reflecting coating. They were placed at the base of each column, at the column beam joints, and in the center of both floors' beams. The system measured the markers' position with accuracies of approximately 0.03 mm and 0.15 mm in terms of the root mean square (RMS) error in static and dynamic (seismic) conditions, respectively. The used accelerometers were PCB 3701G3FA3G, whose broadband capacitive at the column beam joints, and in the center of both floors' beams. The system measured the markers' position with accuracies of approximately 0.03 mm and 0.15 mm in terms of the root mean square (RMS) error in static and dynamic (seismic) conditions, respectively. The used accelerometers were PCB 3701G3FA3G, whose broadband capacitive sensors had a 3 g peak measurement range. They were acquired at a sampling frequency of 200 Hz. Eleven accelerometer channels were positioned on each floor and at the specimen's base, as shown in Figure 5a.    at the column beam joints, and in the center of both floors' beams. The system measured the markers' position with accuracies of approximately 0.03 mm and 0.15 mm in terms of the root mean square (RMS) error in static and dynamic (seismic) conditions, respectively. The used accelerometers were PCB 3701G3FA3G, whose broadband capacitive sensors had a 3 g peak measurement range. They were acquired at a sampling frequency of 200 Hz. Eleven accelerometer channels were positioned on each floor and at the specimen's base, as shown in Figure 5a.  The seismic input was based on an earthquake that occurred on 30 October 2016, the largest event in the context of the 2016-2017 Central Italy seismic sequence, with an estimated magnitude of M w 6.5 that struck 6 km north of Norcia at 07:40 local time (06:40 UTC). In particular, the shake recorded at the Savelli seismic station (NRC, Lat. 42.72492 Lon. 13.12578 WGS84, soil type A), located near Norcia (Figure 7), was considered. This was scaled in acceleration in order to obtain the NRC tests (see the shaking table  sequence in Table 1), with an increasing step of 0.1 g of the peak ground acceleration (PGA) up to the natural shake of 0.87 g of the PGA. Each seismic test was intercalated with dynamic identification tests with the white noise frequency spectrum (also known as "random" tests) at 0.05 g of PGA (Rnd tests in Table 1). The seismic NRC_100 test was repeated to assess degradation cause by damage with same earthquake. The two random Rnd_10 and Rnd_11 tests were executed for checking the repeatability of the modal identification with MM using different camera parameters. All tests were triaxial. The seismic input was based on an earthquake that occurred on 30 October 2016, the largest event in the context of the 2016-2017 Central Italy seismic sequence, with an estimated magnitude of Mw 6.5 that struck 6 km north of Norcia at 07:40 local time (06:40 UTC). In particular, the shake recorded at the Savelli seismic station (NRC, Lat. 42.72492 Lon. 13.12578 WGS84, soil type A), located near Norcia (Figure 7), was considered. This was scaled in acceleration in order to obtain the NRC tests (see the shaking table sequence in Table 1), with an increasing step of 0.1 g of the peak ground acceleration (PGA) up to the natural shake of 0.87 g of the PGA. Each seismic test was intercalated with dynamic identification tests with the white noise frequency spectrum (also known as "random" tests) at 0.05 g of PGA (Rnd tests in Table 1). The seismic NRC_100 test was repeated to assess degradation cause by damage with same earthquake. The two random Rnd_10 and Rnd_11 tests were executed for checking the repeatability of the modal identification with MM using different camera parameters. All tests were triaxial.

Numerical Modeling and Analysis
The dynamic behavior of the structure was evaluated using finite element models to coherently select the seismic input for the shaking table test. The software Midas Gen was used for this purpose. Beam elements with six degrees of freedom per node were used. The floors were not directly modeled, assuming a rigid diaphragm behavior with mass, and their rotary inertia was evaluated separately and applied to the center of mass of each floor. For the vertical action, the mass of the floor was applied to the nodal point of the structure. The mass and inertia of the floor also included the additional point masses placed on the floors to simulate live loads: six of 100 kg and three of 200 kg.
The first analyses were conducted assuming the design values for the concrete Young's modulus, taken from the standard E c = 29.96 GPa; subsequently, the model was updated considering E c = 27.95 GPa, derived from the experimental evaluation of the concrete strength.
However, the analysis of the experimental dynamic data from the first random test (Rnd_01), which was representative of the undamaged structure, showed much lower frequency values than those derived from the initial linear FE model (see Table 2). Instead, a good agreement was obtained by assuming a cracked flexural stiffness in the model, as is reasonable according to the technical codes for seismic analyses, even in the case of service limit states. Therefore, E c was finally assumed to be equal to 20 GPa, and a reduced inertia of the structural members was adopted. In more detail, we considered stiffness ratios of 0.7 for the columns and 0.64 and 0.4 for the x-directed and y-directed beams, respectively. The difference attributed to the beams is related to the different contributions of the one-way floor. Figure 8 shows the first three mode shapes and their associated frequencies after updating the linear model. reasonable according to the technical codes for seismic analyses, even in the case of service limit states. Therefore, Ec was finally assumed to be equal to 20 GPa, and a reduced inertia of the structural members was adopted. In more detail, we considered stiffness ratios of 0.7 for the columns and 0.64 and 0.4 for the x-directed and y-directed beams, respectively. The difference attributed to the beams is related to the different contributions of the one-way floor. Figure 8 shows the first three mode shapes and their associated frequencies after updating the linear model.  For the nonlinear analyses, a Mander constitutive model [36] was adopted for concrete and Menegotto-Pinto [37] for steel. Therefore, three zones characterized the concrete members: (i) not confined for the cover, (ii) confined for the core of the columns, and (iii) confined for the core of the beams. Uncracked initial stiffness was considered in this case. A concrete unconfined strength of f c = 22,217 MPa and a steel yielding strength of f y = 450 MPa were considered.
A seismic analysis was performed with the direct integration of the equations of motion using the earthquake recorded at the Savelli station in Norcia, 30 October 2016 (NRC_100). The final state of the plastic zones due to the rotation around the y-axis in the local reference of the element is shown in Figure 9. No collapse was to be expected, but the structure was expected to exhibit serious damage, according to the objectives of the test. Concerning the top of one of the columns, some of the details of the results are reported in Figure 10, showing the partial cracking of the concrete at the end of the analysis with a scaled input factor of 0.1, which approximately corresponds to NRC_01g.  For the nonlinear analyses, a Mander constitutive model [36] was adopted for concrete and Menegotto-Pinto [37] for steel. Therefore, three zones characterized the concrete members: (i) not confined for the cover, (ii) confined for the core of the columns, and (iii) confined for the core of the beams. Uncracked initial stiffness was considered in this case. A concrete unconfined strength of fc = 22,217 MPa and a steel yielding strength of fy = 450 MPa were considered.
A seismic analysis was performed with the direct integration of the equations of motion using the earthquake recorded at the Savelli station in Norcia, 30 October 2016 (NRC_100). The final state of the plastic zones due to the rotation around the y-axis in the local reference of the element is shown in Figure 9. No collapse was to be expected, but the structure was expected to exhibit serious damage, according to the objectives of the test. Concerning the top of one of the columns, some of the details of the results are reported in Figure 10, showing the partial cracking of the concrete at the end of the analysis with a scaled input factor of 0.1, which approximately corresponds to NRC_01g.

MM-Based Video Processing
Only the videos acquired during the Rnd tests were taken into account, because the MM algorithm works well when the motion is very minimal. Videos in which the motion is strong, such as in seismic tests, are not appropriate for MM processing, as they induce abnormal distortion in the final MM video. However, here, the aim was to extract indications of damage from real structures subjected to ambient vibration, so Rnd tests are more representative of such conditions. In particular, Rnd_02, Rnd_06, and Rnd_11 were acquired in the HD mode (1264 × 1016) at 50 fps, while all other Rnd tests were acquired in the LD mode (632 × 508) at 150 fps.
After the preliminary FEA of the tested structure, four tentative ROIs, named with subscripts from A to D, were selected for the upper floor, where the simulated first bending mode in the x-direction provided the highest values of deformation (Figure 11a). The selected ROIs were assessed in terms of an image contrast by calculating the Michelson index m, which is defined as follows [38]: where Imax and Imin are the highest and lowest pixel values in the ROI, respectively. Then, the ROIs' pixels were analyzed in the frequency domain, and the PSD was calculated. As illustrated in Figure 11b, ROIA has with the highest m value and provided a good identification of the modal frequency. In fact, the contrast in ROIA was determined by the difference in color between the column edge and the white background, while the contrast in the other ROIs was very poor, as it was only because of the slight heterogeneities in the color of the concrete and the presence of markers. Thus, the final ROI was chosen, enlarging ROIA, as shown in Figure 12a. The PSD was then calculated for each Rnd test (Figure 12b).

MM-Based Video Processing
Only the videos acquired during the Rnd tests were taken into account, because the MM algorithm works well when the motion is very minimal. Videos in which the motion is strong, such as in seismic tests, are not appropriate for MM processing, as they induce abnormal distortion in the final MM video. However, here, the aim was to extract indications of damage from real structures subjected to ambient vibration, so Rnd tests are more representative of such conditions. In particular, Rnd_02, Rnd_06, and Rnd_11 were acquired in the HD mode (1264 × 1016) at 50 fps, while all other Rnd tests were acquired in the LD mode (632 × 508) at 150 fps.
After the preliminary FEA of the tested structure, four tentative ROIs, named with subscripts from A to D, were selected for the upper floor, where the simulated first bending mode in the x-direction provided the highest values of deformation (Figure 11a). The selected ROIs were assessed in terms of an image contrast by calculating the Michelson index m, which is defined as follows [38]: where I max and I min are the highest and lowest pixel values in the ROI, respectively. Then, the ROIs' pixels were analyzed in the frequency domain, and the PSD was calculated. As illustrated in Figure 11b, ROI A has with the highest m value and provided a good identification of the modal frequency. In fact, the contrast in ROI A was determined by the difference in color between the column edge and the white background, while the contrast in the other ROIs was very poor, as it was only because of the slight heterogeneities in the color of the concrete and the presence of markers. Thus, the final ROI was chosen, enlarging ROI A , as shown in Figure 12a. The PSD was then calculated for each Rnd test (Figure 12b). An MM video was produced for each Rnd test with an FIR passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz, while α was equal to 100. All MM-processed videos are available at the link in the Supplementary materials section. The MM videos were then processed by searching for the frame with the maximum mean value of the displacements within the ROI, which was assumed as the FMD of the MM video. The frame with the minimum mean value of displacements within the ROI was assumed as the FZD of the MM video. The FMDs and FZDs were An MM video was produced for each Rnd test with an FIR passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz, while α was equal to 100. All MM-processed videos are available at the link in the Supplementary materials section. The MM videos were then processed by searching for the frame with the maximum mean value of the displacements within the ROI, which was assumed as the FMD of the MM video. The frame with the minimum mean value of displacements within the ROI was assumed as the FZD of the MM video. The FMDs and FZDs were An MM video was produced for each Rnd test with an FIR passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz, while α was equal to 100. All MM-processed videos are available at the link in the Supplementary materials section. The MM videos were then processed by searching for the frame with the maximum mean value of the displacements within the ROI, which was assumed as the FMD of the MM video. The frame with the minimum mean value of displacements within the ROI was assumed as the FZD of the MM video. The FMDs and FZDs were processed using color adjustment and contrast maximization. Then, the invert edge detection algorithm was applied to outline the main structural lines of the columns and floor slabs (Figure 13a). In particular, the column lines were considered for the first modal shape identification, as it was expected to be an x-direction bending mode (Figure 13b). The column lines were digitized with 100 points (one point every 4 cm, for a total height of 400 cm). The relative displacements in the x-direction between the corresponding structural lines in the FMD and FZD were calculated to extract the modal shape (Figure 13c). processed using color adjustment and contrast maximization. Then, the invert edge detection algorithm was applied to outline the main structural lines of the columns and floor slabs (Figure 13a). In particular, the column lines were considered for the first modal shape identification, as it was expected to be an x-direction bending mode (Figure 13b). The column lines were digitized with 100 points (one point every 4 cm, for a total height of 400 cm). The relative displacements in the x-direction between the corresponding structural lines in the FMD and FZD were calculated to extract the modal shape (Figure 13c). The extraction of the modal shape was repeated using the three amplest oscillations in the MM video to demonstrate that the result obtained, using only the absolute maximum FMD and minimum FZD frames, was not due to a possible outlier. The corresponding results are indicated with "1", "2", and "3".

Optical 3D MCS Markers
The displacement data from the markers were processed for the extraction of the modal parameters and for the monitoring of the structural deformations. In particular, the markers data of the Rnd tests were processed using experimental modal analysis (EMA) techniques through the multi-input-multi-output (MIMO) method. In fact, the four markers at the column base were considered input signals, while all other markers on the structure were considered output signals. More specifically, the H1 estimator of the frequency response function (FRF) was calculated according to the following common formulation: where Sio is the cross power spectrum of the input and output signals, respectively; Sii is the auto power spectrum of the input signals; and ω is the angular frequency, calculated as 2π·f, where f is the frequency. The H1 estimator, obtained from the marker data, is then compared with the theoretical single-degree-of-freedom (SDOF) response by curve-fitting method for the accurate extraction of the resonance frequency ( Figure 14). The extraction of the modal shape was repeated using the three amplest oscillations in the MM video to demonstrate that the result obtained, using only the absolute maximum FMD and minimum FZD frames, was not due to a possible outlier. The corresponding results are indicated with "1", "2", and "3".

Optical 3D MCS Markers
The displacement data from the markers were processed for the extraction of the modal parameters and for the monitoring of the structural deformations. In particular, the markers data of the Rnd tests were processed using experimental modal analysis (EMA) techniques through the multi-input-multi-output (MIMO) method. In fact, the four markers at the column base were considered input signals, while all other markers on the structure were considered output signals. More specifically, the H 1 estimator of the frequency response function (FRF) was calculated according to the following common formulation: where S io is the cross power spectrum of the input and output signals, respectively; S ii is the auto power spectrum of the input signals; and ω is the angular frequency, calculated as 2π·f, where f is the frequency. The H 1 estimator, obtained from the marker data, is then compared with the theoretical single-degree-of-freedom (SDOF) response by curve-fitting method for the accurate extraction of the resonance frequency ( Figure 14).  In addition, the modal shape was calculated by filtering the markers' displacements at the identified modal frequency. A 4th-order Butterworth passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz was applied.
Relative displacements between markers in the x-and y-directions during the seismic tests were used to obtain the inter-story drift at the first and second floors with respect to the markers at the column bases. The inter-story drift can be defined as the relative translational displacement between two consecutive floors and is widely considered an important engineering demand parameter and indicator of structural performance [39].
The distances between markers were also investigated to detect structural deformations and, notably, a crack's opening and width [40]. In particular, in Figure 15 the typical behavior of a crack's opening during a seismic shake is depicted. Cracks typically open and close during a shake (i.e., dynamic condition) and eventually remain residually open at the end of the test (i.e., static condition) when permanent plastic deformation of the structure is achieved. Markers can detect a crack's width in both dynamic and static conditions so as to accurately characterize the crack's behavior.

Advanced Accelerometers Processing
In addition to the conventional EMA processing of the Rnd tests, the accelerometer data of the seismic tests were also processed using more advanced techniques. In particular, the nonlinear structural behavior of the mockup structure was analyzed, performing frequency (smoothed amplitude Fourier spectra) and time-frequency analyses (using Stockwell Transform [28] and band-variable filter [29]) considering the relative floor accelerations. In addition, the modal shape was calculated by filtering the markers' displacements at the identified modal frequency. A 4th-order Butterworth passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz was applied.
Relative displacements between markers in the x-and y-directions during the seismic tests were used to obtain the inter-story drift at the first and second floors with respect to the markers at the column bases. The inter-story drift can be defined as the relative translational displacement between two consecutive floors and is widely considered an important engineering demand parameter and indicator of structural performance [39].
The distances between markers were also investigated to detect structural deformations and, notably, a crack's opening and width [40]. In particular, in Figure 15 the typical behavior of a crack's opening during a seismic shake is depicted. Cracks typically open and close during a shake (i.e., dynamic condition) and eventually remain residually open at the end of the test (i.e., static condition) when permanent plastic deformation of the structure is achieved. Markers can detect a crack's width in both dynamic and static conditions so as to accurately characterize the crack's behavior. In addition, the modal shape was calculated by filtering the markers' displacements at the identified modal frequency. A 4th-order Butterworth passband filter with the window centered on the identified first frequency and a bandwidth of ±0.2 Hz was applied.
Relative displacements between markers in the x-and y-directions during the seismic tests were used to obtain the inter-story drift at the first and second floors with respect to the markers at the column bases. The inter-story drift can be defined as the relative translational displacement between two consecutive floors and is widely considered an important engineering demand parameter and indicator of structural performance [39].
The distances between markers were also investigated to detect structural deformations and, notably, a crack's opening and width [40]. In particular, in Figure 15 the typical behavior of a crack's opening during a seismic shake is depicted. Cracks typically open and close during a shake (i.e., dynamic condition) and eventually remain residually open at the end of the test (i.e., static condition) when permanent plastic deformation of the structure is achieved. Markers can detect a crack's width in both dynamic and static conditions so as to accurately characterize the crack's behavior.

Advanced Accelerometers Processing
In addition to the conventional EMA processing of the Rnd tests, the accelerometer data of the seismic tests were also processed using more advanced techniques. In particular, the nonlinear structural behavior of the mockup structure was analyzed, performing frequency (smoothed amplitude Fourier spectra) and time-frequency analyses (using Stockwell Transform [28] and band-variable filter [29]) considering the relative floor accelerations.

Advanced Accelerometers Processing
In addition to the conventional EMA processing of the Rnd tests, the accelerometer data of the seismic tests were also processed using more advanced techniques. In particular, the nonlinear structural behavior of the mockup structure was analyzed, performing frequency (smoothed amplitude Fourier spectra) and time-frequency analyses (using Stockwell Transform [28] and band-variable filter [29]) considering the relative floor accelerations.
In order to analyze the accelerometric data referred to the shaking table tests performed using the Rnd and Norcia inputs (see Table 1), all records were first modified using the following processes: All data refer to the accelerations recorded by the sensors, as described in Figure 5, columns 1 and 3. The relative accelerations were evaluated considering as reference stations those located at the base of the model for each horizontal direction. Then, considering only the random tests, the amplitude spectra and related modulus were evaluated using the following relationship: where a r represents the relative floor acceleration. The modulus of the relative acceleration of spectra |A r (ω)| were smoothed using a 75-point moving average filter implemented in MATLAB ® . Figure 16 shows the smoothed amplitude Fourier spectra evaluated in the experimental model, considering the data acquired during the NRC_01g test. The same results were also used to evaluate the mode shapes associated to the first fundamental modal frequencies related to the x-and y-directions. In order to analyze the accelerometric data referred to the shaking table tests performed using the Rnd and Norcia inputs (see Table 1), all records were first modified using the following processes: • Baseline correction; • Trends removal; • 0.1-25 Hz band-pass filter; • Smoothing with a Tukey window.
All data refer to the accelerations recorded by the sensors, as described in Figure 5, columns 1 and 3. The relative accelerations were evaluated considering as reference stations those located at the base of the model for each horizontal direction. Then, considering only the random tests, the amplitude spectra and related modulus were evaluated using the following relationship: where represents the relative floor acceleration. The modulus of the relative acceleration of spectra | | were smoothed using a 75-point moving average filter implemented in MATLAB ® . Figure 16 shows the smoothed amplitude Fourier spectra evaluated in the experimental model, considering the data acquired during the NRC_01g test. The same results were also used to evaluate the mode shapes associated to the first fundamental modal frequencies related to the x-and y-directions. It is worth noting that the spectral characteristics evaluated using this kind of approach are representative of the stationary behavior of the model. When damage processes are under way, the structural modal frequencies change over time during a strong motion excitation; then, the linear modal parameters may not be fully representative of the actual state of damage of the structure. Therefore, structural damage time-frequency analyses allow a more proper evaluation of the actual state of damage, because they are able to provide the frequency changes over time.
With the aim of evaluating the minimum value of the modal frequencies exhibited by the structure during the nonstationary phase, ST analyses were performed on each of the top floor accelerometric recordings (along both the x-and y-directions). The ST was evaluated using the following relationship [28]: It is worth noting that the spectral characteristics evaluated using this kind of approach are representative of the stationary behavior of the model. When damage processes are under way, the structural modal frequencies change over time during a strong motion excitation; then, the linear modal parameters may not be fully representative of the actual state of damage of the structure. Therefore, structural damage time-frequency analyses allow a more proper evaluation of the actual state of damage, because they are able to provide the frequency changes over time.
With the aim of evaluating the minimum value of the modal frequencies exhibited by the structure during the nonstationary phase, ST analyses were performed on each of the top floor accelerometric recordings (along both the x-and y-directions). The ST was evaluated using the following relationship [28]: where t is the time; f is the frequency; and τ is a parameter controlling the position of the Gaussian window along the time axis. An example of the application of the ST is presented in Figure 17, and it is related to the test where the NRC_0.3g earthquake was applied.
where t is the time; f is the frequency; and τ is a parameter controlling the position of the Gaussian window along the time axis. An example of the application of the ST is presented in Figure 17, and it is related to the test where the NRC_0.3g earthquake was applied. The ST analysis allowed for the evaluation of the fundamental frequency variation of the structure during the strong motion test. Along the x-direction, the minimum value of the fundamental frequency exhibited by the structure during the NRC_0.3g test was equal to 1.86 Hz. Similar results were found using STIRF [30] analyses on the same accelerometric recording, not shown here for the sake of brevity. More details concerning the nonlinear behavior of the structure exhibited during the shaking table tests are reported in Section 6.

Results
After the shaking table tests, a visual inspection was carried out in order to assess the crack pattern and state of damage to the structure. The visible cracks were essentially concentrated at the beam column joints on first floor ( Figure 18). Fewer cracks were also visible at the beam column joints on second floor (Figure 19). Only a limited number of small cracks arose at the column's base.
The surveyed crack positions, orientation, size, and openings were considered in order to assess the damage grade of the structure. Initially, the damage grade was detected according to the "Agibilita' e Danno nell'Emergenza Sismica (AeDES)" ("Building Operability and Damage during the Post-Earthquake Emergency") survey form, which is used by the Italian Civil Protection for assessing the usability and damage of buildings after earthquakes [41]. Then, the damage was also assessed with the European Macroseismic Scale of 1998 (EMS98) [42] using the damage grades conversion table proposed in [43]. The damage that resulted was between D3 and D4 for the vertical members (columns) and D1 for the horizontal members (i.e., beams and floors). The ST analysis allowed for the evaluation of the fundamental frequency variation of the structure during the strong motion test. Along the x-direction, the minimum value of the fundamental frequency exhibited by the structure during the NRC_0.3g test was equal to 1.86 Hz. Similar results were found using STIRF [30] analyses on the same accelerometric recording, not shown here for the sake of brevity. More details concerning the nonlinear behavior of the structure exhibited during the shaking table tests are reported in Section 6.

Results
After the shaking table tests, a visual inspection was carried out in order to assess the crack pattern and state of damage to the structure. The visible cracks were essentially concentrated at the beam column joints on first floor ( Figure 18). Fewer cracks were also visible at the beam column joints on second floor (Figure 19). Only a limited number of small cracks arose at the column's base.
The surveyed crack positions, orientation, size, and openings were considered in order to assess the damage grade of the structure. Initially, the damage grade was detected according to the "Agibilita' e Danno nell'Emergenza Sismica (AeDES)" ("Building Operability and Damage during the Post-Earthquake Emergency") survey form, which is used by the Italian Civil Protection for assessing the usability and damage of buildings after earthquakes [41]. Then, the damage was also assessed with the European Macroseismic Scale of 1998 (EMS98) [42] using the damage grades conversion table proposed in [43]. The damage that resulted was between D3 and D4 for the vertical members (columns) and D1 for the horizontal members (i.e., beams and floors).
The distance between the 3D MCS markers was monitored to assess the crack openings and widths. Table 3 summarizes the distribution of the cracks and their widths after the NRC_100 shaking table tests. An analysis of the markers' distances confirmed significantly that the distribution and severity of the cracks were concentrated at the first floor beam column joints. In particular, the number of cracks on the first floor with a width greater than 1 mm was more than double that detected on the second floor.  The distance between the 3D MCS markers was monitored to assess the crack openings and widths. Table 3 summarizes the distribution of the cracks and their widths after the NRC_100 shaking table tests. An analysis of the markers' distances confirmed significantly that the distribution and severity of the cracks were concentrated at the first floor beam column joints. In particular, the number of cracks on the first floor with a width greater than 1 mm was more than double that detected on the second floor. The inter-story drift calculated using the markers during the shaking table sequence is reported in Figure 20. The first and second floor drifts were very similar for the PGA lower than 0.4 g. Then, the second floor drift in the x-direction resulted in the largest until the final   The distance between the 3D MCS markers was monitored to assess the crack openings and widths. Table 3 summarizes the distribution of the cracks and their widths after the NRC_100 shaking table tests. An analysis of the markers' distances confirmed significantly that the distribution and severity of the cracks were concentrated at the first floor beam column joints. In particular, the number of cracks on the first floor with a width greater than 1 mm was more than double that detected on the second floor. The inter-story drift calculated using the markers during the shaking table sequence is reported in Figure 20. The first and second floor drifts were very similar for the PGA lower than 0.4 g. Then, the second floor drift in the x-direction resulted in the largest until the final  The inter-story drift calculated using the markers during the shaking table sequence is reported in Figure 20. The first and second floor drifts were very similar for the PGA lower than 0.4 g. Then, the second floor drift in the x-direction resulted in the largest until the final seismic test. The second floor drift was substantial because of the behavior of the base of the second floor columns, which constituted the upper part of the first floor beam column joints.
The 3D laser scanning survey after the seismic tests confirmed the deformation of the structure concentrated at the beam column joints, causing substantial rotation of the columns, while the floor slabs did not exhibit relevant deformation and remained essentially undamaged.  The 3D laser scanning survey after the seismic tests confirmed the deformation of the structure concentrated at the beam column joints, causing substantial rotation of the columns, while the floor slabs did not exhibit relevant deformation and remained essentially undamaged.
In Figure 21a, a comparison of the fr estimates in the Rnd tests with the accelerometers, markers, and MM videos is depicted vs. the PGA of the previous shaking table test. In order to assess the state of damage of the structure, a damage index (D) can be formulated based on fr, according to the following equation: where f0 is the first modal frequency in Rnd_01, which was performed before the seismic tests so that it was representative of the undamaged structure. Figure 21b shows the estimated D values. In Figure 21a, a comparison of the f r estimates in the Rnd tests with the accelerometers, markers, and MM videos is depicted vs. the PGA of the previous shaking table test. In order to assess the state of damage of the structure, a damage index (D) can be formulated based on f r , according to the following equation: where f 0 is the first modal frequency in Rnd_01, which was performed before the seismic tests so that it was representative of the undamaged structure. Figure 21b shows the estimated D values.
seismic test. The second floor drift was substantial because of the behavior of the base of the second floor columns, which constituted the upper part of the first floor beam column joints. The 3D laser scanning survey after the seismic tests confirmed the deformation of the structure concentrated at the beam column joints, causing substantial rotation of the columns, while the floor slabs did not exhibit relevant deformation and remained essentially undamaged.
In Figure 21a, a comparison of the fr estimates in the Rnd tests with the accelerometers, markers, and MM videos is depicted vs. the PGA of the previous shaking table test. In order to assess the state of damage of the structure, a damage index (D) can be formulated based on fr, according to the following equation: where f0 is the first modal frequency in Rnd_01, which was performed before the seismic tests so that it was representative of the undamaged structure. Figure 21b shows the estimated D values.  Table 4 summarizes the estimated values of f r and the corresponding errors with reference to the frequencies obtained by the accelerometers, assumed as the most conventional and consolidated method. The markers of the used 3D MCS reported errors from -0.10 to +0.08 Hz, with an RMS error of 0.06 Hz. The MM-based estimates provided errors between −0.38 and +0.11 Hz, with an RMS error of 0.16 Hz, with a higher camera speed (150 fps) but LD camera resolution. When using the HD mode resolution and lower speed (50 fps), the error in the MM estimate of f r was in the range −0.08 to −0.22 Hz, with an RMS error of 0.07 Hz. It is interesting to note that the MM result was at 50 fps, which was more representative of the CGCs, slightly better than at a higher speed (150 fps), indicating that when the studied frequencies do not require high-speed acquisitions, the pixel resolution seems to play a more relevant role. In addition, MM slightly underestimated the f r identified using accelerometers, providing negative average error values (-0.08 Hz and -0.15 Hz with the LD and HD modes, respectively). This is probably because of the MM algorithm, which affects the correct timing of the motions.
The first modal shape obtained by the MM videos and 3D MCS markers are illustrated in Figure 22. Here, the modal shape during the Rnd_01 (undamaged structure) and Rnd_12 (final damaged condition) tests can be compared. The modal shape was characterized by markers at only three measurement positions (base, first floor, and second floor), because the marker locations were mainly concentrated at the beam column joints to monitor the cracking where most of the damage was expected. Of course, many more markers could be used on the columns. However, the number of usable markers was limited in comparison to the positions detectable by video pixels. In fact, the modal shape by the MM video was obtained by digitizing 100 points, meaning one point every 4 cm in height. In Figure 22a, the effect of the beam column joints is extremely evident, especially in terms of the relationship between the flexural stiffness of the beams and columns.
As it is often the case in low-rise and mid-rise buildings, the relative stiffness of frame members lies in between flexural-type and shear-type behavior. As a consequence, both beams and columns bend in a double curvature, which typically results in an almost shear-type response. Moreover, the rotational flexibility of the columns at the base of the building is determined by the degree of fixity at the structure's base. The slightly negative rotation detected at the structure's base and both beam column joints of the first and second floors can partially be due to the nonperfectly vertical positioning of the MM camera. However, such rotations were on the order of 0.1 mm in 50 cm, which is very small. In Figure 22b, it is interesting to note that the double curvature of the first floor beam column joint disappeared completely in the Rnd_12 test, which is the effect of advanced damage to the joint, making the column unable to effectively connect to the first floor beam. The curvature at structure's base and the second floor are still visible but clearly reduced with respect to the initial undamaged condition as in Rnd_01, which indicates less damage.
However, such rotations were on the order of 0.1 mm in 50 cm, which is very small. In Figure 22b, it is interesting to note that the double curvature of the first floor beam column joint disappeared completely in the Rnd_12 test, which is the effect of advanced damage to the joint, making the column unable to effectively connect to the first floor beam. The curvature at structure's base and the second floor are still visible but clearly reduced with respect to the initial undamaged condition as in Rnd_01, which indicates less damage.
(a) (b) Figure 22. First modal shape calculated using the MM and markers in the x-direction: (a) modal shape extracted with the "Rnd_01" test; (b) modal shape extracted with the "Rnd_12" test.
In Figure 23a, an analysis of the evolution of the first modal shape is illustrated in terms of the curvature difference between Rnd_01 and Rnd_12. As a confirmation, the curvature difference using MM showed higher values in correspondence with the first floor beam column joint, locating with remarkable accuracy the main damage.
The location of the damage seemed to be even more accurately indicated by the zero crossings of the curvature difference variation in Figure 23b, even if zero crossings cannot provide an estimate of the severity of the damage.
In order to prove the consistency of the obtained results, the modal shape was extracted using the three amplest oscillations (Figure 24a). The corresponding curvature differences depicted in Figure 24b clearly confirm the damage location obtained with the absolute maximum FMD and minimum FZD frames. Figure 22. First modal shape calculated using the MM and markers in the x-direction: (a) modal shape extracted with the "Rnd_01" test; (b) modal shape extracted with the "Rnd_12" test.
In Figure 23a, an analysis of the evolution of the first modal shape is illustrated in terms of the curvature difference between Rnd_01 and Rnd_12. As a confirmation, the curvature difference using MM showed higher values in correspondence with the first floor beam column joint, locating with remarkable accuracy the main damage.
However, such rotations were on the order of 0.1 mm in 50 cm, which is very sma Figure 22b, it is interesting to note that the double curvature of the first floor beam col joint disappeared completely in the Rnd_12 test, which is the effect of advanced dam to the joint, making the column unable to effectively connect to the first floor beam curvature at structure's base and the second floor are still visible but clearly reduced respect to the initial undamaged condition as in Rnd_01, which indicates less damag (a) (b) Figure 22. First modal shape calculated using the MM and markers in the x-direction: (a) m shape extracted with the "Rnd_01" test; (b) modal shape extracted with the "Rnd_12" test.
In Figure 23a, an analysis of the evolution of the first modal shape is illustrate terms of the curvature difference between Rnd_01 and Rnd_12. As a confirmation curvature difference using MM showed higher values in correspondence with the floor beam column joint, locating with remarkable accuracy the main damage. The location of the damage seemed to be even more accurately indicated by the crossings of the curvature difference variation in Figure 23b, even if zero crossings ca provide an estimate of the severity of the damage.
In order to prove the consistency of the obtained results, the modal shape wa tracted using the three amplest oscillations (Figure 24a). The corresponding curvatur ferences depicted in Figure 24b clearly confirm the damage location obtained with absolute maximum FMD and minimum FZD frames. The location of the damage seemed to be even more accurately indicated by the zero crossings of the curvature difference variation in Figure 23b, even if zero crossings cannot provide an estimate of the severity of the damage.
In order to prove the consistency of the obtained results, the modal shape was extracted using the three amplest oscillations (Figure 24a). The corresponding curvature differences depicted in Figure 24b clearly confirm the damage location obtained with the absolute maximum FMD and minimum FZD frames.
Using the modulus of the Fourier spectra of the relative accelerometric Rnd recordings acquired from the top floor, the main modal frequencies were evaluated along both the xand y-directions. Using the NRC accelerometric recordings, time-frequency ST analyses were performed with the aim of evaluating the minimum value of the fundamental frequency exhibited by the structure during each shaking table test. In Figure 25, a comparison of the estimated modal frequencies along the x-and y-directions is shown.  Figure 24. Repetition of the modal shape extraction using the three amplest oscillations ("1", "2", and "3"): (a) modal shape extracted using the "Rnd_12" test; (b) modal shape curvature difference (CE) between the "Rnd_12" and "Rnd_01" tests.
Using the modulus of the Fourier spectra of the relative accelerometric Rnd recordings acquired from the top floor, the main modal frequencies were evaluated along both the x-and y-directions. Using the NRC accelerometric recordings, time-frequency ST analyses were performed with the aim of evaluating the minimum value of the fundamental frequency exhibited by the structure during each shaking table test. In Figure 25, a comparison of the estimated modal frequencies along the x-and y-directions is shown. It is worth noting that the higher the level of structural damage, the greater the difference between the structural natural frequencies estimated using the random tests (stationary behavior) and those evaluated using the Stockwell transform during the strong motion phase.
Considering the modal frequencies values depicted in Figure 25, in Figure 26 the differences are quantified as a function of the nominal PGA. Figure 24. Repetition of the modal shape extraction using the three amplest oscillations ("1", "2", and "3"): (a) modal shape extracted using the "Rnd_12" test; (b) modal shape curvature difference (CE) between the "Rnd_12" and "Rnd_01" tests.
Using the modulus of the Fourier spectra of the relative accelerometric Rnd recordings acquired from the top floor, the main modal frequencies were evaluated along both the x-and y-directions. Using the NRC accelerometric recordings, time-frequency ST analyses were performed with the aim of evaluating the minimum value of the fundamental frequency exhibited by the structure during each shaking table test. In Figure 25, a comparison of the estimated modal frequencies along the x-and y-directions is shown. It is worth noting that the higher the level of structural damage, the greater the difference between the structural natural frequencies estimated using the random tests (stationary behavior) and those evaluated using the Stockwell transform during the strong motion phase.
Considering the modal frequencies values depicted in Figure 25, in Figure 26 the differences are quantified as a function of the nominal PGA. It is worth noting that the higher the level of structural damage, the greater the difference between the structural natural frequencies estimated using the random tests (stationary behavior) and those evaluated using the Stockwell transform during the strong motion phase.
Considering the modal frequencies values depicted in Figure 25, in Figure 26 the differences are quantified as a function of the nominal PGA.
Analyzing the results shown in Figures 25 and 26, the importance of using timefrequency analysis techniques is demonstrated, as the correct evaluation of the structural modal frequencies of the damaged structure and, consequently, the correctly estimation of the relative structural damage can be observed. Using the performed random tests, the fundamental mode shape evolution relative to the x-direction was evaluated and depicted in Figure 27. Analyzing the results shown in Figures 25 and 26, the importance of using time-frequency analysis techniques is demonstrated, as the correct evaluation of the structural modal frequencies of the damaged structure and, consequently, the correctly estimation of the relative structural damage can be observed. Using the performed random tests, the fundamental mode shape evolution relative to the x-direction was evaluated and depicted in Figure 27.   Analyzing the results shown in Figures 25 and 26, the importance of using time-frequency analysis techniques is demonstrated, as the correct evaluation of the structural modal frequencies of the damaged structure and, consequently, the correctly estimation of the relative structural damage can be observed. Using the performed random tests, the fundamental mode shape evolution relative to the x-direction was evaluated and depicted in Figure 27. The curvature variation was evaluated on the interpolated mode shapes retrieved using piecewise cubic Hermite interpolating polynomials, where the most damaged floor was the first floor. The results obtained for each random test in terms of the curvature variation confirmed what was found in the laboratory using traditional visual inspections and using the MM-based technique. The curvature variation was evaluated on the interpolated mode shapes retrieved using piecewise cubic Hermite interpolating polynomials, where the most damaged floor was the first floor. The results obtained for each random test in terms of the curvature variation confirmed what was found in the laboratory using traditional visual inspections and using the MM-based technique.
In order to identify the most damaged floor using the accelerometric dataset, the curvature variations were evaluated using the procedure described in [29,44,45] and shown in Figure 28. In order to identify the most damaged floor using the accelerometric dataset, the curvature variations were evaluated using the procedure described in [29,44,45] and shown in Figure 28.

Conclusions
The present work proposed an MM-based procedure to assess the state of damage of a framed RC building using low-cost video-based equipment and processing. The proposed procedure was validated through shaking table experiments, reproducing the damage to an Italian building typology subjected to a recent earthquake. In particular, the MM-based procedure aimed at extracting the most relevant modal parameters (frequencies and shapes) for the structural health monitoring of buildings.
The results obtained using the proposed procedure were compared for validation with utilizing methods that were more conventional and consolidated measurement systems, including accelerometers, a 3D laser scanner, and an optical 3D MCS.
The advanced analyses of the accelerometers data focused on the detection of structural damage using both natural frequencies and mode shape variations. The former were evaluated using Stockwell transform of NRC accelerometric recordings. Differences greater than 40% with respect to the natural frequencies obtained using Rnd recordings were evaluated, at least for the most damaged conditions. Mode shape variations, on the other hand, were evaluated using Rnd data, calculating the curvature parameters on interpolated mode shapes recovered using piecewise cubic Hermite interpolating polynomial. The results obtained for each random test in terms of the curvature variation located the most damage from the first floor.
Moreover, the 3D laser scanning surveys and optical 3D MCS marker displacements confirmed the location of the main damage on the first floor.
The final 3D laser survey detected the structure's main deformations at the beam column joints, while the floor slabs substantially lacked any deformation. In addition, the 3D MCS markers' relative displacement revealed that the number and width of the main cracks were concentrated at the first floor's beam column joints. Moreover, through the relative displacements of the markers, the structural damage was also evaluated through the evolution of the inter-story drifts in the seismic tests. The modal frequencies of the marker data were also calculated and were in agreement with the accelerometers data.
As for the estimates of the modal frequency, the proposed MM-based procedure provided results with reasonable accuracy and repeatability with respect to the results obtained analyzing the accelerometric recordings in stationary conditions. The MM-based estimates had quite similar errors with either higher camera speed (150 fps) but an LD camera resolution and using an HD mode resolution and a low speed (50 fps). This latter acquisition set-

Conclusions
The present work proposed an MM-based procedure to assess the state of damage of a framed RC building using low-cost video-based equipment and processing. The proposed procedure was validated through shaking table experiments, reproducing the damage to an Italian building typology subjected to a recent earthquake. In particular, the MM-based procedure aimed at extracting the most relevant modal parameters (frequencies and shapes) for the structural health monitoring of buildings.
The results obtained using the proposed procedure were compared for validation with utilizing methods that were more conventional and consolidated measurement systems, including accelerometers, a 3D laser scanner, and an optical 3D MCS.
The advanced analyses of the accelerometers data focused on the detection of structural damage using both natural frequencies and mode shape variations. The former were evaluated using Stockwell transform of NRC accelerometric recordings. Differences greater than 40% with respect to the natural frequencies obtained using Rnd recordings were evaluated, at least for the most damaged conditions. Mode shape variations, on the other hand, were evaluated using Rnd data, calculating the curvature parameters on interpolated mode shapes recovered using piecewise cubic Hermite interpolating polynomial. The results obtained for each random test in terms of the curvature variation located the most damage from the first floor.
Moreover, the 3D laser scanning surveys and optical 3D MCS marker displacements confirmed the location of the main damage on the first floor.
The final 3D laser survey detected the structure's main deformations at the beam column joints, while the floor slabs substantially lacked any deformation. In addition, the 3D MCS markers' relative displacement revealed that the number and width of the main cracks were concentrated at the first floor's beam column joints. Moreover, through the relative displacements of the markers, the structural damage was also evaluated through the evolution of the inter-story drifts in the seismic tests. The modal frequencies of the marker data were also calculated and were in agreement with the accelerometers data.
As for the estimates of the modal frequency, the proposed MM-based procedure provided results with reasonable accuracy and repeatability with respect to the results obtained analyzing the accelerometric recordings in stationary conditions. The MM-based estimates had quite similar errors with either higher camera speed (150 fps) but an LD camera resolution and using an HD mode resolution and a low speed (50 fps). This latter acquisition set-up configuration, which can be considered more representative of CGCs typical camera parameters, as well as low cost, providing even slightly better accuracy than acquisition at higher speed (150 fps), proving that already reasonably affordable cameras have speed levels appropriate for detecting common building frequencies.
The results in terms of the detection and analysis of the modal shape curvature after seismic shocks was probably the most relevant and interesting, because of the potential to locate damage to structures. The MM-based procedure accurately located the main structural damage. It was detected at the first floor's beam column joints, and it was found even by traditional visual inspections, as well as in remarkable accordance with advanced methods of analysis of accelerometric data: 3D laser surveys and 3D MCS markers. Moreover, mode shapes could be obtained using image analysis with greater definition (given its raster nature) than with the use of sparse optical sensors placed on the structure, which makes the video-based procedure a method with high potential even for more complex structural typologies than just framed building.
Therefore, the main novelty of this study was highlighting a simple and low-cost procedure based on the MM method to estimate with good accuracy the intensity and location of damage to a multistory RC framed prototype, which is highly representative of existing buildings in Italy. Even if methods based on conventional contact sensors are still much more accurate and reliable, as well as providing the possibility to analyze the structural behavior during a strong motion phase (i.e., nonlinear conditions), the presented video-based method shows the potential to provide a valuable preliminary estimate of the state and location of damage, almost in stationary conditions, which can help in the management and planning of further extensive monitoring campaigns. Of course, the proposed procedure is also applicable to other construction typologies, such as masonry and steel. The main limitations of this paper are related to the frequency range of interest and the amplitude of the ambient vibration of the structure, such that the equipment's camera speed and resolution should be appropriate. Future steps and improvements are essentially related to the main challenges of these kinds of techniques: (i) the present study was conducted through a laboratory experiment with an almost real-scale physical model, but studies conducted in the field still present more complicated site conditions, such as the presence of nonstructural components; (ii) the quality of the images captured in the field are significantly affected by the surrounding environmental conditions, such as lighting stability, edge shelter by rain, snow and fog, ground vibration, etc.; and (iii) the further development of more effective algorithms to improve the reliability of the results affected by the abovementioned disturbances are currently underway and may take advantage of recent advances in artificial intelligence and other innovative processing techniques.