Modal identification of simple structures with high-speed video using motion magnification

doi:10.1016/j.jsv.2015.01.024

Journal of Sound and Vibration

Volume 345, 9 June 2015, Pages 58-71

https://doi.org/10.1016/j.jsv.2015.01.024 Get rights and content

Abstract

Video cameras offer the unique capability of collecting high density spatial data from a distant scene of interest. They can be employed as remote monitoring or inspection sensors for structures because of their commonplace availability, simplicity, and potentially low cost. An issue is that video data is difficult to interpret into a format familiar to engineers such as displacement. A methodology called motion magnification has been developed for visualizing exaggerated versions of small displacements with an extension of the methodology to obtain the optical flow to measure displacements. In this paper, these methods are extended to modal identification in structures and the measurement of structural vibrations. Camera-based measurements of displacement are compared against laser vibrometer and accelerometer measurements for verification. The methodology is demonstrated on simple structures, a cantilever beam and a pipe, to identify and visualize the operational deflection shapes. Suggestions for applications of this methodology and challenges in real-world implementation are given.

Introduction

Modal analysis of structures depends on the accurate and swift collection of data from a vibrating structure so the data can be analyzed to determine the modal characteristics. The end goal for a sensor system for modal analysis is to be able to quickly and robustly collect data from a vibrating structure. Contact accelerometers are commonly used for modal analysis and are extremely precise, however densely instrumenting a structure is difficult and tedious. Also, when the structure is small compared to the size of an accelerometer, the presence of added mass from accelerometers can affect the result. There are methods for correcting for the effects of accelerometer mass loading, but they are not exact [1], [2]. Non-contact methods of measurement avoid some of these drawbacks and are being researched intensely for the purposes of modal analysis.

Non-contact methods of vibration measurement generally depend on some sort of electromagnetic radiation to transmit the information. Microwave interferometry [3] and laser methods such as laser vibrometry and electronic speckle pattern interferometry have been studied [4], [5]. Cameras measuring visible light provide an interesting method for measuring vibration. They can range from precise instruments for high-frequency and high-resolution video to inexpensive units such as those on cell phones which can be chosen as necessary for the application. However, processing of videos to extract quantitative information remains relatively difficult.

Motion can be quantified in video using a number of image processing techniques. Previously researched methods use edge detection, target objects, or lights to more easily measure any structural motion [6], [7], [8]. More recent methods make use of computer vision techniques, such as the measurement of optical flow to determine the displacements of structures [9], which is related to the techniques in this paper. With a single camera, only measurements of in-plane motion can be made, however a stereo camera setup is capable of measuring both in-plane and out-of-plane motion [10]. A newer method of in-plane and out-of-plane measurement with cameras is the time-of-flight camera [11], however they currently do not provide enough resolution or speed for typical vibration measurement applications.

Recently, new computer vision techniques, collectively called motion magnification, were introduced to magnify small motions in videos [12], [13], [14], [15], [16]. The most recent motion magnification techniques use a signal processing approach to analyze image motions in a way analogous to an Eulerian framework for fluid flow analysis. They are ideal for computing and visualizing mode shapes because they are capable of detecting small subpixel motions that are present in the videos of vibrating structures and because they are able to separate the different modal motions through the use of temporal filtering [17]. This works well with an assumption of a stationary camera and a structure that is barely moving. The use of a video camera for vibration measurement represents a unique capability that would complement existing measurement and sensor systems currently deployed or being researched for non-destructive testing (NDT) and structural health monitoring (SHM).

The objective of this paper is to demonstrate how these newly developed motion magnification computer vision techniques can be used with high-speed camera video to visualize and quantify the vibrational mode shapes of simple structures. The theory behind motion magnification and the extraction of displacement of moving objects in videos are presented to show how videos can be analyzed qualitatively and quantitatively. An experiment comparing a high-speed camera to a laser vibrometer and accelerometer measurement was carried out to confirm that the displacements extracted from the measured video were accurate. Further experiments were conducted on a cantilever beam and a pipe with a high-speed camera. The operational deflection shapes of the cantilever beam were identified by extracting displacements from the raw recorded video and compared to those from accelerometer measurements. Additionally, the motion magnification algorithm was used to visualize the operational deflection shapes of the cantilever beam and pipe test specimens, and screenshots of the generated videos are shown and discussed. A technique for deriving the operational deflection shapes directly from the motion magnified video without extracting displacement through the use of an edge detection algorithm is also demonstrated and compared to accelerometer measurements.

Section snippets

Theory and methods

Videos are made up of a series of images and thus have two domains: the spatial domain corresponding to a 2D field of brightness values in a single image, and the time or temporal domain corresponding to the image as it evolves in time to make a video. Images can be decomposed in the spatial domain by filters into amplitude and phase signals, similar to how an accelerometer signal can be decomposed by a Fourier or wavelet transform. These spatial amplitude and phase signals from all the images

Experimental setup

In order to validate the motion magnification algorithm applied to videos of vibrating structures for the measurement of displacements and operational deflection shapes, an experiment was formulated to compare the results to standard sensors. An accelerometer was mounted on the free end of a steel cantilever beam, and the motion of the accelerometer was simultaneously measured by the accelerometer itself, a laser vibrometer, and a high speed camera, as shown in Fig. 3(a). A screenshot of the

Experimental setup

To test the camera as a sensor for determining the operational deflection shapes of structures, an experiment measuring a steel cantilever beam was formulated. The cantilever beam was instrumented with nine accelerometers so that the operational deflection shapes calculated from the accelerometer data could be compared with those extracted from the high-speed camera video. The beam was excited with an impact hammer near the base and the subsequent vibration was measured by the camera and

Experimental setup

An experiment similar to the instrumented cantilever beam test was performed on a pipe. A section of 4 in schedule 40 PVC pipe was placed on the end of a laboratory bench and held down by hand as shown in Fig. 11(a). The pipe was struck on the free end by an impact hammer as an excitation and the resulting vibrations of the pipe as viewed in the cross-section were recorded with a high-speed camera as shown in Fig. 11(b). The camera recorded video at 20,000 frames per second, with a frame size of

Conclusion

In this paper we have demonstrated motion magnification for extracting displacements from high speed video and demonstrated the algorithm׳s capability of qualitatively identifying the operational deflection shapes of a cantilever beam and a pipe cross section from a video. A verification measurement on a cantilever beam was made and the displacement extracted closely matched those measured by a laser vibrometer. The resulting noise floor of the camera was approximately $1 \times 10^{- 5}$ pixels per root

Acknowledgments

The authors acknowledge the support provided by Royal Dutch Shell through the MIT Energy Initiative, and thank chief scientists Dr. Dirk Smit and Dr. Sergio Kapusta, project managers Dr. Yile Li and Dr. Keng Yap, and Shell-MIT liaison Dr. Jonathan Kane for their oversight of this work. We also acknowledge Draper Laboratory for providing experimental equipment. At the time of this work, Neal Wadhwa was supported by the DoD through the NDSEG fellowship program. Special thanks are due to Reza

References (26)

O. Cakar et al.
Elimination of transducer mass loading effects from frequency response functions
Mechanical Systems and Signal Processing
(2005)
C.R. Farrar et al.
Microwave interferometers for non-contact vibration measurements on large structures
Mechanical Systems and Signal Processing
(1999)
J.J. Lee et al.
A vision-based system for remote sensing of bridge displacement
NDT&E International
(2006)
M.N. Helfrick et al.
3D digital image correlation methods for full-field vibration measurement
Mechanical Systems and Signal Processing
(2011)
L. Hermans et al.
Modal testing and analysis of structures under operational conditions: industrial applications
Mechanical Systems and Signal Processing
(1999)
B. Horn et al.
Determining optical flow
Artificial Intelligence
(1981)
K. Worden
Data processing and experiment design for the restoring force surface method, Part I: Integration and differentiation of measured time data
Mechanical Systems and Signal Processing
(1990)
M. Ashory, Correction of mass-loading effects of transducers and suspension effects in modal testing, Proceedings of...
A. Stanbridge et al.
Modal testing using impact excitation and a scanning LDV
Shock and Vibration
(2000)
H. Van der Auweraer et al.
Application of stroboscopic and pulsed-laser electronic speckle pattern interferometry (ESPI) to modal analysis problems
Measurement Science and Technology
(2002)

S. Patsias et al.

Damage detection using optical measurements and wavelets

Structural Health Monitoring

(2002)

A.M. Wahbeh et al.

A vision-based approach for the direct measurement of displacements in vibrating systems

Smart Materials and Structures

(2003)

E. Caetano et al.

A vision system for vibration monitoring of civil engineering structures

Experimental Techniques

(2011)

Cited by (433)

Towards vision-based structural modal identification at low frame rate using blind source separation
2024, Journal of Infrastructure Intelligence and Resilience
With increasing availability of cost-effective and high-resolution cameras, their use as a non-contact sensing tool has rapidly progressed for structural health monitoring. The cameras offer unique capabilities to provide full-field measurement with high spatial density at low cost. However, extracting high-density temporal data is challenging, as a high-speed camera increases the monitoring cost with high-rate data processing. Recently, motion magnification (MM) has shown significant success in analyzing low-amplitude motion of structural systems. However, previous studies observed that MM methodology performs poorly at low frame rates for modal identifications. In this paper, the influence of low frame rate on phased-based motion magnification (PMM) has been investigated. A novel technique is proposed by combining PMM with zero mean-normalization cross-correlation tracker to determine vibrational responses, and then the spatial Wigner-Ville spectrum-based time-frequency blind source separation method is explored for modal identification using the extracted vibrational responses obtained from the video data. The experimental data of a lumped mass experimental model and a steel bridge is used to test the accuracy of the proposed method. The original and motion-magnified image response data is compared with accelerometer data for modal identification. The proposed method is able to extract the modal parameters with high accuracy for motion-magnified images, even for low frame rates.
Mode-shape magnification in high-speed camera measurements
2024, Mechanical Systems and Signal Processing
If motion, identified using image-based methods, is too small to be seen with the naked eye, motion magnification can be used to help with the visualization. Established motion-magnification methods magnify (typically up to 1000 times) the band-passed content of the image data. Especially at higher frequencies, the amplitudes of measured displacements are often below the noise floor. In this research, a novel method for amplifying vibrations, based on experimental modal analysis (EMA), is introduced. The response of the examined structure to dynamic excitation is measured with a simplified, gradient-based, optical flow method and used to perform a hybrid modal analysis in conjunction with a reference accelerometer response measurement. Such a hybrid approach can: (a) identify structural dynamics significantly in the sub-pixel range, and (b) significantly below the image noise floor. The image of the vibrating structure is subdivided using a planar triangle mesh, which is then warped in accordance with the identified mode shape. A mesh-element-wise affine transformation is performed to obtain an image of the magnified mode shape. In the experimental part, the proposed method achieved magnification factors of approximately 40 thousand times, which is an order of magnitude deeper into noise than available before; additionally, the proposed approach is numerically significantly less demanding.
The introduced mode-shape magnification presents an alternative to existing motion-magnification methods for applications where the harmonic displacement information is hidden by image noise.
Deep learning-based structural health monitoring
2024, Automation in Construction
This article provides a comprehensive review of deep learning-based structural health monitoring (DL-based SHM). It encompasses a broad spectrum of DL theories and applications including nondestructive approaches; computer vision-based methods, digital twins, unmanned aerial vehicles (UAVs), and their integration with DL; vibration-based strategies including sensor fault and data recovery methods; and physics-informed DL approaches. Connections between traditional machine learning and DL-based methods as well as relations of local to global approaches including their extensive integrations are established. The state-of-the-art methods, including their advantages and limitations are presented. The review draws on current literature on the topic, also providing a synergistic analysis leading to the understanding of the evolution of DL as a basis for presenting the future research and development needs. Our overall finding is that despite the rapid progression of digital technology along with the progression of DL, the DL-based SHM appears to be in its infant stages with enormous potential for future developments to bring the SHM technology to a common practical use with wide scope applications, performance reliability, cost, and degree of automation. It is anticipated that this review paper will serve as a basic resource for readers seeking comprehensive and holistic understanding of the subject matter.
A quantitative comparison study for structural flexibility identification using Accelerometric and computer vision-based vibration data
2024, Journal of Sound and Vibration
For civil engineering structures, the dynamics-based structural health monitoring is commonly based on inverse analysis of structural acceleration response data to identify the structural parameters. As the measurement of structural displacement response becomes more feasible and cost-effective thanks to advances in computer vision techniques, a comparison of the displacement and acceleration response data for structural identification has become of considerable interest among researchers. This paper presents a comprehensive analysis and comparison of structural flexibility identification based on displacement versus acceleration response data through uncertainty quantification technique. Firstly, theoretical derivation reveals that the energy spectral density of acceleration response is equal to the fourth power of the frequency of the energy spectral density of displacement response, which affects the distribution of measurement noise across different frequency bands. Secondly, the uncertainty quantification method for modal flexibility from subspace-based system identification is utilized to compare the derivation process of covariance estimation on modal flexibility. Finally, numerical analysis and experimental example are carried out to compare the structural flexibility identification results when using the displacement versus acceleration. It shows that more precise identification results of lower-order modal parameters are obtained when using displacement, which further improves the precision of modal flexibility identification. The displacement is more suitable for identifying the modal flexibility when the measurement noises are at the same level.
Rapid full-field deformation measurements of tall buildings using UAV videos and deep learning
2024, Engineering Structures
Full-field deformation of tall buildings is informative and imperative for structural health condition assessments. This study explores the feasibility of using videos recorded by an unmanned aerial vehicle (UAV) to measure the vibration profiles of tall buildings with the aid of deep neural networks. First, a Transformer network with a self-attention mechanism is used, and its outcome is employed for the homography transformation to minimize the drifting problems induced by the UAV. Second, a boundary-aware salient object detection network is adopted to extract the boundary of the buildings in the stabilized videos for further full-field deformation measurements. The accuracy of the proposed method is experimentally validated using a wood frame structure under harmonic excitations. Under a measuring distance of approximately 4.2 m, the proposed method achieves averages of root-mean-square-errors (RMSEs) for the displacement and rotation measurements of 2.13 mm and 1.74 × 10⁻³ rad, respectively, as compared to that determined by an infrared light tracking system. The study further analyzes the field-test data recorded from the vibration of a 247-m tower. Under a measuring distance of about 55 m, the proposed method is comparable to the vibration measurement at a specific story and estimates the full-field deformation within the region of interest. Overall, the proposed method with rapid UAV deployment is feasible and accurate for monitoring the full-field deformation using the UAV-recorded videos.
Fast and accurate visual vibration measurement via derivative-enhanced phase-based optical flow
2024, Mechanical Systems and Signal Processing
Visual vibration measurement using a digital camera is an emerging non-contact modality capable of enabling flexibility, high spatial resolution, and full field. However, the massive amounts of video data with vibration information (thousands of time-lapse images per measurement) to be handled lead to an actual bottleneck. This prevents the system from extracting the vibration information with high speed and fidelity. Here, a straightforward phase-based optical flow approach is proposed by introducing a derivative operation to perform vibration measurements in a more accurate and efficient manner, in contrast to the state-of-the-art phase-based visual method. This approach generates analytical signals by considering each raw gray image as the real part and its first-order derivative as the imaginary part. As a result, the physical vibration can be extracted from the phase variations of the analytical signals from one to the next. Numerical simulations and experiments are conducted to validate the effectiveness and accuracy of this simple but powerful approach. The results indicate that the proposed approach provides a significantly higher speed than current methods and a correlation coefficient of 99.2 % with the identification results.

View all citing articles on Scopus

View full text

Modal identification of simple structures with high-speed video using motion magnification

Abstract

Introduction

Section snippets

Theory and methods

Experimental setup

Experimental setup

Experimental setup

Conclusion

Acknowledgments

Mechanical Systems and Signal Processing

Mechanical Systems and Signal Processing

NDT&E International

Mechanical Systems and Signal Processing

Mechanical Systems and Signal Processing

Artificial Intelligence

Mechanical Systems and Signal Processing

Modal testing using impact excitation and a scanning LDV

Shock and Vibration

Application of stroboscopic and pulsed-laser electronic speckle pattern interferometry (ESPI) to modal analysis problems

Measurement Science and Technology

Damage detection using optical measurements and wavelets

Structural Health Monitoring

A vision-based approach for the direct measurement of displacements in vibrating systems

Smart Materials and Structures

A vision system for vibration monitoring of civil engineering structures

Experimental Techniques