Skip to main content
SpringerLink
Log in

A lightweight convolutional neural network for multipoint displacement measurements on bridge structures

  • Review
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Emerging computer vision measurement methods overcome the limitations of traditional sensors. However, conventional digital image processing algorithms encounter issues of low precision and timeliness when performing vibration displacement measurements in complex operating conditions. Therefore, this paper builds a deep learning detection algorithm based on a lightweight convolutional neural network to measure the vibration displacement of the structure. In the feature extraction part, the ghost module and depthwise separable convolution are used to condense effective information. In the feature fusion part, it is combined based on the underlying feature similarity principle. The attention mechanism obtains the fine features, edge information and sensitive position information of the target. In the target regression part, preset anchor frames of different sizes are used to return the vibrating targets in the image to obtain precise position information. First, a cable-stayed bridge model is selected for experiments in a laboratory environment, and compared with the measurement results of seven visual algorithms. Furthermore, experiments are conducted outdoors on real bridges to verify the effectiveness of the proposed method. Qualitative and quantitative analysis of the experimental results show that the proposed algorithm can effectively return the small offset of the vibration target, efficiently and precisely measure the vibration displacement signal of the structure in real time, providing an efficient solution for structural damage and health monitoring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

The datasets generated during the current study are available from the corresponding author on reasonable request.

References

  1. Yang, R., Wang, S., Wu, X., Liu, T., Liu, X.: Using lightweight convolutional neural network to track vibration displacement in rotating body video. Mech. Syst. Signal Process. 177, 109137 (2022)

    Article  Google Scholar 

  2. Shen, N., Chen, L., Lu, X., Hu, H., Pan, Y., Gao, Z., Chen, R.: Online displacement extraction and vibration detection based on interactive multiple model algorithm. Mech. Syst. Signal Process. 155, 107581 (2021)

    Article  Google Scholar 

  3. Luo, L., Feng, M.Q., Wu, J., Bi, L.: Modeling and detection of heat haze in computer vision based displacement measurement. Measurement 182, 109772 (2021)

    Article  Google Scholar 

  4. Xu, Y., Brownjohn, J.M.: Review of machine-vision based methodologies for displacement measurement in civil structures. J. Civ. Struct. Health Monit. 8, 91–110 (2018)

    Article  Google Scholar 

  5. Avci, O., Abdeljaber, O., Kiranyaz, S., Hussein, M., Gabbouj, M., Inman, D.J.: A review of vibration-based damage detection in civil structures: from traditional methods to Machine Learning and Deep Learning applications. Mech. Syst. Signal Process. 147, 107077 (2021)

    Article  Google Scholar 

  6. Gao, Y., Karimi, M., Kudreyko, A.A., Song, W.: Spare optimistic based on improved ADMM and the minimum entropy de-convolution for the early weak fault diagnosis of bearings in marine systems. ISA Trans. 78, 98–104 (2018)

    Article  Google Scholar 

  7. Feng, D., Feng, M.Q.: Computer vision for SHM of civil infrastructure: from dynamic response measurement to damage detection–a review. Eng. Struct. 156, 105–117 (2018)

    Article  Google Scholar 

  8. Feng, D., Feng, M.Q.: Experimental validation of cost-effective vision-based structural health monitoring. Mech. Syst. Signal Process. 88, 199–211 (2017)

    Article  Google Scholar 

  9. Shen, M., Yang, J., Jiang, W., Sanjuán, M.A., Zheng, Y.: Stochastic resonance in image denoising as an alternative to traditional methods and deep learning. Nonlinear Dyn. 109(3), 2163–2183 (2022)

    Article  Google Scholar 

  10. Wang, M., Ao, W.K., Bownjohn, J., et al.: Completely non-contact modal testing of full-scale bridge in challenging conditions using vision sensing systems. Eng. Struct. 272, 114994 (2022)

    Article  Google Scholar 

  11. Martini, A., Tronci, E.M., Feng, M.Q., et al.: A computer vision-based method for bridge model updating using displacement influence lines. Eng. Struct. 259, 114129 (2022)

    Article  Google Scholar 

  12. Jana, D., Nagarajaiah, S., Yang, Y.: Computer vision-based real-time cable tension estimation algorithm using complexity pursuit from video and its application in Fred-Hartman cable-stayed bridge. Struct. Control. Health Monit. 29(9), e2985 (2022)

    Article  Google Scholar 

  13. Xu, Y., Zhang, J.: UAV-based bridge geometric shape measurement using automatic bridge component detection and distributed multi-view reconstruction. Autom. Constr. 140, 104376 (2022)

    Article  Google Scholar 

  14. Xiu, C., Weng, Y., Shi, W.: Vision and vibration data fusion-based structural dynamic displacement measurement with test validation. Sensors 23(9), 4547 (2023)

    Article  Google Scholar 

  15. Xiao-wei, Y.E., Chuan-zhi, D.O.N.G.: Review of computer vision-based structural displacement monitoring. China J. Highw. Transp. 32(11), 21 (2019)

    Google Scholar 

  16. Park, J.W., Moon, D.S., Yoon, H., Gomez, F., Spencer, B.F., Jr., Kim, J.R.: Visual–inertial displacement sensing using data fusion of vision-based displacement with acceleration. Struct. Control. Health Monit. 25(3), e2122 (2018)

    Article  Google Scholar 

  17. Min, R., Liu, Z., Pereira, L., Yang, C., Sui, Q., Marques, C.: Optical fiber sensing for marine environment and marine structural health monitoring: a review. Opt. Laser Technol. 140, 107082 (2021)

    Article  Google Scholar 

  18. Janeliukstis, R., Chen, X.: Review of digital image correlation application to large-scale composite structure testing. Compos. Struct. 271, 114143 (2021)

    Article  Google Scholar 

  19. Vicente, M.A., Gonzalez, D.C., Minguez, J., Schumacher, T.: A novel laser and video-based displacement transducer to monitor bridge deflections. Sensors 18(4), 970 (2018)

    Article  Google Scholar 

  20. Dong, C.Z., Ye, X.W., Jin, T.: Identification of structural dynamic characteristics based on machine vision technology. Measurement 126, 405–416 (2018)

    Article  Google Scholar 

  21. Ye, X.W., Dong, C.Z., Liu, T.: Image-based structural dynamic displacement measurement using different multi-object tracking algorithms. Smart Struct. Syst. Int. J. 17(6), 935–956 (2016)

    Article  Google Scholar 

  22. Shao, Y., Li, L., Li, J., An, S., Hao, H.: Computer vision based target-free 3D vibration displacement measurement of structures. Eng. Struct. 246, 113040 (2021)

    Article  Google Scholar 

  23. Ye, X.W., Yi, T.H., Dong, C.Z., Liu, T.: Vision-based structural displacement measurement: system performance evaluation and influence factor analysis. Measurement 88, 372–384 (2016)

    Article  Google Scholar 

  24. Lu, W., Cui, Y., Teng, J.: Structural displacement and strain monitoring based on the edge detection operator. Adv. Struct. Eng. 20(2), 191–201 (2017)

    Article  Google Scholar 

  25. Dizaji, M.S., Harris, D.K., Kassner, B., Hill, J.C.: Full-field non-destructive image-based diagnostics of a structure using 3D digital image correlation and laser scanner techniques. J. Civ. Struct. Health Monit. 11(5), 1415–1428 (2021)

    Article  Google Scholar 

  26. Liu, P., Zhang, L., Wang, M.: Measurement of large-sized-pipe diameter based on stereo vision. Appl. Sci. 12(10), 5277 (2022)

    Article  Google Scholar 

  27. Abe, S.: Fuzzy support vector machines for multilabel classification. Pattern Recogn. 48(6), 2110–2117 (2015)

    Article  Google Scholar 

  28. Tian, Y., Zhang, J., Yu, S.: Rapid impact testing and system identification of footbridges using particle image velocimetry. Comput. Aided Civ. Infrastruct. Eng. 34(2), 130–145 (2019)

    Article  Google Scholar 

  29. Bhowmick, S., Nagarajaiah, S.: Spatiotemporal compressive sensing of full-field Lagrangian continuous displacement response from optical flow of edge: identification of full-field dynamic modes. Mech. Syst. Signal Process. 164, 108232 (2022)

    Article  Google Scholar 

  30. Khaloo, A., Lattanzi, D.: Pixel-wise structural motion tracking from rectified repurposed videos. Struct. Control. Health Monit. 24(11), e2009 (2017)

    Article  Google Scholar 

  31. Wang, W., Li, X., Ahmat, Y., Hu, X., Chen, A.: Vibration measurement method based on point tracking for irregular structures. Optik 176, 482–490 (2019)

    Article  Google Scholar 

  32. Kumar, A.K., Ngọc Mai, N., Guo, S., Han, L.: Entanglement inspired approach for determining the preeminent arrangement of static cameras in a multi-view computer vision system. Vis. Comput. 39, 1–17 (2022)

    Google Scholar 

  33. Xu, Y., Zhang, J., Brownjohn, J.: An accurate and distraction-free vision-based structural displacement measurement method integrating Siamese network based tracker and correlation-based template matching. Measurement 179, 109506 (2021)

    Article  Google Scholar 

  34. Huang, M., Zhang, B., Lou, W., Kareem, A.: A deep learning augmented vision-based method for measuring dynamic displacements of structures in harsh environments. J. Wind Eng. Ind. Aerodyn. 217, 104758 (2021)

    Article  Google Scholar 

  35. Chai, S., Wang, S., Liu, C., Liu, T., Liu, X., Xing, K.: Using semantic segmentation network to measure vibration displacement of rotating body. IEEE Sens. J. 23, 7977–7991 (2023)

    Article  Google Scholar 

  36. Zhu, Y., Wang, S., Zhang, Y., He, Z., Wang, Q.: A dual transformer super-resolution network for improving the definition of vibration image. IEEE Trans. Instrum. Meas. 72, 1–12 (2022)

    Google Scholar 

  37. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: Ghostnet: more features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)

  38. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

  39. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13713–13722 (2021)

  40. Jocher, G.: Yolov5 release v6.2 (2022). https://github.com/ultralytics/yolov5/releases/tag/v6.2

  41. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  42. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759–8768 (2018)

  43. Wang, C.Y., Liao, H.Y.M., Wu, Y.H., Chen, P.Y., Hsieh, J.W., Yeh, I.H.: CSPNet: a new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391 (2020)

  44. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  45. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Wei, X.: YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications (2022). arXiv preprint arXiv:2209.02976

  46. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors (2022). arXiv preprint arXiv:2207.02696

  47. Hisham, M.B., Yaakob, S.N., Raof, R.A.A., Nazren, A.A., Wafi, N.M. Template matching using sum of squared difference and normalized cross correlation. In: 2015 IEEE Student Conference on Research and Development (SCOReD), pp. 100–104. IEEE (2015)

  48. Yao, S., Han, X., Zhang, H., Wang, X., Cao, X.: Learning deep Lucas–Kanade Siamese network for visual tracking. IEEE Trans. Image Process. 30, 4814–4827 (2021)

    Article  Google Scholar 

  49. Jocher, G.: Ultralytics YOLOv8 (2022). https://github.com/ultralytics/ultralytics

  50. Lin, Z., Wang, Y., Zhang, J., Chu, X.: DynamicDet: a unified dynamic architecture for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

  51. Yu, J., Jiang, Y., Wang, Z., Cao, Z., Huang, T. Unitbox: an advanced object detection network. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 516–520 (2016)

  52. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S. Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 658–666 (2019)

  53. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D. Distance-IoU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12993–13000 (2020)

  54. Zheng, Z., Wang, P., Ren, D., Liu, W., Ye, R., Hu, Q., Zuo, W.: Enhancing geometric factors in model learning and inference for object detection and instance segmentation. IEEE Trans. Cybern. 52(8), 8574–8586 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 52065035, the Major Science and Technology Project in Yunnan Province under 202202AC080003.

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming, China

    Sen Wang, Sen Lin & Rongliang Yang

Authors
  1. Sen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rongliang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rongliang Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Lin, S. & Yang, R. A lightweight convolutional neural network for multipoint displacement measurements on bridge structures. Nonlinear Dyn (2024). https://doi.org/10.1007/s11071-024-09673-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11071-024-09673-x

Keywords

Search

Navigation