Skip to main content
SpringerLink
Log in

Deep learning for complex displacement field measurement

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Traction force microscopy (TFM) is one of the most successful and broadly-used force probing technologies to quantify the mechanical forces in living cells. The displacement recovery of the fluorescent beads within the gel substrate, which serve as the fiducial markers, is one of the key processes. The traditional methods of extracting beads displacements, such as PTV, PIV, and DIC, persistently suffer from mismatching and loss of high-frequency information while dealing with the complex deformation around the focal adhesions. However, this information is crucial for the further analysis since the cells mainly transmit the force to the extracellular surroundings through focal adhesions. In this paper, we introduced convolutional neural network (CNN) to solve the problem. We have generated the fluorescent images of the non-deformable fluorescent beads and the displacement fields with different spatial complexity to form the training dataset. Considering the special image feature of the fluorescent images and the deformation with high complexity, we have designed a customized network architecture called U-DICNet for the feature extraction and displacement estimation. The numerical simulation and real experiment show that U-DICNet outperforms the traditional methods (PTV, PIV, and DIC). Particularly, the proposed U-DICNet obtains a more reliable result for the analysis of the local complex deformation around the focal adhesions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Trepat X, Wasserman M R, Angelini T E, et al. Physical forces during collective cell migration. Nat Phys, 2009, 5: 426–430

    Article  Google Scholar 

  2. Plotnikov S V, Pasapera A M, Sabass B, et al. Force fluctuations within focal adhesions mediate ECM-rigidity sensing to guide directed cell migration. Cell, 2012, 151: 1513–1527

    Article  Google Scholar 

  3. Iskratsch T, Wolfenson H, Sheetz M P. Appreciating force and shape —the rise of mechanotransduction in cell biology. Nat Rev Mol Cell Biol, 2014, 15: 825–833

    Article  Google Scholar 

  4. Colin-York H, Shrestha D, Felce J H, et al. Super-resolved traction force microscopy (STFM). Nano Lett, 2016, 16: 2633–2638

    Article  Google Scholar 

  5. Colin-York H, Fritzsche M. The future of traction force microscopy. Curr Opin BioMed Eng, 2018, 5: 1–5

    Article  Google Scholar 

  6. Barbieri L, Colin-York H, Korobchevskaya K, et al. Two-dimensional TIRF-SIM-traction force microscopy (2D TIRF-SIM-TFM). Nat Commun, 2021, 12: 1–4

    Article  Google Scholar 

  7. Bergert M, Lendenmann T, Zündel M, et al. Confocal reference free traction force microscopy. Nat Commun, 2016, 7: 1

    Article  Google Scholar 

  8. Kasagi N, Matsunaga A. Three-dimensional particle-tracking velocimetry measurement of turbulence statistics and energy budget in a backward-facing step flow. Int J Heat Fluid Flow, 1995, 16: 477–485

    Article  Google Scholar 

  9. Sbalzarini I F, Koumoutsakos P. Feature point tracking and trajectory analysis for video imaging in cell biology. J Struct Biol, 2005, 151: 182–195

    Article  Google Scholar 

  10. Willert C E, Gharib M. Digital particle image velocimetry. Exp Fluids, 1991, 10: 181–193

    Article  Google Scholar 

  11. Chu T C, Ranson W F, Sutton M A. Applications of digital-image-correlation techniques to experimental mechanics. Exp Mech, 1985, 25: 232–244

    Article  Google Scholar 

  12. Huang J, Pan X, Peng X, et al. High-efficiency cell-substrate displacement acquisition via digital image correlation method using basis functions. Opt Lasers Eng, 2010, 48: 1058–1066

    Article  Google Scholar 

  13. Zhang X, Chu X, Ji H, et al. Effect of freezing rate on the onion cell deformation evaluated by digital image correlation. Food Anal Methods, 2016, 9: 3125–3132

    Article  Google Scholar 

  14. Gao Y, Cheng T, Su Y, et al. High-efficiency and high-accuracy digital image correlation for three-dimensional measurement. Opt Lasers Eng, 2015, 65: 73–80

    Article  Google Scholar 

  15. Yuan Y, Huang J, Peng X, et al. Accurate displacement measurement via a self-adaptive digital image correlation method based on a weighted ZNSSD criterion. Opt Lasers Eng, 2014, 52: 75–85

    Article  Google Scholar 

  16. Huang J, Pan X, Li S, et al. A digital volume correlation technique for 3-D deformation measurements of soft gels. Int J Appl Mech, 2011, 03: 335–354

    Article  Google Scholar 

  17. Liu Y, Wang J, Su Y, et al. Quantifying 3D cell-matrix interactions during mitosis and the effect of anticancer drugs on the interactions. Nano Res, 2021, 14: 4163–4172

    Article  Google Scholar 

  18. Patel M, Leggett S E, Landauer A K, et al. Rapid, topology-based particle tracking for high-resolution measurements of large complex 3D motion fields. Sci Rep, 2018, 8: 5581

    Article  Google Scholar 

  19. Nogueira J, Lecuona A, Rodríguez P A. Local field correction PIV: On the increase of accuracy of digital PIV systems. Exp Fluids, 1999, 27: 107–116

    Article  Google Scholar 

  20. Schanz D, Gesemann S, Schröder A. Shake-the-box: Lagrangian particle tracking at high particle image densities. Exp Fluids, 2016, 57: 1–27

    Article  Google Scholar 

  21. Schreier H W, Sutton M A. Systematic errors in digital image correlation due to undermatched subset shape functions. Exp Mech, 2002, 42: 303–310

    Article  Google Scholar 

  22. Xu X, Su Y, Cai Y, et al. Effects of various shape functions and subset size in local deformation measurements using DIC. Exp Mech, 2015, 55: 1575–1590

    Article  Google Scholar 

  23. Sabass B, Gardel M L, Waterman C M, et al. High resolution traction force microscopy based on experimental and computational advances. Biophysl J, 2008, 94: 207–220

    Article  Google Scholar 

  24. Quan T, Zhu H, Liu X, et al. High-density localization of active molecules using structured sparse model and Bayesian information criterion. Opt Express, 2011, 19: 16963–16974

    Article  Google Scholar 

  25. Ovesný M, Křížek P, Švindrych Z, et al. High density 3D localization microscopy using sparse support recovery. Opt Express, 2014, 22: 31263–31276

    Article  Google Scholar 

  26. Bar-Kochba E, Toyjanova J, Andrews E, et al. A fast iterative digital volume correlation algorithm for large deformations. Exp Mech, 2015, 55: 261–274

    Article  Google Scholar 

  27. Yang J, Huang J, Jiang Z, et al. SIFT-aided path-independent digital image correlation accelerated by parallel computing. Opt Lasers Eng, 2020, 127: 105964

    Article  Google Scholar 

  28. Zhao L, Wang Q, Zhou P, et al. Digital image correlation method based on surf in airship envelope measurement. In: Proceedings of the International Conference on Aerospace System Science and Engineering 2018. Singapore: Springer, 2018. 105–120

    Google Scholar 

  29. Li W, Li Y, Liang J. Enhanced feature-based path-independent initial value estimation for robust point-wise digital image correlation. Opt Lasers Eng, 2019, 121: 189–202

    Article  Google Scholar 

  30. Stubb A, Laine R F, Miihkinen M, et al. Fluctuation-based superresolution traction force microscopy. Nano Lett, 2020, 20: 2230–2245

    Article  Google Scholar 

  31. Ferrari A. Recent technological advancements in traction force microscopy. Biophys Rev, 2019, 11: 679–681

    Article  Google Scholar 

  32. Li D, Colin-York H, Barbieri L, et al. Astigmatic traction force microscopy (aTFM). Nat Commun, 2021, 12: 2168

    Article  Google Scholar 

  33. Gustafsson N, Culley S, Ashdown G, et al. Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nat Commun, 2016, 7: 1–9

    Article  Google Scholar 

  34. Dertinger T, Colyer R, Iyer G, et al. Fast, background-free, 3D superresolution optical fluctuation imaging (SOFI). Proc Natl Acad Sci USA, 2009, 106: 22287–22292

    Article  Google Scholar 

  35. Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 2481–2495

    Article  Google Scholar 

  36. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Springer International Publishing, 2015. 234–241

  37. Zhang Y, Tian Y, Kong Y, et al. Residual dense network for image super-resolution. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 2472–2481

  38. Dong C, Loy C C, He K, et al. Learning a deep convolutional network for image super-resolution. In: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW). Santiago, 2014. 184–199

  39. Turkoglu M O, Thong W, Spreeuwers L, et al. A layer-based sequential framework for scene generation with gans. AAAI, 2019, 33: 8901–8908

    Article  Google Scholar 

  40. Dosovitskiy A, Fischer P, Ilg E, et al. FlowNet: Learning optical flow with convolutional networks. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, 2015. 2758–2766

  41. Ilg E, Mayer N, Saikia T, et al. FlowNet 2.0: Evolution of optical flow estimation with deep networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, 2017. 1647–1655

  42. Sun D, Yang X, Liu M Y, et al. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 8934–8943

  43. Xu J, Ranftl R, Koltun V. Accurate optical flow via direct cost volume processing. In: Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Honolulu, 2017. 5807–5815

  44. Lagemann C, Lagemann K, Mukherjee S, et al. Deep recurrent optical flow learning for particle image velocimetry data. Nat Mach Intell, 2021, 3: 641–651

    Article  Google Scholar 

  45. Teed Z, Deng J. RAFT: Recurrent all-pairs field transforms for optical flow. In: Proceedings of the European Conference on Computer Vision. Cham: Springer, 2020. 402–419

    Google Scholar 

  46. Cai S, Liang J, Gao Q, et al. Particle image velocimetry based on a deep learning motion estimator. IEEE Trans Instrum Meas, 2020, 69: 3538–3554

    Article  Google Scholar 

  47. Hui T W, Tang X O, Loy C C. LiteFlowNet: A lightweight convolutional neural network for optical flow estimation. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 8981–8989

  48. Boukhtache S, Abdelouahab K, Berry F, et al. When deep learning meets digital image correlation. Opt Lasers Eng, 2021, 136: 106308

    Article  Google Scholar 

  49. Su Y, Gao Z, Fang Z, et al. Theoretical analysis on performance of digital speckle pattern: Uniqueness, accuracy, precision, and spatial resolution. Opt Express, 2019, 27: 22439–22474

    Article  Google Scholar 

  50. Mazzoleni P, Matta F, Zappa E, et al. Gaussian pre-filtering for uncertainty minimization in digital image correlation using numerically-designed speckle patterns. Opt Lasers Eng, 2015, 66: 19–33

    Article  Google Scholar 

  51. Lan S, Gao Y, Xu X, et al. Error analysis of surface-distribution and non-deformation of fluorescent beads for the IC-GN2 DVC algorithm. Opt Lasers Eng, 2021, 140: 106541

    Article  Google Scholar 

  52. Su Y, Zhang Q. Glare: A free and open-source software for generation and assessment of digital speckle pattern. Opt Lasers Eng, 2022, 148: 106766

    Article  Google Scholar 

  53. Thielicke W, Sonntag R. Particle image velocimetry for MATLAB: Accuracy and enhanced algorithms in PIVlab. J Open Res Software, 2021, 9: 12

    Article  Google Scholar 

  54. Thielicke W, Stamhuis E J. PIVlab—Towards user-friendly, affordable and accurate digital particle image velocimetry in MATLAB. J Open Res Software, 2014, 2: 12

    Article  Google Scholar 

  55. Thielicke W. The flapping flight of birds. Dissertation for Doctoral Degree. Groningen: University of Groningen, 2014

    Google Scholar 

  56. Zhang L, Wang T, Jiang Z, et al. High accuracy digital image correlation powered by GPU-based parallel computing. Opt Lasers Eng, 2015, 69: 7–12

    Article  Google Scholar 

  57. Jiang Z, Kemao Q, Miao H, et al. Path-independent digital image correlation with high accuracy, speed and robustness. Opt Lasers Eng, 2015, 65: 93–102

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CAS Key Laboratory of Mechanical Behavior and Design of Materials, Department of Modern Mechanics, University of Science and Technology of China, Hefei, 230027, China

    ShiHai Lan, Yong Su, ZeRen Gao, Ye Chen, Han Tu & QingChuan Zhang

  2. Fuhuang Agile Device, Inc., Hefei, 230000, China

    ShiHai Lan & QingChuan Zhang

Authors
  1. ShiHai Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. ZeRen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ye Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Han Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. QingChuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to QingChuan Zhang.

Additional information

This work was supported by the National Natural Science Foundation of China (Grant Nos. 11872354, 11627803, and 12102423), and the National Science and Technology Major Project (Grant No. J2019-V-0006-0100).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lan, S., Su, Y., Gao, Z. et al. Deep learning for complex displacement field measurement. Sci. China Technol. Sci. 65, 3039–3056 (2022). https://doi.org/10.1007/s11431-022-2122-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-022-2122-y

Keywords

Search

Navigation