Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

When seeing is not believing: application-appropriate validation matters for quantitative bioimage analysis

Nature Methods volume 20, pages 968–970 (2023)Cite this article

Subjects

A key step toward biologically interpretable analysis of microscopy image-based assays is rigorous quantitative validation with metrics appropriate for the particular application in use. Here we describe this challenge for both classical and modern deep learning-based image analysis approaches and discuss possible solutions for automating and streamlining the validation process in the next five to ten years.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Chen, J. et al. Preprint at bioRxiv https://doi.org/10.1101/491035 (2018).

  2. Viana, M. P. et al. Nature 613, 345–354 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Guan, H. & Liu, M. IEEE Trans. Biomed. Eng. 69, 1173–1185 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Yang, J., Zhou, K., Li, Y. & Liu, Z. Preprint at arXiv https://doi.org/10.48550/arXiv.2110.11334 (2021).

  5. Fernando, T., Gammulle, H., Denman, S., Sridharan, S. & Fookes, C. ACM Comput. Surv. 54, 1–37 (2022).

    Article  Google Scholar 

  6. Sander, J., de Vos, B. D. & Išgum, I. Sci. Rep. 10, 21769 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Weigert, M. et al. Nat. Methods 15, 1090–1097 (2018).

    Article  CAS  PubMed  Google Scholar 

  8. Fang, L. et al. Nat. Methods 18, 406–416 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Christiansen, E. M. et al. Cell 173, 792–803.e19 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Nat. Methods 15, 917–920 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Pachitariu, M. & Stringer, C. Nat. Methods 19, 1634–1641 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. LaChance, J. & Cohen, D. J. PLOS Comput. Biol. 16, e1008443 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Reinke, A. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.01790 (2023).

  14. Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. Nat. Methods 18, 203–211 (2021).

    Article  CAS  PubMed  Google Scholar 

  15. Gonzalez, C., Ranem, A., dos Santos, D. P., Othman, A. & Mukhopadhyay, A. Preprint at Research Square https://doi.org/10.21203/rs.3.rs-1582100/v1 (2022).

  16. Larsen, D. D., Gaudreault, N. & Gibbs, H. C. STAR Protoc. 4, 102040 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Schmied, C. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.07005 (2023).

  18. Soltwedel, J. R. & Haase, R. Preprint at Preprints.org https://doi.org/10.20944/preprints202301.0552.v1 (2023).

Download references

Acknowledgements

J.C. is supported by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) in Germany under funding reference 161L0272 and supported by the Ministry of Culture and Science of the State of North Rhine-Westphalia (Ministerium für Kultur und Wissenschaft des Landes Nordrhein-Westfalen, MKW NRW) . We acknowledge the founder of the Allen Institute for Cell Science, Paul G. Allen, for his vision, encouragement and support.

Author information

Authors and Affiliations

  1. Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V., Dortmund, Germany

    Jianxu Chen

  2. Allen Institute for Cell Science, Seattle, WA, USA

    Matheus P. Viana & Susanne M. Rafelski

Authors
  1. Jianxu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matheus P. Viana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susanne M. Rafelski
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.C., M.P.V. and S.M.R. all contributed to all aspects of this work.

Corresponding author

Correspondence to Susanne M. Rafelski.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Viana, M.P. & Rafelski, S.M. When seeing is not believing: application-appropriate validation matters for quantitative bioimage analysis. Nat Methods 20, 968–970 (2023). https://doi.org/10.1038/s41592-023-01881-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-01881-4

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing