Skip to main content
SpringerLink
Log in

Knowledge Distillation with Segment Anything (SAM) Model for Planetary Geological Mapping

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14505))

Abstract

Planetary science research involves analysing vast amounts of remote sensing data, which are often costly and time-consuming to annotate and process. One of the essential tasks in this field is geological mapping, which requires identifying and outlining regions of interest in planetary images, including geological features and landforms. However, manually labelling these images is a complex and challenging task that requires significant domain expertise and effort. To expedite this endeavour, we propose the use of knowledge distillation using the recently introduced cutting-edge Segment Anything (SAM) model. We demonstrate the effectiveness of this prompt-based foundation model for rapid annotation and quick adaptability to a prime use case of mapping planetary skylights. Our work reveals that with a small set of annotations obtained with the right prompts from the model and subsequently training a specialised domain decoder, we can achieve performance comparable to state of the art on this task. Key results indicate that the use of knowledge distillation can significantly reduce the effort required by domain experts for manual annotation and improve the efficiency of image segmentation tasks. This approach has the potential to accelerate extra-terrestrial discovery by automatically detecting and segmenting Martian landforms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    “High-Resolution Imaging Science Experiment” is camera aboard the Mars Reconnaissance Orbiter (MRO) spacecraft, which is designed to capture high-resolution images of the Martian surface and provide detailed information about the planet’s geology and atmosphere.

  2. 2.

    Text prompt is currently not released.

  3. 3.

    https://ode.rsl.wustl.edu/.

  4. 4.

    The SAM decoder is fine-tuned via training with a set of 25 annotated images for 100 epochs.

References

  1. Allemand, P., Delacourt, C., Gasperini, D., Kasperski, J., Pothérat, P.: Thirty years of evolution of the sedrun landslide (swisserland) from multitemporal orthorectified aerial images, differential digital terrain models and field data. Int. J. Remote Sens. Appl 1, 30–36 (2011)

    Google Scholar 

  2. Baker, V.R.: Water and the martian landscape. Nature 412(6843), 228–236 (2001)

    Article  Google Scholar 

  3. Baker, V.R.: Geomorphological evidence for water on mars. Elements 2(3), 139–143 (2006)

    Article  Google Scholar 

  4. Chen, J., Bai, X.: Learning to “segment anything” in thermal infrared images through knowledge distillation with a large scale dataset satir. arXiv preprint arXiv:2304.07969 (2023)

  5. Deng, R., et al.: Segment anything model (sam) for digital pathology: assess zero-shot segmentation on whole slide imaging. arXiv preprint arXiv:2304.04155 (2023)

  6. Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  7. Gou, J., Yu, B., Maybank, S.J., Tao, D.: Knowledge distillation: a survey. Int. J. Comput. Vision 129, 1789–1819 (2021)

    Article  Google Scholar 

  8. He, S., Bao, R., Li, J., Grant, P.E., Ou, Y.: Accuracy of segment-anything model (sam) in medical image segmentation tasks. arXiv preprint arXiv:2304.09324 (2023)

  9. Hu, C., Li, X.: When sam meets medical images: an investigation of segment anything model (sam) on multi-phase liver tumor segmentation. arXiv preprint arXiv:2304.08506 (2023)

  10. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. Adv. Neural Inf. Process. Syst. 28 (2015)

    Google Scholar 

  11. Ji, G.P., Fan, D.P., Xu, P., Cheng, M.M., Zhou, B., Van Gool, L.: Sam struggles in concealed scenes-empirical study on “segment anything”. arXiv preprint arXiv:2304.06022 (2023)

  12. Ji, W., Li, J., Bi, Q., Li, W., Cheng, L.: Segment anything is not always perfect: an investigation of sam on different real-world applications. arXiv preprint arXiv:2304.05750 (2023)

  13. Jia, C., et al.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International Conference on Machine Learning, pp. 4904–4916. PMLR (2021)

    Google Scholar 

  14. Jiang, S., Lian, Z., Yung, K.L., Ip, W., Gao, M.: Automated detection of multitype landforms on mars using a light-weight deep learning-based detector. IEEE Trans. Aerosp. Electron. Syst. 58(6), 5015–5029 (2022)

    Article  Google Scholar 

  15. Julka, S.: An active learning approach for automatic detection of bow shock and magnetopause crossing signatures in mercury’s magnetosphere using messenger magnetometer observations. In: Proceedings of the 2nd Machine Learning in Heliophysics, p. 8 (2022)

    Google Scholar 

  16. Julka, S., Granitzer, M., De Toffoli, B., Penasa, L., Pozzobon, R., Amerstorfer, U.: Generative adversarial networks for automatic detection of mounds in digital terrain models (mars arabia terra). In: EGU General Assembly Conference Abstracts, pp. EGU21-9188 (2021)

    Google Scholar 

  17. Julka, S., Kirschstein, N., Granitzer, M., Lavrukhin, A., Amerstorfer, U.: Deep active learning for detection of mercury’s bow shock and magnetopause crossings. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Grenoble, France, 19–23 September 2022, Proceedings, Part IV. pp. 452–467. Springer, Heidelberg (2023). https://doi.org/10.1007/978-3-031-26412-2_28

  18. Kirillov, A., et al.: Segment anything. arXiv preprint arXiv:2304.02643 (2023)

  19. Lund, B.D., Wang, T.: Chatting about chatgpt: how may ai and gpt impact academia and libraries? Library Hi Tech News (2023)

    Google Scholar 

  20. Nodjoumi, G., Pozzobon, R., Sauro, F., Rossi, A.P.: Deeplandforms: a deep learning computer vision toolset applied to a prime use case for mapping planetary skylights. Earth Space Sci. 10(1), e2022EA002278 (2023)

    Google Scholar 

  21. Palafox, L.F., Hamilton, C.W., Scheidt, S.P., Alvarez, A.M.: Automated detection of geological landforms on mars using convolutional neural networks. Comput. Geosci. 101, 48–56 (2017)

    Article  Google Scholar 

  22. Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., Yang, D.: Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476 (2023)

  23. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)

    Google Scholar 

  24. Ramesh, A., et al.: Zero-shot text-to-image generation. In: International Conference on Machine Learning, pp. 8821–8831. PMLR (2021)

    Google Scholar 

  25. Tang, L., Xiao, H., Li, B.: Can sam segment anything? when sam meets camouflaged object detection. arXiv preprint arXiv:2304.04709 (2023)

Download references

Acknowledgements

The authors acknowledge support from Europlanet 2024 RI that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 871149.

Author information

Authors and Affiliations

  1. Chair of Data Science, University of Passau, 94036, Passau, Germany

    Sahib Julka & Michael Granitzer

Authors
  1. Sahib Julka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Granitzer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sahib Julka .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Catania, Italy

    Giuseppe Nicosia

  2. Newcastle University, Newcastle upon Tyne, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  6. Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Julka, S., Granitzer, M. (2024). Knowledge Distillation with Segment Anything (SAM) Model for Planetary Geological Mapping. In: Nicosia, G., Ojha, V., La Malfa, E., La Malfa, G., Pardalos, P.M., Umeton, R. (eds) Machine Learning, Optimization, and Data Science. LOD 2023. Lecture Notes in Computer Science, vol 14505. Springer, Cham. https://doi.org/10.1007/978-3-031-53969-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53969-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53968-8

  • Online ISBN: 978-3-031-53969-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation