Skip to main content
SpringerLink
Log in

How to Sort Them? A Network for LEGO Bricks Classification

  • Conference paper
  • First Online:
Computational Science – ICCS 2022 (ICCS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13352))

Included in the following conference series:

Abstract

LEGO bricks are highly popular due to the ability to build almost any type of creation. This is possible thanks to availability of multiple shapes and colors of the bricks. For the smooth build process the bricks need to properly sorted and arranged. In our work we aim at creating an automated LEGO bricks sorter. With over 3700 different LEGO parts bricks classification has to be done with deep neural networks. The question arises which model of the available should we use? In this paper we try to answer this question. The paper presents a comparison of 28 models used for image classification trained to classify objects to high number of classes with potentially high level of similarity. For that purpose a dataset consisting of 447 classes was prepared. The paper presents brief description of analyzed models, the training and comparison process and discusses the results obtained. Finally the paper proposes an answer what network architecture should be used for the problem of LEGO bricks classification and other similar problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adam: LEGO sorting chart (2019). https://go.gliffy.com/go/publish/12232322. Accessed 24 Mar 2022

  2. Aplhin, T.: The LEGO storage guide (2020). https://brickarchitect.com/guide/. Accessed 34 Mar 2022

  3. Boiński, T.: Images of LEGO bricks (2021). https://doi.org/10.34808/xz76-ez11. Accessed 24 Mar 2022

  4. Boiński, T., Zaraziński, S., Śledź, B.: LEGO bricks for training classification network (2021). https://doi.org/10.34808/3qfs-rt94. Accessed 24 Mar 2022

  5. Boiński, T., Zawora, K., Zaraziński, S., Śledź, B., Łobacz, B.: LDRAW based renders of LEGO bricks moving on a conveyor belt (2020). https://doi.org/10.34808/jykr-8d71. Accessed 24 Mar 2022

  6. Boiński, T.M.: Hierarchical 2-step neural-based LEGO bricks detection and labeling. In: Proceedings of 37th Business Information Management Association Conference, pp. 1344–1350 (2021)

    Google Scholar 

  7. Chollet, F.: Xception: Deep learning with depthwise separable convolutions (2017)

    Google Scholar 

  8. Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml. Accessed 22 Nov 2021

  9. Foundation, T.B.: Blender (2002). https://www.blender.org/. Accessed 22 Nov 2021

  10. Garcia, P.: LEGO Sorter using TensorFlow on Raspberry Pi (2018). https://medium.com/@pacogarcia3/tensorflow-on-raspbery-pi-lego-sorter-ab60019dcf32. Accessed 22 Nov 2021

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2015)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks (2016)

    Google Scholar 

  13. Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Internat. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998)

    Article  MATH  Google Scholar 

  14. Howard, A., et al.: Searching for mobilenetv3 (2019)

    Google Scholar 

  15. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017)

    Google Scholar 

  16. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely Connected Convolutional Networks (2018)

    Google Scholar 

  17. Jessiman, J.: LDraw. http://www.ldraw.org (1995). Accessed 22 Nov 2021

  18. Jung, A.: imgaug source code. https://github.com/aleju/imgaug. Accessed 22 Nov 2021

  19. Krasin, I., et al.: Openimages: a public dataset for large-scale multi-label and multi-class image classification. Dataset available from https://storage.googleapis.com/openimages/web/index.html (2017)

  20. Krizhevsky, A., Nair, V., Hinton, G.: Cifar-10 and cifar-100 datasets. https://www.cs.toronto.edu/~kriz/cifar.html. Accessed 24 Mar 2022

  21. Kuznetsova, A., et al.: The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. IJCV (2020)

    Google Scholar 

  22. Li, M., et al.: The deep learning compiler: a comprehensive survey. IEEE Trans. Parallel Distrib. Syst. 32(3), 708–727 (2021). https://doi.org/10.1109/tpds.2020.3030548

    Article  Google Scholar 

  23. Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014). http://arxiv.org/abs/1405.0312

  24. Maren, T.: 60 fun LEGO facts every LEGO fan needs to know (2018). https://mamainthenow.com/fun-lego-facts/

  25. Micikevicius, P., et al.: Mixed precision training (2018)

    Google Scholar 

  26. Nelson, T.: ImportLDRaw. https://github.com/TobyLobster/ImportLDraw. Accessed 16 June 2021

  27. Papert, S.A.: The summer vision project (1966). https://dspace.mit.edu/handle/1721.1/6125. Accessed 22 Nov 2021

  28. Le, Q., Zoph, B., Research Scientists, G.B.t.: Using machine learning to explore neural network architecture (2017). https://ai.googleblog.com/2017/05/using-machine-learning-to-explore.html. Accessed 22 Nov 2021

  29. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  30. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: Inverted residuals and linear bottlenecks (2019)

    Google Scholar 

  31. Shukla, N., Fricklas, K.: Machine learning with TensorFlow. Manning Shelter Island, Ny (2018)

    Google Scholar 

  32. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition (2015)

    Google Scholar 

  33. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2016)

    Google Scholar 

  34. Szegedy, C., et al.: Going Deeper with Convolutions (2014)

    Google Scholar 

  35. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the Inception Architecture for Computer Vision (2015)

    Google Scholar 

  36. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., Le, Q.V.: Mnasnet: platform-aware neural architecture search for mobile. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2820–2828 (2019)

    Google Scholar 

  37. Tan, M., Le, Q.V.: EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2020)

    Google Scholar 

  38. West, D.: LEGO sorting machine (2019). https://twitter.com/JustASquid/sta-tus/1201959889943154688. Accessed 08 Feb 2021

  39. Zagoruyko, S., Komodakis, N.: Wide residual networks (2017)

    Google Scholar 

  40. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning (2017)

    Google Scholar 

  41. Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning Transferable Architectures for Scalable Image Recognition (2018)

    Google Scholar 

Download references

Acknowledgment

The authors would like to thank Bartosz Śledź and Sławomir Zaraziński for help with part of the implementation and dataset creation.

Author information

Authors and Affiliations

  1. Faculty of Electronics, Telecommunication and Informatics, Gdańsk University of Technology, 11/12 Narutowicza Street, 80-233, Gdańsk, Poland

    Tomasz Boiński, Konrad Zawora & Julian Szymański

Authors
  1. Tomasz Boiński
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konrad Zawora
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julian Szymański
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz Boiński .

Editor information

Editors and Affiliations

  1. Brunel University London, London, UK

    Derek Groen

  2. University of Amsterdam, Amsterdam, The Netherlands

    Clélia de Mulatier

  3. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  4. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  5. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  6. University of Amsterdam, Amsterdam, The Netherlands

    Peter M. A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Boiński, T., Zawora, K., Szymański, J. (2022). How to Sort Them? A Network for LEGO Bricks Classification. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13352. Springer, Cham. https://doi.org/10.1007/978-3-031-08757-8_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08757-8_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08756-1

  • Online ISBN: 978-3-031-08757-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation