Skip to main content
SpringerLink
Log in

Contour Classification Method for Industrially Oriented Human-Robot Speech Communication

  • Conference paper
  • First Online:
Advanced, Contemporary Control

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1196))

Abstract

This paper describes research dealing with application of image recognition as a tool supporting efficient speech communication between humans and collaborative robots in industrial environment. Image-based recognition of objects in robot’s workspace may provide a context for voice commands. In this way the commands can be shorter and more concise. As the robots “understand” abstract technical terms used by human operators, a user-friendly speech communication can be provided. In order to recognize objects properly, classification of their contours must usually take into account the fact that some objects described by one abstract term may differ in dimensions and shapes, whereas some other objects described by different terms may be very similar. Since the object classification rules are usually application-specific, it is impossible to develop general algorithm applicable in all situations. This problem can be solved using Flexible Editable Contour Templates (FECT). However, the crucial factor determining applicability of contour classification method to speech communication is rapidity of the algorithm used for comparison of real contours against the FECTs. Currently, a computationally-expensive algorithm of segment matching is used. In this paper, we propose an alternative method, based on artificial neural networks (ANN).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pires, J.N.: Robot-by-voice: experiments on commanding an industrial robot using the human voice. Ind. Rob. 32, 505–511 (2005). https://doi.org/10.1108/01439910510629244

    Article  Google Scholar 

  2. Zinchenko, K., Wu, C.Y., Song, K.T.: A study on speech recognition control for a surgical robot. IEEE Trans. Ind. Inform. 13, 607–615 (2017). https://doi.org/10.1109/TII.2016.2625818

    Article  Google Scholar 

  3. Rozo, L., Calinon, S., Caldwell, D.G., Jiménez, P., Torras, C.: Learning physical collaborative robot behaviors from human demonstrations. IEEE Trans. Robot. 32, 513–527 (2016). https://doi.org/10.1109/TRO.2016.2540623

    Article  Google Scholar 

  4. Maurtua, I., Ibarguren, A., Kildal, J., Susperregi, L., Sierra, B.: Human-robot collaboration in industrial applications: safety, interaction and trust. Int. J. Adv. Robot. Syst. 14, 1–10 (2017). https://doi.org/10.1177/1729881417716010

    Article  Google Scholar 

  5. Matthias, B., Kock, S., Jerregard, H., Källman, M., Lundberg, I.: Safety of collaborative industrial robots: certification possibilities for a collaborative assembly robot concept. In: Proceedings of the 2011 IEEE International Symposium on Assembly and Manufacturing, ISAM, Tampere, pp. 1–6 (2011). https://doi.org/10.1109/isam.2011.5942307

  6. Zanchettin, A.M., Bascetta, L., Rocco, P.: Acceptability of robotic manipulators in shared working environments through human-like redundancy resolution. Appl. Ergon. 44, 982–989 (2013). https://doi.org/10.1016/j.apergo.2013.03.028

    Article  Google Scholar 

  7. El Makrini, I., Elprama, S.A., Van Den Bergh, J., Vanderborght, B., Knevels, A.J., Jewell, C.I.C., Stals, F., De Coppel, G., Ravyse, I., Potargent, J., Berte, J., Diericx, B., Waegeman, T., Jacobs, A.: Working with walt: how a cobot was developed and inserted on an auto assembly line. IEEE Robot. Autom. Mag. 25, 51–58 (2018). https://doi.org/10.1109/MRA.2018.2815947

    Article  Google Scholar 

  8. Rogowski, A., Skrobek, P.: Direct human-robot collaboration in welding. Przegląd Spawalnictwa 90, 9–14 (2018). https://doi.org/10.26628/ps.v90i1.847

    Article  Google Scholar 

  9. Rogowski, A.: Can we talk to a welding robot? Przegląd Spawalnictwa 88, 5–8 (2016). https://doi.org/10.26628/wtr.v88i1.557

    Article  Google Scholar 

  10. Gundogdu, K., Bayrakdar, S., Yucedag, I.: Developing and modeling of voice control system for prosthetic robot arm in medical systems. J. King Saud Univ. – Comput. Inf. Sci. 30 (2017). https://doi.org/10.1016/j.jksuci.2017.04.005

  11. Jayawardena, C., Watanabe, K., Izumi, K.: Controlling a robot manipulator with fuzzy voice commands using a probabilistic neural network. Neural Comput. Appl. 16, 155–166 (2007). https://doi.org/10.1007/s00521-006-0056-8

    Article  Google Scholar 

  12. Hamdan, B., Mokhtar, K.: Face recognition using angular radial transform. J. King Saud Univ. – Comput. Inf. Sci. 30, 141–151 (2018). https://doi.org/10.1016/j.jksuci.2016.10.006

    Article  Google Scholar 

  13. Barman, A., Dutta, P.: Facial expression recognition using distance and shape signature features. Pattern Recogn. Lett., 1–8 (2017). https://doi.org/10.1016/j.patrec.2017.06.018

  14. Tsarouchi, P., Athanasatos, A., Makris, S., Chatzigeorgiou, X., Chryssolouris, G.: High level robot programming using body and hand gestures. Procedia CIRP 55, 1–5 (2016). https://doi.org/10.1016/j.procir.2016.09.020

    Article  Google Scholar 

  15. Van Delden, S., Umrysh, M., Rosario, C., Hess, G.: Pick-and-place application development using voice and visual commands. Ind. Rob. 39, 592–600 (2012). https://doi.org/10.1108/01439911211268796

    Article  Google Scholar 

  16. Maurtua, I., Fernández, I., Tellaeche, A., Kildal, J., Susperregi, L., Ibarguren, A., Sierra, B.: Natural multimodal communication for human-robot collaboration. Int. J. Adv. Robot. Syst. 14, 1–12 (2017). https://doi.org/10.1177/1729881417716043

    Article  Google Scholar 

  17. Pleshkova, S., Zahariev, Z.: Development of system model for audio visual control of mobile robots with voice and gesture commands. In: Proceedings of the International Spring Seminar on Electronics Technology, pp. 1–4 (2017). https://doi.org/10.1109/isse.2017.8000979

  18. Tasevski, J., Nikolić, M., Mišković, D.: Integration of an industrial robot with the systems for image and voice recognition. SERBIAN J. Electr. Eng. 10, 219–230 (2013). https://doi.org/10.2298/SJEE1301219T

    Article  Google Scholar 

  19. Karczmarek, P., Kiersztyn, A., Pedrycz, W., Dolecki, M.: An application of chain code-based local descriptor and its extension to face recognition. Pattern Recogn. 65, 26–34 (2017). https://doi.org/10.1016/j.patcog.2016.12.008

    Article  Google Scholar 

  20. Prematilake, C., Ellingson, L.: Evaluation and prediction of polygon approximations of planar contours for shape analysis. J. Appl. Stat. 45, 1227–1246 (2018). https://doi.org/10.1080/02664763.2017.1364716

    Article  MathSciNet  Google Scholar 

  21. Elsalamony, H.A.: Detection and matching of object using proposed signature. In: Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition, pp. 583–596. Elsevier Inc. (2015). https://doi.org/10.1016/b978-0-12-802045-6.00038-7

  22. Elsalamony, H.A.: Automatic object detection and matching based on proposed signature. In: Proceedings of International Conference on Audio, Language and Image Processing, Shanghai, pp. 68–73 (2012). https://doi.org/10.1109/icalip.2012.6376588

  23. Tomakova, R., Komkov, V., Emelianov, E., Tomakov, M.: The use of Fourier descriptors for the classification and analysis of peripheral blood smears image. Appl. Math. 8, 1563–1571 (2017). https://doi.org/10.4236/am.2017.811114

    Article  Google Scholar 

  24. Cai, Z.X., Gu, M.Q.: Traffic sign recognition algorithm based on shape signature and dual-tree complex wavelet transform. J. Cent. South Univ. 20, 433–439 (2013). https://doi.org/10.1007/s11771-013-1504-0

    Article  Google Scholar 

  25. Attalla, E., Siy, P.: Robust shape similarity retrieval based on contour segmentation polygonal multiresolution and elastic matching. Pattern Recogn. 38, 2229–2241 (2005). https://doi.org/10.1016/j.patcog.2005.02.009

    Article  Google Scholar 

  26. Rogowski, A., Skrobek, P.: Object identification for task-oriented communication with collaborative industrial robots. Sensors 20, 1773–1792 (2020). https://doi.org/10.3390/s20061773

  27. Bielecka, M.: Syntactic-geometric-fuzzy hierarchical classifier of contours with application to analysis of bone contours in X-ray images. Appl. Soft Comput. 69, 368–380 (2018). https://doi.org/10.1016/j.asoc.2018.04.038

    Article  Google Scholar 

  28. Firpi, H.A., Ucar, D., Tan, K.: Discover regulatory DNA elements using chromatin signatures and artificial neural network. Bioinformatics 26, 1579–1586 (2010). https://doi.org/10.1093/bioinformatics/btq248

    Article  Google Scholar 

  29. Elsalamony, H.A.: Healthy and unhealthy red blood cell detection in human blood smears using neural networks. Micron 83, 32–41 (2016). https://doi.org/10.1016/j.micron.2016.01.008

    Article  Google Scholar 

  30. Cires, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. https://arxiv.org/pdf/1202.2745.pdf. Accessed 10 Mar 2019

  31. Singh, K.: SVM-BDT PNN and Fourier moment technique for classification of leaf shape. Int. J. Signal Process. Image Process. Pattern Recogn. 3, 67–78 (2011)

    Google Scholar 

  32. Selvaraj, A., Shebiah, N., Ananthi, S., Varthini, S.: Detection of unhealthy region of plant leaves using neural network. Agric. Eng. Int.: CIGR J. 15, 211–217 (2013)

    Google Scholar 

  33. Thanda, A., Venkatesan, S.M.: Audio visual speech recognition using deep recurrent neural networks. In: Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction: 4th IAPR TC 9 Workshop, pp. 98–109 (2016). https://doi.org/10.1007/978-3-319-59259-6_9

Download references

Author information

Authors and Affiliations

  1. Warsaw University of Technology, ul. Narbutta 85, 02-524, Warsaw, Poland

    Piotr Skrobek & Adam Rogowski

Authors
  1. Piotr Skrobek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam Rogowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Rogowski .

Editor information

Editors and Affiliations

  1. Institute of Automatic Control, Lodz University of Technology, Lodz, Poland

    Andrzej Bartoszewicz

  2. Institute of Automatic Control, Lodz University of Technology, Lodz, Poland

    Jacek Kabziński

  3. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Skrobek, P., Rogowski, A. (2020). Contour Classification Method for Industrially Oriented Human-Robot Speech Communication. In: Bartoszewicz, A., Kabziński, J., Kacprzyk, J. (eds) Advanced, Contemporary Control. Advances in Intelligent Systems and Computing, vol 1196. Springer, Cham. https://doi.org/10.1007/978-3-030-50936-1_67

Download citation

Publish with us

Policies and ethics

Search

Navigation