Skip to main content
SpringerLink
Log in
Book cover

New Approaches in Intelligent Image Analysis pp 93–122Cite as

Multi-dimensional Data Clustering and Visualization via Echo State Networks

  • Chapter
  • First Online:

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 108))

Abstract

The chapter summarizes the proposed recently approach for multidimensional data clustering and visualization. It uses a special kind of recurrent networks called Echo state networks (ESN) to generate multiple two-dimensional (2D) projections of the multidimensional original data. For this purpose equilibrium states of all neurons in the ESN are exploited. In order to fit the neurons equilibriums to the data an algorithm for tuning internal weights of the ESN called Intrinsic Plasticity (IP) is applied. Next 2D projections are subjected to selection based on different criteria in dependence on the aim of particular clustering task to be solved. The selected projections are used to cluster and/or to visualize the original data set. Several examples demonstrate possible ways to apply the proposed approach to variety of multidimensional data sets, namely: steel alloys discrimination by their composition; Earth cover classification from hyper spectral satellite images; working regimes classification of an industrial plant using data from multiple measurements; discrimination of patterns of random dot motion on the screen; and clustering and visualization of static and dynamic “sound pictures” taken by multiple randomly placed microphones.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9, 147–169 (1985)

    Article  Google Scholar 

  2. Alexiev, K., Bocheva, N., Stefanov, S.: Assessment of age-related changes in global motion direction discrimination. In: International Conference Automatics and Informatics’11, pp. B277−B280, Sofia, Bulgaria, 3−7 Oct 2011

    Google Scholar 

  3. Beardsley, S.A., Ward, R.L., Vaina, L.M.: A neural network model of spiral-planar motion tuning in MSTd. Vision. Res. 43, 577–595 (2003)

    Article  Google Scholar 

  4. Bocheva, N., Bojilov, L.: Neural network model for visual discrimination of complex motion. Comptes rendus de’l Academie bulgare des Sciences 65(10), 1356–1379 (2012)

    Google Scholar 

  5. Botev, Z.I., Grotowski, J.F., Kroese, D.P.: Kernel density estimation via diffusion. Ann. Stat. 38(5), 2916–2957 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  6. Brody, C.D., Romo, R., Kepecs, A.: Basic mechanisms for graded persistent activity: discrete attractors, continuous attractors, and dynamical representations. Curr. Opin. Neurobiol. 13, 204–211 (2003)

    Article  Google Scholar 

  7. Doukovska, L., Koprinkova-Hristova, P., Beloreshki, S.: Analysis of mill fan system for predictive maintenance. In: International Conference on Automatics and Informatics’11, pp. B-331−B-334, Sofia, Bulgaria, 3–7 Oct 2011

    Google Scholar 

  8. Eli Billauer’s home page: http://billauer.co.il/peakdet.html. Accessed 2013

  9. Fast and accurate state-of-the-art bivariate kernel density estimator by Z. Botev, http://www.mathworks.com/matlabcentral/fileexchange/17204-kernel-density-estimation (updated 2009)

  10. Grossberg, S., Pilly, P.K.: Temporal dynamics of decision-making during motion perception in the visual cortex, Technical report BU CAS/CNS TR-2007-001, Feb 2008

    Google Scholar 

  11. Haddad, W.M., Chellaboina, V.S., Nersesov, S.G.: Thermodynamics: A Dynamical System Approach, Princeton University Press (2005)

    Google Scholar 

  12. Hammouda, K.: A comparative study of data clustering techniques. In: SYDE 625: Tools of Intelligent Systems Design, Course Project, Aug 2000

    Google Scholar 

  13. Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  14. Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state network” approach, GMD Report 159, German National Research Center for Information Technology (2002)

    Google Scholar 

  15. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999)

    Article  Google Scholar 

  16. Koprinkova-Hristova, P., Palm, G.: ESN intrinsic plasticity versus reservoir stability. In: Artificial Neural Networks and Machine Learning—ICANN 2011. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6791, pp. 69−76 (2011)

    Google Scholar 

  17. Koprinkova-Hristova, P., Tontchev, N.: Echo state networks for multi-dimensional data clustering. In: Artificial Neural Networks and Machine Learning—ICANN 2012. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7552 (PART 1), pp. 571–578 (2012)

    Google Scholar 

  18. Koprinkova-Hristova, P., Alexiev, K., Borisova, D., Jelev, G., Atanassov, V.: Recurrent neural networks for automatic clustering of multispectral satellite images, In: Bruzzone, L (ed.) Proceedings of SPIE, Image and Signal Processing for Remote Sensing XIX, vol. 8892, 88920X, 17 Oct 2013. doi:10.1117/12

  19. Koprinkova-Hristova, P., Angelova, D., Borisova, D., Jelev, G.: Clustering of spectral images using Echo state networks. In: IEEE International Symposium on Innovations in Intelligent Systems and Applications, IEEE INISTA 2013, Albena, Bulgaria, 19–21 June 2013. doi:10.1109/INISTA.2013.6577633

  20. Koprinkova-Hristova, P., Doukovska, L., Kostov, P.: Working regimes classification for predictive maintenance of mill fan systems. In: 2013 IEEE International Symposium on Innovations in Intelligent Systems and Applications, IEEE INISTA 2013, Albena, Bulgaria, 19–21 June 2013. doi:10.1109/INISTA.2013.6577632

  21. Koprinkova-Hristova, P., Alexiev, K.: Echo state networks in dynamic data clustering. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8131, pp. 343−350 (2013)

    Google Scholar 

  22. Koprinkova-Hristova, P., Alexiev, K.: Sound fields clusterization via neural networks. In: 2014 IEEE International Symposium on Innovations in Intelligent Systems and Applications, INISTA 2014, pp. 368−374, Alberobello, Italy, 23−25 June 2014

    Google Scholar 

  23. Koprinkova-Hristova, P., Alexiev, K.: Dynamic sound fields clusterization using neuro-fuzzy approach. In: 16th International Conference, AIMSA 2014, Varna, Bulgaria, 11−13 Sept 2014. Artificial Intelligence: Methodology, Systems, and Applications, Lecture Notes in Computer Science, vol. 8722, pp. 194−205 (2014)

    Google Scholar 

  24. Landsat Missions. http://landsat.usgs.gov/. Accessed 2013

  25. Lazar, A., Pipa, G., Triesch, J.: Predictive coding in cortical microcircuits. In: Kurkova, V., et al. (eds.) ICANN 2008, Part II, LNCS 5164, pp. 386–395 (2008)

    Google Scholar 

  26. Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3, 127–149 (2009)

    Article  MATH  Google Scholar 

  27. MRRB GIS System security division, http://gis.mrrb.government.bg/pmapper/map_separate-legend.phtml?winsize=medium&language=bg&config=separate-legend. Accessed 2013

  28. Ozturk, M., Xu, D., Principe, J.: Analysis and design of echo state networks. Neural Comput. 19, 111–138 (2007)

    Google Scholar 

  29. Peng, X., Guo, J., Lei, M., Peng, Y.: Analog circuit fault diagnosis with echo state networks based on corresponding clusters. In: Liu, et al. (eds.) ISNN 2011, Part I, LNCS 6675, pp. 437–444 (2011)

    Google Scholar 

  30. Schrauwen, B., Wandermann, M., Verstraeten, D., Steil, J.J., Stroobandt, D.: Improving reservoirs using intrinsic plasticity. Neurocomputing 71, 1159–1171 (2008)

    Article  Google Scholar 

  31. Steil, J.J.: Online reservoir adaptation by intrinsic plasticity for back-propagation-decoleration and echo state learning. Neural Netw. 20, 353–364 (2007)

    Article  MATH  Google Scholar 

  32. Woodward, A., Ikegami, T.: A reservoir computing approach to image classification using coupled echo state and back-propagation neural networks. In: Proceedings of 26th International Conference on Image and Vision Computing, Auckland, New Zealand, pp. 543–458, 29 Nov−1 Dec 2011 (2011)

    Google Scholar 

  33. Yager, R., Filev, D.: Generation of fuzzy rules by mountain clustering. J. Intell. Fuzzy Syst. 2(3), 209–219 (1994)

    Google Scholar 

Download references

Acknowledgments

The present work is partially supported by the following projects: AComIn “Advanced Computing for Innovation”, grant 316087, funded by the FP7 Capacity Program (Research Potential of Convergence Regions); the projects funded by the Bulgarian Science Fund numbered DDVU02/11, DVU-10-0267/10 and DFNI-I01/8. All real data sets are obtained during work on these projects. The author is grateful to all colleagues and co-authors who contributed by various data sets and comments on the obtained clustering results. Special thanks to Kiril Alexiev, Denica Borisova and Georgi Jelev for their supportive and valuable comments.

Author information

Authors and Affiliations

  1. Bulgarian Academy of Sciences, Institute of Information and Communication Technologies, Acad. G. Bonchev str. bl. 25A, 1113, Sofia, Bulgaria

    Petia Koprinkova-Hristova

Authors
  1. Petia Koprinkova-Hristova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petia Koprinkova-Hristova .

Editor information

Editors and Affiliations

  1. Technical University of Sofia, Sofia, Bulgaria

    Roumen Kountchev

  2. University of Hyogo, Chuo-ku, Japan

    Kazumi Nakamatsu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Koprinkova-Hristova, P. (2016). Multi-dimensional Data Clustering and Visualization via Echo State Networks. In: Kountchev, R., Nakamatsu, K. (eds) New Approaches in Intelligent Image Analysis. Intelligent Systems Reference Library, vol 108. Springer, Cham. https://doi.org/10.1007/978-3-319-32192-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32192-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32190-5

  • Online ISBN: 978-3-319-32192-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation