Skip to main content
SpringerLink
Log in
Book cover

Digital Signal Processing with Matlab Examples, Volume 2 pp 647–835Cite as

Data Analysis and Classification

  • Chapter
  • First Online:

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

Classification is a fundamental activity in many scientific disciplines, and in a large variety of professional applications. In many circumstances, classification is not an easy task. Analysis tools are needed in order to detect distinctive characteristics, and to compare according with suitable measures. The aim of this chapter is to introduce a repertory of important data analysis and classification methods. Some aspects of the chapter have evident connections with pattern recognition techniques, and with data mining. The first sections introduce component analysis (both PCA and ICA), including an interesting example: blind source separation. Next sections focus on clustering and discrimination, introducing linear discriminant analysis (LDA), support vector machines (SVM), K-means, K-nn, and the use of kernels. This is continued with a view of probabilistic contexts, including Bayesian methodology. In this part, the chapter presents the expectation-maximization (EM) algorithm. Bayesian regression, Kriging, Gaussian processes, neurons, etc. The final section on experiments considers face detection, and K-means for picture color reduction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. D.P. Acharya, G. Panda, A review of independent component analysis techniques. IETE Tech. Rev. 25(6), 320–332 (2008)

    Article  Google Scholar 

  2. C.C. Aggarwal, C.K. Reddy, Data Clustering: Algorithms and Applications (Chapman and Hall, 2013)

    Google Scholar 

  3. C.C. Aggarwal, (ed.), Data Classification: Algorithms and Applications (CRC Press, 2015)

    Google Scholar 

  4. S.I. Amari, Natural gradient works efficiently in learning. Neural Comput. 10(2), 251–276 (1998)

    Article  MathSciNet  Google Scholar 

  5. A. Andoni, Nearest neighbor search: the old, the new, and the impossible. Ph.D. thesis, MIT, 2009

    Google Scholar 

  6. S. Andrews, I. Tsochantaridis, T. Hofmann, Support vector machines for multiple-instance learning, in Advances in Neural Information Processing Systems, pp. 561–568 (2002)

    Google Scholar 

  7. P. Anjali, S. Ajay, S.D. Sapre, A review on natural image denoising using independent component analysis (ica) technique. Adv. Comput. Res. 2(1), 06–14 (2010)

    Google Scholar 

  8. M.A. Anusuya, S.K. Katti, Speech Recognition by Machine, a Review (Department of Computer Science and Engineering Sri Jayachamarajendra College of Engineering Mysore, India, 2010). arXiv preprint arXiv:1001.2267

  9. S. Arya, D.M. Mount, N.S. Netanyahu, R. Silverman, A.Y. Wu, An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM 45(6), 891–923 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  10. R. Avnimelech, N. Intrator, Boosted mixture of experts: an ensemble learning scheme. Neural Comput. 11(2), 483–497 (1999)

    Article  Google Scholar 

  11. S. Ayache, G. Quénot, J. Gensel, Classifier fusion for SVM-based multimedia semantic indexing, in Advances in Information Retrieval, pp. 494–504 (Springer, 2007)

    Google Scholar 

  12. F.R. Bach, M.I. Jordan, Kernel independent component analysis. J. Mach. Learn. Res. 3, 1–48 (2002)

    MathSciNet  MATH  Google Scholar 

  13. B. Bahmani, B. Moseley, A. Vattani, R. Kumar, S. Vassilvitskii, Scalable k-means++. Proc. VLDB Endowment 5(7), 622–633 (2012)

    Article  Google Scholar 

  14. S. Balakrishnama, A. Ganapathiraju, Linear Discriminant Analysis-A Brief Tutorial (Institute for Signal and information Processing, Dept. Electrical and Computer Engineering, Mississippi State University, 1998). https://www.researchgate.net/publication/240093048_Linear_Discriminant_ Analysis-A_Brief_Tutorial

  15. M.S. Bartlett, J.R. Movellan, T.J. Sejnowski, Face recognition by independent component analysis. IEEE T. Neural Netw. 13(6), 1450–1464 (2002)

    Article  Google Scholar 

  16. M. Basseville, Divergence Measures for Statistical Data Processing (HAL, INRIA, France, 2010). http://hal.inria.fr/docs/00/54/23/37/PDF/PI-1961.pdf

  17. G. Baudat, F. Anouar, Generalized discriminant analysis using a Kernel approach. Neural Comput. 12(10), 2385–2404 (2000)

    Article  Google Scholar 

  18. S. Bauer, S. Köhler, K. Doll, U. Brunsmann, FPGA-GPU architecture for kernel SVM pedestrian detection, in Proceedings IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 61–68 (2010)

    Google Scholar 

  19. M.J. Beal, Variational algorithms for approximate Bayesian inference. Ph.D. thesis, University of London, 2003

    Google Scholar 

  20. P.N. Belhumeur, J.P. Hespanha, D. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)

    Google Scholar 

  21. A.J. Bell, T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 (1995)

    Article  Google Scholar 

  22. A.J. Bell, T.J. Sejnowski, The “independent components” of natural scenes are edge filters. Vision Res. 37(23), 3327–3338 (1997)

    Article  Google Scholar 

  23. A. Ben-Hur, J. Weston, A user’s guide to support vector machines, in Data Mining Techniques for the Life Sciences, pp. 223–239 (Humana Press, 2010)

    Google Scholar 

  24. K.P. Bennett, C. Campbell, Support vector machines: hype or hallelujah? ACM SIGKDD Explor. Newslett. 2(2), 1–13 (2000)

    Article  Google Scholar 

  25. P. Berkhin, A survey of clustering data mining techniques, in Grouping Multidimensional Data, pp. 25–71 (Springer, 2006)

    Google Scholar 

  26. J.M. Bernardo, M.J. Bayarri, J.O. Berger, A.P. Dawid, D. Heckerman, A.F.M. Smith, M. West, The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures. Bayesian Stat. 7, 453–464 (2003)

    MathSciNet  Google Scholar 

  27. J.C. Bezdek, L.O. Hall, L. Clarke, Review of MR image segmentation techniques using pattern recognition. Med. Phys. 20(4), 1033–1048 (1992)

    Article  Google Scholar 

  28. N. Bhatia, Survey of Nearest Neighbor Techniques (Department of Computer Science DAV College Jalandhar, India, 2010). arXiv preprint arXiv:1007.0085

  29. J. Bi, K. Bennett, M. Embrechts, C. Breneman, M. Song, Dimensionality reduction via sparse support vector machines. J. Mach. Learn. Res. 3, 1229–1243 (2003)

    MATH  Google Scholar 

  30. C.M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995)

    MATH  Google Scholar 

  31. C.M. Bishop, Pattern Recognition and Machine Learning (Springer Verlag, 2010)

    Google Scholar 

  32. R. Blahut, Principles and Practices of Information Theory (Addison-Wesley, 1987)

    Google Scholar 

  33. T. Blaschke, L. Wiskott, Cubica: independent component analysis by simultaneous third- and fourth-order cumulant diagonalization. IEEE T. Sign Process. 52(5), 1250–1256 (2004)

    Article  MathSciNet  Google Scholar 

  34. M. Blum, M. Riedmiller, Optimization of Gaussian process hyperparameters using Rprop, in Proceedings European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2013, pp. 1–6

    Google Scholar 

  35. G. Bohling, Introduction to geostatistics and variogram analysis. Kansas Geol. Survey 1–20 (2005)

    Google Scholar 

  36. G. Bohling, Kriging. C&PE 940, 2005. http://people.ku.edu/~gbohling/cpe940/Kriging.pdf

  37. S. Bose, A. Pal, R. SahaRay, J. Nayak, Generalized quadratic discriminant analysis. Pattern Recogn. 48(8), 2676–2684 (2015)

    Article  Google Scholar 

  38. L. Bottou, C.J. Lin, Support vector machine solvers, eds. by L. Bottou, O. Chapelle, D. DeCoste, J. Weston. Large Scale Kernel Machines, pp. 1–17 (MIT Press, 2007)

    Google Scholar 

  39. P. Boyle, Gaussian processes for regression and optimisation. Ph.D. thesis, Victoria University of Wellington, 2007

    Google Scholar 

  40. S. Bratieres, N. Quadrianto, Z. Ghahramani, Bayesian Structured Prediction Using Gaussian Processes (Department of Engineering, University of Cambridge, 2013). arXiv preprint arXiv:1307.3846

  41. L. Breiman, Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MathSciNet  MATH  Google Scholar 

  42. L. Breiman, Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  43. A.M. Bronstein, M.M. Bronstein, M. Zibulevsky, Y.Y. Zeevi, Sparse ICA for blind separation of transmitted and reflected images. Int. J. Imag. Syst. Technol. 15(1), 84–91 (2005)

    Article  Google Scholar 

  44. C.J. Burges, A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2(2), 121–167 (1998)

    Article  Google Scholar 

  45. H. Byun, S.W. Lee, Applications of support vector machines for pattern recognition: a survey, in Pattern Recognition with Support Vector Machines, pp. 213–236 (Springer, 2002)

    Google Scholar 

  46. D. Cai, X. He, J. Han, Speed up kernel discriminant analysis. The VLDB J. 20(1), 21–33 (2011)

    Article  Google Scholar 

  47. N. Cancedda, E. Gaussier, C. Goutte, J.M. Renders, Word sequence kernels. J. Mach. Learn. Res. 3, 1059–1082 (2003)

    MathSciNet  MATH  Google Scholar 

  48. J.F. Cardoso, Infomax and maximum likelihood for blind source separation. IEEE Sign. Process. Lett. 4, 109–111 (1997)

    Article  Google Scholar 

  49. J.F. Cardoso, High-order contrasts for independent component analysis. Neural Comput. 11, 157–192 (1999)

    Article  Google Scholar 

  50. J.F. Cardoso, B. Laheld, Equivariant adaptive source separation. IEEE T. Sign. Process. 45(2), 434–444 (1996)

    Google Scholar 

  51. J.F. Cardoso, A. Souloumiac, Blind beamforming for non Gaussian signals. IEE Proc.-F 140, 362–370 (1993)

    Google Scholar 

  52. C. Carson, S. Belongie, H. Greenspan, J. Malik, Blobworld: image segmentation using expectation-maximization and its application to image querying. IEEE T. Pattern Anal. Mach. Intell. 24(8), 1026–1038 (2002)

    Google Scholar 

  53. F. Castells, P. Laguna, L. Sörnmo, A. Bollmann, J.M. Roig, Principal component analysis in ECG signal processing. EURASIP J. Appl. Sign. Process. 2007(1), 1–21 (2007)

    MATH  Google Scholar 

  54. M.E. Celebi, Improving the performance of k-means for color quantization. Image Vision Comput. 29(4), 260–271 (2011)

    Article  MathSciNet  Google Scholar 

  55. A.B. Chan, N. Vasconcelos, Counting people with low-level features and Bayesian regression. IEEE T. Image Process. 21(4), 2160–2177 (2012)

    Article  MathSciNet  Google Scholar 

  56. H.P. Chan, D. Wei, M.A. Helvie, B. Sahiner, D.D. Adler, M.M. Goodsitt, N. Petrick, Computer-aided classification of mammographic masses and normal tissue: linear discriminant analysis in texture feature space. Phys. Med. Biol. 40(5), 857–876 (1995)

    Article  Google Scholar 

  57. O. Chapelle, P. Haffner, V.N. Vapnik, Support vector machines for histogram-based image classification. IEEE T. Neural Netw. 10(5), 1055–1064 (1999)

    Article  Google Scholar 

  58. C.H. Chen, On information an distance measures, error bounds, and feature selection. Inf. Sci. 10(2), 159–173 (1976)

    Article  MATH  Google Scholar 

  59. C.W. Chen, J. Luo, K.J. Parker, Image segmentation via adaptive k-mean clustering and knowledge-based morphological operations with biomedical applications. IEEE Trans. Image Process. 7(12), 1673–1683 (1998)

    Article  Google Scholar 

  60. C.Y. Chiu, Y.F. Chen, I. Kuo, H.C. Ku, An intelligent market segmentation system using k-means and particle swarm optimization. Expert Syst. Appl. 36(3), 4558–4565 (2009)

    Article  Google Scholar 

  61. Y. Cho, L.K. Saul, Kernel methods for deep learning, in NIPS Proceedings: Advances in Neural Information Processing Systems, pp. 342–350 (2009)

    Google Scholar 

  62. S. Choi, A. Cichocki, H.M. Park, S.Y. Lee, Blind source separation and independent component analysis: a review. Neural Inf. Process.-Lett. Rev. 6(1), 1–57 (2005)

    Google Scholar 

  63. A. Choudrey, S.J. Roberts, Flexible Bayesian independent component analysis for blind source separation, in Proceedings International Conference on Independent Component Analysis and Signal Separation, (ICA2001), pp. 90–95 (2001)

    Google Scholar 

  64. L. Clemmensen, T. Hastie, D. Witten, B. Ersbøll, Sparse discriminant analysis. Technometrics 53(4), 1–25 (2011)

    Article  MathSciNet  Google Scholar 

  65. A. Coates, A.Y. Ng, Learning feature representations with k-means, in Neural Networks: Tricks of the Trade, pp. 561–580 (Springer Berlin Heidelberg, 2012)

    Google Scholar 

  66. S. Cohen, R. Ben-Ari. Image de-noising by Bayesian regression, in Proceedings Image Analysis and Processing, ICIAP 2011, pp. 19–28 (Springer Berlin Heidelberg, 2011)

    Google Scholar 

  67. P. Comon, Independent component analysis, a new concept? Sign. Process. 36(3), 287–314 (1994)

    Article  MATH  Google Scholar 

  68. D. Cook, A. Buja, J. Cabrera, C. Hurley, Grand tour and projection pursuit. J. Comput. Graph. Stat. 4(3), 155–172 (1995)

    Google Scholar 

  69. G. Coombe, An introduction to principal component analysis and online singular value decomposition. Ph.D. thesis, Dept. of Computer Science, University of North Carolina, 1993

    Google Scholar 

  70. C. Cortes, V. Vapnik, Support-vector network. Mach. Learn. 20, 1–25 (1995)

    MATH  Google Scholar 

  71. A.B. Costello, J. Osborno, Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract. Assess. Res. Eval. 10(7) (2005). http://pareonline.net/getvn.asp?v=10&n=7

  72. N. Cristianini, Kernel Methods for General Pattern Analysis (Lecture Presentation, University of California at Davis, 2004). http://www.kernel-methods.net/tutorials/KMtalk.pdf

  73. N. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines (Cambridge University Press, 2000)

    Google Scholar 

  74. D.R. Cutting, D.R. Karger, J.O. Pedersen, J.W. Tukey, Scatter/gather: a cluster-based approach to browsing large document collections, in Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp. 318–329 (1992)

    Google Scholar 

  75. A. d’Aspremont, L. El Ghaoui, M.I. Jordan, G.R. Lanckriet, A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  76. J. Dauwels, K. Srinivasan, M. Ramasubba Reddy, T. Musha, F.B. Vialatte, C. Latchoumane, A. Cichocki, Slowing and loss of complexity in Alzheimer’s EEG: Two sides of the same coin? Int. J. Alzheimer’s Disease 1–9 (2011)

    Google Scholar 

  77. A. De la Escalera, J.M. Armingol, M. Mata, Traffic sign recognition and analysis for intelligent vehicles. Image Vis. Comput. 21(3), 247–258 (2003)

    Article  Google Scholar 

  78. A.P. Dempster, N.M. Laird, Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. R 39(1), 1–38 (1977)

    Google Scholar 

  79. L. Deng, A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans. Sign. Inf. Process. 3(e2), 1–29 (2014)

    Google Scholar 

  80. D.G.T. Denison, C.C. Holmes, B.K. Mallick, A.F.M. Smith, Bayesian Methods for Nonlinear Classification and Regression (Wiley, 2002)

    Google Scholar 

  81. W. DeSarbo, A. Ansari, P. Chintagunta, C. Himmelberg, K. Jedidi, R. Johnson, M. Wedel, Representing heterogeneity in consumer response models 1996 choice conference participants. Mark. Lett. 8(3), 335–348 (1997)

    Article  Google Scholar 

  82. L. Devroye, L. Györfi, G. Lugosi, A Probabilistic Theory of Pattern Recognition, vol. 31 (Springer Science & Business Media, 2013)

    Google Scholar 

  83. M.M. Deza, E. Deza, Encyclopedia of Distances (Springer Verlag, 2013)

    Google Scholar 

  84. P.M. Dixon, Nearest neighbor methods, in Encyclopedia of Environmetrics (Wiley Online Library, 2002)

    Google Scholar 

  85. C.B. Do, Gaussian Processes (Stanford University, 2007). http://www.see.stanford.edu/materials/aimlcs229/cs229-gp.pdf

  86. M. Dredze, K. Crammer, F. Pereira, Confidence-weighted linear classification, in Proceedings of the 25th ACM International Conference Machine Learning, pp. 264–271 (2008)

    Google Scholar 

  87. R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification (Wiley, 2012)

    Google Scholar 

  88. R. Durrett, Essentials of Stochastic Processes (Springer, 2012)

    Google Scholar 

  89. R. Dybowski, V. Gant, Clinical Applications of Artificial Neural Networks (Cambridge University Press, 2007)

    Google Scholar 

  90. M. Ebden. Gaussian Processes for Regression: A Quick Introduction (Robotics Research Group, University of Oxford, 2008). www.robots.ox.ac.uk/~mebden/reports/GPtutorial.pdf

  91. I. El-Naqa, Y. Yang, M.N. Wernick, N.P. Galatsanos, R.M. Nishikawa, A support vector machine approach for detection of microcalcifications. IEEE T. Med. Imag. 21(12), 1552–1563 (2002)

    Article  Google Scholar 

  92. Y. Engel, S. Mannor, R. Meir, Reinforcement learning with Gaussian processes, in Proceedings of the ACM 22nd International Conference on Machine Learning, pp. 201–208 (2005)

    Google Scholar 

  93. K. Etemad, R. Chellappa, Discriminant analysis for recognition of human face images. JOSA A 14(8), 1724–1733 (1997)

    Article  Google Scholar 

  94. L.A. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6), 510–523 (1988)

    Article  Google Scholar 

  95. G.E. Fasshauer, Positive definite kernels: past, present and future. Dolomite Res. Notes Approximation 4, 21–63 (2011)

    Google Scholar 

  96. L. Feng, Speaker recognition. Ph.D. thesis, Technical University of Denmark, DTU, DK-2800 Kgs. Lyngby, Denmark, 2004

    Google Scholar 

  97. S. Fiori, Overview of independent component analysis technique with an application to synthetic aperture radar (SAR) imagery processing. Neural Netw. 16(3–4), 453–467 (2003)

    Article  Google Scholar 

  98. H. Fleyeh, Traffic and road sign recognition. Ph.D. thesis, Napier University, 2008

    Google Scholar 

  99. Y. Freund, R.E. Schapire, Experiments with a new boosting algorithm, in Proceedings 13th International Conference Machine Learning, vol. 96, pp. 148–156 (1996)

    Google Scholar 

  100. Y. Freund, R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  101. Y. Freund, R.E. Schapire, Large margin classification using the perceptron algorithm. Mach. Learn. 37(3), 277–296 (1999)

    Article  MATH  Google Scholar 

  102. J.H. Friedman, Regularized discriminant analysis. J. Am. Stat. Assoc. 84(405), 165–175 (1989)

    Article  MathSciNet  Google Scholar 

  103. J.H. Friedman, J.W. Tukey, A projection pursuit algorithm for exploratory data analysis. IEEE T. Comput. 23(9), 881–890 (1974)

    Article  MATH  Google Scholar 

  104. F. Fukumizu, Methods with Kernels (Lecture Presentation, The Institute of Statistical Mathematics, Tokyo, 2008). http://www.ism.ac.jp/~fukumizu/ H20_kernel/Kernel_3_methods.pdf

  105. M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, F. Herrera, A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches. IEEE T. Syst. Man Cybern. Part C: Appl. Rev. 42(4), 463–484 (2012)

    Google Scholar 

  106. M. Gales, Multi-Layer Perceptrons (University of Cambridge, 2011). Handout 6, Module 4F10, Engineering Part II B. http://www.mi.eng.cam.ac.uk/~mjfg/local/4F10/lect6.pdf

  107. G. Gan, C. Ma, J. Wu, Data Clustering: Theory, Algorithms, and Applications (SIAM, 2007)

    Google Scholar 

  108. A. Ganapathiraju, J.E. Hamaker, J. Picone, Applications of support vector machines to speech recognition. IEEE T. Sign. Process. 52(8), 2348–2355 (2004)

    Article  Google Scholar 

  109. S.E. Gano, H. Kim, D.E. Brown, Comparison of three surrogate modeling techniques: Datascape, kriging, and second order regression, in Proceedings 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, pp. 1–18 (2006). AIAA-2006–7048 Portsmouth, Virginia

    Google Scholar 

  110. T. Gärtner, A survey of kernels for structured data. ACM SIGKDD Explor. Newslett. 5(1), 49–58 (2003)

    Article  Google Scholar 

  111. G. Gelle, M. Colas, C. Serviere, Blind source separation: a tool for rotating machine monitoring by vibrations analysis? J. Sound Vibr. 248(5), 865–885 (2001)

    Article  Google Scholar 

  112. R. Gonzalez Osuna, Pattern Recognition, Lecture Notes, Course 666 (Texas A&M University, 2014). http://psi.cse.tamu.edu/teaching/lecture_notes/

  113. J.M. Górriz, F. Segovia, J. Ramírez, A. Lassl, D. Salas-Gonzalez, GMM based SPECT image classification for the diagnosis of Alzheimer’s disease. Appl. Soft Comput. 11(2), 2313–2325 (2011)

    Article  Google Scholar 

  114. R.L. Gorsuch, Factor Analysis (Lawrence Erlbaum Associates, 1983)

    Google Scholar 

  115. R.M. Gray, D.L. Neuhoff, Quantization. IEEE Trans. Inf. Theory 44, 2325–2384 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  116. E. Gringarten, C.V. Deutsch, Teacher’s aide variogram interpretation and modeling. Math. Geol. 33(4), 507–534 (2001)

    Article  Google Scholar 

  117. Gaithersburg Statistics Group, NIST/SEMATECH Engineering Statistics Handbook (NIST Information Technology Lab., 2010)

    Google Scholar 

  118. S.R. Gunn, Support vector machines for classification and regression. Technical Report 14, ISIS, 1998

    Google Scholar 

  119. S. Günter, N.N. Schraudolph, S.V.N. Vishwanathan, Fast iterative kernel principal component analysis. J. Mach. Learn. Res. 8, 1893–1918 (2007)

    MathSciNet  MATH  Google Scholar 

  120. C. Guo, Machine learning methods for magnetic resonance imaging analysis. Ph.D. thesis, The University of Michigan, 2012

    Google Scholar 

  121. Y. Guo, T. Hastie, R. Tibshirani, Regularized linear discriminant analysis and its application in microarrays. Biostatistics 8(1), 86–100 (2007)

    Article  MATH  Google Scholar 

  122. H. Gupta, A.K. Agrawal, T. Pruthi, C. Shekhar, R. Chellappa, An experimental evaluation of linear and kernel-based methods for face recognition, in Proceedings IEEE 6th Workshop Applications of Computer Vision,(WACV ), pp. 13–18 (2002)

    Google Scholar 

  123. M.R. Gupta, Y. Chen, Theory and Use of the EM Algorithm (Now Publishers Inc, 2011)

    Google Scholar 

  124. R. Haapanen, A.R. Ek, M.E. Bauer, A.O. Finley, Delineation of forest/nonforest land use classes using nearest neighbor methods. Remote Sens. Environ. 89(3), 265–271 (2004)

    Article  Google Scholar 

  125. G. Hamerly, C. Elkan, Learning the k in k-means. NIPS Proc Adv. Neural Inf. Process. Syst. 16, 281–288 (2004)

    Google Scholar 

  126. C. Hartmann, J. Boedecker, O. Obst, S. Ikemoto, M. Asada, Real-time inverse dynamics learning for musculoskeletal robots based on echo state Gaussian process regression. Robot.: Sci. Syst. (2012)

    Google Scholar 

  127. T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning (Springer, 2013)

    Google Scholar 

  128. R.P. Hauser, D. Booth, Predicting bankruptcy with robust logistic regression. J. Data Sci. 9(4), 565–584 (2011)

    MathSciNet  Google Scholar 

  129. J.D. Haynes, G. Rees, Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 7(7), 523–534 (2006)

    Article  Google Scholar 

  130. H. He, W.C. Siu, Single image super-resolution using Gaussian process regression, in Proceedings IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 449–456 (2011)

    Google Scholar 

  131. B. Heisele, P. Ho, T. Poggio, Face recognition with support vector machines: global versus component-based approach. Proc. IEEE Intl. Conf. Comput. Vision 2, 688–694 (2001)

    Google Scholar 

  132. J. Hensman, N. Fusi, N.D. Lawrence, Gaussian Processes for Big Data (Dept. Computer Science The University of Sheffield, 2013). arXiv preprint arXiv:1309.6835

  133. J. Herault, J. Jutten, Space or time adaptive signal processing by neural network models, ed. by J.S. Denker. Neural Networks for Computing: AIP Conference Proceedings 151 (American Institute of Physics, 1986)

    Google Scholar 

  134. G.G. Herrero, E. Huuppone, Blind Source Separation Techniques for Processing Electroencephalographic Recordings (Tampere University of Technology, 2004). http://www.kasku.org/projects/bss/review/review.pdf

  135. Z.S.J. Hoare, Feature selection and classification of non-traditional data. Examples from veterinary medicine. Ph.D. thesis, University of Wales, Bangor, 2006

    Google Scholar 

  136. L.R. Hochberg, M.D. Serruya, G.M. Friehs, J.A. Mukand, M. Saleh, A.H. Caplan, J.P. Donoghue, Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442(7099), 164–171 (2006)

    Article  Google Scholar 

  137. H. Hoffmann, Kernel pca for novelty detection. Pattern Recogn. 40(3), 863–874 (2007)

    Article  MATH  Google Scholar 

  138. T. Hofmann, B. Schölkopf, A.J. Smola, Kernel methods in machine learning. Ann. Stat. 1171–1220 (2008)

    Google Scholar 

  139. P.J. Huber, Projection pursuit. Ann. Stat. 13, 435–475 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  140. A. Hyvärinen, Survey of independent component analysis. Neural Comput. Surv. 2, 94–128 (1999)

    Google Scholar 

  141. A. Hyvärinen, Independent component analysis: recent advances. Philos. Trans. Roy. Soc. 1–19 (2013). Open Access

    Google Scholar 

  142. A. Hyvärinen, J. Karhunen, E. Oja, Independent Component Analysis (Wiley-Interscience, 2001)

    Google Scholar 

  143. A. Hyvärinen, E. Oja, A fast fixed-point algorithm for independent component analysis. Neural Comput. 9(7), 1483–1492 (1997)

    Article  Google Scholar 

  144. A. Hyvärinen, E. Oja, Independent component analysis: algorithms and applications. Neural Netw. 13(4–5), 411–430 (2000)

    Article  Google Scholar 

  145. L. Ikemoto, O. Arikan, D. Forsyth, Generalizing motion edits with Gaussian processes. ACM Trans. Graph. (TOG) 28(1), 1–12 (2009)

    Article  Google Scholar 

  146. P. Indyk, R. Motwani, Approximate nearest neighbors: towards removing the curse of dimensionality, in Proceedings of the 30th Annual ACM Symposium on Theory of Computing, pp. 604–613 (ACM, 1998)

    Google Scholar 

  147. O. Ivanciuc, Applications of support vector machines in chemistry. Rev. Comput. Chem. 23(291) (2007)

    Google Scholar 

  148. A.J. Izenman, What is independent component analysis? (Temple University, 2003). http://astro.temple.edu/~alan/files/ICA.PDF

  149. A.J. Izenman, Modern Multivariable Statistical Techniques (Springer, 2008)

    Google Scholar 

  150. A.K. Jain, Data clustering: 50 years beyond k-means. Pattern Recogn. 31, 651–666 (2010)

    Article  Google Scholar 

  151. A.K. Jain, M.N. Murty, P.J. Flynn, Data clustering: a review. ACM Comput. Surveys (CSUR) 31(3), 264–323 (1999)

    Article  Google Scholar 

  152. C.J. James, C.W. Hesse, Independent component analysis for biomedical signals. Physiol. Measur. 26(1), 15–39 (2005)

    Article  Google Scholar 

  153. F.A. Jassim, Image Inpainting by Kriging Interpolation Technique (Faculty of Administrative Sciences, Management Information Systems Department, Irbid National University, Jordan, 2013). arXiv preprint arXiv:1306.0139

  154. R. Jenatton, G. Obozinski, F. Bach, Structured Sparse Principal Component Analysis (INRIA, France, 2009). arXiv preprint arXiv:0909.1440

  155. A. Jin, B. Yin, G. Morren, H. Duric, R.M. Aarts, Performance evaluation of a tri-axial accelerometry-based respiration monitoring for ambient assisted living, in Proceedings IEEE 31st Annual International Conference EMBS, pp. 5677–5680 (2009)

    Google Scholar 

  156. I. Jolliffe, Principal component analysis, ed. by Everitt. Encyclopedia of Statistics in Behavioral Science (Wiley, 2005)

    Google Scholar 

  157. T.P. Jung, S. Makeig, T.W. Lee, M.J. McKeown, G. Brown, A.J. Bell, T.J. Sejnowski, Independent component analysis of biomedical signals, in Proceedings International Workshop on Independent Component Analysis and Signal Separation, pp. 633–644 (2000)

    Google Scholar 

  158. C. Jutten, J. Karhunen, Advances in nonlinear blind source separation, in Proceedings 4th International Symposium Independent Component Analysis and Blind Signal Separation, ICA, pp. 245–256 (2003)

    Google Scholar 

  159. C. Jutten, A. Taleb, Source separation: From dusk till dawn, in Proceedings 2nd International Workshop on Independent Component Analysis and Blind Source Separation, (ICA2000), pp. 15–26 (Helsinki, 2000)

    Google Scholar 

  160. T. Kailath, The divergence and Bhattacharyya distance measures in signal selection. IEEE T. Commun. Technol. 15(1), 52–60 (1967)

    Article  Google Scholar 

  161. L. Kanal, Patterns in pattern recognition: 1968–1974. IEEE T. Inf. Theory 20(6), 697–722 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  162. T. Kanungo, D.M. Mount, N.S. Netanyahu, C.D. Piatko, R. Silverman, A.Y. Wu, An efficient k-means clustering algorithm: analysis and implementation. IEEE T. Patt. Anal. Mach. Intell. 24(7), 881–892 (2002)

    Google Scholar 

  163. A. Kapoor, K. Grauman, R. Urtasun, T. Darrell, Active learning with Gaussian processes for object categorization, in Proceedings IEEE 11th International Conference on Computer Vision, ICCV 2007

    Google Scholar 

  164. L. Kaufman, P. Rousseau, Finding Groups in Data (Wiley, 1990)

    Google Scholar 

  165. S. Kay, Intuitive Probability and Random Processes Using MATLAB (Springer, 2006)

    Google Scholar 

  166. S.S. Keerthi, C.J. Lin, Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Comput. 15(7), 1667–1689 (2003)

    Article  MATH  Google Scholar 

  167. H.B. Kekre, T.K. Sarode, New Clustering Algorithm for Vector Quantization Using Rotation of Error Vector (Computer Engineering Mukesh Patel School of Technology Management and Engineering, NMIMS University, Vileparle(w), India, 2010). arXiv preprint arXiv:1004.1686

  168. G. Kerschen, F. Poncelet, J.C. Golinval, Physical interpretation of independent component analysis in structural dynamics. Mech. Syst. Sign. Process. 21, 1561–1575 (2007)

    Article  Google Scholar 

  169. H.C. Kim, J. Lee, Clustering based on Gaussian processes. Neural Comput. 19(11), 3088–3107 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  170. K.I. Kim, M.O. Franz, B. Schölkopf, Iterative kernel principal component analysis for image modeling. IEEE T. Pattern Anal. Mach. Intell. 27(9), 1351–1366 (2005)

    Google Scholar 

  171. S.J. Kim, A. Magnani, S. Boyd, Optimal kernel selection in kernel Fisher discriminant analysis, in Proceedings ACM 23rd International Conference Machine Learning, pp. 465–472 (2006)

    Google Scholar 

  172. R.S. King, Cluster Analysis and Data Mining (Trasatlantic Publishers, 2014)

    Google Scholar 

  173. W.R. Klecka, Discriminant Analysis (Sage Publications, 1980)

    Google Scholar 

  174. J. Kocijan, A. Grancharova, Application of Gaussian processes to the modelling and control in process engineering, in Innovations in Intelligent Machines-5, pp. 155–190 (Springer, 2014)

    Google Scholar 

  175. A. Kocsor, L. Tóth, Kernel-based feature extraction with a speech technology application. IEEE T. Sign. Process. 52(8), 2250–2263 (2004)

    Article  MathSciNet  Google Scholar 

  176. R. Kohn, M. Smith, D. Chan, Nonparametric regression using linear combinations of basis functions. Stat. Comput. 11(4), 313–322 (2001)

    Article  MathSciNet  Google Scholar 

  177. I. Kokkinos, P. Maragos, Synergy between object recognition and image segmentation using the expectation-maximization algorithm. IEEE T. Pattern Anal. Mach. Intell. 31(8), 1486–1501 (2009)

    Google Scholar 

  178. Z. Koldovsky, Fast and accurate methods for independent component analysis. Ph.D. thesis, Czech Technical University in Prague, 2005

    Google Scholar 

  179. S. Koziel, D.E. Ciaurri, L. Leifsson, Surrogate-based methods, in Computational Optimization, Methods and Algorithms, pp. 33–59 (Springer, 2011)

    Google Scholar 

  180. K. Krishna, M.N. Murty, Genetic k-means algorithm. IEEE T. Syst. Man Cybern. Part B Cybern. 29(3), 433–439 (1999)

    Article  Google Scholar 

  181. K. Krivoruchko, Empirical Bayesian Kriging (Software Development Team, Esri, 2012). http://www.esri.com/news/arcuser/1012/empirical-byesian-kriging.html

  182. M. Ku\(\beta \), Gaussian process models. Ph.D. thesis, Technischen Darmstadt, 2006

    Google Scholar 

  183. B. Kulis, M.I. Jordan, Revisiting K-means: New Algorithms Via Bayesian Nonparametrics (Department of CSE, Ohio State University, Columbus, 2011). arXiv preprint arXiv:1111.0352

  184. T. Kumano, S. Jeong, S. Obayashi, Y. Ito, K. Hatanaka, H. Morino, Multidisciplinary design optimization of wing shape for a small jet aircraft using kriging model. AIAA Paper 932, 9–12 (2006)

    Google Scholar 

  185. J. Kumar, R.T. Mills, F.M. Hoffman, W.W. Hargrove, Parallel k-means clustering for quantitative ecoregion delineation using large data sets. Proc. Comput. Sci. 4, 1602–1611 (2011)

    Article  Google Scholar 

  186. M. Kuss, C.E. Rasmussen, Assesing approximate inference for binary Gaussian process classification. J. Mach. Learn. Res. 6, 1679–1704 (2005)

    MathSciNet  MATH  Google Scholar 

  187. N. Kwak, Feature extraction for classification problems and its application to face recognition. Pattern Recogn. 41(5), 1701–1717 (2008)

    Article  MATH  Google Scholar 

  188. V. Lakshmanan, R. Rabin, V. DeBrunner, Multiscale storm identification and forecast. Atmos. Res. 67, 367–380 (2003)

    Article  Google Scholar 

  189. L.D. Lathauwer, B.D. Moor, J. Vandewalle, An introduction to independent component analysis. J. Chemom. 14, 123–149 (2000)

    Article  Google Scholar 

  190. G.F. Lawler, Introduction to Stochastic Processes (Chapman and Hall, 2006)

    Google Scholar 

  191. J.H. Lee, H.Y. Jung, T.W. Lee, S.Y. Lee, Speech feature extraction using independent component analysis, in Proceedings IEEE International Conference Acoustics, Speech, and Signal Processing, ICASSP’00, vol. 3, pp. 1631–1634 (2000)

    Google Scholar 

  192. C.S. Leslie, E. Eskin, A. Cohen, J. Weston, W.S. Noble, Mismatch string kernels for discriminative protein classification. Bioinformatics 20(4), 467–476 (2004)

    Article  Google Scholar 

  193. A. Levey, M. Lindenbaum, Sequential Karhunen-Loeve basis extraction and its application to images. IEEE T. Image Process. 9(8), 1371–1374 (2000)

    Article  MATH  Google Scholar 

  194. E. Ley, M.F. Steel, On the effect of prior assumptions in Bayesian model averaging with applications to growth regression. J. Appl. Econ. 24(4), 651–674 (2009)

    Article  MathSciNet  Google Scholar 

  195. J. Li, A.D. Heap, A review of spatial interpolation methods for environmental scientists. Geosci. Australia Record 23 (2008)

    Google Scholar 

  196. S.Z. Li, A.K. Jain, Handbook of Face Recognition (Springer, 2005)

    Google Scholar 

  197. T. Li, S. Zhu, M. Ogihara, Using discriminant analysis for multi-class classification: an experimental investigation. Knowl. Inf. Syst. 10(4), 453–472 (2006)

    Article  Google Scholar 

  198. X. Li, L. Wang, E. Sung, Adaboost with SVM-based component classifiers. Eng. Appl. Artif. Intell. 21(5), 785–795 (2008)

    Article  Google Scholar 

  199. A. Lichtenstern, Kriging Methods in Spatial Statistics (Bachelor’s Thesis, Technische Universität München, 2013). http://ww.mediatum.ub.tum.de/doc/1173364/1173364.pdf

  200. C.F. Lin, S.D. Wang, Fuzzy support vector machines. IEEE T. Neural Netw. 13(2), 464–471 (2002)

    Article  Google Scholar 

  201. H.T. Lin, Adaptive Boosting – AdaBoosting (Lecture Notes, Machine Learning, National Taiwan University, 2008). https://www.csie.ntu.edu.tw/~b92109/course/

  202. J. Lin, Divergence measures based on the Shannon entropy. IEEE T. Inf. Theory 37(1), 145–151 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  203. R. Linsker, Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Comput. 4, 691–702 (1992)

    Article  Google Scholar 

  204. C. Liu, H. Wechsler, Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition. IEEE Trans. Image Process. 11(4), 467–476 (2002)

    Article  Google Scholar 

  205. H.X. Liu, R.S. Zhang, F. Luan, X.J. Yao, M.C. Liu, Z.D. Hu, B.T. Fan, Diagnosing breast cancer based on support vector machines. J. Chem. Inf. Comput. Sci. 43(3), 900–907 (2003)

    Article  Google Scholar 

  206. Y. Liu, R. Emery, D. Chakrabarti, W. Burgard, S. Thrun, Using EM to learn 3D models of indoor environments with mobile robots, in Proceedingd 18th International Conference Machine Learning, vol. 1, pp. 329–336 (2001)

    Google Scholar 

  207. Z. Liu, D. Chen, H. Bensmail, Gene expression data classification with Kernel principal component analysis. BioMed. Res. Int. 2005(2), 155–159 (2005)

    Google Scholar 

  208. M. Liwicki, A. Graves, H. Bunke, J. Schmidhuber, A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks, in Proceedings 9th International Conference on Document Analysis and Recognition, vol. 1, pp. 367–371 (2007)

    Google Scholar 

  209. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, C. Watkins, Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)

    MATH  Google Scholar 

  210. A.T. Lora, J.M.R. Santos, A.G. Expósito, J.L.M. Ramos, J.C.R. Santos, Electricity market price forecasting based on weighted nearest neighbors techniques. IEEE T. Power Syst. 22(3), 1294–1301 (2007)

    Article  Google Scholar 

  211. F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 4, 1–25 (2007)

    Article  Google Scholar 

  212. J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Face recognition using LDA-based algorithms. IEEE T. Neural Netw. 14(1), 195–200 (2003)

    Article  Google Scholar 

  213. J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, J. Wang, An efficient kernel discriminant analysis method. Pattern Recogn. 38(10), 1788–1790 (2005)

    Article  MATH  Google Scholar 

  214. T.C. Lu, C.Y. Chang, A survey of VQ codebook generation. J. Inf. Hiding Multimed. Sign. Process. 1(3), 190–203 (2010)

    Google Scholar 

  215. M. Luo, Y.F. Ma, H.J. Zhang, A spatial constrained k-means approach to image segmentation, in Proceedings IEEE 4th Conference Information, Communications and Signal Processing, vol. 2, pp. 738–742 (2003)

    Google Scholar 

  216. Y.Z. Ma, J.J. Royer, H. Wang, Y. Wang, T. Zhang, Factorial kriging for multi-scale modeling. J. Southern African Instit. Mining Metall. 114, 651–657 (2014)

    Google Scholar 

  217. O. Makhnin, Introduction to Kriging (Lecture 10, Math 586, New Mexico Institute of Mining and Technology, Department of Mathematics, 2013). http://infohost.nmt.edu/~olegm/586/HYD10.pdf

  218. J. Makhoul, S. Roucos, H. Gish, Vector quantization in speech coding. Proc. IEEE 73(11), 1551–1588 (1985)

    Article  Google Scholar 

  219. A. Mansour, M. Kawamoto, ICA papers classified according to their applications and performances. IEICE T. Fundam. E86-A(3), 620–633 (2003)

    Google Scholar 

  220. A.M. Martínez, A.C. Kak, PCA versus LDA. IEEE T. Pattern Anal. Mach. Intell. 23(2), 228–233 (2001)

    Google Scholar 

  221. B. Matei, A Review of Independent Component Analysis Techniques (Rutgers University School of Engineering, 2000). http://coewww.rutgers.edu/riul/research/tutorials/tutorialica.pdf

  222. G. McLachlan, T. Krishnan, The EM Algorithm and Extensions (Wiley, 2007)

    Google Scholar 

  223. S.A. Medjahed, T.A. Saadi, A. Benyettou, Breast cancer diagnosis by using k-nearest neighbor with different distances and classification rules. Intl. J. Comput. Appl. 62(1) (2013)

    Google Scholar 

  224. S. Mika, J. Ratsch, G. Weston, B. Scholkopf, Fisher discriminant analysis with kernels, in Proceedings IEEE Workshop Neural Networks for Signal Processing IX, vol. 1, pp. 41–48 (1999)

    Google Scholar 

  225. S. Mika, B. Schölkopf, A.J. Smola, K.R. Müller, M. Scholz, G. Rätsch, Kernel PCA and de-noising in feature spaces. NIPS 4(5), 1–7 (1998)

    Google Scholar 

  226. B. Moghaddam, T. Jebara, A. Pentland, Bayesian face recognition. Pattern Recogn. 33(11), 1771–1782 (2000)

    Article  Google Scholar 

  227. T.K. Moon, The expectation-maximization algorithm. IEEE Sign. Process. Mag. 13(6), 47–60 (1996)

    Article  Google Scholar 

  228. L. Morissette, S. Chartier, The k-means clustering technique: general considerations and implementation in mathematica. Tutorials Quant. Meth. Psychol. 91(1), 15–24 (2013)

    Article  Google Scholar 

  229. J. Morra, Z. Tu, A. Toga, P. Thompson, Machine learning for brain image segmentation, eds. by Gonzalez and Romero. Biomedical Image Analysis and Machine Learning Technologies: Applications and Techniques (Medical Information Science Reference, 2009)

    Google Scholar 

  230. J.H. Morra, Z. Tu, L.G. Apostolova, A.E. Green, A.W. Toga, P.M. Thompson, Comparison of AdaBoost and support vector machines for detecting Alzheimer’s disease through automated hippocampal segmentation. IEEE T. Med. Imag. 29(1), 30–43 (2010)

    Article  Google Scholar 

  231. D.G. Morrison, On the interpretation of discriminant analysis. J. Market. Res. 156–163 (1969)

    Google Scholar 

  232. M. Muja, D.G. Lowe, Fast approximate nearest neighbors with automatic algorithm configuration. Proc. VISAPP 1, 331–340 (2009)

    Google Scholar 

  233. S.A. Mulaik, Foundations of Factor Analysis (CRC Press, 2009)

    Google Scholar 

  234. K. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, An introduction to kernel-based learning algorithms. IEEE T. Neural Netw. 12(2), 181–201 (2001)

    Article  Google Scholar 

  235. K. Murphy, Machine Learning: A Probabilistic Perspective (MIT Press, 2012)

    Google Scholar 

  236. G.R. Naik, D.K. Kumar, An overview of independent component analysis and its applications. Informatica 35, 63–81 (2011)

    MATH  Google Scholar 

  237. G.P. Nason, Design and choice of projection indices. Ph.D. thesis, University of Bath, UK, 1992

    Google Scholar 

  238. R.M. Neal, G.E. Hinton, A view of the EM algorithm that justifies incremental, sparse, and other variants, in Learning in Graphical Models, pp. 355–368 (Springer, 1998)

    Google Scholar 

  239. H.P. Ng, S.H. Ong, K.W.C. Foong, P.S. Goh, W.L. Nowinski, Medical image segmentation using k-means clustering and improved watershed algorithm, in Proceedings IEEE Southwest Symposium o Image Analysis and Interpretation, pp. 61–65 (2006)

    Google Scholar 

  240. M.H. Nguyen, F. Torre, Robust kernel principal component analysis, in Proceedings Advances in Neural Information Processing Systems, pp. 1185–1192 (2009)

    Google Scholar 

  241. D. Nguyen-Tuong, M. Seeger, J. Peters, Model learning with local Gaussian process regression. Adv. Robot. 23(15), 2015–2034 (2009)

    Article  Google Scholar 

  242. H. Nickisch, C.E. Rasmussen, Approximations for binary Gaussian process classification. J. Mach. Learn. Res. 9, 2035–2078 (2008)

    MathSciNet  MATH  Google Scholar 

  243. M.G. Omran, A.P. Engelbrecht, A. Salman, An overview of clustering methods. Intell. Data Anal. 11(5), 583–605 (2007)

    Google Scholar 

  244. F. Orabona, J. Keshet, B. Caputo, The projectron: a bounded kernel-based perceptron, in Proceedings of the ACM 25th International Conference Machine Learning, pp. 720–727 (2008)

    Google Scholar 

  245. E. Osuna, R. Freund, F. Girosi, Support vector machines: training and applications. Technical report, MIT, 1997. AI Memo 1602

    Google Scholar 

  246. A. Oursland, J. De Paula, N. Mahmood, Case Studies of Independent Component Analysis (2013). Numerical Analysis of Linear Algebra, CS383C. http://www.oursland.net/tutorials/ica/ica-report.pdf

  247. B. Pardo, Machine Learning, Topic 6: Clustering (Lecture Presentation, Northwestern University, 2009). http://www.cs.northwestern.edu/~pardo/courses/eecs349/lectures/NUEECS349Falltopic6-clustering.pdf

  248. I. Pardoe, X. Yin, R.D. Cook, Graphical tools for quadratic discriminant analysis. Technometrics 49(2) (2007)

    Google Scholar 

  249. H.S. Park, C.H. Jun, A simple and fast algorithm for k-medoids clustering. Expert Syst. Appl. 36(2), 3336–3341 (2009)

    Article  Google Scholar 

  250. L. Parra, P. Sajda, Blind source separation via generalized eigenvalue decomposition. J. Mach. Learn. Res. 4, 1261–1269 (2003)

    MathSciNet  MATH  Google Scholar 

  251. D. Pelleg, A.W. Moore, X-means: extending k-means with efficient estimation of the number of clusters, in Proceedings ICML, pp. 727–734 (2000)

    Google Scholar 

  252. W. Penny, S. Kiebel, K. Friston, Variational bayes, eds. by K. Friston, J. Ashburner, S. Kiebel, T. Nichols, W. Penny. Statistical Parametric Mapping: The Analysis of Functional Brain Images (Elsevier, 2006)

    Google Scholar 

  253. F. Pereira, T. Mitchell, M. Botvinick, Machine learning classifiers and FMRI: a tutorial overview. Neuroimage 45(1), S199–S209 (2009)

    Article  Google Scholar 

  254. D. Petelin, B. Filipi, J. Kocijan, Optimization of Gaussian process models with evolutionary algorithms, in Adaptive and Natural Computing Algorithms, pp. 420–429 (Springer, 2011)

    Google Scholar 

  255. D.L. Pham, C. Xu, J.L. Prince, Current methods in medical image segmentation 1. Ann. Rev. Biomed. Eng. 2(1), 315–337 (2000)

    Article  Google Scholar 

  256. R. Plamondon, S.N. Srihari, Online and off-line handwriting recognition: a comprehensive survey. IEEE T. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)

    Google Scholar 

  257. J.H. Plasse, The EM algorithm in multivariate Gaussian mixture models using Anderson acceleration. Master’s thesis, Worcester Polytechnic Institute, 2013

    Google Scholar 

  258. M. Pohar, M. Blas, S. Turk, Comparison of logistic regression and linear discriminant analysis. Metodoloki Zvezki 1(1), 143–161 (2004)

    Google Scholar 

  259. C. Posse, Tools for two-dimensional exploratory projection pursuit. J. Comput. Graph. Stat. 4(2), 83–100 (1995)

    Google Scholar 

  260. C.E. Rasmussen, C.K.I. Williams, Gaussian Processes for Machine Learning (MIT Press, 2006)

    Google Scholar 

  261. J.D. Rennie, L. Shih, J. Teevan, D.R. Karger, Tackling the poor assumptions of naive Bayes text classifiers. ICML 3, 616–623 (2003)

    Google Scholar 

  262. B.D. Ripley, Pattern Recognition and Neural Networks (Cambridge University Press, 2008)

    Google Scholar 

  263. I. Rish, An empirical study of the naive Bayes classifier, in Proceedings IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, vol. 3, pp. 41–46 (2001)

    Google Scholar 

  264. S. Roberts, M. Osborne, M. Ebden, S. Reece, N. Gibson, S. Aigrain, Gaussian processes for time-series modeling. Philos. Trans. Royal Soc. A: Math. Phys. Eng. Sci. 371(1984), 20110550 (2013)

    Google Scholar 

  265. D.N. Rutledge, D.J-R Bouveresse, Independent component analysis with the JADE algorithm. Trends Anal. Chem. 50, 22–32 (2013)

    Google Scholar 

  266. S. Ryali, K. Supekar, D.A. Abrams, V. Menon, Sparse logistic regression for whole-brain classification of FMRI data. NeuroImage 51(2), 752–764 (2010)

    Article  Google Scholar 

  267. S. Cha, Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Meth. Appl. Sci. 1(4), 300–307 (2007)

    Google Scholar 

  268. H. Sahbi, Kernel PCA for similarity invariant shape recognition. Neurocomputing 70(16), 3034–3045 (2007)

    Article  Google Scholar 

  269. S. Samarasinghe, Neural Networks for Applied Sciences and Engineering (Auerbach Publications, 2006)

    Google Scholar 

  270. J. Sankaranarayanan, H. Samet, A. Varshney, A fast all nearest neighbor algorithm for applications involving large point-clouds. Comput. Graph. 31(2), 157–174 (2007)

    Article  Google Scholar 

  271. G. Schalk, D.J. McFarland, T. Hinterberger, N. Birbaumer, J.R. Wolpaw, BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE T. Biomed. Eng. 51(6), 1034–1043 (2004)

    Article  Google Scholar 

  272. B. Schölkopf, A. Smola, K.R. Müller, Kernel principal component analysis, in Artificial Neural Networks—ICANN’97, pp. 583–588 (Springer, 1997)

    Google Scholar 

  273. B. Schölkopf, A. Smola, K.R. Müller, Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)

    Article  Google Scholar 

  274. M. Seeger, Gaussian processes for machine learning. Int. J. Neural Syst. 14(2), 69–106 (2004)

    Article  Google Scholar 

  275. N. Seo, Eigenfaces and Fisherfaces (University of Maryland, ENEE633 Pattern Recognition, 2007). http://note.sonots.com/SciSoftware/FaceRecognition.html

  276. G. Shakhnarovich, P. Indyk, T. Darrell, Nearest Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006)

    Google Scholar 

  277. S. Shan, B. Cao, W. Gao, D. Zhao, Extended Fisherface for face recognition from a single example image per person, in Proceedings IEEE International Symposium Circuits and Systems, ISCAS 2002, vol. 2 (2002). II-81

    Google Scholar 

  278. J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis (Cambridge University Press, 2004)

    Google Scholar 

  279. H. Shen, J.Z. Huang, Sparse principal component analysis via regularized low rank matrix approximation. J. Multivar. Anal. 99(6), 1015–1034 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  280. Y. Shi, D. Dai, C. Liu, H. Yan, Sparse discriminant analysis for breast cancer biomarker identification and classification. Progr. Nat. Sci. 19(11), 1635–1641 (2009)

    Article  Google Scholar 

  281. J. Shlens, A tutorial on principal component analysis. J. Comput. Graph. Stat. 4(2), 83–100 (2003)

    Google Scholar 

  282. V.K. Singh, N. Tiwari, S. Garg, Document clustering using k-means, heuristic k-means and fuzzy c-means, in Proceedings International Conference Computational Intelligence and Communication, Networks, 2011, pp. 297–301

    Google Scholar 

  283. T.E. Smith, Notebook on Spatial Data Analysis (SEAS, Penn Engineering, 2014). http://www.seas.upenn.edu/~ese502/#notebook

  284. A.J. Smola, B. Schölkopf, A tutorial on support vector regression. Stat. Comput. 14(3), 199–222 (2004)

    Article  MathSciNet  Google Scholar 

  285. S. Sonnenburg, G. Rätsch, B. Schölkopf, Large scale genomic sequence SVM classifiers, in Proceedings 22nd ACM International Conference Machine Learning, pp. 848–855 (2005)

    Google Scholar 

  286. S. Srivastava, M.R. Gupta, B.A. Frigyik, Bayesian quadratic discriminant analysis. J. Mach. Learn. Res. 8(6), 1277–1305 (2007)

    MathSciNet  MATH  Google Scholar 

  287. O. Stegle, S.V. Fallert, D.J. MacKay, S. Brage, Gaussian process robust regression for noisy heart rate data. IEEE T. Biomed. Eng. 55(9), 2143–2151 (2008)

    Article  Google Scholar 

  288. M. Steinbach, G. Karypis, V. Kumar, A comparison of document clustering techniques, in Proceedings KDD Workshop on Text Mining, vol. 400, pp. 525–526 (2000)

    Google Scholar 

  289. J.V. Stone, Independent Component Analysis (MIT Press, 2004)

    Google Scholar 

  290. K. Suzuki, Artificial Neural Networks – Industrial and Control Engineering Applications (InTech, 2011)

    Google Scholar 

  291. R. Tandon, A Survey of Sparse PCA (The University of Texas at Austin, 2012). http://www.cs.utexas.edu/~rashish/sparse_pca.pdf

  292. Y. Tang, Deep Learning Using Linear Support Vector Machines (Department of Computer Science, University of Toronto, 2013). arXiv preprint arXiv:1306.0239

  293. D.M. Taylor, S.I.H. Tillery, A.B. Schwartz, Direct cortical control of 3D neuroprosthetic devices. Science 296(5574), 1829–1832 (2002)

    Article  Google Scholar 

  294. A. Teynor, H. Burkhardt, Fast codebook generation by sequential data analysis for object classification, in Advances in Visual Computing, pp. 610–620 (Springer, 2007)

    Google Scholar 

  295. S. Theodoridis, A. Pikrakis, K. Koutroumbas, D. Cavouras, Introduction to Pattern Recognition (Academic Press, 2010)

    Google Scholar 

  296. P. Tichavsky, Z. Koldovsky, E. Oja, Performance analysis of the FastICA algorithm and Cramer-Rao bounds for linear independent component analysis. IEEE T. Sign. Process. 54(4), 1189–1197 (2006)

    Article  MathSciNet  Google Scholar 

  297. M.E. Tipping, Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1, 211–244 (2001)

    MathSciNet  MATH  Google Scholar 

  298. F. Tombari, L. Di Stefano, A. Lanza, S. Mattoccia, Non-linear parametric Bayesian regression for robust background subtraction, in Proceedings IEEE Workshop onMotion and Video Computing, WMVC’09, pp. 1–7 (2009)

    Google Scholar 

  299. S. Tong, D. Koller, Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2, 45–66 (2002)

    MATH  Google Scholar 

  300. A. Tsai, J. Zhang, A.S. Willsky, Expectation-maximization algorithms for image processing using multiscale models and mean-field theory, with applications to laser radar range profiling and segmentation. Opt. Eng. 40(7), 1287–1301 (2001)

    Article  Google Scholar 

  301. I. Tsochantaridis, T. Joachims, T. Hofmann, Y. Altun, Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 1453–1484 (2005)

    Google Scholar 

  302. D. Tuia, M. Volpi, M. Dalla Mura, A. Rakotomamonjy, R. Flamary, Automatic feature learning for spatio-spectral image classification with sparse SVM. IEEE T. Geosci. Remote Sens. 52(10), 6062–6074 (2014)

    Google Scholar 

  303. M. Turk, A. Pentland, Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  304. I.Y. Turner, E.M. Huff, Principal components analysis of triaxial vibration data from helicopter transmissions, in Proceedings 56th Meeting of the Society for Machinery Failure Prevention Technology, 2002

    Google Scholar 

  305. J.H. van Hateren, A. van der Schaaf, Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. Roy. Soc. London. Series B: Biol. Sci. 265(1394), 359–366 (1998)

    Google Scholar 

  306. V. Vapnik, The Nature of Statistical Learning Theory (Springer, 2000)

    Google Scholar 

  307. A. Vellido, P.J. Lisboa, J. Vaughan, Neural networks in business: a survey of applications (1992–1998). Expert Syst. Appl. 17(1), 51–70 (1999)

    Article  Google Scholar 

  308. E. Vincent, R. Gribonval, C. Févotte, Performance measurement in blind audio source separation. IEEE T. Audio, Speech, Lang. Process. 14(4), 1462–1469 (2006)

    Google Scholar 

  309. U. Von Luxburg, R.C. Williamson, I. Guyon, Clustering: Science or art? ICML Unsupervised and Transfer, Learning, pp. 65–80 (2012)

    Google Scholar 

  310. P. Wagner, Face Recognition with GNU Octave/MATLAB (Cracow University of Technology, Poland, 2012). http://mars.iti.pk.edu.pl/~chmaj/APSC/facerec_octave.pdf

  311. J. Wakefield, Non-linear regression modelling and inference. Meth. Models Stat. 119–153 (2004)

    Google Scholar 

  312. J.Y. Wang, Application of support vector machines in bioinformatics. Ph.D. thesis, National Taiwan University, 2002

    Google Scholar 

  313. Q. Wang, Kernel Principal Component Analysis and Its Applications in Face Recognition and Active Shape Models (Rensselaer Polytechnic Institute, 2012). arXiv preprint arXiv:1207.3538

  314. R. Wang, Adaboost for feature selection, classification and its relation with SVM, a review. Phys. Proc. 25, 800–807 (2012)

    Article  Google Scholar 

  315. K. Wayne, Tutorial 2: Numerical Linear Algebra (Computer Science Dept., Princeton University, 2007. SEAS Short Course Programming in MATLAB). https://www.cs.princeton.edu/~wayne/teaching/linear-algebra.pdf

  316. W.M. Wells III, W.E.L. Grimson, R. Kikinis, F.A. Jolesz, Adaptive segmentation of MRI data. IEEE T. Med. Imag. 15(4), 429–442 (1996)

    Article  Google Scholar 

  317. J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, T.M. Vaughan, Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Article  Google Scholar 

  318. K.P. Wong, D. Feng, S.R. Meikle, M.J. Fulham, Segmentation of dynamic PET images using cluster analysis. IEEE T. Nucl. Sci. 49(1), 200–207 (2002)

    Article  Google Scholar 

  319. J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, Y. Ma, Robust face recognition via sparse representation. IEEE T. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Google Scholar 

  320. M. Xiao, An improved background reconstruction algorithm based on basic sequential clustering. Inf. Technol. J. 7(3), 522–527 (2008)

    Article  Google Scholar 

  321. R. Xu, D. Wunsch, Clustering, vol. 10 (Wiley, 2008)

    Google Scholar 

  322. R. Xu, D.C. Wunsch, Clustering algorithms in biomedical research: a review. IEEE Rev. Biomed. Eng. 3, 120–154 (2010)

    Article  Google Scholar 

  323. I. Yamaguchi, T. Kuzuyoshi, An algebraic solution to independent component analysis. Opt. Commun. 178, 59–64 (2000)

    Article  Google Scholar 

  324. J. Yang, Z. Jin, J.Y. Yang, D. Zhang, A.F. Frangi, Essence of kernel Fisher discriminant: KPCA plus LDA. Pattern Recogn. 37(10), 2097–2100 (2004)

    Article  Google Scholar 

  325. P. Yang, Y. Hwa, Yang, B.B Zhou, A.Y. Zomaya, A review of ensemble methods in bioinformatics. Curr. Bioinf. 5(4), 296–308 (2010)

    Google Scholar 

  326. P.N. Yianilos, Data structures and algorithms for nearest neighbor search in general metric spaces, in Proceedings of the fourth annual ACM-SIAM Symposium on Discrete algorithms, pp. 311–321 (Society for Industrial and Applied Mathematics, 1993)

    Google Scholar 

  327. D. You, O.C. Hamsici, A.M. Martinez, Kernel optimization in discriminant analysis. IEEE T. Pattern Anal. Mach. Intell. 33(3), 631–638 (2011)

    Google Scholar 

  328. V. Zarzoso, P. Comon, M. Kallel, How fast is FastICA? in Proc. EUSIPCO-2006 (2006)

    Google Scholar 

  329. C. Zhang, Z. Zhang, A survey of recent advances in face detection. Technical report, Microsoft Research, 2010

    Google Scholar 

  330. S.X. Zhang, C. Liu, K. Yao, Y. Gong, Deep Neural Support Vector Machines for Speech Recognition (Microsoft Research, 2015). http://research.microsoft.com/pubs/244711/0004275.pdf

  331. W. Zhao, R. Chellappa, P.J. Phillips, A. Rosenfeld, Face recognition: a literature survey. ACM Comput. Surv. (CSUR) 35(4), 399–458 (2003)

    Article  Google Scholar 

  332. J. Zhu, H. Zou, S. Rosset, T. Hastie, Multi-class AdaBoost. Stat. Interf. 2(3), 349–360 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  333. M. Zibulevsky, B. Pearlmutter, Blind source separation by sparse decomposition in a signal dictionary. Neural Comput. 13(4), 863–882 (2001)

    Article  MATH  Google Scholar 

  334. H. Zou, T. Hastie, R. Tibshirani, Sparse principal component analysis. J. Comput. Graph. Stat. 15(2), 265–286 (2006)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Systems Engineering and Automatic Control, Universidad Complutense de Madrid, Madrid, Spain

    Jose Maria Giron-Sierra

Authors
  1. Jose Maria Giron-Sierra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose Maria Giron-Sierra .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Giron-Sierra, J.M. (2017). Data Analysis and Classification. In: Digital Signal Processing with Matlab Examples, Volume 2. Signals and Communication Technology. Springer, Singapore. https://doi.org/10.1007/978-981-10-2537-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-2537-2_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-2536-5

  • Online ISBN: 978-981-10-2537-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation