Skip to main content
SpringerLink
Log in

Nonlinear multi-output regression on unknown input manifold

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Consider unknown smooth function which maps high-dimensional inputs to multidimensional outputs and whose domain of definition is unknown low-dimensional input manifold embedded in an ambient high-dimensional input space. Given training dataset consisting of ‘input-output’ pairs, regression on input manifold problem is to estimate the unknown function and its Jacobian matrix, as well to estimate the input manifold. By transforming high-dimensional inputs in their low-dimensional features, initial regression problem is reduced to certain regression on feature space problem. The paper presents a new geometrically motivated method for solving both interrelated regression problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Vapnik, V.: Statistical Learning Theory. John Wiley, New York (1998)

    MATH  Google Scholar 

  2. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)

    Book  MATH  Google Scholar 

  3. James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning with Applications in R. Springer Texts in Statistics. Springer, New York (2013)

    MATH  Google Scholar 

  4. Bishop, C. M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2007)

    MATH  Google Scholar 

  5. Deng, L., Yu, D.: Deep Learning: Methods and Applications. NOW Publishers, Boston (2014)

    MATH  Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  7. Friedman, J. H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  8. Rasmussen, C. E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  9. Loader, C.: Local Regression and Likelihood. Springer, New York (1999)

    MATH  Google Scholar 

  10. Bernstein, A. V., Kuleshov, A. P., Yanovich, Y.A.: Manifold learning in regression tasks. In: Gammerman, A., Vovk, V., Papadopoulos, H (eds.) SLDS 2015. LNCS, vol. 9047, pp 414–423. Springer, Heidelberg (2015)

    Google Scholar 

  11. Burnaev, E., Vovk, V.: Efficiency of conformalized ridge regression. JMLR W&,CP 35, 605–622 (2014)

    Google Scholar 

  12. Burnaev, E., Panov, M., Zaytsev, A.: Regression on the basis of nonstationary gaussian processes with bayesian regularization. J. Commun. Technol. Electron. 61 (6), 661–671 (2016)

    Article  Google Scholar 

  13. Burnaev, E., Nazarov, I.: Conformalized Kernel Ridge Regression Proceedings of the 15 th IEEE International Conference on Machine Learning and Applications (ICMLA 16), pp 45–52. IEEE Computer Society, USA (2016)

    Google Scholar 

  14. Polyak, B. T.: Introduction to optimization. Translations Series in Mathematics and Engineering. Optimization Software Inc. Publications Division, New York (1987)

    Google Scholar 

  15. Bertsekas, D. P.: Nonlinear Programming, 2nd edn. Belmont MA, Athena Scientific (1999)

    MATH  Google Scholar 

  16. Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004)

    Book  MATH  Google Scholar 

  17. Wang, G., Gary Shan, S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(3), 370–381 (2007)

    Article  Google Scholar 

  18. Forrester, A. I. J., Sobester, A., Keane, A. J.: Engineering Design via Surrogate Modelling. A Practical Guide. Wiley, New York (2008)

    Book  Google Scholar 

  19. Kuleshov, A. P., Bernstein, A. V.: Cognitive technologies in adaptive models of complex plants. Inf. Control. Probl. Manuf. 13(1), 1441–1452 (2009)

    Google Scholar 

  20. Stone, C. J.: Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  21. Stone, C. J.: Optimal global rates of convergence for nonparametric regression. Ann. Stat. 10, 1040–1053 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  22. Rajaram, D., Pant, R. S.: An improved methodology for airfoil shape optimization using surrogate based design optimization. In: Rodrigues, H. et al. (eds.) Engineering Optimization IV, pp 147–152. CRC Press, Taylor & Francis Group, London (2015)

    Google Scholar 

  23. Bernstein, A., Kuleshov, A., Sviridenko, Y., Vyshinsky, V.: Fast aerodynamic model for design technology Proceedings of West-East High Speed Flow Field Conference (WEHSFF-2007), Moscow, Russia (2007). http://wehsff.imamod.ru/pages/s7.htm

    Google Scholar 

  24. Zhu, F., Qin, N., Burnaev, E. V., Bernstein, A. V., Chernova, S. S.: Comparison of three geometric parameterization methods and their effect on aerodynamic optimization. In: Poloni, C. et al. (eds.) Eurogen 2011, Optimization and Control with Applications to Industrial and Societal Problems International Conference on Proceedings - Evolutionary and Deterministic Methods for Design, pp 758–772. Sira, Capua, Italy (2011)

    Google Scholar 

  25. Hu, J., Tian, J., Yang, L.: Functional feature embedded space mapping of fMRI data Proceedings of the 28 th annual IEEE International Conference Engineering in Medicine and Biology Society (EMBS’2006), pp. 1014–1017 (2006)

    Google Scholar 

  26. Shen, X., Meyer, F. G.: Analysis of Event-Related fMRI Data Using Diffusion Maps. Lecture Notes in Computer Science, vol. 3565/2005 Information Processing in Medical Imaging, pp. 652–663. Springer, Berlin/Heidelberg (2005)

    Google Scholar 

  27. Thirion, B., Faugeras, O.: Nonlinear dimension reduction of fMRI data: The Laplacian embedding approach Proceedings of the IEEE International Symposium on Biomedical Imaging: Nano to Macro, vol. 1, pp 372–375. IEEE, Publisher (2004)

    Google Scholar 

  28. Thirion, B., Faugeras, O.: Low Dimensional Embedding of fMRI datasets arXiv:0709.3121v2 [stat.ML], 16 Jan 2008, pp. 1–16 (2008)

    Google Scholar 

  29. Mannfolk, P., Wirestam, R., Nilsson, M., Ståhlberg, F., Olsrud, J.: Dimensionality reduction of fMRI time series data using locally linear embedding. Magn. Reson. Mater. Phys., Biol. Med. 23(5), 327–338 (2010)

    Article  Google Scholar 

  30. Gerber, S., Tasdizena, T., Fletcher, P. T., Joshia, S., Whitaker, R.: On the manifold structure of the space of brain images. Lecture Notes in Computer Science, vol. 5761 Medical Image Computing and Computer-Assisted Intervention, pp. 305–312. Springer, Heidelberg (2009)

    Google Scholar 

  31. Gerber, S., Tasdizena, T., Fletcher, P. T., Joshia, S., Whitaker, R.: Manifold modeling for brain population analysis. Med. Image Anal. 14(5), 643–653 (2010)

    Article  Google Scholar 

  32. Fletcher, P. T.: Geodesic regression on Riemannian manifolds Proceedings of International Workshop on Mathematical Foundations of Computational Anatomy (MFCA), pp. 75–86 (2011)

    Google Scholar 

  33. Fletcher, P. T.: Geodesic regression and the theory of least squares on riemannian manifolds. Int. J. Comput. Vis. 105(2), 171–185 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  34. Banerjee, M., Chakraborty, R., Ofori, E., Vaillancourt, D., Vemuri, B. C.: Nonlinear regression on Riemannian manifolds and its applications to Neuro-image analysis. Lecture Notes in Computer Science, vol. 9349 Medical Image Computing and Computer-Assisted Intervention, Part I, pp 719–727. Springer, Heidelberg (2015)

    Google Scholar 

  35. Marcus, D. S., Wang, T. H., et al.: OASIS: Cross-Sectional mri data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19 (9), 1498–1507 (2007)

    Article  Google Scholar 

  36. Seung, H. S., Lee, D. D.: The manifold ways of perception. Science 290 (5500), 2268–2269 (2000)

    Article  Google Scholar 

  37. Huo, X., Ni, X., Smith, A. K.: Survey of manifold-based learning methods. In: Liao, T. W., Triantaphyllou, E. (eds.) Recent Advances in Data Mining of Enterprise Data, pp 691–745. World Scientific, Singapore (2007)

    Google Scholar 

  38. Ma, Y., Fu, Y. (eds.): Manifold Learning Theory and Applications. CRC Press, London (2011)

  39. Bernstein, A. V., Kuleshov, A. P.: Low-Dimensional Data representation in data analysis. In: EI Gayar, N., Schwenker, F., Suen, C (eds.) ANNPR 2014. LNCS, vol. 8774, pp 47–58. Springer, Heidelberg (2014)

  40. Kuleshov, A. P., Bernstein, A. V.: Manifold learning in data mining tasks. In: Perner, P (ed.) MLDM 2014. LNCS, vol. 8556, pp 119–133. Springer, Heidelberg (2014)

  41. Pelletier, B.: Nonparametric regression estimation on closed Riemannian manifolds. J. Nonparametric Stat. 18(1), 57–67 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  42. Loubes, J. -M., Pelletier, B.: A kernel-based classifier on a Riemannian manifold. Statistics and Decisions 26(1), 35–51 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  43. Steinke, F., Hein, M., Schölkopf, B.: Nonparametric regression between general Riemannian manifolds. SIAM J. Imaging Sci. 3(3), 527–563 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  44. Levina, E., Bickel, P. J.: Maximum likelihood estimation of intrinsic dimension. In: Saul, L., Weiss, Y., Bottou, L (eds.) Advances in Neural Information Processing Systems, vol. 17, pp 777–784. MIT Press, Cambridge (2005)

  45. Fan, M., Qiao, H., Zhang, B.: Intrinsic dimension estimation of manifolds by incising balls. Pattern Recogn. 42, 780–787 (2009)

    Article  MATH  Google Scholar 

  46. Fan, M., Gu, N., Qiao, H., Zhang, B.: Intrinsic dimension estimation of data by principal component analysis arXiv:1002.2050v1 [cs.CV], 10 Feb 2010, pp 1–8 (2010)

    Google Scholar 

  47. Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E., Campadelli, P.: IDEA: Intrinsic Dimension estimation algorithm. In: Maino, G., Foresti, G. (eds.) ICIAP 2011, Part I. LNCS, vol. 6978, pp 433–442. Springer, Heidelberg (2011)

    Google Scholar 

  48. Campadelli, P., Casiraghi, E., Ceruti, C., Rozza, A.: Intrinsic Dimension Estimation: Relevant Techniques and a Benchmark Framework. Mathematical Problems in Engineering vol. 2015, Article ID 759567, pp. 1–21 (2015)

  49. Banerjee, M., Chakraborty, R., Ofori, E., Okun, M. S., Vaillancourt, D., Vemuri, B. C.: A nonlinear regression technique for manifold valued data with applications to Medical Image Analysis Conference: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4424–4432 (2016)

    Google Scholar 

  50. Hinkle, J., Muralidharan, P., Fletcher, P. T.: Polynomial regression on riemannian manifolds arXiv:1201.2395v2 [math.ST], 1 Mar 2012, pp. 1–14 (2012)

    Google Scholar 

  51. Liu, G., Lin, Z., Yu, Y.: Multi-output regression on the output manifold. Pattern Recogn. 42(11), 2737–2743 (2009)

    Article  MATH  Google Scholar 

  52. Shi, X., Styner, M., Lieberman, J., Ibrahim, J. G., Lin, W., Zhu, H.: Intrinsic regression models for manifold-valued data. J. Amer. Stat. Assoc. 5762(1), 192–199 (2009)

    Google Scholar 

  53. Kim, H. J., Bendlin, B. B., et al.: Multivariate general linear models (MGLM) on riemannian manifolds with applications to statistical analysis of diffusion weighted images Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2705–2712 (2014)

    Google Scholar 

  54. Kim, H. J., Adluru, N., et al.: Canonical Correlation analysis on Riemannian Manifolds and its Applications. Lecture Notes in Computer Science, vol. 8690 Computer Vision – ECCV 2014, pp. 251–267. Springer, Heidelberg (2014)

    Google Scholar 

  55. Bickel, P., Li, B.: Local polynomial regression on unknown manifolds. IMS Lecture notes – Monograph Series, vol. 54 Complex Datasets and Inverse Problems: Tomography Networks and Beyond, pp. 177–186 (2007)

  56. Aswani, A., Bickel, P., Tomlin, C.: Regression on manifolds: Estimation of the exterior derivative. Ann. Stat. 39(1), 48–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  57. Cheng, M. -Y., Wu, H. -T.: Local Linear Regression on Manifolds and its Geometric Interpretation. J. Am. Stat. Assoc. 108(504), 1421–1434 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  58. Yang, Y., Dunson, D. B.: Bayesian manifold regression arXiv:1305.0167v2 [math.ST], 16 June 2014, pp. 1–40 (2014)

    Google Scholar 

  59. Einbeck, J., Evers, L.: Localized regression on principal manifolds Proceedings of the 25 th International Workshop on Statistical Modeling (IWSM 2010), pp. 1–6 (2010)

    Google Scholar 

  60. Niyogi, P., Smale, S., Weinberger, S.: Finding the homology of submanifolds with high confidence from random samples. Discrete Comput. Geom. 39, 419–441 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  61. Pennec, X.: Probabilities and statistics on riemannian manifolds: Basic tools for geometric measurements Proceedings of the International Workshop on Nonlinear Signal and Image Processing (NSIP’99), pp. 194–198 (1999)

    Google Scholar 

  62. Yanovich, Y.U.: Asymptotic properties of local sampling on manifolds. J. Math. Stat. 12(3), 157–175 (2016)

    Article  Google Scholar 

  63. Adragni, K. P., Cook, R. D.: Sufficient Dimension Reduction and prediction in Regression. Phil. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 367(1906), 4385–4405 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  64. Lee, K. -Y., Li, B., Chiaromonte, F.: A general theory for nonlinear sufficient dimension reduction: formulation and estimation. Ann. Stat. 41(1), 221–249 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  65. Chen, J., Deng, S. -J., Huo, X.: Electricity price curve modeling and forecasting by manifold learning. IEEE Trans. Power Syst. 23(3), 877–888 (2008)

    Article  Google Scholar 

  66. Lee, J. A., Verleysen, M.: Quality assessment of dimensionality reduction based on k-ary neighborhoods. In: Saeys, Y. et al. (eds.) JMLR Workshop and Conference Proceedings New Challenges for Feature Selection in Data Mining and Knowledge Discovery, vol. 4, pp 21–35. Antwerpen, Belgium (2008)

    Google Scholar 

  67. Lee, J. A., Verleysen, M.: Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing 72(7-9), 1431–1443 (2009)

    Article  Google Scholar 

  68. Bernstein, A.V., Kuleshov, A.P.: Data-based manifold reconstruction via tangent bundle manifold learning ICML-2014 Workshop Topological Methods for Machine Learning, Beijing, China, June 2014. http://topology.cs.wisc.edu/KuleshovBernstein.pdf(2014)

    Google Scholar 

  69. Freedman, D.: Efficient simplicial reconstructions of manifold from their samples. IEEE Trans. Pattern Anal. Mach. Intell. 24(10), 1349–1357 (2002)

    Article  Google Scholar 

  70. Levin, D.: Mesh-independent surface interpolation. In: Hamann, Brunnett, Mueller (eds.) Mathematics and Visualization Series, Geometric Modeling for Scientific Visualization, part 1, pp 37–49. Springer, Berlin, Heidelberg (2003)

  71. Kolluri, R., Shewchuk, J. R., O’Brien, J. F.: Spectral surface reconstruction from noisy point clouds Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP’04), pp 11–21. ACM Press, New York (2004)

  72. Boissonnat, J. -D., Ghosh, A.: Manifold reconstruction using tangential Delaunay complexes. Discrete Comput. Geom. 51(1), 221–267 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  73. Karygianni, S., Frossard, P.: Tangent-based manifold approximation with locally linear models. Signal Process. 104, 232–247 (2014)

    Article  Google Scholar 

  74. Canas, G. D., Poggio, T., Rosasco, L.: Learning manifolds with K-Means and K-Flats arXiv:1209.1121v4 [cs.LG], 19 Feb 2013, pp. 1–19 (2013)

  75. Jollie, T.: Principal component analysis. Springer, New York (2002)

  76. Kramer, M.: Nonlinear Principal Component Analysis using autoassociative neural networks. AIChE J 37(2), 233–243 (1991)

    Article  Google Scholar 

  77. Hinton, G. E., Salakhutdinov, R. R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  78. Saul, L. K., Roweis, S. T.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  79. Saul, L. K., Roweis, S. T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res. 4, 119–155 (2003)

    MathSciNet  MATH  Google Scholar 

  80. Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimension reduction via local tangent space alignment. SIAM J. Sci. Comput. 26(1), 313–338 (2005)

    Article  MATH  Google Scholar 

  81. Bernstein, A. V., Kuleshov, A. P.: Tangent bundle manifold learning via Grassmann&Stiefel eigenmaps arXiv:1212.6031v1 [cs.LG], December 2012, pp. 1–25 (2012)

  82. Kuleshov, A., Bernstein, A., Yanovich, Y.U.: Asymptotically optimal method in Manifold estimation. In: Márkus, L., Prokaj, V. (eds.) Abstracts of the 29th European Meeting of Statisticians (EMS-2003), Budapest, Hungary, 20-25 July 2013, p. 325 (2013)

  83. Law, M. H. C., Jain, A. K.: Nonlinear manifold learning for data stream. In: Berry, M., Dayal, U., Kamath, C., Skillicorn, D (eds.) Proceedings of the 4 th SIAM International Conference on Data Mining, Like Buena Vista, Florida, USA, 33–44 (2004)

  84. Law, M. H. C., Jain, A. K.: Incremental nonlinear dimensionality reduction by manifold learning. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 377–391 (2006)

    Article  Google Scholar 

  85. Gao, X., Jiye Liang, J.: An improved incremental nonlinear dimensionality reduction for isometric data embedding. Inf. Process. Lett. 115(4), 492–501 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  86. Kouropteva, O., Okun, O., Pietikäinen, M.: Incremental locally linear embedding algorithm. In: Kalviainen, H. et al. (eds.) 14th Scandinavian Conference on Image Analysis (SCIA) 2005, LNCS, vol. 3540, pp 521–530. Springer-Verlag, Berlin, Heidelberg (2005)

  87. Kouropteva, O., Okun, O., Pietikäinen, M.: Incremental locally linear embedding. Pattern Recogn. 38(10), 1764–1767 (2005)

    Article  MATH  Google Scholar 

  88. Schuon, S., Ðurković, M., Diepold, K., Scheuerle, J., Markward, S.: Truly Incremental Locally Linear Embedding Proceedings of the CoTeSys 1st International Workshop on Cognition for Technical Systems (6-8 October 2008), munich, Germany, 5 pp (2008)

  89. Jia, P., Yin, J., Huang, X., Hu, D.: Incremental Laplacian eigenmaps by preserving adjacent information between data points. Pattern Recogn. Lett. 30(16), 1457–1463 (2009)

    Article  Google Scholar 

  90. Liu, X., Yin, J., Feng, Z., Dong, J.: Incremental Manifold Learning Via Tangent Space Alignment. Schwenker, F., Marinai, S. (eds.) ANNPR 2006, LNAI, Vol. 4087, pp. 107–121. Springer-Verlag. Berlin, Heidelberg (2006)

  91. Abdel-Mannan, O., Ben Hamza, A., Youssef, A.: Incremental line tangent space alignment algorithm Proceedings of 2007 Canadian Conference on Electrical and Computer Engineering (CCECE 2007), 22-26 April 2007, Vancouver, pp. 1329–1332. IEEE (2007)

  92. Han, Z., Meng, D.-Y., Xu, Z.-B., Gu, N.-N.: Incremental alignment manifold learning. J. Comput. Sci. Technol. 26(1), 153–165 (2011)

    Article  Google Scholar 

  93. Chao Tan, C., Guan, J.: A new manifold learning algorithm based on incremental spectral decomposition. In: Zhou, S., Zhang, S., Karypis, G (eds.) 8th International Conference on Advanced Data Mining and Applications (ADMA) Nanjing, China, December 15-18, 2012, LNCS, vol. 7713, pp 149–160. Springer-Verlag, Berlin, Heidelberg (2012)

  94. Kuleshov, A., Bernstein, A.: Incremental construction of low-dimensional data representations. In: Schwenker, F., EI Gayar, N., Abbas, H. M., Trentin, E (eds.) ANNPR 2016. LNCS, vol. 9896, pp 55– 67. Springer, Heidelberg (2016)

  95. Golub, G. H., Van Loan, C. F.: Matrix Computation, 3rd edn. Johns Hopkins University Press, Baltimore, MD (1996)

  96. Bernstein, A. V., Kuleshov, A. P.: Manifold Learning: generalizing ability and tangent proximity. Int. J. Softw. Eng. Inform. 7(3), 359–390 (2013)

    Google Scholar 

  97. Jost, J.: Riemannian Geometry and Geometric Analysis, 6th edn. Springer-Verlag, Berlin, Heidelberg (2011)

  98. Lee, J. M.: Manifolds and Differential Geometry Graduate Studies in Mathematics, vol. 107. American Mathematical Society, Providence (2009)

  99. Perrault-Joncas, D., Meilă, M.: Non-linear dimensionality reduction: riemannian metric estimation and the problem of geometric recovery arXiv:1305.7255v1 [stat.ML], 30 May 2013, pp. 1–25 (2013)

  100. Kuleshov, A. P., Bernstein, A. V.: Extended Regression on Manifolds Estimation. Lecture Notes in Artificial Intelligence, vol. 9653 Conformal and Probabilistic Prediction with Applications, pp. 208–228. Springer, Heidelberg (2016)

  101. Singer, A., Wu, H. -T.: Vector diffusion maps and the connection laplacian. Commun. Pure. Appl. Math. 65(8), 1067–1144 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  102. Tyagi, H., Vural, E., Frossard, P.: Tangent space estimation for smooth embeddings of Riemannian manifold arXiv:1208.1065v2 [stat.CO], 17 May 2013, pp. 1–35 (2013)

  103. Kaslovsky, D. N., Meyer, F. G.: Non-asymptotic analysis of tangent space perturbation. Inf. Inf. J. IMA 3(2), 134–187 (2014)

    MathSciNet  MATH  Google Scholar 

  104. Hamm, J.: Lee, Daniel D.: Grassmann discriminant analysis: a unifying view on subspace-based learning Proceedings of the 25 th International Conference on Machine Learning (ICML 2008), pp. 376–383 (2008)

  105. Wolf, L., Shashua, A.: Learning over sets using kernel principal angles. J. Mach. Learn. Res. 4, 913–931 (2003)

    MathSciNet  MATH  Google Scholar 

  106. Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to Algorithms. MIT Press, Cambridge (2001)

  107. Fredman, M., Tarjan, R. E.: Fibonacci heaps and their uses in improved network optimization algorithms. J. Assoc. Comput. Mach. 34(3), 596–615 (1987)

    Article  MathSciNet  Google Scholar 

  108. Bernstein, A. V., Kuleshov, A. P., Yanovich, Y.A.: Information preserving and locally isometric&conformal embedding via tangent manifold learning Proceedings of the International IEEE Conference on Data Science and Advanced Analytics (DSAA 2015), pp 1–9. IEEE Computer Society, Piscataway, USA (2015)

  109. Trefethen, L.N., Bau, D. III: Numerical. Linear Algebra. SIAM, Philadelphia (1997)

  110. Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics, Berlin (2007)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Skolkovo Institute of Science and Technology, Moscow, Russia

    Alexander Kuleshov & Alexander Bernstein

  2. National Research University Higher School of Economics, Moscow, Russia

    Alexander Kuleshov & Alexander Bernstein

Authors
  1. Alexander Kuleshov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Bernstein.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuleshov, A., Bernstein, A. Nonlinear multi-output regression on unknown input manifold. Ann Math Artif Intell 81, 209–240 (2017). https://doi.org/10.1007/s10472-017-9551-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-017-9551-0

Keywords

Mathematics Subject Classification (2010)

Search

Navigation