Abstract
The theory of multivariate neural network operators in a Kantorovich type version is here introduced and studied. The main results concerns the approximation of multivariate data, with respect to the uniform and Lp norms, for continuous and Lp functions, respectively. The above family of operators, are based upon kernels generated by sigmoidal functions. Multivariate approximation by constructive neural network algorithms are useful for applications to neurocomputing processes involving high dimensional data. At the end of the paper, several examples of sigmoidal functions for which the above theory holds have been presented.
The authors are members of the Gruppo Nazionale per l’Analisi Matematica, la Probabilitá e le loro Applicazioni (GNAMPA) of the Istituto Nazionale di Alta Matematica (INdAM). This work was supported by University of Perugia - Department of Mathematics and Computer Sciences. Moreover, the first author of the paper has been partially supported within the GNAMPA-INdAM Project 2016 “Problemi di regolarità nel Calcolo delle Variazioni e di Approssimazione”.
Danilo Costarelli orcid ID: 0000-0001-8834-8877
Gianluca Vinti orcid ID: 0000-0002-9875-2790
References
[1] Amari, S.—Cichocki, A.—Yang, H. H.: A new learning algorithm for blind signal separation, Advances in neural information processing systems (1996), 757–763.Search in Google Scholar
[2] Anastassiou, G. A.: Intelligent Systems: Approximation by Artificial Neural Networks. In: Intelligent Systems Reference Library 19, Springer-Verlag, Berlin, 2011.10.1007/978-3-642-21431-8Search in Google Scholar
[3] Anastassiou, G. A.—Coroianu, L.—Gal, S. G.: Approximation by a nonlinear Cardaliaguet-Euvrard neural network operator of max-product kind, J. Comp. Anal. Appl. 12 (2010), 396–406.10.1007/978-3-642-17098-0_18Search in Google Scholar
[4] Angeloni, L.—Vinti, G.: Approximation in variation by homothetic operators in multidimensional setting, Differential Integral Equations 26 (2013), 655–674.10.57262/die/1363266083Search in Google Scholar
[5] Angeloni, L.—Vinti, G.: Convergence and rate of approximation in
[6] Angeloni, L.—Vinti, G.: A characterization of absolute continuity by means of Mellin integral operators, Z. Anal. Anwend. 34 (2015), 343–356.10.4171/ZAA/1543Search in Google Scholar
[7] Bardaro, C.—Butzer, P. L.—Stens, R. L.—Vinti, G.: Kantorovich-type generalized sampling series in the setting of Orlicz spaces, Samp. Theory Signal Image Proc. 6 (2007), 29–52.10.1007/BF03549462Search in Google Scholar
[8] Bardaro, C.—Karsli, H.—Vinti, G.: Nonlinear integral operators with homogeneous kernels: Pointwise approximation theorems, Appl. Anal. 90 (2011), 463–474.10.1080/00036811.2010.499506Search in Google Scholar
[9] Bardaro, C.—Musielak, J.—Vinti, G.: Nonlinear Integral Operators and Applications. De Gruyter Ser. Nonlinear Anal. Appl. 9, New York, Berlin, 2003.10.1515/9783110199277Search in Google Scholar
[10] Barron, A. R.: Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inform. Theory 39 (1993), 930–945.10.1109/18.256500Search in Google Scholar
[11] Boccuto, A.— Bukhvalov, A. V.—Sambucini, A. R.: Inequalities in classical spaces with mixed norms, Positivity 6 (2002), 393–411.10.1023/A:1021353215312Search in Google Scholar
[12] Boccuto, A.— Candeloro, D.—Sambucini, A. R.: Vitali-type theorems for filter convergence related to Riesz space-valued modulars and applications to stochastic processes, J. Math. Anal. Appl. 419 (2014), 818–838.10.1016/j.jmaa.2014.05.014Search in Google Scholar
[13] Cao, F.—Chen, Z.: The approximation operators with sigmoidal functions, Comput. Math. Appl. 58 (2009), 758–765.10.1016/j.camwa.2009.05.001Search in Google Scholar
[14] Cao, F.—Chen, Z.: The construction and approximation of a class of neural networks operators with ramp functions, J. Comput. Anal. Appl. 14 (2012), 101–112.Search in Google Scholar
[15] Cao, F.—Chen, Z.: Scattered data approximation by neural networks operators, in print in: Neurocomputing (2016), https://doi.org/10.1016/j.neucom.2016.01.013.10.1016/j.neucom.2016.01.013Search in Google Scholar
[16] Cao, F.—Liu, B.—Park, D. S.: Image classification based on effective extreme learning machine, Neurocomputing 102 (2013), 90–97.10.1016/j.neucom.2012.02.042Search in Google Scholar
[17] Cardaliaguet, P.—Euvrard, G.: Approximation of a function and its derivative with a neural network, Neural Networks 5 (1992), 207–220.10.1016/S0893-6080(05)80020-6Search in Google Scholar
[18] Cheang, G. H. L.: Approximation with neural networks activated by ramp sigmoids, J. Approx. Theory 162 (2010), 1450–1465.10.1016/j.jat.2010.03.004Search in Google Scholar
[19] Cheney, E. W.—Light, W. A.—Xu, Y.: Constructive methods of approximation by ridge functions and radial functions, Numer. Algorithms 4 (1993), 205–223.10.1007/BF02144104Search in Google Scholar
[20] Cluni, F.—Costarelli, D.—Minotti, A. M.—Vinti, G.: Applications of sampling Kantorovich operators to thermographic images for seismic engineering, J. Comp. Anal. Appl. 19 (2015), 602–617.Search in Google Scholar
[21] Cluni, F.—Costarelli, D.—Minotti, A. M.—Vinti, G.: Enhancement of thermographic images as tool for structural analysis in earthquake engineering, NDT & E International 70 (2015), 60–72.10.1016/j.ndteint.2014.10.001Search in Google Scholar
[22] Coroianu, L.—Gal, S. G.: Approximation by nonlinear generalized sampling operators of max-product kind, Sampl. Theory Signal Image Process. 9 (2010), 59–75.10.1007/BF03549524Search in Google Scholar
[23] Coroianu, L.—Gal, S. G.: Approximation by max-product sampling operators based on sinc-type kernels, Sampl. Theory Signal Image Process. 10 (2011), 211–230.10.1007/BF03549542Search in Google Scholar
[24] Coroianu, L.—Gal, S. G.: Saturation results for the truncated max-product sampling operators based on sinc and Fejér-type kernels, Sampl. Theory Signal Image Process. 11 (2012), 113–132.10.1007/BF03549552Search in Google Scholar
[25] Coroianu, L.—Gal, S. G.: Saturation and inverse results for the Bernstein max-product operator, Period. Math. Hungar. 69 (2014), 126–133.10.1007/s10998-014-0062-zSearch in Google Scholar
[26] Costarelli, D.: Interpolation by neural network operators activated by ramp functions, J. Math. Anal. Appl. 419 (2014), 574–582.10.1016/j.jmaa.2014.05.013Search in Google Scholar
[27] Costarelli, D.: Neural network operators: constructive interpolation of multivariate functions, Neural Networks 67 (2015), 28–36.10.1016/j.neunet.2015.02.002Search in Google Scholar PubMed
[28] Costarelli, D.—Spigler, R.: Solving Volterra integral equations of the second kind by sigmoidal functions approximation, J. Integral Equations Appl. 25 (2013), 193–222.10.1216/JIE-2013-25-2-193Search in Google Scholar
[29] Costarelli, D.—Spigler, R.: Convergence of a family of neural network operators of the Kantorovich type, J. Approx. Theory 185 (2014), 80–90.10.1016/j.jat.2014.06.004Search in Google Scholar
[30] Costarelli, D.—Spigler, R.: A collocation method for solving nonlinear Volterra integro-differential equations of the neutral type by sigmoidal functions, J. Integral Equations Appl. 26 (2014), 15–52.10.1216/JIE-2014-26-1-15Search in Google Scholar
[31] Costarelli, D.—Spigler, R.: Approximation by series of sigmoidal functions with applications to neural networks, Ann. Mat. Pura Appl. 194 (2015), 289–306.10.1007/s10231-013-0378-ySearch in Google Scholar
[32] Costarelli, D.—Spigler, R.: Solving numerically nonlinear systems of balance laws by multivariate sigmoidal functions approximation, in print in: Comp. Appl. Math. (2016). https://doi.org/10.1007/s40314-016-0334-8.10.1007/s40314-016-0334-8Search in Google Scholar
[33] Costarelli, D.—Vinti, G.: Order of approximation for sampling Kantorovich operators, J. Integral Equations Appl. 26 (2014), 345–368.10.1216/JIE-2014-26-3-345Search in Google Scholar
[34] Costarelli, D.—Vinti, G.: Rate of approximation for multivariate sampling Kantorovich operators on some functions spaces, J. Integral Equations Appl. 26 (2014), 455–481.10.1216/JIE-2014-26-4-455Search in Google Scholar
[35] Costarelli, D.—Vinti, G.: Degree of approximation for nonlinear multivariate sampling Kantorovich operators on some functions spaces, Num. Funct. Anal. Optim. 36 (2015), 964–990.10.1080/01630563.2015.1040888Search in Google Scholar
[36] Costarelli, D.—Vinti, G.: Max-product neural network and quasi interpolation operators activated by sigmoidal functions, J. Approx. Theory 209 (2016), 1–22.10.1016/j.jat.2016.05.001Search in Google Scholar
[37] Costarelli, D.—Vinti, G.: Pointwise and uniform approximation by multivariate neural network operators of the max-product type, Neural Networks 81 (2016), 81–90.10.1016/j.neunet.2016.06.002Search in Google Scholar
[38] Costarelli, D.—Vinti, G.: Approximation by max-product neural network operators of Kantorovich type, Results Math. 69 (2016), 505–519.10.1007/s00025-016-0546-7Search in Google Scholar
[39] Cybenko, G.: Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems 2 (1989), 303–314.10.1007/BF02551274Search in Google Scholar
[40] Di Marco, M.—Forti, M.—Grazzini, M.—Pancioni, L.: Necessary and sufficient condition for multistability of neural networks evolving on a closed hypercube, Neural Networks 54 (2014), 38–48.10.1016/j.neunet.2014.02.010Search in Google Scholar
[41] Di Marco, M.—Forti, M.—Nistri, P.—Pancioni, L.: Discontinuous neural networks for finite-time solution of time-dependent linear equations, IEEE Trans. on Cybernetics, PP (99) (2015), 1-12. https://doi.org/10.1109/TCYB.2015.2479118.10.1109/TCYB.2015.2479118Search in Google Scholar
[42] Goh, A. T. C.: Back-propagation neural networks for modeling complex systems, Artificial Intelligence in Engineering 9 (1995), 143–151.10.1016/0954-1810(94)00011-SSearch in Google Scholar
[43] Gripenberg, G.: Approximation by neural network with a bounded number of nodes at each level, J. Approx. Theory 122 (2003), 260–266.10.1016/S0021-9045(03)00078-9Search in Google Scholar
[44] Ismailov, V. E.: On the approximation by neural networks with bounded number of neurons in hidden layers, J. Math. Anal. Appl. 417 (2014), 963–969.10.1016/j.jmaa.2014.03.092Search in Google Scholar
[45] Ito, Y.: Independence of unscaled basis functions and finite mappings by neural networks, Math. Sci. 26 (2001), 117–126.Search in Google Scholar
[46] Kainen, P. C.—Kurková, V.: An integral upper bound for neural network approximation, Neural Comput. 21 (2009), 2970–2989.10.1162/neco.2009.04-08-745Search in Google Scholar PubMed
[47] Kurková, V.—Sanguineti, M.: Model complexities of shallow networks representing highly varying functions, Neurocomputing 171 (2016), 598–604.10.1016/j.neucom.2015.07.014Search in Google Scholar
[48] Kyurkchiev, N.—Markov, S.: Sigmoid Functions: Some Approximation and Modelling Aspects. Some Moduli in Programming Environment Mathematica, LAP (Lambert Acad. Publ.), 2015.10.11145/j.bmc.2015.03.081Search in Google Scholar
[49] Lin, S.—Xu, Z.—Zeng, J.: Error estimate for spherical neural networks interpolation, Neural Processing Letters 42 (2015), 369–379.10.1007/s11063-014-9361-xSearch in Google Scholar
[50] Llanas, B.—Sainz, F. J.: Constructive approximate interpolation by neural networks, J. Comput. Appl. Math. 188 (2006), 283–308.10.1016/j.cam.2005.04.019Search in Google Scholar
[51] Maiorov, V.: Approximation by neural networks and learning theory, J. Complexity 22 (2006), 102–117.10.1016/j.jco.2005.09.001Search in Google Scholar
[52] Makovoz, Y.: Uniform approximation by neural networks, J. Approx. Theory 95 (1998), 215–228.10.1006/jath.1997.3217Search in Google Scholar
[53] Moller, M. F.: A scaled conjugate gradient algorithm for fast supervised learning, Neural networks 6 (1993), 525–533.10.1016/S0893-6080(05)80056-5Search in Google Scholar
[54] Vinti, G.—Zampogni, L.: Approximation results for a general class of Kantorovich type operators, Adv. Nonlinear Stud. 14 (2014), 991–1011.10.1515/ans-2014-0410Search in Google Scholar
© 2017 Mathematical Institute Slovak Academy of Sciences
