Skip to main content
SpringerLink
Log in

Learning Radial Basis Function Networks with the Trust Region Method for Boundary Problems

  • Intellectual Control Systems, Data Analysis
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

We consider the solution of boundary value problems of mathematical physics with neural networks of a special form, namely radial basis function networks. This approach does not require one to construct a difference grid and allows to obtain an approximate analytic solution at an arbitrary point of the solution domain. We analyze learning algorithms for such networks. We propose an algorithm for learning neural networks based on the method of trust region. The algorithm allows to significantly reduce the learning time of the network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Tolstykh, A.I. and Shirobokov, D.A., Mesh-Free Method Based on Radial Basis Functions, Zh. Vychisl. Mat. Mat. Fiz., 2005, vol. 45, no. 8, pp. 1498–1505.

    MathSciNet  MATH  Google Scholar 

  2. Liu, G.R., Mesh-Free Methods: Moving Beyond the Finite Element Method, Boca Raton: CRC Press, 2003.

    MATH  Google Scholar 

  3. Kansa, E.J., Motivation for Using Radial Basis Function to Solve PDEs. http://www.cityu.edu.hk/rbf-pde/files/overview-pdf.pdf

  4. Buhmann, M.D., Radial Basis Functions: Theory and Implementations, Cambridge: Cambridge Univ. Press, 2004.

    MATH  Google Scholar 

  5. Fasshauer, G.E., Meshfree Approximation Methods with MATLAB, River Edge: World Scientific, 2007.

    Book  MATH  Google Scholar 

  6. Chen, W. and Fu, Z.J., Recent Advances in Radial Basis Function Collocation Methods, New York: Springer, 2014.

    Book  MATH  Google Scholar 

  7. Fornberg, B. and Flyer, N., Solving PDEs with Radial Basis Functions, Acta Numerica, 2015, vol. 4, pp. 215–258.

    Article  MathSciNet  MATH  Google Scholar 

  8. Jianyu, L., Siwei, L., Yingjian, Q., and Yaping, H., Numerical Solution of Differential Equations by Radial Basis Function Neural Networks, Proc. Int. Joint Conf. on Neural Networks, 2002, vol. 1, pp. 773–777.

    Google Scholar 

  9. Mai-Duy, N. and Tran-Cong, T., Solving High Order Ordinary Differential Equations with Radial Basis Function Networks, Int. J. Numer. Methods Eng., 2005, vol. 62, pp. 824–852.

    Article  MathSciNet  MATH  Google Scholar 

  10. Sarra, S., Adaptive Radial Basis Function Methods for Time Dependent Partial Differential Equations, Appl. Numer. Math., 2005, vol. 54, no. 1, pp. 79–94.

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen, H., Kong, L., and Leng, W., Numerical Solution of PDEs via Integrated Radial Basis Function Networks with Adaptive Training Algorithm, Appl. Soft Comput., 2011, vol. 11, no. 1, pp. 855–860.

    Article  Google Scholar 

  12. Kumar, M. and Yadav, N., Multilayer Perceptrons and Radial Basis Function Neural Network Methods for the Solution of Differential Equations: A Survey, Comput. Math. Appl., 2011, vol. 62, pp. 3796–3811.

    Article  MathSciNet  MATH  Google Scholar 

  13. Yadav, N., Yadav, A., and Kumar, M., An Introduction to Neural Network Methods for Differential Equations, New York: Springer, 2015.

    Book  MATH  Google Scholar 

  14. Tarkhov, D.A., Neirosetevye modeli i algoritmy. Spravochnik (Neural Network Models and Algorithms. Reference), Moscow: Radiotekhnika, 2014.

    Google Scholar 

  15. Gorbachenko, V.I. and Artyukhina, E.V., Mesh-Free Methods and Their Implementation with Radial Basis Neural Networks, Neirokomp’yut.: Razrabotka, Primen., 2010, no. 11, pp. 4–10.

    Google Scholar 

  16. Gorbachenko, V.I. and Zhukov, M.V., Solving Boundary Value Problems of Mathematical Physics Using Radial Basis Function Networks, Comput. Math. Math. Phys., 2017, vol. 57, no. 1, pp. 145–155.

    Article  MathSciNet  MATH  Google Scholar 

  17. Goodfellow, I., Bengio, Y., and Courville, A., Deep Learning, Boston: MIT Press, 2017. Translated under the title Glubokoe obuchenie, Moscow: DMK Press, 2018.

    MATH  Google Scholar 

  18. Jia, W., Zhao, D., Shen, T., Su, C., Hu, C., and Zhao, Y., A New Optimized GA-RBF Neural Network Algorithm, Comput. Intelligence Neurosci., 2014, Article ID982045.

    Google Scholar 

  19. Gill, P.E., Murray, W., and Wright, M.H., Practical Optimization, London: Academic, 1981. Translated under the title Prakticheskaya optimizatsiya, Moscow: Mir, 1985.

    MATH  Google Scholar 

  20. Sutskever, I., Martens, J., Dahl, G., and Hinton, G., On the Importance of Initialization and Momentum in Deep Learning, Proc. 30th Int. Conf. on Machine Learning, 2013, vol. 28, pp. 1139–1147.

    Google Scholar 

  21. Alkezuini, M.M. and Gorbachenko, V.I., Training Networks of Radial Basis Functions with Nesterov’s Method for Solving Boundary Problems of Mathematical Physics, Proc. XII Intl. Sci.-Tech. Conf. Analytic and Numerical Methods of Modeling Natural Science and Social Problems, Penza: PGU, 2017, pp. 171–175.

    Google Scholar 

  22. Fletcher, R. and Reeves, C.M., Function Minimization by Conjugate Gradients, Comput. J., 1964, vol. 7, pp. 149–154.

    Article  MathSciNet  MATH  Google Scholar 

  23. Polak, E. and Ribiére, G., Note sur la convergence de méthodes de directions conjugués, Revue Française d’Inform. Recherche Opération., Série Rouge, 1969, vol. 3, no. 1, pp. 35–43.

    MATH  Google Scholar 

  24. Zhang, L., Li, K., He, H., and Irwin, G.W., A New Discrete-Continuous Algorithm for Radial Basis Function Networks Construction, IEEE Trans. Neural Networks Learning Syst., 2013, vol. 24, no. 11, pp. 1785–1798.

    Article  Google Scholar 

  25. Xie, T., Yu, H., Hewlett, J., Rozycki, P., and Wilamowski, B., Fast and Efficient Second-Order Method for Training Radial Basis Function Networks, IEEE Trans. Neural Networks Learning Syst., 2012, vol. 23, no. 4, pp. 609–619.

    Article  Google Scholar 

  26. Markopoulos, A.P., Georgiopoulos, S., and Manolakos, D.E., On the Use of Back Propagation and Radial Basis Function Neural Networks in Surface Roughness Prediction, J. Indust. Engin. Int., 2016, vol. 12, no. 3, pp. 389–400.

    Article  Google Scholar 

  27. Zhang, L., Li, K., and Wang, W., An Improved Conjugate Gradient Algorithm for Radial Basis Function (RBF) Networks Modelling, in Proc. UKACC Int. Conf. on Control, 2012, pp. 19–23.

    Google Scholar 

  28. Sadeghi, M., Pashaie, M., and Jafarian, A., RBF Neural Networks Based on BFGS Optimization Method for Solving Integral Equations, Adv. Appl. Math. Biosci., 2016, vol. 7, no. 1, pp. 1–22.

    Google Scholar 

  29. Conn, A.R., Gould, N.I.M., and Toint, P.L., Trust-Region Methods, MPS-SIAM, 1987.

    MATH  Google Scholar 

  30. Wild, S.M., Regis, R.G., and Shoemaker, C.A., ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions, SIAM J. Scientific Comput., 2008, vol. 30, no. 6, pp. 3197–3219.

    Article  MathSciNet  MATH  Google Scholar 

  31. Bernal, F., Trust-Region Methods for Nonlinear Elliptic Equations with Radial Basis Functions, Comput. Math. Appl., 2016, vol. 72, no. 7, pp. 1743–1763.

    Article  MathSciNet  MATH  Google Scholar 

  32. Gorbachenko, V.I. and Zhukov, M.V., The Learning of Radial Basis Function Network Using Trust Region Method to Solve Poisson’s Equation, Inform. Tekhnol., 2013, no. 9, pp. 65–70.

    Google Scholar 

  33. Brink, H., Richards, J.W., and Fetherolf, M., Real-World Machine Learning, Shelter Island: Manning, 2016. Translated under the title Mashinnoe obuchenie, St. Petersburg: Piter, 2017.

    Google Scholar 

  34. Steihaug, T., The Conjugate Gradient Method and Trust Region in Large Scale Optimization, SIAM J. Numer. Anal., 1983, vol. 20, no. 3, pp. 626–637.

    Article  MathSciNet  MATH  Google Scholar 

  35. Watkins, D.S., Fundamentals of Matrix Computations, New York: Springer, 1991. Translated under the title Osnovy matrichnykh vychislenii, Moscow: Binom, 2012.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moscow State Technical University of Civil Aviation, Moscow, Russia

    L. N. Elisov

  2. Penza State University, Penza, Russia

    V. I. Gorbachenko & M. V. Zhukov

Authors
  1. L. N. Elisov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. I. Gorbachenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. V. Zhukov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. N. Elisov.

Additional information

Original Russian Text © L.N. Elisov, V.I. Gorbachenko, M.V. Zhukov, 2018, published in Avtomatika i Telemekhanika, 2018, No. 9, pp. 95–105.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elisov, L.N., Gorbachenko, V.I. & Zhukov, M.V. Learning Radial Basis Function Networks with the Trust Region Method for Boundary Problems. Autom Remote Control 79, 1621–1629 (2018). https://doi.org/10.1134/S0005117918090072

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117918090072

Keywords

Search

Navigation