Skip to main content
SpringerLink
Log in

Adaptive neural control using reinforcement learning for a class of robot manipulator

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, an adaptive control algorithm is proposed for a class of robot manipulator systems with unknown functions and dead-zone input by using a reinforcement learning scheme. The parameters of the dead zone are supposed to be unknown but bounded. The unknown functions can be approximated based on the neural networks, which is one part of the reinforcement learning scheme, namely an action network. The other part is called critic network which is used to approximate the reinforcement signal. Then, the prominent advantage of the proposed approach is that an optimal control input can be obtained by using two networks compared with the results of robot manipulator with dead zone: an additional term is given to compensate for the effect of the dead zone, and a special design procedure to solve the difficulties in constructing the controllers and adaptation laws. Based on the Lyapunov analysis theory, all the signals of the closed-loop system are proved to be bounded and the system output can track the reference signal to a bounded compact set. Finally, a simulation example is given to illustrate the effectiveness of the approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Li ZJ, Ge SS, Adams M, Wijesoma WS (2008) Robust adaptive control of uncertain force/motion constrained nonholonomic mobile manipulators. Automatica 44(3):776–784

    Article  MathSciNet  Google Scholar 

  2. Li ZJ, Yang YP, Li JX (2010) Adaptive motion/force control of mobile under-actuated manipulators with dynamics uncertainties by dynamic coupling and output feedback. IEEE Trans Control Syst Technol 18(5):1068–1079

    Article  Google Scholar 

  3. Chen CLP, LeClair SR, Pao YH (1998) An incremental adaptive implementation of functional-link processing for function approximation, time-series prediction, and system identification. Neurocompting 18(1–3):11–31

    Article  Google Scholar 

  4. Tong SC, He XL, Zhang HG (2009) A combined backstepping and small-gain approach to robust adaptive fuzzy output feedback control. IEEE Trans Fuzzy Syst 17(5):1059–1069

    Article  Google Scholar 

  5. Chen WS, Jiao LC, Li J, Li RH (2010) Adaptive NN backstepping output-feedback control for stochastic nonlinear strict-feedback systems with time-varying delays. IEEE Trans Syst Man Cybern B Cybern 40(3):939–950

    Article  Google Scholar 

  6. Li ZJ, Cao XQ, Ding N (2011) Adaptive fuzzy control for synchronization of nonlinear teleoperators with stochastic time-varying communication delays. IEEE Trans Fuzzy Syst 19(4):745–757

    Article  Google Scholar 

  7. Chen WS, Jiao LC (2011) Adaptive tracking for periodically time varying and nonlinearly parameterized systems using multilayer neural networks. IEEE Trans Neural Netw 21(2):345–351

    Article  MathSciNet  Google Scholar 

  8. Liu YJ, Tong SC, Wang D, Li TS, Chen CLP (2011) Adaptive neural output feedback controller design with reduced-order observer for a class of uncertain nonlinear SISO systems. IEEE Trans Neural Netw 22(8):1328–1334

    Article  Google Scholar 

  9. Zhou Q, Shi P, Lu J, Xu S (2011) Adaptive output feedback fuzzy tracking control for a class of nonlinear systems. IEEE Trans Fuzzy Syst 19(5):972–982

    Article  Google Scholar 

  10. Li TS, Wang D, Feng G (2010) A DSC approach to robust adaptive NN tracking control for strict-feedback nonlinear systems. IEEE Trans Syst Man Cybern B Cybern 40(3):915–927

    Article  Google Scholar 

  11. Tong SC, Li Y, Li YM, Liu YJ (2011) Observer-based adaptive fuzzy backstepping control for a class of stochastic nonlinear strict-feedback systems. IEEE Trans Syst Man Cybern B Cybern 41(6):1693–1704

    Article  MathSciNet  Google Scholar 

  12. Li HY, Yu J, Hilton C, Liu H (2012) Adaptive sliding mode control for nonlinear active suspension systems using T-S fuzzy model. IEEE Trans Ind Electron. doi:10.1109/TIE.2012.2202354

  13. Yang CG, Li ZJ, Li J (2012) Trajectory planning and optimized adaptive control for a class of wheeled inverted pendulum vehicle models. IEEE Trans Syst Man Cybern B Cybern. doi:10.1109/TSMCB.2012.2198813

  14. Li ZJ, Xu CQ (2009) Adaptive fuzzy logic control of dynamic balance and motion for wheeled inverted pendulums. Fuzzy Sets Syst 160(12):1787–1803

    Article  MATH  Google Scholar 

  15. Sun FC, Li L, Li HX, Liu HP (2007) Neuro-fuzzy dynamic-inversion-based adaptive control for robotic manipulators-discrete time case. IEEE Trans Ind Electron 54(3):1342–1351

    Article  MathSciNet  Google Scholar 

  16. Cui RX (2012) Mutual synchronization of multiple robot manipulators with unknown dynamics. J Intell Rob Syst 68(2):105–119

    Article  MATH  Google Scholar 

  17. Piltan F, Sulaiman N, Soltani S, Marhaban MH, Ramli R (2011) An adaptive sliding surface slope adjustment in PD sliding mode fuzzy control for robot manipulator. Int J Control Autom 4(3):65–76

    Google Scholar 

  18. Zhou J, Wen C, Zhang Y (2006) Adaptive output control of nonlinear systems with uncertain dead-zone nonlinearity. IEEE Trans Autom Control 51(3):504–511

    Article  MathSciNet  Google Scholar 

  19. Xu JX, Viswanathan B (2000) Adaptive robust iterative learning control with dead zone scheme. Automatica 36(1):91–99

    Article  MATH  MathSciNet  Google Scholar 

  20. Ibrir S, Xie WF, Su CY (2007) Adaptive tracking of nonlinear systems with non-symmetric dead-zone input. Automatica 43(3):522–530

    Article  MATH  MathSciNet  Google Scholar 

  21. Wang XS, Su CY, Hong H (2004) Robust adaptive control of a class of nonlinear systems with unknown dead-zone. Automatica 40(3):407–413

    Article  MATH  MathSciNet  Google Scholar 

  22. Liu L, Liu YJ, Chen CLP (2012) Adaptive neural network control for a DC motor system with dead-zone. Nonlinear Dyn. doi:10.1007/s11071-012-0698-2

  23. Zhang TP, Ge SS (2008) Adaptive dynamic surface control of nonlinear systems with unknown dead zone in pure feedback form. Automatica 44(7):1895–1903

    Article  MATH  MathSciNet  Google Scholar 

  24. Song Q, Hu WJ, Yin L, Soh YC (2002) Robust adaptive dead zone technology for fault-tolerant control of robot manipulators using neural networks. J Intell Rob Syst 33(2):113–137

    Article  MATH  Google Scholar 

  25. Chen M, Ge SS, Ren BB (2011) Adaptive tracking control of uncertain MIMO nonlinear systems with input constraints. Automatica 47(3):452–465

    Article  MATH  MathSciNet  Google Scholar 

  26. Chen M, Ge SS, Ren BB (2010) Robust adaptive neural network control for a class of uncertain MIMO nonlinear systems with input nonlinearities. IEEE Trans Neural Netw 21(5):796–812

    Article  Google Scholar 

  27. Ren BB, Sam SZ, Lee TH, Su CY (2009) Adaptive neural control for a class of nonlinear systems with uncertain hysteresis inputs and time-varying state delays. IEEE Trans Neural Netw 20(7):1148–1164

    Article  Google Scholar 

  28. Zhang HG, Luo YH, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503

    Article  Google Scholar 

  29. Wang FY, Jin N, Liu D, Wei QL (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans Neural Netw 22(1):24–36

    Article  Google Scholar 

  30. Yang QM, Jagannathan S (2012) Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators. IEEE Trans Syst Man Cybern Part B Cybern 42(2):377–390

    Article  Google Scholar 

  31. Lin CK (2003) A reinforcement learning adaptive fuzzy controller for robots. Fuzzy Sets Syst 137:339–352

    Article  MATH  Google Scholar 

  32. Zhang X, Zhang HG, Liu D, Kim YS (2009) Neural-network-based reinforcement learning controller for nonlinear systems with non-symmetric dead-zone inputs. IEEE symposium on adaptive dynamic programming and reinforcement learning, pp 124–129

  33. Liu YJ, Zhou N (2010) Observer-based adaptive fuzzy-neural control for a class of uncertain nonlinear systems with unknown dead-zone input. ISA Trans 49:462–469

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61074014, 61104017); Program for Liaoning Innovative Research Team in University LT2012013; Program for Liaoning Excellent Talents in University under grant LJQ2011064.

Author information

Authors and Affiliations

  1. College of Science, Liaoning University of Technology, Jinzhou, 121001, Liaoning, China

    Li Tang, Yan-Jun Liu & Shaocheng Tong

Authors
  1. Li Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaocheng Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan-Jun Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tang, L., Liu, YJ. & Tong, S. Adaptive neural control using reinforcement learning for a class of robot manipulator. Neural Comput & Applic 25, 135–141 (2014). https://doi.org/10.1007/s00521-013-1455-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-013-1455-2

Keywords

Search

Navigation