Skip to main content
SpringerLink
Log in

Reinforcement learning-based flow management of gas turbine parts under stochastic failures

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

For maintenance of gas turbines (GTs) in oil and gas applications, capital parts are removed and replaced by parts of the same type taken from the warehouse. When the removed parts are found not broken, they are repaired at the workshop and returned to the warehouse, ready to be used in future maintenance. The management of this flow is of great importance for the profitability of a GT plant. In this paper, we adopt a previously developed formalized framework of the part flow and reinforcement learning (RL) to optimize part flow management. The formal framework and RL algorithm are extended to account for the stochastic failure process of the involved parts. An application to a scaled-down case study derived from an industrial application is illustrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yang K, He C, Huang Q, Huang ZY, Wang C, Wang Q, Liu YJ, Zhong B (2016) Very high cycle fatigue behaviors of a turbine engine blade alloy at various stress ratios. International Journal of Fatigue

  2. Morini M, Pinelli M, Spina PR, Venturini M (2010) Influence of blade deterioration on compressor and turbine performance. J Eng Gas Turb Power 132:3

    Google Scholar 

  3. Boyce BL, Ritchie RO (2001) Effect of load ratio and maximum stress intensity on the fatigue threshold in Ti–6Al–4V. Eng Fract Mech 68(2):129–147

    Article  Google Scholar 

  4. Bodrov AI, Stalder JP (1998) An analysis of axial compressor fouling and a blade cleaning method. J Turbomach 120:256–61

    Article  Google Scholar 

  5. Kurz R, Brun K (2012) Fouling mechanisms in axial compressors. J Eng Gas Turb Power 134:3

    Article  Google Scholar 

  6. Peters JO, Ritchie RO (2000) Influence of foreign-object damage on crack initiation and early crack growth during high-cycle fatigue of Ti–6Al–4V. Eng Fract Mech 67(3):193–207

    Article  Google Scholar 

  7. Eliaz N, Shemesh G, Latanision RM (2002) Hot corrosion in gas turbine components. J Eng Gas Turb Power 9(1):31–43

    Google Scholar 

  8. Goward GW (1998) Progress in coatings for gas turbine airfoils. Surf Coat Technol 108:73–79

    Article  Google Scholar 

  9. Compare M, Martini F, Mattafirri S, Carlevaro F, Zio E (2016) Semi-Markov model for the oxidation degradation mechanism in gas turbine nozzles. IEEE Trans Reliab 65(2):574–581

    Article  Google Scholar 

  10. Ng I, Parry G, Wild P, McFarlane D, Tasker P (2011) Complex engineering service systems: concepts and research. Springer, London

    Book  Google Scholar 

  11. Wang W (2010) A model for maintenance service contract design, negotiation and optimization. Eur J Oper Res 201:239–246

    Article  Google Scholar 

  12. Murthy DNP, Asgharizadeh E (1999) Optimal decision making in a maintenance service operation. Eur J Oper Res 116:259–273

    Article  Google Scholar 

  13. Godoy DR, Pascual R, Knights P (2014) A decision-making framework to integrate maintenance contract conditions with critical spares management. Reliab Eng Syst Safety 131:102–108

    Article  Google Scholar 

  14. Jin T, Tian Z, Xie M (2015) A game-theoretical approach for optimizing maintenance, spares and service capacity in performance contracting. Int J Prod Econ 161:31–43

    Article  Google Scholar 

  15. Jackson C, Pascual R, Knights P (2008) Optimal maintenance service contract negotiation with ageing equipment. Eur J Oper Res 189(2):387–398

    Article  Google Scholar 

  16. Bollapragada S, Gupta A, Lawsirirat C (2007) Managing a portfolio of long term service agreements. Eur J Oper Res 182:1399–1411

    Article  Google Scholar 

  17. Hu Q, Boylan JE, Chen H, Labib A (2018) in spare parts management: a review. Eur J Oper Res 266:395–414

    Article  Google Scholar 

  18. Sutton R, Barto A (1998) Introduction to reinforcement learning, vol 135. MIT Press, Cambridge

    Google Scholar 

  19. Lieckens KT, Colen PJ, Lambrecht MR (2015) Network and contract optimization for maintenance services with remanufacturing. Comput Oper Res 54:232–244

    Article  MathSciNet  Google Scholar 

  20. Kurz J (2016) Capacity planning for a maintenance service provider with advanced information. Eur J Oper Res 251:466–477

    Article  MathSciNet  Google Scholar 

  21. Olde Keizer MCA, Teunter RH, Veldman J (2017) Joint condition-based maintenance and inventory optimization for systems with multiple components. Eur J Oper Res 257:209–222

    Article  MathSciNet  Google Scholar 

  22. Van Horenbeek A, Buré J, Cattrysse D, Pintelon L, Vansteenwegen P (2013) Joint maintenance and inventory optimization systems: a review. Int J Prod Econ 143:499–508

    Article  Google Scholar 

  23. Jaturonnatee J, Murthy DNP, Boondiskulchok R (2006) Optimal preventive maintenance of leased equipment with corrective minimal repairs. Eur J Oper Res 174:201–215

    Article  Google Scholar 

  24. Zanjani MK, Nourelfath M (2014) Integrated spare parts logistics and operations planning for maintenance service providers. Int J Prod Econ 158:44–53

    Article  Google Scholar 

  25. Compare M, Bellani L, Cobelli E, Zio E, Annunziata F, Sepe M, Carlevaro F A Reinforcement Learning approach to optimal part flow management for gas turbine maintenance, submitted for publication to European Journal of Operational Research

  26. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Article  Google Scholar 

  27. Szepesvári Cs (2010) Algorithms for reinforcement learning. Morgan and Claypool

  28. Kuznetsova E, Li YF, Ruiz C, Zio E, Ault G, Bell K (2013) Reinforcement learning for microgrid energy management. Energy Elsevier 59:133–146

    Article  Google Scholar 

  29. Rahimiyan M, Mashhadi HR (2010) An adaptive Q-learning algorithm developed for agent-based computational modeling of electricity market. IEEE Trans Syst Man Cybrn Part C, Appl Rev 40(5):547

    Article  Google Scholar 

  30. Barde S, Yacout S, Shin H (2016) Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks. J Intell Manuf, 1–15

  31. Aissani N, Beldjilali B, Trentesaux D (2009) Dynamic scheduling of maintenance tasks in the petroleum industry: a reinforcement approach. Eng Appl Artif Intell 22(7):1089–1103

    Article  Google Scholar 

  32. Pontrandolfo P, Gosavi A, Okogbaa OG, Das T (2002) Global supply chain management: a reinforcement learning approach. Int J Prod Res 40(6):1299–1317

    Article  Google Scholar 

  33. Giannoccaro I, Pontrandolfo P (2002) Inventory management in supply chains: a reinforcement learning approach. Int J Prod Econ 78(2):153–161

    Article  Google Scholar 

  34. Kim CO, Jun J, Baek JK, Smith RL, Kim YD (2005) Adaptive inventory control models for supply chain management. Int J Adv Manuf Technol 26(9–10):1184–1192

    Article  Google Scholar 

  35. Ehleiter A, Jaehn F (2016) Housekeeping: foresightful container repositioning. Int J Prod Econ 179:203–211

    Article  Google Scholar 

  36. Kim KH, Lee KM, Hwang H (2003) Receiving operations for yard cranes in port container terminals sequencing delivery. Int J Prod Econ 84:283–292

    Article  Google Scholar 

  37. Zio E (2007) An introduction to the basics of reliability and risk analysis, vol 13. World Scientific

  38. Wang YH, Li TH, Lin CJ (2013) Backward Q-learning: the combination of Sarsa algorithm and Q-learning. Eng Appl Artif Intel 26:2184–2193

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Energy Department, Politecnico di Milano, Milan, Italy

    Michele Compare, Enrico Cobelli & Enrico Zio

  2. Aramis S.r.l., Milan, Italy

    Michele Compare, Luca Bellani & Enrico Zio

  3. Chair on Systems Science and the Energetic Challenge, Foundation Electricité de France at CentraleSupélec, Paris, France

    Enrico Zio

Authors
  1. Michele Compare
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Bellani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Enrico Cobelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enrico Zio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michele Compare.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Compare, M., Bellani, L., Cobelli, E. et al. Reinforcement learning-based flow management of gas turbine parts under stochastic failures. Int J Adv Manuf Technol 99, 2981–2992 (2018). https://doi.org/10.1007/s00170-018-2690-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-018-2690-6

Keywords

Search

Navigation