Skip to main content

Advertisement

Log in

Reinforcement learning for opportunistic maintenance optimization

  • Production Management
  • Published:
Production Engineering Aims and scope Submit manuscript

Abstract

Intelligent systems, that support the maintenance of production resources, offer real-time data-based approaches to optimize the maintenance effort and to reduce the usage of resources within production systems. However, unused potentials remain regarding maintenance schedules with minimal opportunity costs of the measures taken. This work provides a novel, machine-learning-based approach for the exploitation of these remaining optimization opportunities as an exemplary extension of the current state of the art. The determination of an optimal maintenance schedule for parallel working machines, is based on the data of a production system. The main result of this work is the performance of the implemented reinforcement learning algorithms, both in terms of downtime reduction, which increases the production output, and in terms of reducing maintenance costs compared to existing maintenance strategies. Hence, this work provides a holistic approach to the optimization of maintenance strategies and gives further evidence of a meaningful applicability of reinforcement learning algorithms in manufacturing processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Wuest T (2016) Machine learning in manufacturing: advantages, challenges, and applications. Prod Manuf Res 4:23-45

    Google Scholar 

  2. Colledani M, Magnanini MC, Tolio T (2018) Impact of opportunistic maintenance on manufacturing system performance. CIRP Ann 67(1):499–502

    Article  Google Scholar 

  3. Hashemian HM, Bean Wendell C (2011) State-of-the-art predictive maintenance techniques. IEEE Trans Instrum Meas 60(10):3480–3492

    Article  Google Scholar 

  4. Lindström J, Larsson H, Jonsson M, Leyon E (2017) Towards intelligent and sustainable production: combining and integratingonline predictive maintenance and continuous quality control. Procedia CIRP 63:443–448

    Article  Google Scholar 

  5. Yang L et al (2018) Opportunistic maintenance of production systems subject to random wait time and multiple control limits. J Manuf Syst 47:12–34

    Article  Google Scholar 

  6. Stricker N et al (2018) Reinforcement learning for adaptive order dispatching in the semiconductor industry. CIRP Ann 67(1):511–514

    Article  Google Scholar 

  7. Wang X et al (2014) Reinforcement learning based predictive maintenance for a machine with multiple deteriorating yield levels. J Comput Inf Syst 10(1):9–19

    Google Scholar 

  8. Wang J (2016) Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. J Intell Manuf 27(2):325–333

    Article  Google Scholar 

  9. Sutton RS, Barto AG (2017) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  10. Crites RH, Barto AG (1995) Improving elevator performance using reinforcement-learning. Adv Neural Inf Process Syst 8:1017–1023

    Google Scholar 

  11. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. CoRR. arXiv:abs/1312.5602

  12. Williams RJ (1992) Simple statistic gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256

    MATH  Google Scholar 

  13. Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust region policy optimization. In: Proceedings of the 31st International Conference on Machine Learning, vol 37, pp 1889–1897

  14. Schulman J (2017) Proximal policy optimization algorithms. Adv Neural Inf Process Syst 8:1017–1023

    Google Scholar 

  15. Schijve J (2009) Fatigue of structures and materials. Springer, Amsterdam, pp 15–21

    Book  MATH  Google Scholar 

  16. Xie M, Lai CD (1996) Reliability analysis using an additive Weibull model with bathtub-shaped failure rate function. Reliab Eng Syst Saf 52(1):87–93

    Article  Google Scholar 

Download references

Acknowledgements

We extend our sincere thanks to the German Federal Ministry of Education and Research (BMBF) for supporting this research project 02K16C082 Produktionsbezogene Dienstleistungssysteme auf Basis von Big-Data-Analysen (ProData).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andreas Kuhnle.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuhnle, A., Jakubik, J. & Lanza, G. Reinforcement learning for opportunistic maintenance optimization. Prod. Eng. Res. Devel. 13, 33–41 (2019). https://doi.org/10.1007/s11740-018-0855-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11740-018-0855-7

Keywords

Navigation