Abstract
Intelligent systems, that support the maintenance of production resources, offer real-time data-based approaches to optimize the maintenance effort and to reduce the usage of resources within production systems. However, unused potentials remain regarding maintenance schedules with minimal opportunity costs of the measures taken. This work provides a novel, machine-learning-based approach for the exploitation of these remaining optimization opportunities as an exemplary extension of the current state of the art. The determination of an optimal maintenance schedule for parallel working machines, is based on the data of a production system. The main result of this work is the performance of the implemented reinforcement learning algorithms, both in terms of downtime reduction, which increases the production output, and in terms of reducing maintenance costs compared to existing maintenance strategies. Hence, this work provides a holistic approach to the optimization of maintenance strategies and gives further evidence of a meaningful applicability of reinforcement learning algorithms in manufacturing processes.
Similar content being viewed by others
References
Wuest T (2016) Machine learning in manufacturing: advantages, challenges, and applications. Prod Manuf Res 4:23-45
Colledani M, Magnanini MC, Tolio T (2018) Impact of opportunistic maintenance on manufacturing system performance. CIRP Ann 67(1):499–502
Hashemian HM, Bean Wendell C (2011) State-of-the-art predictive maintenance techniques. IEEE Trans Instrum Meas 60(10):3480–3492
Lindström J, Larsson H, Jonsson M, Leyon E (2017) Towards intelligent and sustainable production: combining and integratingonline predictive maintenance and continuous quality control. Procedia CIRP 63:443–448
Yang L et al (2018) Opportunistic maintenance of production systems subject to random wait time and multiple control limits. J Manuf Syst 47:12–34
Stricker N et al (2018) Reinforcement learning for adaptive order dispatching in the semiconductor industry. CIRP Ann 67(1):511–514
Wang X et al (2014) Reinforcement learning based predictive maintenance for a machine with multiple deteriorating yield levels. J Comput Inf Syst 10(1):9–19
Wang J (2016) Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. J Intell Manuf 27(2):325–333
Sutton RS, Barto AG (2017) Reinforcement learning: an introduction. MIT Press, Cambridge
Crites RH, Barto AG (1995) Improving elevator performance using reinforcement-learning. Adv Neural Inf Process Syst 8:1017–1023
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. CoRR. arXiv:abs/1312.5602
Williams RJ (1992) Simple statistic gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256
Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust region policy optimization. In: Proceedings of the 31st International Conference on Machine Learning, vol 37, pp 1889–1897
Schulman J (2017) Proximal policy optimization algorithms. Adv Neural Inf Process Syst 8:1017–1023
Schijve J (2009) Fatigue of structures and materials. Springer, Amsterdam, pp 15–21
Xie M, Lai CD (1996) Reliability analysis using an additive Weibull model with bathtub-shaped failure rate function. Reliab Eng Syst Saf 52(1):87–93
Acknowledgements
We extend our sincere thanks to the German Federal Ministry of Education and Research (BMBF) for supporting this research project 02K16C082 Produktionsbezogene Dienstleistungssysteme auf Basis von Big-Data-Analysen (ProData).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kuhnle, A., Jakubik, J. & Lanza, G. Reinforcement learning for opportunistic maintenance optimization. Prod. Eng. Res. Devel. 13, 33–41 (2019). https://doi.org/10.1007/s11740-018-0855-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11740-018-0855-7