We develop a decision support framework based on Markov decision processes to maximize the profit from the operation of a multi-state system. This framework enables a comprehensive management of the multi-state system, which considers the maintenance decisions together with those on the multi-state system operation setting, that is, its loading condition and configuration. The decisions are informed by a condition monitoring system, which estimates the health state of the multi-state system components. The approach is shown with reference to a mechanical system made up of components affected by fatigue.
DingYLisnianskiA.Fuzzy universal generating functions for multi-state system reliability assessment. Fuzzy Set Syst2008; 159(3): 307–324.
2.
LiuCChenNYanjJ.New method for multi-state system reliability analysis based on linear algebraic representation. Proc IMechE, Part O: J Risk and Reliability2015; 229: 469–482.
3.
WijnmalenDJDHontelezJAM. Coordinated condition-based repair strategies for components of a multi-component maintenance system with discounts. Eur J Oper Res1997; 98: 52–63.
4.
ZilleVBérenguerCGrallAet al. Modelling multicomponent systems to quantify reliability centred maintenance strategies. Proc IMechE, Part O: J Risk and Reliability2011; 225(2): 141–160.
5.
HoppWJKuoYL.An optimal structured policy for maintenance of partially observable aircraft engine components. Nav Res Log1998; 45: 335–352.
6.
BianLGebraeelN.Stochastic modeling and real-time prognostics for multi-component systems with degradation rate interactions. IIE Trans2014; 46(5): 470–482.
7.
ShinstineDSAhmedILanseyKE.Reliability/availability analysis of municipal water distribution networks: case studies. J Water Res Pl2002; 128(2): 140–151.
8.
WangSHongLChenXet al. Review of interdependent infrastructure systems vulnerability analysis. In: Proceedings of the 2011 2nd international conference on intelligent control and information processing, Harbin, China, 25–28 July 2011, pp.446–451. New York: IEEE.
9.
TingaTJannsenR.The interplay between deployment and optimal maintenance intervals for complex multi-component systems. Proc IMechE, Part O: J Risk and Reliability2013; 227(3): 227–240.
10.
HaurieAL’EcuyerP.A stochastic control approach to group preventive replacement in a multicomponent system. IEEE T Automat Contr1982; 27(2): 387–393.
11.
WangRChenN. A survey of condition-based maintenance modeling of multi-component systems. In: Proceedings of the 2016 IEEE international conference on industrial engineering and engineering management (IEEM), Bali, Indonesia, 4–7 December 2016, pp.1664–1668. New York: IEEE.
12.
HarlowDGPhoenixSL.The chain-of-bundles probability model for the strength of fibrous materials I: analysis and conjectures. J Compos Mater1978; 12(2): 195–214.
13.
TrappeyAJTrappeyCVNiW-C.A multi-agent collaborative maintenance platform applying game theory negotiation strategies. J Intell Manuf2013; 24(3): 613–623.
14.
SigaudOBuffetO.Markov decision processes in artificial intelligence (Chapters 1 and 2). Hoboken, NJ: Wiley, 2010.
15.
SuttonRSBartoAG.Introduction to reinforcement learning. 1st ed. Cambridge, MA: MIT Press, 1998.
16.
BaraldiPZioE.Guest editorial. Proc IMechE, Part O: J Risk and Reliability2015; 229(4): 277–278.
17.
SharpMCobleJNamAet al. Lifecycle prognostics: transitioning between information types. Proc IMechE, Part O: J Risk and Reliability2015; 229(4): 279–290.
18.
JardineAKSLinDBanjevicD.A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mech Syst Signal Pr2006; 20(7): 1483–1510.
19.
ZioE. Prognostics and health management of industrial equipment. In: KadryS (ed.) Diagnostics and prognostics of engineering systems: methods and techniques. Hershey, PA: IGI Global, 2012, pp.333–356.
20.
ZioECompareM.Evaluating maintenance policies by quantitative modeling and analysis. Reliab Eng Syst Safe2013; 109: 53–65.
21.
GrallABérenguerCDieulleL.A condition-based maintenance policy for stochastically deteriorating systems. Reliab Eng Syst Safe2002; 76(2): 167–180.
22.
PipeK. Practical prognostics for condition based maintenance. In: Proceedings of the international conference on prognostics and health management (PHM 2008), Denver, CO, 6–9 October 2008. New York: IEEE.
23.
ZontaDGlisicBAdriaenssensS.Value of information: impact of monitoring on decision-making. Struct Control Hlth2014; 21: 1043–1056.
24.
PozziMZontaDWangWet al. A framework for evaluating the impact of structural health monitoring on bridge management. In: Proceedings of the 5th international conference on bridge maintenance, safety and management (IABMAS2010), Philadelphia, PA, 11–15 July 2010, pp.646–653. Florida: CRC Press.
25.
MemarzadehMPozziM.Value of information in sequential decision making: component inspection, permanent monitoring and system-level scheduling. Reliab Eng Syst Safe2016; 154: 137–151.
26.
PapakostantinouKGShinozuaM.Planning optimal inspection and maintenance policies via dynamic programming and Markov processes. Part I: theory. Reliab Eng Syst Safe2014; 130: 202–213.
27.
PapakostantinouKGShinozuaM.Planning optimal inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation. Reliab Eng Syst Safe2014; 130: 214–224.
28.
PapakostantinouKGShinozuaM.Optimum inspection and maintenance policies for corroded structures using partially observable Markov decision processes and stochastic, physically based models. Probabilist Eng Mech2014; 37: 93–108.
NielsenJSSørensenJD.Methods for risk-based planning of O&M of wind turbines. Energies2014; 7: 6645–6664.
31.
DawidRMcMillanDRevieM. Review of Markov models for maintenance optimization in the context of offshore wind. In: Proceedings of the annual conference of the prognostics and health management society, Coronado, CA, 18–24 October 2015, pp. 269–279. PHM Society.
32.
Olde KeizerMCATeunterRHVeldmanJ.Joint condition-based maintenance and inventory optimization for systems with multiple components. Eur J Oper Res2017; 257: 209–222.
33.
MitchellM.An introduction to genetic algorithms. Cambridge, MA: MIT Press, 1999.
34.
ZioE.Computational methods for reliability and risk analysis, vol. 14 (Series on quality, reliability and engineering statistics). Singapore: World Scientific, 2009.
35.
FangS-CPuthenpuraS.Linear optimization and extensions: theory and algorithms. Upper Saddle River, NJ: Prentice Hall, 1993.
36.
LisnianskiALevitinG.Multi-state system reliability: assessment, optimization and applications (Chapter 1). Hackensack, NJ; London; Singapore; Kowloon, Hong Kong: World Scientific, 2003, pp.15–50.
37.
LisnianskiALaredoDHaimHB. Multi-state Markov model for reliability analysis of a combined cycle gas turbine power plant. In: Proceedings of the 2016 2nd international symposium on stochastic models in reliability engineering, life science and operations management (SMRLO), Beer-Sheva, 15–18 February 2016, pp.131–135. New York: IEEE.
38.
Van OtterloMWieringM. Chapter 1: reinforcement learning and Markov decision processes. In: Van OtterloMWieringM (eds) Reinforcement learning: state-of-the-art. Berlin; Heidelberg: Springer, 2012, pp.3–42.
39.
MansourYSinghS. On the complexity of policy iteration. In: Proceedings of the 15th conference on uncertainty in artificial intelligence (UAI’99), Stockholm, 30 July–1 August 1999, pp.401–408. California: Morgan Kaufmann Publishers Inc.
40.
LittmanMLDeanTLKaelblingLP. On the complexity of solving Markov decision problems. In: Proceedings of the 11th conference on uncertainty in artificial intelligence (UAI’95), Montreal, QC, Canada, 18–20 August 1995, pp.394–402. New York: ACM.
41.
CadiniFZioEAvramD.Monte Carlo-based filtering for fatigue crack growth estimation. Probabilist Eng Mech2009; 24(3): 367–373.
