Chapter13 Maintenance Planning Under Uncertainties Using a Continuous-State POMDP Framework Roland Schöbi and Eleni Chatzi Abstract Planning under uncertainty is an area that has attracted significant attention in recent years. Partially Observable Markov Decision Process (POMDP) is a sequential decision making framework particularly suited for tackling this problem. POMDP has this far mainly been used in robotics for a discrete-state formulation. Only few authors have dealt with the solution of the continuous-state POMDPs. This paper introduces the concept of approximating the continuous state using a mixture of Gaussians in order to render this methodology suitable for the problem of optimal maintenance planning in civil structures. Presently, a large part of existing infrastructure is reaching the end of its expected lifespan. The POMDP framework is used herein in order to take deterioration processes into account and to accordingly plan the optimal maintenance strategy for the remaining lifespan. The capabilities of the method are demonstrated through an example application on a bridge structure. Keywords POMDP • Maintenance planning • Uncertainty • Normalized unscented transform • Continuous-state space 13.1 Introduction Partially Observable Markov Decision Processes (POMDPs) constitute an efficient methodology for planning under uncertainty. This formulation relies on a two-stage approach: in a first stage the agent executes an action which changes the state of its surrounding environment, whilst in a second stage the agent receives feedback from the system in terms of, firstly, information and secondly, rewards. All processes include uncertainty so as to account for the inaccuracies and lack of knowledge pertaining to real world systems. An advantage of the POMDP framework is its versatility. In recent years the method has been applied in the fields of portfolio management [1], optimal teaching strategies [2], gesture recognition [3], and fishery management [4] just to name a few. In the field of engineering it has been used mainly in robotics [5, 6]. In civil engineering, POMDP has been used in very few instances for the purpose of maintenance planning. Optimal maintenance planning is mainly performed using alternative methods like Bayesian ones [7–9], probability trees [10], or risk based inspection methods [8, 11]. One of the first implementations of POMDPs in civil engineering was performed by Ellis et al. [12] who used a discrete-state formulation. Due to its versatility, the POMDP approach can be adapted for the case of optimal maintenance planning as will be demonstrated in the current paper. This paper initiates by illustrating the continuous-state POMDP approach. Later, the adaptation to a general problem of maintenance planning is enabled by introducing the nonlinear stochastic action model. This is followed by an illustrative implementation on the maintenance planning of a bridge system. R. Schöbi ( ) • E. Chatzi ETH Zurich, Wolfgang-Pauli-Strasse 15, 8093 Zurich, Switzerland e-mail: schoebi@ibk.baug.ethz.ch H.S. Atamturktur et al. (eds.), Model Validation and Uncertainty Quantification, Volume 3: Proceedings of the 32nd IMAC, A Conference and Exposition on Structural Dynamics, 2014, Conference Proceedings of the Society for Experimental Mechanics Series, DOI 10.1007/978-3-319-04552-8__13, © The Society for Experimental Mechanics, Inc. 2014 135
RkJQdWJsaXNoZXIy MTMzNzEzMQ==