Markov decision process for sewer rehabilitation

Finding the optimal solution to address problems in sewer management systems has always challenged asset managers. An understanding of deterioration mechanisms in sewers can help asset managers in developing prediction models for estimating whether or not sewer collapse is likely. The effective use of deterioration prediction models along with the development and use of life cycle cost analysis (LCCA) can contribute to the goals of reducing construction, operation and maintenance costs in sewer systems. When sewer system maintenance/rehabilitation options are viewed as investment alternatives, it is important, and in some cases, imperative, to make decisions based on life cycle costs instead of relying totally on initial construction costs. The objective of this paper is to discuss the application of deterioration modelling and life cycle cost principles in sewer system management, and to explore the role of the Markov chain model in decision making regarding sewer rehabilitation. A test case is used to demonstrate the application of the Markov chain decision model for sewer system management. The analysis includes evaluation of this concept using dynamic programming and the policy improvement algorithm.