Discussion: Strategic asset management modelling of infrastructure assets

The authors are to be congratulated for publishing this excellent description of a very stimulating project (Stratford et al., 2010). It would be very helpful if Network Rail could describe the subsequent development work done to update the modelling and future plans since the paper was first written in 2008.

It is important to recognise that the Office of Rail Regulation (ORR) was not able to accept the output from CECASE (Civil Engineering Cost and Strategy Evaluation modelling tool) in reaching conclusions for the 2008 periodic review. They are aware that further development work on the CECASE models is being undertaken for the 2013 periodic review.

The contributors reviewed CECASE for the 2008 periodic review, as well as SACP (Structures Annual Cost Profile) which was used to inform the 2003 determination of Network Rail's funding by the, then, Office of the Rail Regulator. The objective now is to ask a few questions which it is hoped will be of interest to the wider audience.

It is worth quoting paragraph 5.56 in the contributors' final conclusions for the 2008 periodic review (see http://www.rail-reg.gov.uk/upload/pdf/383.pdf).

We have therefore concluded that although the CECASE model provides informed and useful analysis of future activity volume and expenditure requirements, it does not yet do so with the robustness that we consider necessary for us to be able to treat its outputs with sufficient certainty.

In all such decision support tools, at whatever their stage of development and reliability, it is worth keeping in mind that they are just that – a support to the wider decision-making process. It is also worth noting that the water industry relies less on statistical approaches and more on bottom-up approaches. It puts more effort into collecting sound information about the asset, its condition, historical work and associated costs.

Network Rail was unable to demonstrate satisfactorily that the outcomes from CECASE in terms of funding requirements were justified and, as a consequence, ORR was unable to support the full amount requested.

The issues arose from a number of areas

• • the sample asset base selected

• • the linkage to required key performance indicators on the railway

• • the overall accuracy given the large levels of funding being sought (Network Rail were seeking around £2·2 billion for repair and renewal of civil assets in the period 2009–2013), whereas the ORR allowed £1·9 billion; this was an increase of £129 million over CP3 (rounded figures at 2006–2007 prices).

A number of assets were not considered suitable for CECASE modelling, for example those of small population where a ‘bottom-up' analysis was both feasible and more accurate, and rock cuttings where Network Rail had doubts over the output.

Would the authors care to expand on the limitations of the model, particularly linkage to actual asset condition and performance? It is important to emphasise that the short-term modelling (i.e. years 1–5 programme) relies on actual work programmes. Some further discussion on calibration of the output would also be informative.

The ORR commissioned the independent reporter, Asset Management Consultancy Limited (AMCL), to carry out a review of CECASE and details may be found on the ORR website (see http://www.rail-reg.gov.uk/upload/pdf/amc-statistical-sampling-review-220508.pdf). This report includes comment on

• • the sufficiency of the sample size

• • the degree of accuracy and likely tolerance

• • the sensitivity of sample size, asset selection and policy application.

The AMCL report found that the accuracy of the individual asset groups varied by as much as ±33–74%, resulting in an overall variance of ±17%. The report suggests that a greater sample would be required to get the variance down to, say, 10%. AMCL also highlighted concerns over the lack of historical work and cost information, and how realistic the modelling assumptions were against real-life interventions. There were also concerns about linear deterioration assumptions used in the model.

Have the authors taken into account the likely pessimism-bias by engineers who are naturally more likely to be cautious in their outlook?

In Section 4.4.4, the authors note that an accuracy of ±15% was selected as a compromise with the effort required to improve by just a modest amount. Does this accuracy arise from just statistical considerations or does it also make some allowance for accuracy of the cost database, which is referred to in Section 6. Would the authors agree that accuracy of the cost database is also crucial to models of this type?

It would be interesting to find out to what extent the authors found a correlation between the SCMI (Structures Condition Marking Index, as defined by Network Rail) scores and model assumptions, and what interventions have actually taken place in real life. In this respect the model is not used to continually learn and update strategic forecasts.

In Section 7 the reference to asset condition marking is well understood for bridges (structures condition marking index, etc.), but could the authors expand on what they were using for other assets such as earthworks, retaining walls and tunnels.

In Figure 5, why is policy D referred to here? Earlier in the paper the authors note that policy D was discarded.

Figure 6 raises a question, given that for policy B the operator spends more up front but faces a peak around the 55-year mark. However, with policy C, which spends less up front, the peak comes at the 70-year mark? It appears that a policy review at the 30-year mark would have a significant effect on reducing a future bow wave; would the authors concur with this view?

Would the authors care to expand on the ability of CECASE to illustrate sustainable investment over long periods? For example, 35 years was taken in the periodic public review 2008 (PR08), but longer periods, as illustrated in Figure 6, would be better. The reason that ABLE (A Bayes Linear Estimator) is more reliable over the 30 years is the uncertainty over the exact degradation/intervention point(s). Hence would the authors not agree that an ongoing programme would be beneficial in order to ‘learn' from actual work carried out?

Given that technology changes over time, have the authors considered the effect of new materials (e.g. carbon fibre) in maintaining the structures?

With reference to Section 10, could the authors expand on the effect of improvement works carried out on sample structures? It could be argued that such works imply an improvement to the whole asset population that the samples represent. In control period 3, a significant increase in structures funding allowed a number of structures to be repaired or rebuilt, including some in the sample studies.

One of the acknowledged limitations of the CECASE model is that it is not accurate in the short 1–5-year horizon, where more accurate ‘bottom-up' workbank studies provide a useful sanity check. For longer-term scenarios and given current funding constraints the ability of tools such as CECASE to run alternative funding scenarios and demonstrate the impact on asset reliability and performance is a commendable objective, always providing that the likely accuracy is understood.