Using Statistical Models and Machine Learning Techniques to Process Big Data from the Forth Road Bridge

Transport infrastructure like bridges play an important role in modern society. Most bridges were designed to have a life span of 120 years. However, the engineering community has discovered that some of these bridges will not live up to the designed life span and are approaching the premature end of their operational life. In the US, it was found that 9.1% bridges which carried 188 million trips per day were structurally deficient in 2016. (ASCE, 2017) Structurally deficient means these bridges require significant maintenance, rehabilitation or replacement. This can be caused by:

• Discrepancies between the designed and constructed structures (Lopez & Love, 2012)

• An unanticipated rate of material deterioration. The material deficiency can be caused by manufacturing errors provided that bridge components are prefabricated. For example, lower grade materials are used during manufacture. (Wardhana & Hadipriono, 2003)

• An Unexpected increase in traffic volume.

Assessments and monitoring on existing bridges are required for maintenance purposes. Conventional assessment techniques on living bridges rely on visual inspections, which could be inaccurate due to the subjectivity of the inspectors, inaccessibility of the structure and irregular inspections. (Phares, et al., 2001) Therefore, a continuous monitoring technique for damage detection purposes is required. Inspired by the disadvantages of conventional evaluation techniques and developments in the informatics industry, the Structural Health Monitoring (SHM) method has drawn attention in recent years. SHM can be described as a combination of multiple systems including sensory, data manipulating, damage detection and modelling to acquire information about the integrity of in-service structures continuously. (Bakht & Mufti, 2015)

There are two approaches for data-interpretation algorithms for SHM systems data: model-based (inverse strategy) or data-driven (pattern recognition). (Worden & Dulieu-Barton, 2004) In the model-based method, data collected from SHM systems are used to develop detailed numerical models (e.g. finite element models). By comparing the predicted responses and measured ones, anomalies can be discovered. However, for bridges, these kinds of models have proved difficult and expensive. (Cavadas, et al., 2013) By contrast, in the data-driven method, no numerical models of the structures themselves are required. Anomalous behaviours are detected through patterns in the data. Thus, data-driven methods are faster and more efficient than model-based methods. Pattern recognition becomes critical in this method.

Farrar and Worde (2006) presented the use of statistical pattern recognition for analysing the data collected by SHM systems. This can be summarised in four steps:

• Operational evaluation - justification of employing SHM techniques

• Data pre-processing - preparation for data analysis

• Feature extraction - perform data analysis to extract relevant information

• Statistical model development - apply algorithms to determine the state of the structure

To extend beyond the application of monitoring, patterns and trends can be detected by applying statistical pattern recognition algorithms. There are a few commonly used methods for anomalies and trend detections in time-series analyses:

Principal component analysis (PCA) – can be used to extract critical information from the data set and present it in a set of uncorrelated variables named principal components. (Abdi & Williams, 2010) Hence, by using a small number of principal components to represent the original data set, further analysis time and computational power can be saved.

Autoregressive integrated moving average (ARIMA) model – can be used in time series data analysis for anomaly detection, notably in seasonal varying responses.

Artificial Neural Networks (ANNs) – machine learning algorithms can be used to generate a predictive model based on exploring the correlations of existing data, e.g. images and time-series data.

Autoregressive moving average (ARMA) models were used to perform time-series classifications for a four-storey frame structure with experimental data. (Carden & Brownjohn, 2008) Nine test configurations were investigated. The results showed the ARMA classifier can identify structural changes in all configurations.

Cavadas, et al. (2013) applied data-driven methods on moving-load data to identify anomalies and their positions on a simple frame subjected to a point-load, including the use of Principal Component Analysis (PCA) and Robust regression analysis (RRA). The results showed successful early discoveries on changes of structural responses and anomaly positions using PCA. RRA reduced the detection time compared with PCA but didn’t provide damage positions.

Omenzetter & Brownjohn (2006) analysed static strain data recorded by the SHM system installed on the Singapore-Malaysia Second Link during construction and in-service period. An Autoregressive Integrated Moving Average (ARIMA) model was used in the analysis. The model was able to detect significant changes during construction and post-construction periods, matched with corresponding events such as cable tensioning and maintenance. This shows the effectiveness of ARIMA models on the detecting structural changes.

Chalouhi, et al. (2017) applied machine learning techniques on data collected from the SHM system installed on a railway bridge located in Northern Italy. The research employed Artificial Neural Networks (ANNs) training and Gaussian processes for pattern recognition and compared predicted deck accelerations with measured ones at each train passage. Any difference between predicted and measured responses can be treated as an anomaly. However, in their research, only deck accelerations were discussed under given environmental and operational conditions. The relation between environmental conditions and bridge responses was not investigated. Furthermore, for a railway bridge, passing train information is relatively easy to acquire than a road bridge where traffic can be hard to anticipate.

Sohn, et al. (2002) applied a combination of time-series analyses, ANNs and statistical inference techniques for damage classification of an 8-DOF experimental system subject to virtually changing environmental and operational conditions. Their results indicated that the proposed methods enabled damage detection when the system is under a range of normal environmental and operational conditions.

Despite the fact that some researchers have successfully used ANNs for damage detection of SHM data, improvements can be made. Firstly, many researches used experimental specimens which cannot represent the complex operational conditions for a living structure. Secondly, the volume of data the researchers collected was insufficient to reliably train an ANN model. Therefore, some researchers only used ANNs for data normalisation purposes. (Sohn, et al., 2002) Furthermore, the potential of ANNs in forecasting is enormous compared with other statistical forecasting models. (Zhang, et al., 1998) ANNs can have non-linear input data with no restrictions on their value or distribution, matching the behaviours of real-world systems. In theory, with some tweaking to the parameters and optimisations, this technique can be applied to different SHM systems. However, a large amount of data is required to accurately train an ANN model. This often increases the cost of monitoring the structure since the accuracy of the model varies with the size and range of the captured data. Furthermore, the underlying mechanisms of a data-driven, non-linear black-box model is difficult to interpret, increasing the difficulty of examining the model to improve our understanding of the theory of a structure.

In this paper, several statistical analyses were used to explore patterns and trends in the data collected by the SHM system installed on the Forth Road Bridge (FRB). The Forth Road Bridge, constructed and opened during the 1960s, is a 2.5 km long suspension bridge located in South Queensferry, Scotland. (Figure 1) The main tower is 156 m above the sea level, with a steel stiffening truss suspended deck with a width of 33 m. (Figure 2) The main span and side spans have a length of 1006 m and 408m respectively. Expansion joints were used and installed under the main towers.

The truss end links of the bridge were discovered to be significantly overstressed during certain combinations of load during a full assessment of the bridge by the Forth Estuary Transport Authority (FETA). Also, FETA suggested the full replacement of these truss end links in 2011. (SPICe, 2016) Following a truss end link failure, the bridge was closed in 2015 for major maintenance. The SHM system was installed during that maintenance.

Around 20 GB of data is produced every month from 180 sensors on the Forth Road Bridge. Hence, the high-performance computing facilities (Eddie 3 and Terra Correlator) from the University of Edinburgh are required for analysis purpose. The data used in this paper covers from December 2015 to September 2017.

The layout of this paper: The “Methodology” section will discuss the background of the statistical models and machine learning methods used for the analysis of the SHM data. The “Results & Discussion” section will present the analysis results and discuss the accuracy of the prediction models. The “Conclusions” section will summarise the paper and suggest the future work.

PCA maps high dimensional data sets to lower dimensions with minimal loss of information.

Firstly, mean normalization is calculated:

Where $xji$ is the value of j with index i, μ_j is the mean of j

The covariance matrix is computed to reduce the data from n-dimension x⁽ⁱ⁾ ∈ ℝⁿ to k-dimension z⁽ⁱ⁾ ∈ ℝ^k,

Where Σ is a n x × n covariance matrix and m is the number of data points.

The eigenvectors of the covariance matrix are calculated using a singular-value decomposition function (svd), giving covariance U size n × n.

To reduce the size of the matrix, k columns are extracted from matrix U, giving a U_reduced matrix with size n × k.

Where z is the transformed values from set x.

An Autoregressive Integrated Moving Average (ARIMA) model will be developed to improve our understanding of the data, estimate the trends and set threshold levels to detect anomalies.

Assumptions The fundamental assumption behind ARIMA is that the time series data will behave and correlate in the future values in the same way as in the past, allowing it as a valid model to detect anomalies. However, the anomalies (e.g. bridge closure) in the past needs to be labelled and maybe normalized so that it does not violate the assumption and drastically change the fitted model.

Theory ARIMA model, as known as the Box-Jenkins method, is a composite of time-series models (Naylor & Seaks, 1972):

AR(p) Autoregressive (AR) model, where the predicted value is based on its previous (lagged) value, with the number of iterations with p, can be represented by:

Where Y_t is the value of data at time index t in the series and ϕ is the AR coefficient.

MA(q) Moving-average (MA) model, where the model is composed of the mean and white noise values, with the number of iterations as q, can be represented by:

Where μ is the mean value, ∈ is the error term and θ is the MA coefficient.

ARMA (p,q) Combining the AR and MA model, we can represent the behaviour as a function of the past values and errors with the following equation:

I(d) Integrated (I) value to remove the non-stationarity of the series into stationary series by the degree of differentiation d:

ARIMA (p,d,q) The ARIMA model can be given by:

Figure 3 is an illustration of the ARIMA model, where the shaded areas are the range of predicted values with a predefined confidence interval.

The stationarity and parameters of the series can be determined using the autocorrelation function (ACF) and partial autocorrelation function (PACF). On the other hand, an iterative approach can be used to determine the parameters instead by comparing the predicted value with the actual value.

Random forest works as an ensemble learning method for classification, regression or other problems. In this paper, it is the regression problem that has been addressed. At the training stage, a random forest contains various decision trees.

Decision Tree A decision tree is a tree-like or flow-chart like structure of decisions and represents possible consequences. In summary, the tree model determines the importance (sensitivity) ranking of the features, then uses a probabilistic approach to generate a predictive outcome.

Random Forest is an ensemble learning method that uses multiple decision trees together, reducing the chance of over-fitting. However, Random Forest models are slow in making predictions, which might be a significant drawback for realtime applications. Also, they have poor performance on predicting the outcome of rare observations, requiring a large range of observations in the training data.

An Artificial Neural Network (ANN) is a computational model inspired by biological neural networks. An ANN consists of artificial neurons, processing elements and connections (like biological synapse) between these neurons. (Shanmuganathan & Samarasinghe, 2016)

The output of an artificial neuron is calculated by a non-linear function of the sum of its inputs. Also, the connections passing “signals” between neurons always have weights to adjust the inputs of a neuron. The weight will influence the strength of inputs to a neuron in its calculations. The “signals” are real numbers. Also, each neuron will have a threshold value to determine whether the neuron should become “active” by the sum of computed inputs. If the neuron is “activated”, the sum of computed inputs will become an output and pass to the next layer.

Therefore, for a supervised learning process (input and output are known for the training set), the following procedures are taken in the ANN:

• Weights are randomly assigned to all connections at the beginning

• Computed outputs can be then acquired through multiple hidden layers

• Calculate the errors between the actual outputs and the computed outputs

• Adjust the weights by the errors

• Iterate the steps above until the computed outputs are highly similar to the actual outputs

The ANNs machine learning method has been widely applied in computer vision and voice.

Long Short-Term Memory (LSTM) Models with Long Short-Term Memory (LSTM) cells are a type of Recurrent Neural Network (RNN) - ANN with connections between neurons within the same layer. As the name suggested, LSTM models can retain and use information from previous time steps. This is particularly useful in time-series analysis, as future values correlate with previous.

It is shown that LSTM is able to model complex nonlinear feature interactions with a large volume high-dimensional dataset. (Ogunmolu, et al., 2016) This gives more flexibility for LSTM compared with traditional ANNs.

Using the data on February 2017, component scores between the sensors were computed. Figure 4 shows the cumulative sum of explained variance at the n_th component.

Figure 5 shows that the 1st component can represent approximately 58% of the variance in 127 columns. Furthermore, 90% of the data in the 127 dimensions can be represented by 3 components, showing some features in the dataset strongly correlate to others, especially for sensors in the same sensor group. PCA also has the ability to normalize extreme values, reducing the noise in the dataset but potentially losing valuable data that describes uncommon behaviours.

This analysis shows the ability to use PCA to reduce the dimensionality of the dataset, reducing computational power required while maintaining the majority of the information in the dataset.

Results from PCA indicates a strong correlation between sensors, which might suggest the structural response is influenced by other factors.

In this paper, the ARIMA model and some machine learning algorithms are used to impute missing data due to faulty sensors and maintenance.

Among all kinds of monitoring data from the SHM system on the FRB, the traffic volume behaves in a seasonal manner with minor anomalies and variations. Therefore, a univariate ARIMA model is suitable to serve as a prediction model. Figure 6 shows a 6-month period traffic volume with 1 month of missing records. Data prior to the missing part was used to train the ARIMA model. The 7-day moving average shows the traffic volume on the bridge has a weekly varied cycle with an indication of seasonal patterns. An additive model was used, representing the time series as a combination of trend, seasonality and noise, to explore the seasonality of the data. (Figure 7)

The trend plot suggests the general traffic volume has no noticeable change throughout the 6 months, except for a significant reduction during the Christmas holiday period.

The seasonality plot suggests a repetitive cycle in the daily traffic volume, with traffic volume on weekdays larger than on weekends, with the highest on Fridays.

The residual plot gives the similar information as the trend plot.

Using the differencing of 336 samples (1 week), the transformed series is visualized in Figure 8. The plot suggests a pattern of randomness, so the parameter d in the ARIMA model can be set to 1. The ACF and PACF plots of the series are shown in Figure 9. The series reaches the critical level at lag 4 in the ACF plot and lag 2 in the PACF plot, indicating the moving-average (MA) parameter of 4 and the autoregressive (AR) parameter of 2 can be used.

Therefore, an ARIMA(2,1,4) model was fitted using the transformed series. By applying a reverse differencing process, the predicted series without differencing can be calculated. Figure 10 shows the predicted and the original traffic volume variation over a period of 11 weeks with approximately 2600 samples, with a coefficient of determination (R²) of 0.981, indicating the predicted series was able to match the seasonal pattern of the original series.

Using the same model, the missing data from 2017-01 to 2017-02 was filled in and visualized in Figure 11. The plot shows the model predicted a similar seasonal pattern.

Event and inspection log can be used to fine-tune the predicted value manually. For example, the traffic volume might be lower if there was an accident or no traffic at all if there was a bridge closure.

Compared with traffic volume data, other types of data, e.g. displacement and strain, are more influenced by environmental and operational conditions of the bridge. Therefore, the seasonality and trends are not as explicit as traffic data. For example, data from strain gauge (id 136) shows weak seasonality and periodical trends. (Figure 12) The ARIMA model is incapable of making predictions on such a dataset. Therefore, other models are required. By using machine learning, it is possible to develop prediction models on this kind of data.

To evaluate the machine learning algorithms, one target sensor and a correlated group with 3 sensors plus traffic data are selected. The details of these sensors are given in Table 1.

The correlation coefficients between all sensors are given in Table 2, where 1 represents total positive linear correlation, -1 for total negative linear correlation and 0 for no correlation. It can be discovered that correlations are generally small among sensors, ranging from -0.2 to 0.2. However, the data of displacement sensor 93 and thermometer 266 have a relatively high correlation, nearly -0.9. As a group, the correlation between the predicting group and the target sensor is higher.

Figure 13 and Figure 14 show the observations of correlated group and target sensor respectively. The whole sample set is chosen as the machine learning training set and a subset containing a part of the missing values is chosen as the predicting set, shown in Figure 15. Two machine learning techniques are applied for the training set data: Artificial Neural Networks and Random Forest Regression.

Artificial Neural Network The number of hidden layers and neurons per hidden layer can be manually set to optimise the results. The model accuracy can also be computed where the best prediction score is 1 and the worst is 0. It is found that training the model with 2 hidden layers and 100 neurons gives the best model accuracy, 0.415. The prediction results on the predicting set is shown in Figure 16.

Random Forest The number of decision trees in the random forest can be manually set to acquire the best model accuracy. The model accuracy achieves over 0.92 with 10 trees. The prediction results on the predicting set is shown in Figure 17.

Model Comparison Artificial neural networks (ANNs) models show a relatively lower accuracy on recognising data patterns. The random forest regression model can deliver more accurate predictions. Therefore, random forest regression behaves better in this scenario. However, the ANNs used in this paper are the basic type which may not be efficient for time-series analysis. Prediction models based on Recurrent Neural Networks (RNN) with Long Short Term Memory cells (LSTM) are still under development by the authors at this moment.

This paper presents an overview of analysis methods used for the very large amount of data collected by the bridge structural health monitoring system. The PCA results suggest that the high-dimensional monitoring data can be aggressively reduced while maintaining the integrity of its information. The use of the ARIMA in prediction models for time-series data with high seasonality and strong periodical trends has proved successful. For more complex data, machine learning methods, random forest and ANNs, are proposed. Random forest shows more accurate prediction results compared with ANNs.

Regardless of the poor behaviours of traditional ANNs on time-series analysis, it cannot be concluded that ANNs are not suitable for this kind of prediction. The use of Recurrent Neural Networks (RNN) is still being investigated by the authors. This kind of neural networks will have better performance on time-series data. On the other hand, RNN makes predictions based on history of time-series data itself. Therefore, preparing a correlated group of sensors is not necessary. This will reduce the dependency on highly correlated datasets which could be difficult to discover for SHM data of real structures.