Visual inspection remains key for assessing the condition of bridges and hence assisting with planning and maintenance activities. There have been many efforts to improve or supplement visual inspection processes using new sensing technologies and data capture methods to usher in an era of ‘smart bridges’ or ‘smart infrastructure’. One method to improve data capture is a ‘remote inspection’ where inspectors use digital photographs of a bridge to identify and grade structural defects to the standard of a ‘general inspection’. In this paper, survey data are presented to help formulate a preliminary assessment of the potential for engineers to implement this possible evolution of the visual inspection process. A potential schema for remote visual inspections is developed and presented as a conceptual web application. The focus on the development of the schema includes the need for ease of use by inspectors and integration of collected digital data into bridge management systems. The suggested platform is seen as a transitional method to aid in the long-term implementation of further automation of the inspection process. The system architecture is provided along with possible technologies that may support or enhance it, as well as a discussion of the potential barriers to implementation.
1. Introduction
1.1 Background
An array of technologies and novel methods exist that can be used in routine bridge inspection to enhance data gathering, improve inspector safety and reduce the number and impact of potential road closures (Nepomuceno et al., 2022). Recent technological advances may include systems that perform structural health assessment or condition monitoring based on a variety of devices, such as data collected through wireless sensors (e.g. Gunner et al., 2017; Martać et al., 2020), video surveillance (Waterfall et al., 2012) and fibre optics (Alexakis et al., 2019). Another possible method is ‘remote inspection’, where an inspector uses digital images of a bridge structure to identify and grade defects to the standard of a ‘general inspection’ (GI) (see Bennetts et al. (2020) for more details on the visual inspection regime in the UK). In this paper, survey data are presented to evaluate the readiness of potential users to trial and/or accept proposed technological evolutions to the visual inspection process and to investigate how the proposed system can be made more user-friendly and implementable. The work proposes a potential schema for this alternative method of routine inspection as a conceptual web application. A focus is placed on developing a framework that would be easier to deploy during day-to-day operations as part of the transition to a more automated process. The overall system architecture is outlined, along with the prospective technologies that could support it, as well as the potential difficulties for implementation.
Innovation of various aspects of the process of bridge condition inspection has been the topic of recent studies (e.g. Achuthan et al., 2021; Javadnejad et al., 2017; Kruachottikul et al., 2019, 2021; Perry et al., 2020; Sacks et al., 2018). The research presented in Perry et al. (2020) and Achuthan et al. (2021) focuses on processes involving the usage of unmanned aerial vehicles (UAVs), which are gaining considerable attention in bridge management operations and other civil engineering applications (e.g. Freeman et al., 2021); although these studies are highly topical, the majority of the content is UAV-specific, and less applicable within the scope of this paper. In Sections 1.2 to 1.4, three of the aforementioned studies are addressed in greater detail, as these serve as inspiration for the development of the novel schema proposed in this study.
1.2 Bridge visual defect quality control assisted mobile application (Kruachottikul et al., 2021)
For Thailand's Department of Highways, Kruachottikul et al. (2021) presented a ‘user-centric bridge visual defect quality control mobile application’ to facilitate communication and aid field professionals with inspection of visual defects. The app developed by Kruachottikul et al. (2021) is aimed at two key stakeholders, defined as ‘users’: project managers and bridge inspectors (Kruachottikul et al., 2021). Project managers can add projects and bridges that require inspection, and assign specific tasks to users during the inspection processes (they also have the authority to approve the result of the artificial intelligence (AI)-assisted inspection) (Kruachottikul et al., 2021). Each inspection generates data that are saved on a cloud server and may be used to create maintenance plans and increase the accuracy of the AI (Kruachottikul et al., 2021). Users are restricted to task-specific functions, where they can add and amend information about damage to the bridge's, as well as upload a photograph for AI analysis (Kruachottikul et al., 2021). When trialled with 14 users from Thailand's Department of Highways, an overall satisfaction score of 4.024 was received (on a scale of 0 (unsatisfactory) to 5 (very satisfied)) (Kruachottikul et al., 2021).
1.3 Semantic enrichment engine for bridges (Sacks et al., 2018)
Sacks et al. (2018) developed a novel method called a ‘semantic enrichment engine for bridges (SeeBridge)’ for compilation of data for bridge inspection management, which utilises building information modelling (BIM) and point cloud data processing. Sacks et al. (2018: p. 135) proposed four enhancements to the process:
- (a)
a data gathering system for bridges that makes use of remote sensing approaches
- (b)
a structural object recognition and classification software that automates the creation of three-dimensional geometry from remote sensing data
- (c)
a semantic enrichment engine that uses forward chaining rules developed from expert knowledge to produce a ‘semantically rich BIM model’ from the three-dimensional model
- (d)
a damage detection tool for identifying, quantifying, classifying and integrating damage information into the BIM model.
1.4 BridgeDex (Javadnejad et al., 2017)
Identifying the need to organise and manage inspection data in a systematic manner, Javadnejad et al. (2017: p. 10) developed ‘BridgeDex’, a web-based application for managing and querying ‘multiscale/multiyear bridge inspection images, bridge reports, and other relevant metadata’. BridgeDex primarily comprised two components (Javadnejad et al., 2017): (a) BridgeDex-map, a web-based geographical information system (GIS) tool that displays the locations of bridges in a database in plan view, and (b) BridgeDex-profile, a ‘profile view’ of each structure enabling users to view large numbers of photographs. In addition, the user can make use of a library of metadata related to scanned bridge inventories, which includes ‘inspection notes, non-destructive test results, and structural drawings’ (Javadnejad et al., 2017: p. 10).
1.5 Scope and aims
This paper has two primary aims: (a) to use new survey data to establish criteria for design of the new schema for bridge inspection and (b) to present the new system architecture and explain the intended functionality. The developed schema incorporates key elements of those proposed by Javadnejad et al. (2017), Sacks et al. (2018) and Kruachottikul et al. (2021), but does not call for use of any algorithm, AI process or otherwise to identify and rate defects – this is left to the human ‘off-site inspector’ and therefore the developed schema represents a pathway to move to a more automated visual inspection system.
2. Candidate system architecture
Figure 1 illustrates the proposed system architecture for a potential inspection workflow. This workflow is subject to the demonstration that the separation of image capture from defect interpretation provides adequately comparable results to on-site inspection. In this system, the role of on-site teams is to ensure high-quality image capture. This process may involve the use of high-resolution cameras, drones (Tomiczek et al., 2019), light detection and ranging (Lidar) (Omer et al., 2019) and/or an automated image capture system such as the one employed in McRobbie et al. (2015). The resulting images are sent to the service provider office off-site. There may be a pre-processing team that stitches the images together to form a three-dimensional model or to double check the quality of the images. The resulting high-quality images can also be stored onto an internal service provider database for future reference. Over time, as more bridges are inspected using this new regime, the number of high-quality image models for each structure would increase.
System architecture for potential future use of the proposed inspection workflow (created in Lucidchart (2022))
System architecture for potential future use of the proposed inspection workflow (created in Lucidchart (2022))
This inspection workflow (Figure 1) has potential advantages over the current labour-intensive workflow. The first time a bridge is inspected in this way may be time consuming, as optimum image capture point positions are determined. However, once these have been accurately measured and stored, the subsequent inspection should take less time. Advantages of less time spent on-site include the reduction in health and safety risk to the bridge inspector and network users, and a reduced need for traffic management. Depending on the technology employed (e.g. drones), closer imagery for deck soffits over rivers and valleys may also be obtained without the need for roped access or under-bridge units.
Bridge structures are unique and can vary greatly in complexity from bridge to bridge. A steel lattice girder bridge, for example, would be complex to photograph at all the angles needed to check for corrosion traps and deterioration. Bridges with numerous bearings would need them all to be photographed so the off-site assessor could view every single one. However, in the current process, an inspector looks at each bearing, makes notes on their condition, but only photographs typical examples of sound and defective bearings – as opposed to all of them. The trade-off in time between photographing every part of a bridge against the current inspection process should be explored. Perhaps this workflow could be applied to bridges of less complex design initially. This potential inspection workflow may have implications on the current business model. A service provider could theoretically build up a large database of high-quality images for structures under their remit, and start to market it as a product to asset owners. This process would, however, bring up thematic issues such as data ownership and security. For example, it could be questioned who has ownership of image data that represent an asset owner's structure, but were captured by a service provider. If this theoretical database increases in value to match the cost of current service contracts, increased security measures will have to be put in place to safeguard the data.
3. Bridge visual inspection photographs
Visual inspections conducted on a routine basis are a vital part of the UK's highways inspection regime for UK highway infrastructure and remain the key source of structural condition information (see Bennetts, 2019; Bennetts et al., 2016, 2020). Inspectors are expected to evaluate prior inspection reports on the structure throughout the planning and preparation phase, as well as becoming familiar with the basic design of the structure and its location. The inspectors then physically assess the bridge for problems, and practically all inspections include the taking of digital photographs as they travel around the structure.
According to a consultation process conducted by McRobbie et al. (2015), engineers accept photographs as evidence to assist them in assessing bridge condition. However, these images are often being recorded and provided to engineers in an ad hoc fashion and in an unsystematic manner (McRobbie et al., 2015). This approach results in a ‘partial image record’ of the bridge and makes it challenging to compare images taken at subsequent inspections (McRobbie et al., 2015).
4. Industry perspectives: survey results
The viability of a photograph-based remote inspection process to obtain defect ratings was examined. As part of the trial, a questionnaire was given to the ten participants. The participants included industry practitioners and academics with expertise in the bridge management sector (see the forthcoming thesis of Nepomuceno (2022) for more details on the survey conducted). The third author of the present paper was one of the academic representatives who participated in the survey for benchmarking purposes (the results are aggregated in the results shown in this paper). It is acknowledged that with a larger sample size the results presented in this paper may change.
In this questionnaire, a section was included to elicit participants’ initial impressions on the acceptability of photo-based remote inspection in industry practice. The relevant questions for this paper are outlined in the Appendix. This section was optional, and only individuals who regarded themselves to be ‘working bridge inspectors’ were to respond. This section received eight responses. Participants were asked to identify potential benefits of implementing a remote inspection process in one of the questions. The lead author presented a list of possible benefits, and respondents were asked to select all that applied. Figure 2 illustrates the results.
Questionnaire responses to the question ‘Which of the following benefits could arise from adopting a photo-based inspection?’
Questionnaire responses to the question ‘Which of the following benefits could arise from adopting a photo-based inspection?’
All eight respondents indicated ‘Improved health and safety’, making it the most anticipated benefit. Following that, ‘reduced inspection cost’ and ‘reduced network disruption’ were both cited by seven of the eight respondents. One of the eight respondents saw no value in using a photograph-based inspection. The findings of this poll are promising, demonstrating that engineers responsible for bridge maintenance see real benefits in implementing a remote inspection process.
Another survey question asked participants to rate their confidence in using a photograph-based inspection, as opposed to a typical in-person inspection. Confidence was rated using a scale of 1 (‘no confidence’) to 5 (‘extremely confident’). A rating of 3, implied a neutral opinion. Another question was also given to reflect the participant's level of confidence in a photograph-based inspection, but with major enhancements (such as those detailed in Table 1).
Mean ratings for each potential improvement in Figure 4
| Improvement to process | Description | Mean rating |
|---|---|---|
| Detailed structure drawings | Access to detailed structural drawing | 4.1 |
| 360 cameras | 360° camera photos allowing you to orient yourself | 4.0 |
| Compass/GPS | Having images overlaid with compass and/or GPS coordinates | 3.9 |
| Grouped photos | Having photographs grouped in folders by element/location | 3.7 |
| Higher image resolution | This would minimise low-quality, blurry photos | 3.3 |
| Standardised images | Standardised image framing (e.g. images captured with consistent focus, lighting, exposure, etc.) | 2.8 |
| Improvement to process | Description | Mean rating |
|---|---|---|
| Detailed structure drawings | Access to detailed structural drawing | 4.1 |
| 360 cameras | 360° camera photos allowing you to orient yourself | 4.0 |
| Compass/GPS | Having images overlaid with compass and/or GPS coordinates | 3.9 |
| Grouped photos | Having photographs grouped in folders by element/location | 3.7 |
| Higher image resolution | This would minimise low-quality, blurry photos | 3.3 |
| Standardised images | Standardised image framing (e.g. images captured with consistent focus, lighting, exposure, etc.) | 2.8 |
Figure 3 presents the responses to both questions. If major changes are achieved, a favourable increase in confidence can be noted, with seven respondents giving a confidence level of 4. This result appears to indicate that respondents might be receptive to modifications to the visual inspection process, especially with new technology incorporated into the process. Most working inspectors in the study perceive potential benefits and would be willing to accept a refined approach in the future.
The results from the questions outlined in the Appendix were used in the development of the current schema. Specifically, these were the responses to a question in which participants were presented with a list of possible modifications to the photograph-based inspection that could increase defect-grading accuracy. Table 1 summarises the potential enhancements. Participants were asked to rate the perceived effectiveness of each prospective enhancement on a scale ranging from 1 (ineffective) to 5 (very effective). Figure 4 shows the results. For each future enhancement, a broad variety of responses were received. Table 1 displays the mean ratings and sorts them by perceived effectiveness. Having access to detailed structural plans during an inspection received a 4.1 effectiveness rating, followed by the usage of 360° cameras and attaching a compass/global positioning system (GPS) data to each photograph, which received 4.0 and 3.9 effectiveness ratings, respectively. It is worth noting that respondents viewed higher image resolutions and having standardised images as having the least potential for efficacy, with considerably more divided responses for each.
Questionnaire responses to the question ‘Please rate the effectiveness of these potential adjustments to the photo-based inspection that could improve defect-grading accuracy.’
Questionnaire responses to the question ‘Please rate the effectiveness of these potential adjustments to the photo-based inspection that could improve defect-grading accuracy.’
From these results, the following criteria for a schema for remote inspections was formed:
- ▪
remote inspectors should have access to detailed structural drawings for a bridge structure where possible
- ▪
remote inspectors should have the ability to know their orientation and position relative the structure when examining photographs
- ▪
360° cameras should be considered for use during the image capture phase.
5. Proposed schema
The proposed schema draws on the work presented in Javadnejad et al. (2017), Sacks et al. (2018) and Kruachottikul et al. (2021) and incorporates some elements from the systems proposed in the aforementioned studies. The key difference is that in Kruachottikul et al. (2021) AI is used to identify defects and evaluate severity and in Javadnejad et al. (2017) the BridgeDex system utilises an algorithm to evaluate change of aspects of the bridge. Sacks et al. (2018) explains that SeeBridge contains an algorithm for damage detection. The schema proposed in this paper retains a human inspector (located off-site) to identify defects and rate their severity as an intermediate arrangement until it is established that the automated systems can be relied upon for decision-making processes and that bridge engineers accept their use for day-to-day bridge management decisions.
5.1 Overview
Figure 5 shows the key stakeholder actions in the remote inspection process. This process is illustrated using a unified modelling language (UML) use case diagram. In UML use case diagrams, the central box (or ‘container’) represents a system or process which contains the system's key actions. The figures surrounding the central container represent ‘actors’ (i.e. stakeholders) that interact with the system. Actors on the left-hand side denote stakeholders that directly affect and influence the system, while the actors on the right are more passive and likely to act in response to a system process.
UML use case diagram of key stakeholder actions in remote inspection process (created in Lucidchart (2022))
UML use case diagram of key stakeholder actions in remote inspection process (created in Lucidchart (2022))
The key stakeholders in this remote inspection schema are now described, along with their respective duties. The on-site photographer's primary responsibility is to travel to the bridge site, capture high-quality photographs, and upload them to a database. The off-site inspector would then examine the captured images and use them to identify and rate defects. The data generated would be in the form of a table including all of the defects discovered and their associated attributes, as well as any associated defect images. These output data would then be used to input defect ratings with the bridge management system (BMS) used by the asset owner. The final ‘active’ stakeholder in this schema is a project manager, whose function is to organise the bridge structures that require inspection (in collaboration with the asset owner) and assign the off-site and on-site tasks. Finally, the asset owner can then take appropriate measures based on the inspection reports generated by the system.
Figure 6 elaborates on the UML case diagram, by showing the workflow of this remote inspection process. The workflow is structured using the six fundamental stages of visual inspection: (a) ‘planning and preparation’; (b) ‘image capture’; (c) ‘defect identification’; (d) ‘defect grading’; (e) ‘interpretation of change over time’; and (f) ‘making maintenance decisions’ (expanded from Bennetts (2019) in Nepomuceno et al. (2021)).
Workflow diagram of remote inspection schema (created in Lucidchart (2022))
During the ‘planning and preparation’ stage, it may be expected that both the on-site photographer and the off-site inspector review any relevant documents related to past inspections of the bridge structure being assessed. This procedure would entail past inspection reports as well as photographs from previous inspections (such as GIs and principal inspections (PIs)). For on-site photographers, this stage will allow them to become familiar with the basic design of the structure and the types of defects to expect. Following on from that, the on-site photographer travels to the bridge site to take digital photographs, preferably using a high-resolution GPS camera equipped with geotagging capabilities. In addition, a camera equipped with a magno-compass would be desirable, allowing for the acquisition of orientation data (i.e. knowing which way the camera is facing at the time of capture). The on-site photographers are expected to have greater competence in high-resolution photography (to reduce the number of low-quality photographs) and will not be required to make judgements about the type or magnitude of defects discovered.
Once the necessary photographs have been acquired for inspection, they can be uploaded to a database that the off-site inspector can access. This upload would be accomplished using a bespoke remote inspection web application, the intended functions of which are detailed in Section 5.2. This upload may be completed promptly ‘on-site’ for a small number of photographs (e.g. less than 20). After this upload phase, the photographs are processed to overlay a compass and any GPS/orientation data onto the image.
Once uploaded, the off-site inspector can use the remote inspection app to view the processed images in a systematic manner. The inspector can attribute a defect's class, severity, extent and priority rating to each photograph (see Figure 7). The off-site inspector would also have access to a map view within the app, which offers an aerial view of the overall structure, displaying the location and orientation of the on-site photographer for a specific image. Together with the superimposed compass, this visualisation should aid inspectors in determining the location of defects. While navigating through the photographs, the off-site inspector will have access to any necessary documentation pertaining to the bridge structure. As ratings are assigned, the data are aggregated into a table (csv format) that can be exported from the app and used to input into the appropriate BMS.
As this system would be used to conduct consecutive inspections, images from multiple years will be collected. Within the inspection app, images of significant defects dating back several years can be examined to aid in interpreting how a defect has changed over time by the off-site inspector.
Finally, more standard procedures are used to make maintenance decisions. The off-site inspector can compute performance indices for a structure and the pertinent BMS can help analyse a network of structures. This analysis would then assist the asset owner with planning maintenance activities.
5.2 Web application
The functionality and high-level network architecture for the web application in the proposed remote inspection schema are described in this section.
5.2.1 Functionality
The UML use case diagram for the remote inspection app is shown in Figure 8. This diagram also depicts the relationships between key actions using ‘include’ and ‘extend’ terminologies. ‘Include’ denotes when a follow-up action is conducted immediately after a preceding action. The term ‘extend’ refers to an optional activity that may occur because of a preceding action. All permitted users of the app will have their log-in information created for them by the system's administrator. The key actions within the app are shown in Figure 8. Users will be restricted to certain actions based on the following authorisation levels:
- ▪
level 1: log-in, view structures
- ▪
level 2: log-in, view structures, upload photographs
- ▪
level 3: log-in, view structures, upload photographs, inspect photographs, change settings.
UML use case diagram of remote inspection app (created in Lucidchart (2022))
Level 1 access will be granted to asset owners, allowing them to view structures and pertinent photographs within their stock. On-site photographers will be granted level 2 access, allowing them to upload images for a specific structure. They will, however, be unable to rate defects. Finally, project managers and off-site inspectors will be granted level 3 access, enabling them to use all functions within the app. By establishing these authorisations, the possibility of assigning incorrect ratings is reduced.
5.2.2 Graphical user interface
This proposed web application is intended to enable inspectors to assign ratings rapidly to defects visible in photographs. The graphical user interface (GUI) is discussed in this section. Figure 9 shows a concept GUI for the inspection capability of the web app. The screen would be divided into three sections: (a) a left-hand toolbar; (b) the central image display; and (c) a right-hand toolbar. Users can navigate through a list of all available photographs for a structure using the left-hand toolbar. The selected photograph would be displayed in the centre of the screen. Using the buttons at the top of the screen, inspectors could assign values to defect attributes. When a button is clicked, a drop-down list of all available options would be displayed. Figure 9(a) demonstrates this for severity. Additionally, a compass is overlaid in the top right corner of the display, indicating the camera's orientation for the relevant photograph. The position can be changed within the app's settings. The left-hand toolbar would allow users to switch to a map view (see Figure 9(b)). By selecting this option, an aerial view of the structure is displayed, along with a red dot-and-arrow representing the camera's location and orientation at the time of capture. This visualisation may assist inspectors in determining the defect's location.
(a) GUI example of standard photo view (photo courtesy of WSP). (b) GUI example of map view (inset image © Google Maps 2022)
(a) GUI example of standard photo view (photo courtesy of WSP). (b) GUI example of map view (inset image © Google Maps 2022)
The right-hand toolbar can be used to toggle between historical comparisons, related documents and image meta-data. Using ‘History’ enables the user to access previous years’ photographs of the same defect or element. However, only ratings for the current year of inspection could be assigned. The image registration techniques presented in Bush et al. (2022) could be used to allow comparison of defect photographs over time. This functionality is possible using the image raster data system described in Javadnejad et al. (2017). In addition, users will be able to view available reports and structural documents pertaining to the structure under inspection. This feature was indicated as a useful improvement in the questionnaire responses outlined in Table 1. Access to these relevant documents will have to be considered together with the asset owner. Finally, meta-data extracted from the exchangeable image format (EXIF) data of a digital image could be viewed. This visualisation could include information about the time and date of the capture, the GPS coordinates, the image resolution, the orientation of the image and the camera serial model.
5.2.3 Network architecture (based on Javadnejad et al. (2017))
The network structure for the remote inspection could be configured as seen in Figure 10 (to some extent based on the BridgeDex system of Javadnejad et al. (2017)). As a web-based application, the architecture will include the use of a web server, with which the client will interact primarily. Cloud-based storage has become increasingly prevalent in recent years and enables document storage to be centralised. The folder structure of the website will contain all the web pages (and components) for each bridge, including scripts, bridge photographs and assigned defect rating data. The HyperText markup language (HTML) files will contain the page structure, as well as the text and headers for the interactive forms and toolbars contained within the app. Cascading style sheets (CSS) files will be used to style and lay out each page of the website. To add interactivity to web pages, JavaScript (JS) code can be used and stored in JS files. Additionally, HTML files can be used to store images, documents and ratings. To develop the web application, additional work on the software development will be required.
Network structure for remote inspection app (created in Lucidchart (2022))
6. Discussion
6.1 Benefits
The risk to inspectors’ health and safety would be reduced when they are not required to be on-site. On-site photographers, on the other hand, are expected to spend less time on-site than a traditional inspector because they will be less concerned with rating defects and will spend less effort remembering defect types and measurements. Similarly, there would be a potential reduction in the need for highway closures. This outcome is especially relevant if 360° cameras are used to inspect the soffits of bridges. In this case, a system in which a 360° camera is mounted on a slow-moving vehicle and captures the underside of a bridge could be designed.
Such a system would introduce a division of labour into the process, potentially resulting in more efficient use of the workforce. By employing on-site photographers with photographic expertise to capture the images, images of higher quality are better ensured. Simultaneously, by exclusively delegating defect grading to qualified and experienced inspectors, inspectors can focus their time on tasks that best utilise their expertise. A suitable analogy would be the process of medical doctors performing X-ray examinations. Qualified radiographers acquire the patient's X-ray images and send them to a qualified medical doctor for diagnosis. While both individuals understand each other's field, they are primarily assigned to tasks that require their specific expertise. In addition, this ensures that inspectors can evaluate defects during business hours and in an office environment, where engineering judgement is more likely to be at its peak. This remote inspection schema enables systematic data analysis procedures to be implemented. For example, there is the potential to introduce calibration by having a portion of defects double graded or graded by multiple inspectors. This improvement would be made possible by the app's intended ability to assign defect attribute values rapidly.
6.2 Considerations
6.2.1 User qualifications
The schema does not remove the need for inspector training. If deployed in the UK, it is recommended that project managers and off-site inspectors are certified under the bridge inspector certification scheme (BICS) (Lantra, 2016) before undertaking their respective roles. It may be the case that on-site photographers must complete phase 1 of the BICS, or at the very least achieve a rating of A (awareness) in the following specific core units: (a) ‘Introduction to inspections’; (b) ‘Structure types and elements/Behaviour of Structures’; (c) ‘Inspection process’; and (d) ‘Defects descriptions and causes’. To ensure high-quality image capture, an appropriate training/selection process for on-site photographers should be put in place.
6.2.2 Viable structures
It may be the case that this schema is implemented for certain structure types only. Several comments were provided in the ‘industry adoption’ section of questionnaire B (Appendix C) (see Nepomuceno (2022), forthcoming). One respondent stated: ‘Once structures get above a certain size, … the number of photos to be reviewed would get very daunting, and the inspector could easily get lost amongst the data. I have experience of uploading others’ inspection notes that this can easily happen’. This comment perhaps indicates that only bridges below a certain length should be inspected in this way, or that a method to position images on a three-dimensional visual of a bridge is needed (see e.g. the ‘SeeBridge’ system described in Sacks et al. (2018)).
However, the subjectivity on the number of photographs to be taken during inspections is made apparent in the following comment from the same section of the questionnaire: ‘Most inspectors typically take photos of defects, rather than areas where there are not defects. Would probably need a larger quantity of photos of the structure to enable a ‘remote inspector’ to have confidence that areas of concern had not been missed’.
6.2.3 Communication between on-site and off-site teams
Even though the tasks of image capture and defect grading are separated and exclusively delegated in this schema, communication channels would still be needed between on-site photographers and off-site inspectors. One questionnaire respondent stated that: ‘A 5 min chat with the person that took the photos’ would be helpful in ensuring the remote inspector's confidence in their ratings. In light of this observation, it may be useful to record a brief audio or video clip of the on-site photographer's remarks or a short transcript.
6.3 Future proofing
The schema outlined in this paper intentionally holds back from mandating a need for any novel technology or algorithm. By doing so, it theoretically makes it easier to implement in current day-to-day operations and procedures practised in industry today. Figure 11 depicts a system architecture structured around the six fundamental stages of visual inspection. This system includes the environmental and social factors that must be considered in each stage. For this schema to be effective, it is likely that its users will need to adapt to a new way of working, which is perhaps a significant obstacle to implementation.
Future remote inspection process based on the fundamental stages of visual inspection. For each stage, potential technologies and environmental and social factors are outlined (created in Lucidchart (2022)) (note: InSAR, interferometric synthetic aperture radar; 3D, three-dimensional)
Future remote inspection process based on the fundamental stages of visual inspection. For each stage, potential technologies and environmental and social factors are outlined (created in Lucidchart (2022)) (note: InSAR, interferometric synthetic aperture radar; 3D, three-dimensional)
6.4 Innovative aspects of the proposed schema
According to the survey responses, both academic and industrial practitioners in the field of bridge management are considering novel techniques to improve the visual inspection process. The schema in this paper incorporates some elements of the systems described in Section 1 (Javadnejad et al., 2017; Kruachottikul et al., 2021; Sacks et al., 2018), and the authors suggest that the novelty of this schema lies in: (a) its basis in and applicability to the visual inspection process in the UK, and (b) the overarching nature of the framework in which other innovative systems can fit in to. By including the six fundamental stages of visual inspection (Nepomuceno et al., 2021), this framework can assist with the development and implementation of the promising web applications and data processes described in Section 1.
However, it is noted that the important next steps in this research work are as follows: (a) to consult working bridge inspectors to develop aspects of the schema for practical use; and (b) to develop a prototype of the proposed web application and compare the resulting defect rating data to traditional inspection results.
6.5 Limitations
The main limitation of the proposed schema is that a human inspector is still required to view the captured images to identify and score defects, and this process remains time-consuming and may lead to ‘bottlenecks’ as the system focuses on ‘image acquisition’ only (cf. Kruachottikul et al., 2019). It is also noted that the user needs as identified by the survey may change with a larger survey population. However, the proposed schema would allow for further automation in the future, as new systems are trialled and potentially are accepted by the bridge management community.
7. Summary and conclusions
Survey data were used to determine the key criteria for developing a new schema for off-site bridge inspection using a photograph-based remote inspection process. The separation of image capture tasks from defect identification and grading is critical to this process. Four critical stakeholders were identified: (a) a project manager; (b) an on-site photographer; (c) an off-site inspector; and (d) the asset owner. On-site photographers are solely responsible for image capture, whereas off-site inspectors are solely responsible for defect identification and grading.
A conceptual web application that would serve as a tool for the schema is presented. The app's GUI and capabilities were described, as well as a high-level network architecture. It is a web-based application that would assist in the management of bridge inspection photographs, prior inspection reports and other pertinent data useful to off-site inspectors. In addition, it would enable off-site inspectors to quickly rate defects and view images of defects from previous inspections. Overall, the proposed app would make bridge inspection data more accessible and help track evolution of defects with time.
Data availability statement
To maintain anonymity of the survey participants the survey data cannot be made available in a data repository. No other experimental data were generated during this study.
Acknowledgements
The first author would like to acknowledge financial support from WSP, as well as from the Engineering and Physical Sciences Research Council (EPSRC) through the National Productivity Investment Fund (grant number EP/R51245X/1). The authors would also like to acknowledge the UKCRIC Urban Observatories (grant number EP/P016782/1), also funded by EPSRC. The authors express thanks to the anonymous individuals who gave their time to participate in this study. The statements made in this paper do not necessarily reflect the opinions of the organisations involved.
Appendix
The questions in this Appendix (Tables 2–5) have been taken from Appendix C – questionnaire B in Nepomuceno (2022).
Question 5 from Appendix C – Questionnaire B in Nepomuceno (2022)
| Please rate the effectiveness of these potential adjustments to the photo-based inspection that could improve defect-grading accuracy | 1 (Would make no difference) | 2 | 3 | 4 | 5 (Would be very effective) |
|---|---|---|---|---|---|
| Having images overlaid with compass and/or GPS co-ordinates | |||||
| 360° camera photos allowing you to orient yourself | |||||
| Higher image resolution | |||||
| Having photographs grouped in folders by element/location | |||||
| Standardised image framing (e.g. images captured with consistent focus, lighting, exposure etc.) | |||||
| Access to detailed structure drawings |
| Please rate the effectiveness of these potential adjustments to the photo-based inspection that could improve defect-grading accuracy | 1 (Would make no difference) | 2 | 3 | 4 | 5 (Would be very effective) |
|---|---|---|---|---|---|
| Having images overlaid with compass and/or GPS co-ordinates | |||||
| 360° camera photos allowing you to orient yourself | |||||
| Higher image resolution | |||||
| Having photographs grouped in folders by element/location | |||||
| Standardised image framing (e.g. images captured with consistent focus, lighting, exposure etc.) | |||||
| Access to detailed structure drawings |
Question 8 from Appendix C – Questionnaire B in Nepomuceno (2022)
| Compared to current industry practice, how confident would you be in a photo-based inspection procedure exactly like the one you completed? | 1 (No confidence) | 2 | 3 | 4 | 5 (Extremely confident) |
| Compared to current industry practice, how confident would you be in a photo-based inspection procedure exactly like the one you completed? | 1 (No confidence) | 2 | 3 | 4 | 5 (Extremely confident) |
Question 9 from Appendix C – Questionnaire B in Nepomuceno (2022)
| Compared to current industry practice, how confident would you be in a photo-based inspection procedure if significant improvements were made (such as those in Q5)? | 1 (No confidence) | 2 | 3 | 4 | 5 (Extremely confident) |
| Compared to current industry practice, how confident would you be in a photo-based inspection procedure if significant improvements were made (such as those in Q5)? | 1 (No confidence) | 2 | 3 | 4 | 5 (Extremely confident) |
Question 10 from Appendix C – Questionnaire B in Nepomuceno (2022)
| Which of the following benefits could arise from adopting a photo-based inspection? (Select all that apply.) | Reduced potential network disruption |
| More efficient use of labour | |
| Improved health and safety | |
| Reduced inspection cost | |
| Reduced staff travel | |
| None |
| Which of the following benefits could arise from adopting a photo-based inspection? (Select all that apply.) | Reduced potential network disruption |
| More efficient use of labour | |
| Improved health and safety | |
| Reduced inspection cost | |
| Reduced staff travel | |
| None |











